When you're trying to quantify something that isn't currently being quantified, you can't just focus on the outputs.
Users care about outputs.
But if you're building, auditing, or improving AI systems at scale, you need granular visibility into the system itself.
How do you filter through thousands of queries to detect patterns?
I first explored this in materials optimisation models. The more I work with purpose-built AI systems, the more I realise just how important metadata is.
System analysts need to understand why those outputs emerged. Not just the what. Manual review is impossible over thousands of queries. Metadata enables pattern detection 😉.
I needed to consider the metadata in my current research, when aiming to comply with the EU AI Act.
Without metadata tagging queries by different sub-groups, I can’t measure at scale.
This is implementing NIST AI Risk Management Framework’s MAP function requirement: “Without contextual knowledge, and awareness of risks within the identified contexts, risk management is difficult to perform.” It’s also an ISO 42001 requirement for audit trails in high-stakes AI systems.
If your AI system feeds into decision making (whether materials science, educational assessments, biomedical applications) or just complying with regulations, metadata isn't documentation or an add-on. It is the loop that enables continuous improvement.
Do you obsess over metadata for recognising patterns like I do?
