Avijit Ghosh(@evijit):New @evaluatingevals blog: “Mapping Evaluation Cards to Emerging AI Governance Requirements in California, the EU, and the UK”, with @davidmanheim, @Andrxwtran, @Sree_Harsha_N, Jessica Ji, @batznerjan, Wm. Matthew Kennedy, @AnkaReuel, and @IreneSolaiman. Many recent AI governance proposals rely on evaluations, independent auditors, and external verification. But one very practical problem keeps coming up: once an evaluation has actually been run, how should the evidence be recorded and communicated in a way that regulators, evaluators, and developers can all use? Right now, oversight bodies often have to work through provider documentation in many different formats, making it much harder to compare evaluations or automatically parse the information they need. In this work, we took 40 governance requirements across California (AB 1405, SB 813), the EU (AI Act, GPAI Code of Practice), and the UK (JCHR Report), broke them down into 281 individual pieces of evidence, and then asked which of those actually belong in an evaluation record. We identified 60 such pieces, covering topics such as evaluation methodology, configuration, benchmark information, results, and the conditions under which an evaluation was conducted. We find that Evaluation Cards, through a mix of reported and auto-pulled fields, can already represent 56 of those 60 evidence elements! I think the broader opportunity here is that Evaluation Cards can provide a shared, open format for communicating evaluation evidence across developers, evaluators, and policymakers. Instead of every governance process asking for similar information in a different format, a common reporting layer can make that evidence easier to compare, reuse, and analyze. Evaluation Cards are also only one part of a broader ecosystem around AI evaluation and independent assessment. Other standards cover topics such as evaluator independence, model documentation, and incident reporting. The goal is for these pieces to work together, with each capturing the information it is best suited for while making the underlying evidence interoperable across different governance processes. We are already starting to put this into practice through our collaboration with the UK AI Security Institute (with more in the pipeline), and I think there is a real opportunity for evaluation infrastructure and governance requirements to shape each other. Policymakers can help expose where our schemas are incomplete and push them to better reflect global needs, while open schemas can give regulators a much clearer picture of what standardized, machine-readable evaluation evidence can actually look like. Would love feedback from people working on evaluations, auditing, regulation, or assurance. There is a lot more to build here :) Read the blog and explore the interactive map:

2026.09.28 15:27

New @evaluatingevals blog: “Mapping Evaluation Cards to Emerging AI Governance Requirements in California, the EU, and the UK”, with @davidmanheim, @Andrxwtran, @Sree_Harsha_N, Jessica Ji, @batznerjan, Wm. Matthew Kennedy, @AnkaReuel, and @IreneSolaiman. Many recent AI governance proposals rely on evaluations, independent auditors, and external verification. But one very practical problem keeps coming up: once an evaluation has actually been run, how should the evidence be recorded and communicated in a way that regulators, evaluators, and developers can all use? Right now, oversight bodies often have to work through provider documentation in many different formats, making it much harder to compare evaluations or automatically parse the information they need. In this work, we took 40 governance requirements across California (AB 1405, SB 813), the EU (AI Act, GPAI Code of Practice), and the UK (JCHR Report), broke them down into 281 individual pieces of evidence, and then asked which of those actually belong in an evaluation record. We identified 60 such pieces, covering topics such as evaluation methodology, configuration, benchmark information, results, and the conditions under which an evaluation was conducted. We find that Evaluation Cards, through a mix of reported and auto-pulled fields, can already represent 56 of those 60 evidence elements! I think the broader opportunity here is that Evaluation Cards can provide a shared, open format for communicating evaluation evidence across developers, evaluators, and policymakers. Instead of every governance process asking for similar information in a different format, a common reporting layer can make that evidence easier to compare, reuse, and analyze. Evaluation Cards are also only one part of a broader ecosystem around AI evaluation and independent assessment. Other standards cover topics such as evaluator independence, model documentation, and incident reporting. The goal is for these pieces to work together, with each capturing the information it is best suited for while making the underlying evidence interoperable across different governance processes. We are already starting to put this into practice through our collaboration with the UK AI Security Institute (with more in the pipeline), and I think there is a real opportunity for evaluation infrastructure and governance requirements to shape each other. Policymakers can help expose where our schemas are incomplete and push them to better reflect global needs, while open schemas can give regulators a much clearer picture of what standardized, machine-readable evaluation evidence can actually look like. Would love feedback from people working on evaluations, auditing, regulation, or assurance. There is a lot more to build here :) Read the blog and explore the interactive map: