AVERI standards director @prpaskov says the "only 50 AI evaluators" narrative is misleading because it ignores an entire hidden pool of auditors, cyber experts, and NDA-bound teams already doing the work: "When people are giving this 50 evaluators number, they're largely focusing on the evaluation of the model or the system level. At AVERI, we think of it in three parts. You can audit the model or the system, you can audit compute, and you can audit governance." "A lot of frontier evaluations to date have been done under NDAs. So there are more even on the model and system level than are visible." "The skillsets of traditional auditors like the Big Four, KPMG, EY, there are a lot of skillsets in governance auditing and risk management auditing that we can pull from. In the cybersecurity world, tons of skillsets." "There is actually a lot of talent out there. We just need to do a better job at funneling it." @AVERIorg

I spoke with @NPR on the current state of third-party evaluations, alongside experts including @DKokotajlo @AnkaReuel and others. I think this is a fantastic write-by @Jingnan_Huo that captures the nuances of this evolving landscape for a broad and general audience.

AVERI is glad to see that independent auditing of frontier AI was prominently featured in the White House accord signed today. The details of such auditing matter greatly, which is why AVERI works with other members of the @aievalforum to identify best practices and develop standards in areas such as auditor independence, minimum access requirements, and audit scope. We believe that audits conforming to such standards should ultimately be required for leading AI companies, though voluntary commitments are a step forward.

“When companies promise that they’re going to do a better job on safety, we shouldn’t have to take their word for it.” - @random_walker on @CBSNews discussing today’s AEF letter and our call for credible, independent AI evaluations.

over the past week, I’ve gotten a lot of questions about what independent AI evaluations should actually look like in practice. what about conflicts of interest between labs and evaluators? will evaluators actually get meaningful access? what if companies just block the publication of unfavorable findings? these are good and important questions. and we shouldn’t assume voluntary arrangements alone will produce the conditions that credible independent evaluation requires. so i’m very excited to see 100+ researchers, evaluators, and experts — including us at @AVERIorg — laying out minimum conditions for embedded evaluation, including: meaningful independence, strong access, transparency / editorial control, protections against retaliation, and a diversity of perspectives and competencies. and we shouldn’t try to incorporate those principles in ad hoc, one-off agreements, policy is critical to standardizing these safeguards across the industry. glad to see @aievalforum leading on putting these principles on paper.

i'm heartened to see ai companies agree *that* we need embedded evaluations; now the question becomes *how.* answering this question requires many voices. today 100+ respond, calling for five minimum requirements to make embedded evaluations credible. we have a lot of work to do. to meet this demand, @aievalforum -- where i act as senior advisor -- is embarking on an ambitious path ahead to: (1) develop standards for embedded evaluation; (2) create independence-preserving funding mechanisms; and (3) establish a talent pipeline for independent evaluation. we are actively expanding our membership. we also welcome conversations on support and funding. please reach out if you would like to collaborate with us on any of these initiatives.

Today, more than 100 leading AI experts endorsed a set of minimum requirements to take seriously AI companies' recent call to embed external evaluators. These evaluators need to be genuinely independent, transparent, and represent a range of expertise areas. They also need to be guaranteed employee-level access and to be protected from retaliation for findings that make companies look bad. We welcome model developers’ recent calls for independent oversight, but it’s what they do next that matters. The labs must be accountable for ensuring these requirements are met, so that the public can have faith in the process and the outcomes. Over the past week, the AI community has debated the appropriate role of external evaluation, including who should do it and on what terms. We may not agree on everything, but there is a lot of common ground. To make embedded evaluations credible, more than 100 experts with varying backgrounds and ideas about AI risk agree in today’s letter that frontier AI developers should: 1. Guarantee embedded evaluators full editorial independence and mitigate conflicts of interest 2. Rely on multiple evaluators with differing viewpoints and areas of expertise 3. Publicly document the terms under which evaluators operate, as well as facilitating permissive publication of methods and findings 4. Shield evaluators from retaliation 5. Grant access equivalent to that of highly privileged employees There is a thriving and growing ecosystem of independent AI evaluators who are advancing this science every day – but we need aligned standards, guaranteed protections, and independent funding. That’s why we created the AI Evaluator Forum. Today we are entering our next phase. We’re launching an open call for new members, collaborators, and independent funding sources to help evaluators meet this moment and demand accountability from developers. Join us in building the evaluator ecosystem. See the public letter here: Learn more at

The AI Evaluator Forum (AEF) welcomes recent statements regarding the importance of embedded independent experts to verify AI safety and security claims. This is a first step towards trustworthy oversight of frontier AI. No single evaluator can do this work alone. A vibrant ecosystem of independent evaluators from distinct backgrounds can offer a range of expertise and methodology, help to ensure rigor, and avoid a single point of failure. The Forum exists to strengthen this ecosystem. Realizing the benefits of third-party evaluations also requires independence, access, and transparency. AEF brings evaluators together to address these questions. Our first standard, AEF-1: Minimum Operating Conditions for Independent Third-Party AI Evaluations, envisions a minimum floor for access, managing conflicts of interest, funding relationships, recusal requirements, and transparency of evaluation terms. Read more here: The long-term success of third-party evaluation relies on standards and frameworks. AEF is committed to developing these with its members and collaborators. We welcome engagement from organizations committed to rigorous, independent work. Learn more about AEF, express interest in joining, and raise questions for us here:

AVERI commends recent statements by the leaders of Anthropic, OpenAI, and SpaceX regarding the importance of embedding independent experts within frontier AI companies. We are committed to working with various stakeholders to advance standards for frontier AI auditing, and moving towards binding requirements for auditing that aligns with those standards. Our flagship paper, “Frontier AI Auditing,” published in collaboration with experts in industry, civil society, and academia, lays out a vision for third-party auditing of frontier AI companies’ systems and practices and includes embedding as a key component. Embedding helps facilitate deep, secure access to non-public information. Moving embedded auditing from a voluntary practice to an effective and universal industry requirement requires not only pilot projects, but also standards, tooling, and thoughtful policy design. AVERI exists to provide these, and we are conducting pilot projects aimed at informing standards for different aspects of AI audits. Embedded auditing will require clear standards for who counts as a qualified and independent auditor, how audits should be conducted and reported, and what safety and security standards companies should be audited against. Fortunately, we are not starting from scratch: with evaluators across the ecosystem, we coauthored a standard (AEF-1) which covers, e.g., conflict of interest disclosure and management in an AI context. Learn more about vision for frontier AI auditing here:

anthropic unilaterally committing to having embedded independent evaluators is BIG and worthy of commendation. they are not the only ones building the frontier. this should be standard practice across frontier companies and neolabs building similarly capable systems — independent third-party evaluation should be required by law.

Anthropic is unilaterially committing to embedding evaluators to verify safety practices and report incidents. Evaluators will get access badges, company laptops, and "permissions and tools similar to those of internal employees who do comparable risk assessments." This is a remarkable step for frontier AI auditing, transparency, safety, and security. I hope that other companies follow suit. And I look forward to continuing to build the standards, practices, tools, and policy at @AVERIorg to make this work rigorous, effective, and universal.

Want to shape the future of frontier AI auditing? Join us. @AVERIorg is hiring a Security Research Engineer, Pilot Audit Project Manager, and accepts general expressions of interests. Apply here:

This is a first, and (we think) is super important. Confidentiality concerns wrt both proprietary models and sensitive benchmarks are legitimate. Having the ability to run both in secure enclaves gives privacy guarantees which are stronger than contractual obligations, and will hopefully create greater opportunities for external evaluations and subsequently safer AI. A pleasure working with @GoogleDeepMind @openminedorg and @MLCommons on this.