This is a first, and (we think) is super important.
Confidentiality concerns wrt both proprietary models and sensitive benchmarks are legitimate. Having the ability to run both in secure enclaves gives privacy guarantees which are stronger than contractual obligations, and will hopefully create greater opportunities for external evaluations and subsequently safer AI.
A pleasure working with
@GoogleDeepMind @openminedorg and
@MLCommons on this.