CALL FOR AI DETECTION BENCHMARKS I believe it is vital that third parties are able to audit AI detectors like Pangram. Our largest sample size benchmark, from UChicago, was run on Pangram 3 and is a year old now. New LLMs are being released all the time, and the binary task doesn't capture the full spectrum of real world performance. A great benchmark will help shape how the world interacts with AI over the coming years. I am putting out a call for new third-party evaluations that attempt to answer one or more of the following questions: - How well do current detectors catch text from the latest frontier AI models? What about in high context settings, like deep research or an agent harness? - How well do current detectors do on mixed authorship text? Ideally, research in this area explores the full spectrum of AI assistance from light grammatical edits to full co-authorship. - Any sort of sliced evals around language, background, or ethnicity. Null results that show no bias are still interesting to us. What I value in a benchmark is extraordinarily clean data, frontier LLM data, and a high enough difficulty that the benchmark is not saturated by most detectors. If you want credits to run your benchmark, please apply here: