I'm surprised at how little AI text is in S&P 500 filings. Take Microsoft. About 2% of the text in its 2026 filings were marked as AI by Pangram. My read on this is that individuals are asking AI assistants for snippets of text, but nobody is close to using agents to draft significant portions of these documents.

CALL FOR AI DETECTION BENCHMARKS I believe it is vital that third parties are able to audit AI detectors like Pangram. Our largest sample size benchmark, from UChicago, was run on Pangram 3 and is a year old now. New LLMs are being released all the time, and the binary task doesn't capture the full spectrum of real world performance. A great benchmark will help shape how the world interacts with AI over the coming years. I am putting out a call for new third-party evaluations that attempt to answer one or more of the following questions: - How well do current detectors catch text from the latest frontier AI models? What about in high context settings, like deep research or an agent harness? - How well do current detectors do on mixed authorship text? Ideally, research in this area explores the full spectrum of AI assistance from light grammatical edits to full co-authorship. - Any sort of sliced evals around language, background, or ethnicity. Null results that show no bias are still interesting to us. What I value in a benchmark is extraordinarily clean data, frontier LLM data, and a high enough difficulty that the benchmark is not saturated by most detectors. If you want credits to run your benchmark, please apply here:

I’ve had a few people write to ask me to follow up on a (reply!) tweet I wrote that’s gotten a lot of retweets. I considered deleting it just to avoid the headache, but I think that would make people more paranoid. So let me share my thoughts.

Another really cool paper out of @MohitIyyer’s lab! We’ve been thinking about idea provenance for a while and while I think there is a lot more grey area here than in text provenance, this is an impressive proof of concept for what could be really interesting technology

OpenAI's watermarking technical report is out. One thing that stands out to me is that they evaluate it at a target false positive rate of ~1%! That's surprisingly high for what some would assume to be irrefutable proof of AI.

Interesting—Pangram (intentionally) does not flag AI-translated texts as AI-generated. This is great for non-native English speakers, and makes sense once you think about it. Here are Pangram scores on pre-2022 French/Spanish writing, translated with a series of AI prompts. 1/

SF friends, come meet the Pangram team next week! Caveat: I won't be there, I am on vacation eating egg tarts with my family. Counter-caveat: most of the research team will be there including our elusive CTO who is almost never online or on twitter

This is a cool result showing that today’s frontier LLMs are much better at detecting AI content than they used to be. In my opinion, it’s very impressive that Opus 5.5 has such a sub-1% false positive rate. We’ve found that LLMs-as-AI-detectors still struggle on our more challenging eval sets, but perhaps this is a blog post for another time.

"We build humanizers to help make the Pangram model more robust, but we're never going to release it because I think that would just be an evil thing to do." @pangram CEO @max_spero_ tells @semaforben and @maxwelltani why the company makes the tools it's trying to stop.