Turing Award winner Andrew Yao and Moonshot AI founder Zhilin Yang have co-authored a new paper.
They're asking a question: Can today's frontier LLMs do real theoretical computer science research?
To find out, the team built TCSExam, a new benchmark for AI research.
It covers four areas:
• Theorem Proving
• Solver Engineering
• Mathematical Constructions
• Open Conjectures
Unlike typical benchmarks, TCSExam doesn't just test whether a model can solve a problem. It checks whether its work actually holds up.
The team put together 100 tasks for the Full test and 20 for Lite, evaluating 8 agents on Full and 9 on Lite.
Every submission is checked by the Lean kernel or an executable checker.
Here's what they found.
On theorem proving, GPT-6 Astra hit a 70% success rate with no hints.
Kimi K3.1 Pro came second at 38.3%.
No other model scored above 11.7%.
Constructions results:
GPT-6 Astra scored 104.9%, beating the existing reference results.
Kimi K3 also pushed two records forward:
• Graph theory: 630 edges in a 124-vertex graph with no 3- or 4-cycles, up from 629.
• Quantum error correction: Improved a code's parameters from [[25,7,5]] to [[25,7,6]].
What about problems that are still unsolved?
The team tested 30 open conjectures.
No model independently solved any of them from start to finish.
AI isn't ready to do frontier research on its own. But it's already producing results that researchers can check and build on.
🚨BREAKING: The Alan Turing Institute has developed a proposal to train an open-source British AI base model from scratch and release it as a public good, according to a UK government advisory letter published today.
The model would be UK-governed national infrastructure available to researchers, startups and public services, rather than a proprietary product owned by a single company.
Britain's top science advisers want ministers to consider the proposal and allocate national computing capacity for AI trials. They warn that without urgent action, the UK risks becoming a 'high-value AI customer' rather than a country able to shape the technology.