Niket Patel
@niketnpatel
Deep Learning & Math, phding @ NYU (Prev. UCLA)
258 Following 218 Followers
“An accumulation of facts is no more a science than a heap of stones is a house.” - Poincaré
LLMs are now proving results that have resisted mathematicians for decades. Finding interesting theorems without human guidance is a new bottleneck. We show that we can teach an LLM to do it!
TL;DR:
→ a quantitative notion of interestingness
→ 4.3× higher interestingness
→ a self-expanding discovery loop
[1/5]
Show more
Repeated sampling is the default way to scale LLM reasoning at test time. But token level noise often produces many near duplicate attempts that follow the same high level idea.
🧵 How can we cover more of the solution space without sacrificing throughput?
Show more
Unpopular opinion, but the main point of math and science is discovering and proving true statements
Buckmaster + @__alpoge__ just posted a counterexample to Euler's equation! They show that the equations of fluid dynamics, in this case 3d Euler, can develop singularities.
Here's a super rough sketch of the most basic version of the punchline.
Post:
Show more
1/ Can LLMs introspect, i.e., reason about their internal states? Recent work claims LLMs notice when their "thoughts" get tampered with, and can report their content. We looked closely and we think it's too early to say that. Work led by @shashwat_s19 , with @tallinzen and me.
Show more
Excited to announce the my first project @AIatMeta! - this was a super exciting collaboration with an amazing team. Stay tuned for future work!
Our team at @AIatMeta is excited to announce ATLAS: one of the largest automated formalization efforts to date.
ATLAS contains Lean 4 formalizations of both statements and proofs from 25+ mathematics textbooks, spanning dozens of domains, for a total of 500k lines of code. We are also releasing a flexible formalization harness and a companion paper.
External contributions are welcome!
Joint work spearheaded by our amazing PhD student Ahmad Rammal (@Ahmad3Rammal), together with Niket Patel (@niketnpatel ), Fabian Gloeckle (@FabianGloeckle), Amaury Hayat (@Amaury_Hayat), Remi Munos (@MunosRemi), Julia Kempe (@KempeLab), Vivien Cabannes, and myself from @AIatMeta, @NYUDataScience , and Ecole des Ponts. This is an ongoing effort; more details in the thread below.
(1/9)
Show more
Our team at @AIatMeta is excited to announce ATLAS: one of the largest automated formalization efforts to date.
ATLAS contains Lean 4 formalizations of both statements and proofs from 25+ mathematics textbooks, spanning dozens of domains, for a total of 500k lines of code. We are also releasing a flexible formalization harness and a companion paper.
External contributions are welcome!
Joint work spearheaded by our amazing PhD student Ahmad Rammal (@Ahmad3Rammal), together with Niket Patel (@niketnpatel ), Fabian Gloeckle (@FabianGloeckle), Amaury Hayat (@Amaury_Hayat), Remi Munos (@MunosRemi), Julia Kempe (@KempeLab), Vivien Cabannes, and myself from @AIatMeta, @NYUDataScience , and Ecole des Ponts. This is an ongoing effort; more details in the thread below.
(1/9)
Show more
Most people with “AI/ML” in their bios don’t even know a real symmetric matrix always has real eigenvalues.
Can we train LLMs with RL using the same next token prediction loss as pre-training?
(yes)
We conduct a study on (log)prob rewards and show they give a simple way to bridge verifiable and non-verifiable settings with a single reward, broadly applicable for fine-tuning LLMs.
Show more
Can a model learn to break its own reasoning plateau?
In our new paper, we show that LLMs can be taught with meta-RL to generate their own "stepping stones" that kickstart learning on hard math problems (0/128 success rate) where direct RL fails.
Paper 📝:
Blog post 🌐:
(1/n)
Show more
