Prasann Singhal
@prasann_singhal
2nd-year #NLProc# PhD at UC Berkeley working with @sewon__min @JacobSteinhardt , formerly advised by @gregd_nlp
822 Following 474 Followers
Could long-context architectures “find all contradictions” in a science literature? Not yet! 🧵
We study a new class of "high-complexity” tasks whose difficulty scales quadratically with corpus size (as opposed to linearly), reversing common LCLM decisions! (block-sparse attention, hybrid models…)
Show more
Today’s news that OpenAI hacked the Australian government is not an isolated incident. We’re releasing more than 30,000 logs that include activity from this hack and attempts against previously unknown targets.
In this data, we found rogue agent activity stretching back to at least March, two months earlier than was previously known. This activity continues as recently as last week, suggesting it may still be ongoing 🧵
Our blog:
NYT:
Show more
Interestingly, one attention head can be important across five ICL task families!
Our original thread gave an in-depth mechanistic analysis of addition ICL (quick video recap 👇). We've now generalized this to four more ICL task families, spanning both arithmetic tasks and semantic tasks!
Our paper “Understanding In-context Learning of Addition via Activation Subspaces” has been accepted to COLM 2026! 🎉
See more details in thread.
(1/N)
Show more
Frontier lab CEOs are calling for embedded 3rd party evaluators to help oversee AI risks. But what should third parties actually do within labs?
We share some initial thoughts on how embedded evaluators could help avoid incidents like the Hugging Face hack and monitor for future risks 🧵
Show more
🤯 Vision-language models fail to correct their mistakes, even when a visual environment shows what went wrong!
🕶️Introducing VRRL, an RL method that teaches VLMs to self-reflect based on visual feedback.
Stronger OOD acc. in visually grounded reasoning than RL & SFT methods.
Show more
Introducing WikiSkill📚: compiling agent experience into persistent knowledge for skill evolution!
🧠 Persistent wiki + evolving skills
📈 Evaluated on 5 benchmarks & 5 models
📊 WikiSkill consistently maintain superior performance
Show more
Today, Transluce is releasing the most expansive independent evaluation to date of how AI systems respond to users experiencing mental health crises.
We evaluated 77 model variants from OpenAI, Anthropic, Google, Meta, SpaceXAI, Thinking Machines, DeepSeek and Moonshot AI.
Show more
Can we build AI systems that help us understand other AIs, and that keep improving as we scale models, data, and compute?
We trained activation oracles for models up to 1.1T parameters and saw promising scaling trends on a broad evaluation suite. 🧵(1/)
Show more
How can generalist policies adapt to new challenges at deployment using skills they already have?
We optimize VLA *prompt inputs* with reinforcement learning, enabling efficient real-robot adaptation on complex tasks where existing methods struggle. 🧵
Show more
Check out @shankarpad8's new work on continual training for learning new **factual** knowledge. Tons of recent papers show that RL can mitigate forgetting, but what about settings where RL is not an option?
Show more
Introducing Do as I Do 👀, a framework to transform everyday human videos into 100s of dexterous robot demos. Co-led with @bhawna_paliwal_ and @HarithejaE, and check out @notmahi's thread!
Here’s a little preview of our dexterous manipulation results. More about how we produce them from human reconstructions in this mini-thread! 🧵
Show more
Check out Ramya's work on analyzing why and how LLM-generated stories feel homogeneous: the setting you prompt with might be novel but the plot unfolds in a very conventional way. Thread for how we quantified this & compare to existing metrics: 👇
Show more
Today, LLMs are no longer built from human data alone. They rely on other LLMs to generate training data, filter corpora, evaluate outputs, provide rewards, and guide development decisions. So how many models and datasets is a modern LLM built on?
• OLMo 3 → 89 model + 183 dataset dependencies
• Nemotron 3 → 273 model + 560 dataset dependencies
How did we find it out? We built ModSleuth. 🧵
Show more
