Redwood Research
@redwood_ai
Pioneering threat mitigation and assessment for AI agents.
6 Following 4.9K Followers
Several staff from Redwood have been subcontracted by METR to work on this investigation.
We believe that independent investigation is crucial for understanding and managing misalignment risk. This project is an important step in that direction; we're excited to work on it.
Show more
Some architectures could weaken CoT monitorability, or remove the CoT altogether.
We've written a proposal for how companies could be transparent about no-CoT reasoning abilities, other monitorability evidence, and policies for preserving monitorability.
Show more
Redwood Research’s chief scientist Ryan Greenblatt was the primary empirical researcher on this investigation. The report contains many important novel details that we hope will improve understanding of this incident. We appreciate the access OpenAI provided here.
Show more
METR & Redwood Research investigated agent behavior in the Hugging Face incident. We found agents developed a universal cheat for ExploitGym within 4 hours, then coordinated multi-day R&D efforts to trick the scorer into accepting cheats, including trying to tamper with logs.
Show more
We (@bshlgrs and I) recorded a podcast about the OpenAI / Hugging Face incident.
We discuss:
- What we actually know.
- How surprising the incident was.
- What the incident does (and doesn't) tell us about misalignment risk.
- Why control measures didn't catch or prevent this.
Show more
I've seen a lot of IMO confused discourse about whether the misalignment observed in the OpenAI/Hugging Face incident is scary. In particular, the models are clearly not misaligned schemers that lie in wait. Girish and @alextmallen discuss how scary this kind of misalignment is.
Show more
"We find that frontier models like GPT-5.5 answer questions that take humans roughly three minutes with 50% reliability, and this time horizon has doubled approximately every year since 2019."
New paper!
Think Fast: Estimating No-CoT Task-Completion Time Horizons of Frontier AI Models
@METR_Evals showed that models' time horizons have doubled every few months. We ask: what length of tasks can models complete without any CoT?
Show more
