Reilly Haskins
@ReillyHaskins02
AI safety research @ METR
30 Following 271 Followers
My learnings from this exercise:
- even if you have a policy that flagged actions should be reviewed by a human, if you use a coding agent to launch a bunch of evals it might decide to "approve" any flagged actions itself
- tracking down and accounting for all the inference an organization is doing is non-trivial, even at METR (where we use a custom centralized API router for most usage)
- lots of implementation details in making your monitor robust (e.g. in earlier versions of Inspect, subagent calls weren't monitored)
Show more
We run lots of evals at METR. Sometimes, agents attempt harmful actions.
I built a monitor that blocks suspicious tool calls until a human reviews them.
Writing out a case for why it's effective surfaced hidden assumptions. I'd recommend it to anyone building monitors!
Show more
We believe it's important to track and investigate misalignment incidents: cases where an AI agent autonomously took sophisticated, sustained actions in violation of human intent. In a new post, we lay out how independent propensity investigations of such incidents could be conducted.
Show more
Text diffusion models are fast, but are less transparent than today's LLMs because they do many forward passes before outputting text.
We audit the transparency of DiffusionGemma and find that the intermediates are interpretable. This recovers many of the benefits of CoT!
🧵
Show more
Could an AI company lose control of its own agents? To find out, Anthropic, Google, Meta, and OpenAI let us (1) test their best internal models with CoT access, (2) review non-public info about capabilities, alignment, and control.
The result: our first Frontier Risk Report.
Show more
