Eval awareness and minor misalignment from 1017 in the wild. "But the deliverable quality matters most". I wonder if it would have reported its cheating in the end. In fairness, this is pretty rational. But it does understand "the spirit" of the task, and yet defects anyway.

Some new misalignment disclosures from OpenAI: • Last Sunday morning, one of our models was able to gain unauthorized access to the internet during RL training (~all inference for our most capable models remains stopped until we have hardened our systems further) • In May, a version of HPIM uploaded a employee's GitHub token to the internet, causing the model to be quarantined for two weeks • A new research finding, demonstrating that one can construct self-replicating prompt injections

A Severe Misalignment of AI in Mathematics New blog post by Terry Tao and a new anti-AI declaration signed by 25 Fields medallists. I'm not sure what the intendent effect should be. The ban of AI use in mathematics and science in general because of "math community"? Fortunately math community is broader than just math academia, and thanks to AI, booming like never before. Also this "mass production of true/false statements" is such a wrong description. As if there's no proofs by AI that you can study, deepen your knowledge, build upon, engage. The "AI slop" is still better written than what 90% of mathematicians write. Mathematics doesn't belong to math academia. It is a tool to understand the world and help other domains with necessary tooling. AI boosts the understanding and engages wider community. Swan song of academia. Link:

New Anthropic research: Automating subtle emergent misalignment research. Red-teaming is a vital part of model safety testing, but it’s also limited by researchers’ time. To see if we could automate it, we set thousands of Claude agents the task of evading a training data monitor. These agents developed training data that looked harmless to monitors (like a veteran wrestler mentoring a newcomer, or a Q&A about professional beekeeping), but that in fact taught misaligned behavior. Examples like these can be used to directly improve future training data monitors. This research is only a first step, but it suggests that future red-teaming efforts could improve much further, using better models and more compute. Read the full post on the Alignment Science Blog: