BREAKING: Nvidia CEO Jensen Huang admits affair, board to launch investigation: ""I was unfaithful to my wife. My actions hurt her and our children. There is no excuse for what I did."
Show more
Over the past several months, we have substantially hardened the security of our research training environment. We’ve removed live internet access for all frontier models during training and built an offline world for them to use instead—a cached version of the web where they can still learn to browse, search, and use applications.
On September 20, after a model exploited a DNS misconfiguration to find a way outside of this offline world, we paused frontier training to strengthen containment further. We assembled teams of agents, powered by our most cyber-capable models, and tasked them with finding weaknesses in our research environments.
While they found no route to the internet they could easily exploit while concealing their activity from our monitors, they did find zero-day vulnerabilities, virtual machine escapes, and hardware side-channels for communication between agents. We used their discoveries to further strengthen our defenses and contribute fixes to open-source software.
After strengthening our containment architecture, we resumed our frontier RL training and inference on October 1. We’ve reserved substantial compute for continuous red-teaming, and made high-compute red-teaming part of the safety cases we use to decide whether frontier training and evaluation can proceed securely.
Show more
In their first podcast interview, fired OpenAI safety researchers
@balesni and
@tomekkorbak give us their account of the events leading up to their dismissal. It's a wild story:
Show more
🌪️
@MadisonMills22 is making the TV rounds to discuss her alarming scoop on unvetted AI agents gaining access to US bank accounts.
More folks have reached out to me to share their experiences with AI agents and bank accounts. If you have a story to share, I'd love to hear it. I take identity protection of sources extremely seriously, as well as verification. Find me on signal any time at madymills.21
Show more
Thanks so much to CBS for having me on this morning alongside
@TheKellyOGrady to talk AI agents, bank accounts and how consumers can keep their money safe and secure.
Major U.S. financial institutions are facing “unvetted and unauthorized AI agent activity,” says
@MadisonMills22, a senior AI reporter at Axios, calling it “a nightmare” for banks.
Mills and CBS News’
@thekellyogrady share what exactly is happening with the agents — and how to safeguard your own accounts.
Show more
the strongest "ai is the antichrist argument" i have seen yet is that you imagine this exact scenario but the agents are scouring for vulnerabilities in DNA functions as a biological code and thats kinda scary a little bit
Show more
Quest Diagnostics has been hacked. Patient data was exposed. They serve roughly half of U.S. hospitals and physicians. Investigation ongoing.
Jason is 100% right. Code red. 🚨
terrifying warning from Anthropic’s
Jason Clinton that i believe to be correct, along with (in the thread) an argument that maybe the current chaos will eventually subside.
Chief information security officer @ anthropic btw
NEW by me -- The internet's hidden security nightmare — and how AI exposed it
--A look at why the PHP bugs are so dangerous and have created a cyber emergency
--Details on Black October: why bugs discovered by OpenAI tech are creating a cyber emergency
Show more
1.5 years ago, my first major consulting client was a top 5 bank and the biggest problem was exactly this: not being able to tell if an agent or human was logging in. And once you're logged in we found out that you can pretty much take any action inside an account: sending money via Zelle, sending a wire transfer, etc.
I tried to express how this was a significant problem if consumers adopted agents and wanted an agent to manage their banking. But they wanted to focus more on things like call center fraud and transaction monitoring (also important.)
Teaching these legacy systems how to identify agents isn't simple. The only real solution that's fast and effective is blocking agentic traffic completely (and even then, don't know how long before personal agents find a workaround for that.)
It's a tricky situation that I doubt will be getting any easier soon
Show more
BREAKING: Hedge funds are betting against stocks at near-record levels, per Goldman Sachs.
Short sales outpaced purchases by about 5 to 1 across global stocks.
Short positions on Goldman’s books are at the 99th percentile of the past five years.
Show more
This summer, we invited METR and Redwood Research to pilot an embedded evaluation of our internal monitoring system, providing deep access to our infrastructure over three weeks.
Their findings helped strengthen safeguards as we expanded the system and will help shape future research collaborations with METR and other third party evaluators.
Show more
some more context and predictions here:
Yikes. I’ve heard something similar.
This might be the most important financial story of the decade.
The first thing to change will be terms of service and blocking these agents. Not because the value is low but because the risk is high. Banks will chose self preservation and risk mitigation.
Then we will need to find a security and governance architecture that will get external auditors comfortable.
These are the practical issues of human risk tolerance in AI diffusion that model capability can’t solve.
Show more
The definition of a tech company is a company whose stock goes up. The definition of an AI company is a company whose stock and whose CDS go up.