★Sunshine Chaser★ Kicking it into high gear! Hiya♪ Welcome to my exclusive summer channel, my Chief Navigator! You have just one mission today: basking in the sunshine with me! USS Bennington is changing into her new L2D+ attire. She will grace your dock in the near future, Commander. #AzurLane# #Yostar# #AzurLane8thAnniv#

Theorem co-founder @diagram_chaser reveals the one-line change that took verifying real-world HTTPS code from 4,000 millennia to seconds: "The project that I worked on in my PhD was verifying the code that runs HTTPS in browsers like Chrome and Firefox." "You plot this beautiful graph that is an exponential in the number of bits in the prime, where you're like, it takes a couple seconds on my tiny toy examples, and on real-world examples, it would take over 4,000 millennia." "The thing that it's actually checking is that you wrote the same thing in two different ways. This should be very fast. It's doing the operations in the wrong order." "So you tweak one line, and then it drops down to a couple seconds." @theoremlabs

Theorem co-founders @rajashree + @diagram_chaser explain why formal verification could break cybersecurity’s endless whack-a-mole and stop reward hacking during AI training: Rajashree Agrawal: "Formal verification is asymmetric security. Currently you have this whack-a-mole problem. As the attackers get better, the defenders have to catch up. You keep doing this forever." "Once you prove this particular property holds of your program, you don't need to check that again. This feels like the asymmetric approach that you need if you're going to avoid this cyber apocalypse." "If you wanna move beyond one-off interactions, build a very complex world, lots of automations, you're going to have structured things coming out of models which looks like software. Being able to reason about software seems like a good hammer to have." Jason Gross: "If you can verify the RL environments and the graders that you're running, then the models won't have any reward hacks during training, and so you can potentially train them to not be as reward hacky in general." @theoremlabs

Theorem co-founders @rajashree + @diagram_chaser on how you actually prove an AI agent can’t escape its sandbox, and what happens when the proof exposes a way out: Jason Gross: "Anything that the agent does inside the sandbox will not result in some canary file outside the sandbox getting changed." Rajashree Agrawal: "We intend to have one by the end of the year. We've got a verified sandbox now, and the goal is to keep adding features in collaboration with the AI labs." Jason Gross: "If the AI can guess the secret root key of the package server, then it can do anything. Either I try to prove that the AI is not going to be able to guess that, or I design the system so that the channel just doesn't allow it to authenticate that way." "You also want to prove that on most inputs there's no change in behavior. Because otherwise it could make it inescapable by saying, well, sandbox just shuts down as soon as it starts." @theoremlabs

Theorem co-founders @rajashree + @diagram_chaser explain why verified AI sandboxes are still months away despite the latest breakthroughs in automated theorem proving: Rajashree Agrawal: "The models just got good enough to prove these theorem statements, or they're still getting there. One of the costs is just tokens." Jason Gross: "Taking the recent Navier-Stokes news as a baseline, the models wrote something like 600,000 lines of Lean in about 17 hours. This works out to between one kilobyte and 30 kilobytes verified per hour." "The smallest version of Linux with a sandbox that we can make is about five megabytes. So that's still a handful of months away, even at the rate of the Navier-Stokes auto-formalization." "We need to build a pipeline from verifying the software back into RL-ing the models, so that if you want to verify all production software that exists in the world, it costs you less than $10 trillion." @theoremlabs