Check out how the team at @SentinelOne uses Pydantic Evals to score their security agents.
this is so fucking cool.
carter church gave gpt-6 astra a scanned image of a 217-year-old encrypted letter sent to one of napoleon’s generals.
six hours later it had transcribed ~1,300 cipher symbols, found a partial historical key, built a solver for the rest and decoded the letter.
and while doing it, it figured out the fucking letter had been dated incorrectly.
what an insane time to be alive.
Show more
Mentioned on @ambXover that I asked ChatGPT about the prospects of solving standing mysteries "with one prompt." And here you go:
We are living in the craziest time: Carter Church used GPT-6 Astra to decipher a 217-year-old letter to one of Napoleon’s generals, reporting about six hours of model work.
The difficulty lay primarily in extracting decipherable text at all from a poorly legible historical image.
Starting with a scanned image, Astra transcribed 1,300 cipher units, found a published partial key, wrote a solver for the remaining code, and cross-referenced historical letters to correct the document’s date.
Cryptiana now lists the cipher as solved. Really cool use case!
Show more
JUST IN: GPT-6 Astra cracks unsolved 217-year-old Napoleonic military cipher, revealing a secret briefing sent to one of Napoleon’s generals ahead of Austria’s 1809 invasion.
We need more data centers.
I used GPT-6 Astra to break an unsolved cipher to one of Napoleon's generals that had gone unread for 217 years.
What makes this impressive isn't actually the codebreaking, but that Astra completed the entire multi-modal workflow in ~6 hours from a single image and goal.
1/6🧵
Show more
It's true. The principle of least privilege, separation of duties, input sanitization, logging & monitoring, defense in depth -- all apply to AI.
In Cybersecurity, unmeasurable capability is unshippable capability. @Gabeincognito and I discuss what it takes to measure real agentic performance and why a correct verdict for the wrong reasons is still a failure.
Show more
A 95% match rate between an AI agent's verdict and an analyst's can still mask pattern-matching behavior. Carter Church & Gabriel Bernadett-Shapiro explain why SentinelOne builds security agents backwards from evaluation, testing competency by competency to confirm real investigative reasoning:
Show more
A 95% match rate between an AI agent's verdict and an analyst's can still mask pattern-matching behavior. Carter Church & Gabriel Bernadett-Shapiro explain why SentinelOne builds security agents backwards from evaluation, testing competency by competency to confirm real investigative reasoning:
Show more
The first glimpses of a revolution in intelligence historical affairs.
I used GPT-6 Astra to break an unsolved cipher to one of Napoleon's generals that had gone unread for 217 years.
What makes this impressive isn't actually the codebreaking, but that Astra completed the entire multi-modal workflow in ~6 hours from a single image and goal.
1/6🧵
Show more
JUST IN: Apple CEO John Ternus is reportedly considering a major overhaul to “accelerate product development” by cutting management layers and moving toward a more engineering led structure.
Building agents backwards from evaluation!
Latest from me and my colleague Carter Church - How to build agents from evaluations.
A short read if you are building an agent for your security program and have questions of where to start or how to measure success.
Show more
ppl still have no idea what this means for the world.
literally going to change everything.
