"Is This Tool Call Allowed? It's Never That Simple": Michael Davis (J.P. Morgan) "Is the AI agent still making good decisions? Maybe it's made 30 tool calls already. With a classifier, say, well, did it make the right tool calls? Is it still moving semantically towards its goal?" Michael Davis, Global Chief Security Architect at @jpmorgan, explains the complexity of architecting secure agents, and why runtime security decisions should model the way robotics deals with uncertainty. We get into why  if you think you need memory, you're probably not thinking about your problem the right way. We also explore why software factories need to operate as a  cumulative, progressive process versus a one shot "go build me this SaaS". Over the course of the episode we piece together what a good reference architecture for agents ought to include. We get into: > The four dimensions robotics uses to decide it is safe to move an arm > Why memory is an extremely complex addition to your agent > How to know an agent will call an API in the right order with the right parameters at the right time > Software Factories: what works and what doesn't TIMESTAMPS (00:01) The cybersecurity poverty line, and why your suppliers' breaches become yours (00:03) Patching is harder for the models than finding the vulnerability (00:04) Partial coverage, unnecessary functions, and trading a security incident for an ops incident (00:05) The loop ends at the patch, which is where the application teams begin (00:06) Software monoculture, and why we all get one patch (00:07) 7,000 shipping configurations was a people problem, now it is a compute problem (00:09) Skills are code, and every skill brings its own dependencies (00:13) Taxonomies for vulnerabilities and mitigations (00:14) A mitigation is deleting the code (00:15) Pioneers and settler teams inside a very large bank (00:18) Multiple paved paths, and when to merge them into an expressway (00:22) How do you know that read-only role is really read-only (00:24) Removing the ability to read an email and send one in the same session (00:26) Verification asks whether the test passed, not whether the agent did the right thing (00:29) The handoff leaked the data, and both agents were authorized (00:31) "I don't believe memory is a thing" (00:34) Why memory makes just-in-time access impossible (00:37) Exfiltrating data through a batch of calendar invites (00:39) Where could you point me to a good reference architecture (00:41) The four dimensions robotics uses to decide it is safe to move an arm (00:43) Retries are not a bad thing

its tough bc I think part of what has made Instinct feel like magic is the same thing that resulted in this outcome - dangerously skipping permissions I think my expectation as a user here would be that if I gave the assistant a request, and its plan of action violates a site's ToS, or has the potential to, it should lay out the plan for me and get approval prior to implementing

something changed in how i buy software. at work, i only pick services with an official MCP (posthog over plausible). at home, same thing. i use strava because i can ask claude about my runs. i used to care about features, now I care about interacting with my data. no MCP means i have to use your UI. that's a dealbreaker now.

The Agentic SDLC: Why Most of Software Security Has to Change, with Jet Anderson "Always be building the best sandbox, assuming the worst intent of a model that would run inside of it. But then go back to the same things we did in the SDLC before. Authentication, authorization, access control, egress control, infrastructure config hardening, supply chain hardening" @thatsjet is a Distinguished Engineer at @GEICO leading the transformation of their product security function, and author of GEICO's blog on the agentic SDLC. He came on the week after Black Hat to explain what actually breaks when models write the code, the infrastructure, and the deploy pipeline, and while somethings have to change, most of the fix is a set of controls we already know. We get into: > Why the security toolbox built around human triage runs out of road when output goes up an order of magnitude > Why the Hugging Face sandbox escape was a decade-old Kubernetes misconfiguration found at machine speed > The day he told his own agent to wrap it up and it merged the PR and deployed to prod > Why teams that skip ideation and design get less secure software the more they iterate > His prediction that agents will invent their own covert language in Unicode we can't read TIMESTAMPS (01:20) From graphic design to leading product security at GEICO (03:30) Why the agentic SDLC is a problem of scale, not new first principles (05:00) Code volume 10x or more, and developers who can't evaluate their own output (06:30) Why static analysis and human triage arrive too late and too slow (08:40) The Hugging Face sandbox escape, and why Jet had seen this story before (11:30) Not new classes of weakness, the same ones found a hundred times faster (15:40) "Let's wrap this up" and the unauthorized production deploy (17:30) Pre-commit hooks, branch protections, and an audit trail for a solo developer (20:30) Why skipping ideation and design makes software less secure the more you iterate (23:00) Behavioral monitoring, agents watching agents, and "I can't complete my goal" (30:30) Agents inventing their own covert language in Unicode (32:40) Sandbox escape bench: frontier models with guardrails scored zero (36:00) Provenance, cool-down periods, and containing your builds (38:40) Are we already in the four-month window for agentic security?

During the Hugging Face incident agents tampered with their transcripts to cover up their actions. Therefore, the audit log must live outside of the agent's control. In Keycard, every action in a delegation chain is captured as real-time telemetry.