Karpathy’s Agentic Engineering finally has proper DevTools!
When an agent stops working, the model is only one possible cause.
The problem could be a failed tool, a lost connection, an interface update that never appeared, or something earlier in the conversation.
@CopilotKit has rebuilt its open-source Inspector around this problem.
It sits inside the application and watches the full interaction between the user, the interface, and the agent.
When something fails, the Inspector button turns red, names the failure, and opens the spot where it happened.
But even a much harder problem is reproducing that failure.
This is because agents do not always follow the same path twice. Running the same prompt again may trigger another tool, produce another response, or start with a different application state.
Inspector handles this through saved Threads and an isolated Agent Playground.
A developer can open the conversation where the problem occurred, choose "Try from here," and copy everything up to that point into the Playground.
The original conversation remains unchanged while another response or tool path is tested.
Threads can open inside the live application, while assistant messages can jump back to their matching context in Inspector.
Everything runs underneath AG-UI, the protocol that carries messages, tool activity, state changes, and other events between the agent and the interface.
Those interactions can also feed CopilotKit Intelligence so that when the same issue or useful behavior appears repeatedly, it can become an evidence-backed Insight and a proposed SKILL(.)md file.
Developers can review, edit, and approve the improvement before the agent inherits it.
The tool also helps developers find a problem, recreate its surrounding context, test another path, and turn repeated lessons into agent improvements.
It is open source and included with CopilotKit development builds. Intelligence setup starts with one prompt.
Here are the docs:
I am testing this extensively and will cover this in more detail soon with a hands-on demo.