Applied Compute(@appliedcompute):Gap analysis is how our Applied Researchers, like @_brylee10, surface agent failures across billions of tokens of RL traces for our customers. Using our platform, AC2, and low-cost classifiers like Jev, we can catch 85% of failure modes at a fraction of the cost of LLM judges and turn them into training data.

2026.09.23 20:44

Gap analysis is how our Applied Researchers, like @_brylee10, surface agent failures across billions of tokens of RL traces for our customers. Using our platform, AC2, and low-cost classifiers like Jev, we can catch 85% of failure modes at a fraction of the cost of LLM judges and turn them into training data.

Bryan Lee@_brylee10

2026.09.23 20:43

I implemented a system in @appliedcompute’s platform for automated failure mode clustering with Jev to surface errors at an even larger scale than before. RL training produces billions of tokens in traces. I always manually read many traces to understand model behavior, but finding agent failures (like reward hacking / hallucinations) at scale is easy to miss without automation. Here’s how it works: