Bryan Lee(@_brylee10):I implemented a system in @appliedcompute’s platform for automated failure mode clustering with Jev to surface errors at an even larger scale than before. RL training produces billions of tokens in traces. I always manually read many traces to understand model behavior, but finding agent failures (like reward hacking / hallucinations) at scale is easy to miss without automation. Here’s how it works:

Bryan Lee

@_brylee10

daily 1%. building @appliedcompute. prev co-founder Sentinel (YC), @twosigma, @harvard

参加 October 2011

282 フォロー中 334 ファン

Bryan Lee@_brylee10

2026.09.23 20:43

I implemented a system in @appliedcompute’s platform for automated failure mode clustering with Jev to surface errors at an even larger scale than before. RL training produces billions of tokens in traces. I always manually read many traces to understand model behavior, but finding agent failures (like reward hacking / hallucinations) at scale is easy to miss without automation. Here’s how it works: