Greg Pstrucha(@grichadev):I wasn't able to get it to a 100%, but I found that if I run it as a confidence check, jev is pretty consistently flagging *all* my eval'ed issue and rejecting red herrings at 0.65 confidence or over. i can still send the remaining issues for double check to a larger model and reap the benefits of it being cheap. more interestingly, it being confidence level is a lever i can pull, if I want my pipeline to catch things more conservatively at the trade-off of slightly larger cost (for a second pass), that's very configurable and valuable to me.

2026.09.17 22:56

I wasn't able to get it to a 100%, but I found that if I run it as a confidence check, jev is pretty consistently flagging *all* my eval'ed issue and rejecting red herrings at 0.65 confidence or over. i can still send the remaining issues for double check to a larger model and reap the benefits of it being cheap. more interestingly, it being confidence level is a lever i can pull, if I want my pipeline to catch things more conservatively at the trade-off of slightly larger cost (for a second pass), that's very configurable and valuable to me.