가입 후 초대 링크를 공유하면 동영상 재생 및 초대 보상을 받을 수 있습니다.

Ishan Mukherjee
@ishanmkh
dad + founder. now: cofounder/ceo @rox_ai past: 4 newcos + 3 bigcos after @mit
가입 April 2009
586 팔로잉 중    1.8K 팬
Every AI team says they have evals, but most are just guesses about what users will do. Using @DevinAI and @braintrust, the team built an eval suite that grows automatically with the product. Proud of @gopalgoel, @XuBriGuy, @AksharSarvesh for building this. This is the kind of problem we hire for.
더 보기
Nobody at Rox writes evals anymore. Devin now QAs our agent every week, on real traffic. It reads real conversations with our agent, rebuilds each failure in a full copy of Rox, and turns it into an eval. We just review the PR. The eval suite already covers 72% of real usage happening in the product, vs. 43–57% for the evals we used to write by hand. And it’s growing week over week with Devin. Built with @cognition and @braintrust:
더 보기