註冊並分享邀請連結,可獲得影片播放與邀請獎勵。

Ryan Brooks
@rbro112
Staff Eng @sentry on AI/ML (all things evals) Formerly employee 1 @emergetools (YC W21), mobile @airbnb
加入 June 2020
209 正在關注    329 粉絲
It’s been a week since I moved one of eval datasets from Gemini 3.1 to Jev by @typesafeai for LLM judging. The results so far: - No meaningful change in scoring accuracy - ~200× cheaper (~$0.01 → ~$0.00005 per judge) - ~50× faster (~10s → ~0.2s median) - ~50% fewer input tokens, 87% fewer output tokens But not everything’s perfect, more details in the thread
顯示更多
0
15
110
15
轉發到社區