註冊並分享邀請連結,可獲得影片播放與邀請獎勵。

Yarchi
@undefinedKi
AI & tech researcher | Data engineer | Building cool stuff | Sharing everything I learn
加入 March 2015
519 正在關注    20.7K 粉絲
I put Jev in front of my agent, and now it cuts about 90% of my token spend without any drop in quality. Here is the setup Most of what an agent gets is a decision. Is this email worth answering, does this ticket need a human, etc. I was paying frontier prices for every one of them. So I added a gate. Every request goes to Jev first. Jev does not write anything. You send it two things: > State: any text, like with any other model > Questions: not open ones, only ones with a known set of answers There are three types of questions: > Noul: yes or no > Choice: one option from a list > Score: a scale from 2 to 10 It answers in 70 to 500 milliseconds, at $0.042 per million input tokens, and output is free. So, how my setup works: 1. A request comes in and goes to Jev, never straight to the main model 2. Jev answers one question: does this need real work 3. If it is confident the answer is no, the request is closed right there 4. If yes, or if it is not sure, it goes to the frontier model. Only that model writes anything About 10% of requests make it through. The other 90% never touch the expensive model, and the ones that matter still get it. The saving comes from what you skip.
顯示更多
0
24
113
13
轉發到社區