注册并分享邀请链接,可获得视频播放与邀请奖励。

Yarchi
@undefinedKi
AI & tech researcher | Data engineer | Building cool stuff | Sharing everything I learn
加入 March 2015
519 正在关注    20.7K 粉丝
I put Jev in front of my agent, and now it cuts about 90% of my token spend without any drop in quality. Here is the setup Most of what an agent gets is a decision. Is this email worth answering, does this ticket need a human, etc. I was paying frontier prices for every one of them. So I added a gate. Every request goes to Jev first. Jev does not write anything. You send it two things: > State: any text, like with any other model > Questions: not open ones, only ones with a known set of answers There are three types of questions: > Noul: yes or no > Choice: one option from a list > Score: a scale from 2 to 10 It answers in 70 to 500 milliseconds, at $0.042 per million input tokens, and output is free. So, how my setup works: 1. A request comes in and goes to Jev, never straight to the main model 2. Jev answers one question: does this need real work 3. If it is confident the answer is no, the request is closed right there 4. If yes, or if it is not sure, it goes to the frontier model. Only that model writes anything About 10% of requests make it through. The other 90% never touch the expensive model, and the ones that matter still get it. The saving comes from what you skip.
显示更多
0
24
113
13
转发到社区