fun weekend project: AutoJev.
i was curious to see if i could train a competitive Jev-like model completely autonomously with a swarm of agents using our internal system. turns out you can get quite far!
some details:
- gave the swarm a devbox with an h200 gpu
- the swarm is a mix of astra and fable; used sol and luna for synthetic data and filtering; ran for 20 hours
- spent $3.1k in total ($1.9k on agents, $1.2k on data)
- used qwen3.8-27b as the backbone. it underperforms Jev initially, but after training it is competitive/better, and it also supports 260k context and multimodal input
- the agents tried RL, but it hasn't worked out so far, so they fell back to SFT on high-quality synthetic data
- both accuracy and calibration are better
- jev-compatible api, playground, and inference. on an h200 it gets around 120ms p95. that will be higher under contention, but still quite fast for an unoptimized version
code:
model:
going to let it run longer, and will also bring up inference so folks can try it out.
显示更多