We spent the last few days testing JEV internally. We tried to reproduce it, break it, and improve it.
5 things we learned:
1. JEVâs core insight is simple - and probably right. If the answer space is bounded, donât generate the answer token by token. Remove autoregressive decoding and you can remove 1â2 orders of magnitude of inference work.
2. RLCD isnât the moat. Data might be. Laya already open-sourced the implementation + weights. The missing piece is the synthetic data recipe. And our experiments strongly point in the same direction.
3. âOpen source already beat JEVâ is a benchmark illusion.
Same checkpoint:
In-distribution: 0.769
OOD: 0.541
Change the distribution and the apparent breakthrough largely disappears.
4. Compute-optimal â learnability-optimal.
We moved state outside the problem sequence to save compute. Result: -27 points. Likely because we saved FLOPs while throwing away useful MLM pretraining priors.
Then we tested 6 more ideas in one night: anchor selection, continuity smoothing, asymmetric windows, bucketed temperature, two-stage retrieval, full RLCD.
All failed or failed to generalize. Only one thing consistently worked: More data.
1,200 â 123,475 examples
OOD: 0.4069 â 0.5498
5. The weirdest result: A decoder with zero additional training still beat every trained small encoder:
0.5671 vs 0.5628 / 0.5411 / 0.4069
Our takeaway: JEVâs breakthrough may not be a particular architecture or training algorithm.
It may be a much simpler idea: When the answer space is bounded, search it. Donât generate it. And right now, the biggest bottleneck to making that work broadly doesnât look like architecture.
It looks like data.