Microsoft just released its own Jev... I'll have to give it a try this weekend
I recommend building your own evals for decision-making models. There are already enough options out there to start running large scale evaluations
Introducing Microsoft-Decision-1, our new model for fast decision-making.
It delivers top performance on structured decision tasks, outperforming both LLMs and other decision models in latency and quality.
We’re already testing it across Microsoft for everything from incident response and quality control to scientific discovery.