Cool approach, i think less latency and cost is the way to go. Here's how this would have likely been made. - Tiny transformer or equivalent model (maybe a few bn or so?), explaining latency and cost - Questions are sent in parallel to multiple copies of it (I'm sure they're edge located) - The model is post-trained for calibration in a wide variety of data (the recipe is relatively simple, and likely targeted on distillation of logprobs / confidence of a bigger model) Notice how cost is ONLY for input tokens as output is merely numbers (few tokens) because input could be huge (questions and options). At 0.042-per-million price they have, Astra estimates the model to be 3bn parameters. One could replicate this by post training Qwen 3.5 2Bn. I expect people to do so soon!