Braintrust(@braintrust):When you eval open-weight models, you also need to eval the serving stack. The inference engine, model revision, precision, caching, and account limits can all impact the system you're testing. We ran an eval on Kimi K3 served on @FireworksAI_HQ vs @Kimi_Moonshot. They performed equally well on quality, but median time to first token was 55% faster on Fireworks. Read more →

Braintrust

@braintrust

Active observability for agents in production.

参加 August 2023

60 フォロー中 7.7K ファン

Braintrust@braintrust

2026.09.10 19:15

When you eval open-weight models, you also need to eval the serving stack. The inference engine, model revision, precision, caching, and account limits can all impact the system you're testing. We ran an eval on Kimi K3 served on @FireworksAI_HQ vs @Kimi_Moonshot. They performed equally well on quality, but median time to first token was 55% faster on Fireworks. Read more →