Braintrust(@braintrust):When you eval open-weight models, you also need to eval the serving stack. The inference engine, model revision, precision, caching, and account limits can all impact the system you're testing. We ran an eval on Kimi K3 served on @FireworksAI_HQ vs @Kimi_Moonshot. They performed equally well on quality, but median time to first token was 55% faster on Fireworks. Read more →

Braintrust

@braintrust

Active observability for agents in production.

Joined August 2023

60 Following 7.7K Followers

Braintrust@braintrust

2026.09.10 19:15

When you eval open-weight models, you also need to eval the serving stack. The inference engine, model revision, precision, caching, and account limits can all impact the system you're testing. We ran an eval on Kimi K3 served on @FireworksAI_HQ vs @Kimi_Moonshot. They performed equally well on quality, but median time to first token was 55% faster on Fireworks. Read more →