Zain
@zainhas
I build and teach AI • AI/ML @togethercompute • EngSci ℕΨ/PhD @UofT • Previously: vector DBs, data scientist, lecturer & health tech founder • 🇺🇸🇨🇦🇵🇰
Joined August 2012
2.1K Following 8.3K Followers
probably the most important thing about this release:
"Compared with the previous generation, V4.1-Flash’s KV cache needs just:
🔹 1/4 the HBM
🔹 1/8 the SSD storage"
4x smaller than even v4 flash
Show more
💾 Smaller KV cache. Bigger savings.
Compared with the previous generation, V4.1-Flash’s KV cache needs just:
🔹 1/4 the HBM
🔹 1/8 the SSD storage
Cache-hit charges often account for a large share of agent costs. Compressing the cache cuts those costs significantly.
3/6
Show more
