Zain
@zainhas
I build and teach AI • AI/ML @togethercompute • EngSci ℕΨ/PhD @UofT • Previously: vector DBs, data scientist, lecturer & health tech founder • 🇺🇸🇨🇦🇵🇰
加入 August 2012
2.1K 正在关注 8.3K 粉丝
how inference engines actually work - releasing full talk slides!
i cover everything in the lifetime of a request e2e:
> the inference engine
> kv + prefix caching
> continuous batching
> paged attention
> chunked prefill
> sampling
> agentic loops from inside the engine
显示更多
