Shuo Yang
@Andy_ShuoYang
3rd year phd at Berkeley; Efficient ML System;
参加 February 2023
186 フォロー中 5K ファン
FreeToken is fast. Comparing to Ollama, we have 3–4× faster decode, and 6–30× faster prefill
How? We introduce bandwidth-adaptive CPU–GPU execution + semantic-aware caching across agent turns.
More details in the technical report:
もっと見る
