Shuo Yang
@Andy_ShuoYang
3rd year phd at Berkeley; Efficient ML System;
Joined February 2023
186 Following 5K Followers
FreeToken is fast. Comparing to Ollama, we have 3–4× faster decode, and 6–30× faster prefill
How? We introduce bandwidth-adaptive CPU–GPU execution + semantic-aware caching across agent turns.
More details in the technical report:
Show more
