Shuo Yang
@Andy_ShuoYang
3rd year phd at Berkeley; Efficient ML System;
가입 February 2023
186 팔로잉 중 5K 팬
FreeToken is fast. Comparing to Ollama, we have 3–4× faster decode, and 6–30× faster prefill
How? We introduce bandwidth-adaptive CPU–GPU execution + semantic-aware caching across agent turns.
More details in the technical report:
더 보기
