Prince Canuma
@Prince_Canuma
Joined July 2012
1.3K Following 22.9K Followers
Based
Your are making claims that extrapolate the actual impact of this.
1. There is no mega kernel equivalent for GPU
2. Its a best case with 6 tokens accepted from draft model
3. Only valid for low volume of request. Is that a real use case?
vLLM is cooking, yes, but not it’s not as easy as that
Show more
