Joe Barrow
@barrowjoseph
NLP + Machine Learning
Prev: NLP Researcher @AdobeResearch, PhD @ClipUmd.
Joined December 2012
517 Following 5K Followers
Good engineering is about observability.
Speculative decoding accelerates LLM inference, but you're running it blind. Every rejected draft token is wasted compute, but are you looking at the drafts?
This weekend I wrote specspecs to solve that!
Show more
