TheDavidTai
@TheDavidTai
Founder, hacker, stochastic tool. @microsoft escapee. Currently @eigenlabs.
Checkout my github actions emulator:
648 Following 1.1K Followers
Since a lot of the community is interested in actual performance of the current community kernel after 5 days. Here's some quick tests with MTP at depth 3 and temperature 0 on a basic 1024 token programming prompt.
You can compare with something like @redp314's vllm tuning here (third screenshot)
Show more
Checkout @eigenlabs and our first paper about @yukonresearch, the future of crowdsourced research!
99.1 tps after @morgymcg made a late breakthrough much like he did in Laguna. Final push for getting this over 100 tps this weekend!
How did @MiaAI_lab and @0xSero get this running similarly to 27b 8bit speeds? Just ported to MLX for test runs. 👀. .
Its really great that Eigen is gamifying frontier software research. We are in an era where LLMs let everyone try and solve problems without needing to understand the tooling just the problem itself.
Being able to iterate and apply theory without worrying so much about the medium can empower every curious mind.
Show more
