Mike Key
@1337hero
Senior Software Engineer going hard on AI.
Building local LLMs & AI agents
Homelab - 3x AMD R9700 & Strix Halo
Shipping code @
参加 March 2023
279 フォロー中 2.2K ファン
Figured it out eventually
Muse is the slowest dense model I've ever attempted to run at Q8_0 and I don't know what the problem is. All day and I can't get it faster than 16tok/s / 661.7 ppt - pretty much WORSE than me trying to run DSv4 1 bit.
もっと見る
