So thrilled to have Elliot join the team! Absolutely love his agency to go extremely deep into what ever he is passionate about and his passion for lowering the floor for others through his teaching! Welcome @elliotarledge ! Looking forward to see all the great things you achieve!

custom megakernels coming for the following nvidia chip families: 16 (so 1660 as example), 20, 30, 40, 50. ill have numbers for all these on windows 11 and ubuntu 26 also figuring out profiling tools for android devices so we can squeeze maximum quality/perf/watt from your device (completely airgapped)

18 of the 20 best open models today are Mixture-of-Experts. Dense is basically over. And yet the know-how to train a good MoE still lives mostly in the heads of a handful of teams. So we spent 800k GPU hours running the experiments ourselves and wrote everything down. Super happy to finally release "The Ultimate MoE Training Handbook", free and open, organized around the questions training teams hit: - how many experts, how big, shared or not? - why is half my step spent in all-to-all, and how do I hide it? - why is my run 25x slower with no error in the logs? - softmax, sigmoid or sqrt(softplus) router, and why keep it in fp32? - aux loss or DeepSeek's loss-free bias? - upcycle a dense checkpoint or train from scratch? - can I serve with fewer experts per token than I trained with? ⚠️ Warning: it's long (~70k words, budget a few days, sorry) but with 55 interactive figures, code you can run and numbers from our own runs. Read it here:

phonon is doing well! I'm: - scaling up the data recipe - adding support for many accents, increasing the amount of languages it can by upgrading to Parakeet v3 and training the refiner on more diverse examples - adding mobile support, meaning Phonon and Android - testing on Windows and Linux

i reverse engineered wispr flow, then used deepseek to fine-tune LFM3-1.7B. the result is on par with the wispr flow models you pay for, and it's free, open source and runs entirely on your mac. key-up latency is 65 ms on my m4 max vs ~520 ms for wispr flow on the same clips. a 6 minute yap session pacing around my office still lands in about a quarter second. > rust backend with a native metal engine: each decode step runs as one fused megakernel > a dflash2 speculative drafter is built into the engine, trained on my local 4 x rtx pro 6000 server > parakeet transcribes, LFM3-1.7B refines. both stream while you talk, so key-up only waits on the last chunk > never leaves your macbook, so slow internet or bad ping is no longer an issue > import your own dictionary of terms for your workflow > i picked every default myself. it feels amazing out of the box > adapts to the app you're in: discord is lowercase casual, slack and email are formal, claude code gets clear prompts > training data: 77k transcripts of public audio and synthetic dictations, each rewritten by deepseek v4.1 flash 1017 following a written spec of what "what you meant" means, in 4 writing modes. none of my own dictations went into it > blind judge (claude opus) on an hour of lectures: phonon's text preferred 90 times, wispr's 23 > built with claude opus 5.5 [xhigh] brew install --cask infatoshi/tap/phonon

this is really interesting as I was not expecting a 65 billion parameter model to be performing as well as models over 10x its size on agentic coding tasks. but the intuition comes in when we think about how much of the parameters are just stored knowledge and memory, as opposed to this cognitive core that Karpathy described a little while back. in this case, it has less ability to store knowledge, and we can think of it as 65 billion active parameters for cognitive functioning. not to mention that this is a great recipe for super fast pre-training because they don't have to worry about the complexities of balancing and stabilizing an MoE. this bad boy can fit on a single RTX Pro 6000 Blackwell at NVFP4 precision with the DFlash drafter getting probably really good performance for local setups. I bet there will be some interesting post-trains done on this model which will be extremely good at certain types of agentic coding tasks given that they said they had a light amount of post-training claude code snapshot attached

these days, i spend my time zooming in on why certain commands or tools take so long to execute from a hardware standpoint. i always find crazy inefficiencies in so many tools i use every day. this is why reverse engineering is so lovely. the tools we have, including long training runs and evals, etc... are bottlenecked by assembly factories (or compilers!). training is limited by the MFU achieved, compilers like bun or cargo or nvcc are bottlenecked by parts of their own compilers like lowering, which such code was compiled by something else. everything just goes back to how good our compilers are and what decisions were made around them given the humans or use cases they were intended for. except now we are using them in weird ways the humans who built them didnt expect, and they are slowing us down a lot. have you ever asked a model to look back into its own traces to see what was taking the most wall clock time? > thinking > code gen > tool calls > specific commands > waiting on something to finish theres a pretty high chance you are never actually bottlenecked by the tokens per second. those in a lot of my cases end up being less than 10% of the time. having a slower more intelligent model which is aware of its own time and how efficient the tool is that its about to call is a serious gap for progress. of course this can be RLed... but the tuning will be a beast of its own for the labs. giving my models some skills around this and hooks for awareness of time has been paying off well in my workflow. hope u enjoyed the ramble haha