I models can learn to read without a tokenizer. It's called in-context tokenization: context changes how the model groups information into chunks. And the fact that it can work for DNA prediction is mind-blowing! @cartesia gave its new H-Net model a made-up word. At first, it broke it into little pieces. But the more it saw the word, the more of it it grouped together. The same thing happened with patterns and names that kept showing up in code. Then Cartesia trained a separate H-Net on bacterial DNA, and showed it a human gene. And somehow, it picked up on the gene’s structure too. That's wild. So, here's a question: Can we build systems that learn how to organize each kind of data for themselves? Let's see ↓ In this video, I also get into how H-Net works, how it compares to SmolLM3, and why it needed much less training data to reach similar results ->

🚨 OpenAI new model appeared in arena now "Fe-pichu" runs on OpenAI Tokenizer They're testing it on arena rn also a new model from qwen is on arena Will test both and see it

GEAR: 10× faster autoregressive image generation Tencent Hunyuan's new method jointly trains VQ tokenizers and AR generators end-to-end, beating LlamaGen-REPA with a novel dual read-out. All tokenizers are on Hugging Face.

Imagine AI agents turning messy data into a symphony of insights. Tokenizers could be the maestros, transforming noise into knowledge while they automate the mundane. What if agents learned and adapted faster than we can dream?

K2 Horizon got another llama.cpp-family path, this time through TurboQuant. Spark-X2.5 already landed in upstream llama.cpp. I’ve actually got it running myself. A brand-new K2 Horizon TurboQuant PR adds … 🧠 K2 Horizon model support 📦 Hugging Face → GGUF conversion 💬 tokenizer + chat templates ⚙️ llama.cpp-style inference And they’ve already tested 🧠 K2-Horizon-3.7B Q4_K_M 🍎 M3 Pro / 18GB ⚡ ~35 tok/s 📚 64K configured context The PR also adds Spark-X2.5 to the TurboQuant fork: ⚡ ~36 tps 📚 256K configured context Caveats ⚠️ This is still an OPEN PR ⚠️ Only Metal has been tested ⚠️ Other K2 sizes + MoVA/MoE variants remain untested K2 Horizon already has an IFM llama.cpp fork, but it still isn’t supported in normal upstream llama.cpp. So what I’m watching here is whether TurboQuant becomes another path for K2.

AI Toolkit now supports training LoRAs for YuE2-3B, an amazing music model. Support would not be possible without the tokenizer and help from @sin_ceriously . Thank you! Also special thanks to @Machinedelusion. More in 🧵

The complete roadmap for becoming dangerous at LLM engineering from zero to production : Watching tutorials won’t make you an LLM engineer. Building these projects will: - Train a small working model - Master embeddings from scratch - Create your own tokenizer - Code RoPE and ALiBi yourself - Write attention by hand - Construct multi-head attention - Assemble a complete Transformer block - Test different training objectives - Implement generation strategies - Add speculative decoding - Build efficient KV caching - Explore MQA, GQA, and MLA - Solve long-context challenges - Understand FlashAttention deeply - Calculate real hardware limits - Build a simple Mixture-of-Experts - Study sparse vs dense trade-offs - Experiment with state-space models - Try diffusion-based language models - Design solid data pipelines - Generate and use synthetic data - Verify scaling laws yourself - Go through SFT to DPO to RLHF to GRPO - Apply proper quantization - Set up serving infrastructure - Build reliable evaluation systems - Create RAG pipelines - Add tool use and agent loops - Connect vision adapters - Practice interpretability techniques - Run serious red-teaming - Ship one complete end-to-end system Finish even half of these and you’ll be ahead of 95% of people talking about LLMs.

ox alpha ran the best AI marketing campaign of 2026. completely anonymous, no company has claimed it. dropped on openrouter for free with insane rate limits. the community went crazy. people fingerprinting the tokenizer. tracking server locations. everyone was convinced it was a frontier model beating fable. turns out it scored roughly same as opus 4.8 on deepswe. impressive for what it is. but not the frontier mogging everyone thought. but the hype is real. the engagement is real. the mystery did exactly what it was supposed to do.

🚨 WE MAY HAVE JUST CAUGHT OX ALPHA. An Ox Alpha session reportedly surfaced saying: “Powered by GLM, trained by And it gets crazier… Tokenizer fingerprinting reportedly matched GLM on 11/11 probes. So what exactly is Ox Alpha hiding? GLM 5.5? 👀

We built our entire voice stack in-house, training our VAD, tokenizer, and audio models from scratch. In blind head-to-head human evaluations against OpenAI, Grok is consistently rated as the preferred model across axes such as pronunciation, accent, and prosody.