Daniel Han
@danielhanchen
Building @UnslothAI • Making open-source LLMs faster, better & more accessible • YC S24 • ex-NVIDIA ML
2K Following 37.2K Followers
Unsloths support for Deepseek is here! 👀
+ Improvements on Strix
+ Improvements on parallel requests
+ Improvements on DGX Spark
+ Improvements on Intel B-series
See more at:
Show more
We're running 128K for all benchmarks - LCB and Aider are finished - the results are even more obvious
We have noticed recent comparisons published by Unsloth that do not align with our results for DeepSeek-V4.1-Flash-1017. So just if it is not obvious: we did of course not modify any hyperparameters to favor our models. We used standard settings appropriate for a model of this scale (763B parameters) and its 1M context length capability: a 128K context window and a reasoning effort set to 50.
To the best of our knowledge, the discrepancy in performance stems from a difference in benchmark constraints - not in the model being worse. Unsloth's comparisons has a strict 32K token reasoning budget, whereas our benchmarks allow up to 128K output tokens. This significant difference in compute explains the variance in results; it is not due to any hidden adjustments on our part. Furthermore, imposing such a low reasoning budget (32K) appears inconsistent for a model designed with a 1M context window and 763B params.
Our primary goal is open collaboration and transparency. All of our code is available on GitHub, and our papers are on arXiv. We are genuinely happy that other quantizations are performing well, as this was a key motivation for publishing these models in the first place. We invite anyone with the compute resources to reproduce our results using our exact settings (128K output budget, reasoning effort 50) for independent verification. Our previous models were benchmarked by third parties like @bnjmn_marie, who do great work for the community. We welcome rigorous testing of our claims. Furthermore, we are currently running the comparisons on the models within our setup and will post results here soon.
Thanks and stay tuned!
Show more
We're releasing new Unsloth v3.1 GGUF quants for DeepSeek-V4.1-Flash-1017. We ran benchmarks and v3 + v3.1 do better than GSQ-RCO.
We also cannot reproduce numbers ran by GSQ-RCO - AIME25 for 1bit is not 100% but rather 91.88%. Livecodebench and GPQA are similarly incorrect.
There are also claims that KLD is not a good metric, and that people can "game" KLD. The pirmary goal of quantization is to minimize KLD but also minimize disk space. If you get a KLD of 0, this means your quantized model == 16bit model, as logits are perfectly modelled.
We ran benchmarks on multiple benchmarks including AIME25, Aider Polyglot, LiveCodeBench v6, HMMT 2026, GPQA Diamond, MMLU-Pro and IFEval, and also recorded reasoning length to see if lower bit quants think for longer.
In we show that lower bit quants do think somewhat longer than the unquantized model - we plan to investigate and make quants even better.
However, we're still much ahead vs GSQ-RCO on even reasoning length as well.
We provide full benchmarking data, benchmarking methods (for eg we ran AIME25 16 times per question and took an average) and more in our updated blog at
Show more
We’re releasing new DeepSeek-V4.1-Flash-1017 GGUFs, ranking highest across benchmarks.
Dynamic V3.1 leads across nearly all benchmarks: GPQA Diamond, AIME25, LiveCodeBench v6, MMLU-Pro, Aider & more.
Run on 64GB+ RAM.
GGUF:
Blog:
Show more
Mistral Large 4 is here! We made some Unsloth quants & tested across many domains! 1bit retains 91% of top-1 accuracy across coding, agentic tasks, math etc whilst being 80% smaller (223GB vs 1.1TB FP8)
They all work in Unsloth Desktop now! Thank you Mistral for day zero access!
Show more
Mistral Le Chonk can now run locally!
The 1 trillion parameter model runs on a Mac Studio with our 1-bit GGUF which retains 91% top-1 accuracy.
Le Chonk is the strongest Western open model to date.
GGUF:
Guide:
Show more
Unsloth now has browser use for all local models! It can take screenshots and act as well! We also added some cool Halloween themes and effects!
If you click Trick or Treat, there are over 11 themes to choose from + animations 👻🎃!
Show more
Your local LLM can now take control of a browser and complete tasks autonomously via Unsloth!
Qwen3.8-27B successfully controlled the browser to complete tasks like chess, painting, with no human intervention.
GitHub:
Guide:
Show more
MiniMax M3.1 Flash meets the @UnslothAI community. 🐙🦥✨
Unsloth’s Dynamic GGUFs bring our 428B MoE model to a 143.9GB 2-bit footprint, with 91.5% top-1 token agreement in their evaluation across seven domains.
That means a lower memory barrier to self-hosting, deployment through Unsloth Desktop and llama.cpp, and an OpenAI-compatible API for existing apps and agents.
With 78K+ GitHub stars and over 500M model downloads on Hugging Face, Unsloth gives M3.1 Flash a path into an established community of developers running, evaluating and building with open models.
Open weights. Local control. More ways to build.
Show more
Unsloth works seamlessly in RTX Spark! You can do video, image, audio generation and all multimodal llm coding tasks plus training for all!
Native Windows ARM64 + PyTorch + triton kernels are all enabled!
Show more
Getting started with @UnslothAI on RTX Spark? Just run the .exe.
@danielhanchen and @NaderLikeLadder got hands-on with NVIDIA RTX Spark to talk about running and training AI models locally, exploring reinforcement learning, and making it easier to start building with Unsloth.
Show more
Getting started with @UnslothAI on RTX Spark? Just run the .exe.
@danielhanchen and @NaderLikeLadder got hands-on with NVIDIA RTX Spark to talk about running and training AI models locally, exploring reinforcement learning, and making it easier to start building with Unsloth.
Show more
We made Unsloth UD3 MiniMax-M3.1-Flash quants! We tested across multiple domains to ensure our quants work great for them as well!
By default, MiniMax-M3.1-Flash uses NVFP4 for MoE layers and BF16 / MXFP8 for the rest. UD-IQ1_M retains ~89.9% top-1%. Works in Unsloth Desktop!
Show more
MiniMax-M3.1-Flash can now run locally!
We made 2-bit GGUF retain 91.5% top 1 accuracy while being 45% smaller. Run on 150GB RAM.
MiniMax-M3.1-Flash is the strongest open model for its size & among the best overall.
GGUF:
Guide:
Show more
Your agents have spoken. They want more compute. We’ve answered: a new Mac Mini M6 is being raffled off TONIGHT at the Open Source AI Awards. Keep your agents happy. 🏆
I’ll be speaking about why open source AI and open models are essential to keeping innovation accessible, AI accountable, and the freedom to build in everyone’s hands.
Joining me on stage:
• Peter Walker @PeterJ_Walker — @OpenRouter
• Ian Ballantyne @IanBallantyne — @GoogleDeepMind
• Daniel Han @danielhanchen — @UnslothAI
• Irene Solaiman @IreneSolaiman — @huggingface
• Ahmad M. Osman @TheAhmadOsman — @OsmanticAI
• Alex Newman @thedotmack — @Claude_Memory
• Lucas Atkins @latkins — @arcee_ai
• Basil Chatha @realbasilchatha — @latentspacepod, moderating the fireside with Peter
Then we celebrate the builders behind the breakthroughs with @open_intel_fdn.
🏆 Project awards:
The Bedrock Award • The Reward Signal Award • The Guardrails Award • The Reality Check Award • The Mission Control Award • The Glass Box Award
🏆 Individual awards:
The Community Lifeline Award • The Unsung Maintainer Award • The Open Knowledge Award • The Naming Things Is Hard Award
And yes, there are prizes for the “Best OSAI Week Post” on X. Share your highlights using BOTH #OpenSourceAIWeek# and #OpenIntelligence# to enter.
🍽️ Doors open at 5 PM with hot food
🎤 Program starts at 6 PM sharp
Interest is high and entry is first come, first served. Arrive early!
Registration is required:
Let’s give the people building open AI their moment.
#OpenSourceAIWeek# #OpenIntelligence#
Show more
NVIDIA releases Nemotron-3.5-Bolt-E4B, a new model that runs locally on 6GB RAM. 💚
Bolt-E4B can run on edge devices like phones and delivers top-tier performance for its size. Trained on Gemma 4, it supports vision.
GGUF:
Guide:
Show more
Available now: NVIDIA Nemotron 3.5 Bolt E4B 4GB ⚡️
Our newest small open model is built for local assistants, interactive characters and robots that can carry on a conversation and use tools to get things done.
Watch Reachy Mini chat and move, with Bolt handling conversation and NVIDIA Cosmos handling vision. Both models run locally on Jetson Orin Nano 8GB.
Learn more:
Show more
We plugged Beam into @opencode, asked it to look at @UnslothAI, and create a fine-tuning notebook for the latest and smallest Gemma-4 model on a Text2SQL task – an out of distribution domain for Beam.
The model successfully researches the model cards, Unsloth documentation, and available READMEs, and implements the fine-tuning logic and surrounding machinery. As a result, Beam is able to successfully finetune Gemma, increasing Gemma’s accuracy on its heldout test set by 66.5%.
Show more
We made Flint Unsloth GGUFs - Recursive's new model - it's an MoE that's the best perf for its size!
It works great for tool calls and agentic coding in Unsloth Desktop! We also now have a beta browser + browser-use, skills, more themes and more! Flint can use them all!
Show more
Run Flint locally on 24GB RAM via Unsloth Dynamic GGUFs!
Flint is the strongest MoE model under 50B parameters.
Recursive’s Apache-2.0 model has 39.8B parameters (2.5B active). Run at 125 t/s on Mac, 150 on RTX Spark
GGUF:
Guide:
Show more
We made some Unsloth UD3 dynamic quants for Ornith's new 122B release! UD-Q2_K_XL is 44.5GB and retains around 84% top-1% for wikitext general and 89% ish for chat whilst being 82% smaller (BF16 = 249GB)
Show more
🥳 Ornith-1.5-122B Unsloth GGUFs are now available!
🧡 Thanks for the awesome work by @UnslothAI
🔗
We worked on a llama.cpp impl for DS4.1-Flash + made Unsloth dynamic GGUFs for it!
The engram / PLE can be swapped out from MXFP8 to Q6_K to Q5_K and IQ4_NL precision - we found Q6_K approx = MXFP8 and IQ4_NL loses a tiny bit of accuracy.
The engram can stream fully from disk!
Show more
DeepSeek V4.1-Flash-1017 can now run locally! 🐳
We made 1-bit retain ~88.2% accuracy while shrinking size from 508GB to 250GB (−52%).
Run on a 128GB+ RAM device.
V4.1 Flash 1017 is the strongest open model to date
Guide:
GGUF:
Show more
Unsloth now works with ARM64 + Windows + CUDA on RTX Spark! We made [running, training] X [diffusion, multimodal, language, decision] models all run natively + we optimized it to make it 2.7x faster!
Get it via winget or
Show more
You can now run and train 120B+ parameter models locally on a laptop! NVIDIA RTX Spark is here. 💚
With Unsloth and RTX Spark, train diffusion, decision, multimodal models ~2.7× faster with optimized kernels and binaries for CUDA on ARM. All local with up to 128GB unified memory
Show more
how do you fine-tune a decision model on your own data?
in these two notebooks, you'll learn:
• how to prepare the training data
• how to fine-tune our d1-3B decision model
• using @UnslothAI and @huggingface TRL
all within 30 minutes on a free-tier GPU.
unsloth:
TRL:
ps: how adorable are these halloween kitties? 👻
Show more
Muse Spark Open from Meta is here! We made Unsloth UD-3 quants for the model with UD-IQ1_S being 176GB
We also did some KLD benchmarks and recovered as much accuracy as we could for the smaller quants!
Muse Spark Open is fully multimodal and works in Unsloth Desktop!
Show more
Meta releases Muse Spark Open, a new SOTA open model.
Run on a Mac Studio via Dynamic 2-bit Unsloth GGUFs.
Muse-Spark-Open-816B-A42B supports text, images, video and audio input with 1M context.
GGUF:
Guide:
Show more
We created some Unsloth UD-3 GGUF quants for NVIDIA's Nemotron 3.5 Super and Bolt model!
Bolt runs great in RTX Spark!
We also made NVFP4 GGUFs for Super!
We also ran KLD benchmarks for both - docs at and
Show more
NVIDIA releases 2 new models: Nemotron-3.5-Bolt and Nemotron-3.5-Super-VL.
Nemotron-3.5-Bolt-74B-A9B runs on 24GB RAM. It delivers top-tier performance for its size.
Super-VL-120B-A12B runs on 82GB RAM.
Bolt GGUF:
Guide:
Super GGUF:
Show more
Yesterday I trained my own Jev style decision model by finetuning Qwen3.5-4B on the LocalLLaMA/typed-decisions dataset with @UnslothAI
added a decision head + trained with 4-bit QLoRA on a single A100 within 15 mins, 4 epochs -> 80% accuracy (2 Epochs are enough though)
Merged the LoRA and quantized to Q4_K_M.
Today its so satisfying to watch it play Flappy Bird locally on a single RTX 4060 via llama.cpp.
250ms latency (faster than most APIs I've tested) in just 4GB VRAM during inference.
What should I finetune + test next?
Show more
