Zain
@zainhas
I build and teach AI • AI/ML @togethercompute • EngSci ℕΨ/PhD @UofT • Previously: vector DBs, data scientist, lecturer & health tech founder • 🇺🇸🇨🇦🇵🇰
2.1K Following 8.3K Followers
stabilizing RL using a simple second-year probability undergrad trick!
Cov(A, B) = E[AB] − E[A] E[B] ⇒ E[AB] = E[A] E[B] + Cov(A, B)
show this to all the undergrads complaining about the applicability of boring courses😅
Show more
harness choice impacts the cost alot more than it does the quality
> Quality delta is about 3-4%
> Cost delta is 2-4x
True for swebench lite i wonder how this behaves for non hillclimbed domains outside code.
Show more
how inference engines actually work - releasing full talk slides!
i cover everything in the lifetime of a request e2e:
> the inference engine
> kv + prefix caching
> continuous batching
> paged attention
> chunked prefill
> sampling
> agentic loops from inside the engine
Show more
Somebody page @HamelHusain
Deepseek is looking at the data
Hmm, data is still and will always be the biggest moat.
Deepseek is clearly saying that as of now better post-training comes more from "better data + better environments" than from novelties in RL algorithm.
Show more
probably the most important thing about this release:
"Compared with the previous generation, V4.1-Flash’s KV cache needs just:
🔹 1/4 the HBM
🔹 1/8 the SSD storage"
4x smaller than even v4 flash
Show more
💾 Smaller KV cache. Bigger savings.
Compared with the previous generation, V4.1-Flash’s KV cache needs just:
🔹 1/4 the HBM
🔹 1/8 the SSD storage
Cache-hit charges often account for a large share of agent costs. Compressing the cache cuts those costs significantly.
3/6
Show more
i appreciate that they provide a nice mapping for API backward compat
I am always correct about the big picture, because I understand how they roll. DeepSeek V4.1 tosses out V4's cludge on every level. HCA is ditched, CSA generalized. no sparse attention warmup. simpler single-pass mHC. Engram, etc etc.
Welcome to the new Transformer.
Show more
theres absolutely nothing wrong with gpt-4o-mini-realtime-audio-turbo-preview-v2-2024-12-17
One of the greatest joys is debating new model names with researchers. They come up with the most hilarious stuff
By the way, before you get too excited
objectively speaking, V4.1 Flash is a ≈718B model, ie almost the same size as GLM 5.3 (just very, very sparse). 552B mostly-FP4 backbone (307GB) + 196B FP8 E4M3 Engrams. ≈500GB on disk. It's a looot bigger than V4Flash or GLM 5.3 Flash.
Show more
make "encoder-decoder" great again😉
just a quick guide on how to use the 0-100 reasoning_effort setting on ds v4.1 flash
oh wow have not seen this before for any model... this complicates things
> "DeepSeek-V4.1-Flash supports a continuously controllable reasoning effort from 1 to 100."
reasoning_effort = [1 to 100]
Show more
i appreciate that they provide a nice mapping for API backward compat
need to normalize releasing these "model performance across harnesses" eval
> depending on the harness you're using a new model with you have no idea if you're using a frontier model or the equivalent of a one from a year ago
Show more
deepseek-v4.1 flash -- for once a model where the reasoning_effort scales the output quality and token output nice and ~linearly
> quality improves nicely with reasoning effort setting 1 to 100
> Avg output tokens also scale nicely with effort
Show more
+1 point for the whale on felony bench, I suppose?
deepseek eval'd their new v4.1 flash model in 8 different harness configs
> model performs best in minimal harnesses -- mini-SWE & minimal deepseek harness
> underperforms in both claude code and codex
Show more
oh wow have not seen this before for any model... this complicates things
> "DeepSeek-V4.1-Flash supports a continuously controllable reasoning effort from 1 to 100."
reasoning_effort = [1 to 100]
Show more
god those deepseek-v4.1 flash numbers look good👀
> sota on deepswe up there with sol and fable5
> very competitive on terminal bench 3
DeepSeek-V4.1 Flash tech report out now too!
🚀 Introducing DeepSeek-V4.1-Flash: smarter, faster, more efficient.
🔹 Introducing the smallest model in our new architecture family, with native visual understanding.
🔹 Designed for greater capability, faster inference, higher throughput, and scaling to larger models.
1/6
Show more
