Remember when you asked me where did this bottle to? It goes inside ur stupid and gaped asshole. I'm sure you'll be able to keep it inside for a bit while I play with you. How about I open it and pour myself a glass while it's inside ur asshole. Just to humiliate u more slave
Gated DeltaNet-2 (GDN-2) is now a lot faster on NVIDIA GPUs 🚀
Thanks to an amazing effort by our cuDNN team, we now have full cuDNN support for GDN-2. This covers not just prefill (forward), but also a highly optimized backward pass built on CUTLASS primitives, available now as part of the CUTLASS 4.7.0 release.
On GB300 (BF16, batch 4, 64 heads, d=128), compared to the FLA Triton implementation:
⚡ Forward: up to 6.4x faster
⚡ Backward: up to 2.8x faster
⚡ End-to-end: ~3x faster training
Throughput stays flat across sequence lengths from 2K to 32K, meaning the kernels are compute-bound where they should be.
Benchmarks below. 👇
If you are interested to know more about GDN-2, see our manuscript below:
GAMEDAY! Our Golden Eagles are taking on the Provo Bulldogs at home today! Join us for a thrilling lineup: S-3:30p, JV-5:00p, V-6:30. Let’s show our support! #maplemountainsports#