Interesting HuggingFace post. What if the easiest way to make long-context Local AI faster is to simply STOP feeding it everything? A new HF forum experiment tested 600 distractor-heavy long-context prompts with Qwen2.5-7B 4-bit. Instead of sending the entire ~14K-token context, a MiniLM semantic selector kept only the most relevant ~60%. The result ... 📚 Input tokens 13,124 → 7,702 ➡️ -41% ⚡ Prefill 871 ms → 463 ms ➡️ -47% 🚀 End-to-end latency 944 ms → 519 ms ➡️ -45% 💾 Peak VRAM 11.71 GiB → 9.16 GiB And here's the weird part... 🎯 Token F1 actually IMPROVED: 0.534 → 0.581 So the model got: ✅ less context ✅ lower VRAM use ✅ almost 2× faster prefill ✅ lower total latency ✅ BETTER answers Why? Because more context isn't always better context. Coding agents and RAG systems often drag around enormous histories filled with irrelevant junk. ⚠️ This was a controlled distractor-heavy HotpotQA stress test, not proof that deleting 40% of every real-world context will improve accuracy. 🔗 Link in ALT.

Does anyone know if @huggingface (owned by @NVIDIA) still has access to the MI455X that AMD gave them under embargo? Does anyone know if AMD continues to give exclusive access to next-gen hardware to Hugging Face (owned by NVIDIA)? Will Hugging Face be working on making Nemotron3 Ultra work on MI455X?

Ever since NVIDIA acquired @HuggingFace, we have been looking into migrating some of our work off of HuggingFace and to alternative solutions like ModelScope. Even though NVIDIA's announcement claims they will continue allowing HuggingFace to be accelerator-agnostic, NVIDIA does not have a good track record of developing hardware-agnostic software. We love HuggingFace and hope we are wrong, but at the same time, we are also finding the UX of ModelScope to be great!