Register and share your invite link to earn from video plays and referrals.

Michael Helcig
@mhelcig
Working on local intelligence. CS @ ETH Zurich, Research @ ISTA | EPFL Co-Creator of GSQ-RCO
123 Following    3.4K Followers
We have noticed recent comparisons published by Unsloth that do not align with our results for DeepSeek-V4.1-Flash-1017. So just if it is not obvious: we did of course not modify any hyperparameters to favor our models. We used standard settings appropriate for a model of this scale (763B parameters) and its 1M context length capability: a 128K context window and a reasoning effort set to 50. To the best of our knowledge, the discrepancy in performance stems from a difference in benchmark constraints - not in the model being worse. Unsloth's comparisons has a strict 32K token reasoning budget, whereas our benchmarks allow up to 128K output tokens. This significant difference in compute explains the variance in results; it is not due to any hidden adjustments on our part. Furthermore, imposing such a low reasoning budget (32K) appears inconsistent for a model designed with a 1M context window and 763B params. Our primary goal is open collaboration and transparency. All of our code is available on GitHub, and our papers are on arXiv. We are genuinely happy that other quantizations are performing well, as this was a key motivation for publishing these models in the first place. We invite anyone with the compute resources to reproduce our results using our exact settings (128K output budget, reasoning effort 50) for independent verification. Our previous models were benchmarked by third parties like @bnjmn_marie, who do great work for the community. We welcome rigorous testing of our claims. Furthermore, we are currently running the comparisons on the models within our setup and will post results here soon. Thanks and stay tuned!
Show more
Run a 763B-parameter model locally. DeepSeek-V4.1-Flash-1017 on a 12GB GPU. Frontier quality with GSQ-RCO quants, directly supported in the Strata inference engine. Further info below. Much more to come. Follow for updates.
Show more
0
98
1.8K
190
Forward to community