Register and share your invite link to earn from video plays and referrals.

Cruz
@ViC305
God-fearing husband & father. AI Engineer. Fine-tuning + day-zero EXL3 & GGUF quants + benchmarks • Local LLMs • honest tok/s • fitting big models on small GPUs
Joined June 2009
976 Following    2.9K Followers
🚀 FULL GLM-5.3. FOUR DGX SPARKS. A NEW PARETO-FRONTIER CONTENDER. This is a HUGE personal milestone for my SAGE work! 🔥 My GLM-5.3 SAGE EXL3 3.38 bpw brings the full model down to 319 GB while preserving ALL 19,200 routed experts. No pruning. No expert merging. Every bit matters. 🎯 INDEPENDENT-SET FIDELITY • KL divergence: 0.082388 nats • BF16 top-1 agreement: 91.1523% • 10,240 scored positions on text excluded from calibration ⚡ FOUR-SPARK PERFORMANCE Around 41 tok/s average (69 tok/s burst) across six real prompts with DFlash2, with drafted replies matching serial decoding token for token. 256K-context serving demonstrated. Sized toward a 1M-context memory target, with full-length inference validation still ahead. The goal is the highest-fidelity full GLM-5.3 possible on four Sparks WITHOUT sacrificing the memory needed for 256K–1M context. Not just making the weights fit, but leaving room to actually USE the model. 🙏 A MASSIVE thank you to @cfontes, who personally funded the compute required to make this model possible. This release would not have happened without his support! And we’re not done. He’ll be releasing a recipe soon that’s going to blow everyone’s minds! 🤯🔥 This is what community-backed local AI looks like. More capability, less memory, and people coming together to push what’s possible. 💪 Model 👇
Show more