Ridark(@ridark_eth):vLLM shipped a GPU speculator, and full-precision lost vLLM v0.32.0 on Oct 18, GPU n-gram speculator in Model Runner V2. speculators v0.9.0 the next day, 50% less hidden-state storage. 10 open-source GitHub repos that turn full-precision weights into cheap tokens: 167.8k combined stars. 9 Apache-2.0, 1 MIT. 8 of 10 pushed this week. Checked Oct 22. SHRINK THE WEIGHTS 01. llm-compressor ★3.9k https://t.co/sAgzsurVKf -> Quantize and prune for vLLM 02. GPTQModel ★1.3k https://t.co/oiW7WG4lSQ -> GPTQ and AWQ on any hardware 03. Model-Optimizer ★5.4k https://t.co/RkF5y7k4Py -> Quantize, prune, distill for vLLM 04. bitsandbytes ★8.5k https://t.co/EgAZg2iDfj -> 4-bit and 8-bit PyTorch kernels SPECULATE AHEAD 05. speculators ★914 https://t.co/11QSbaKjzS -> Train draft models for vLLM 06. ArcticInference ★490 https://t.co/CiTWU54q4o -> vLLM throughput and latency plugin BATCH AND SERVE 07. vllm ★94.9k https://t.co/pfZl1IMgP3 -> Continuous batching, prefix caching, quantized serving 08. sglang ★37.6k https://t.co/OHRWIObDc5 -> Fast serving for agentic workloads 09. lmdeploy ★8.1k https://t.co/TIqWjm4qwS -> Compress, deploy, serve in one toolkit 10. flashinfer ★6.7k https://t.co/bWEsw2m4RZ -> Attention, GEMM and MoE kernels the stack: full-precision model → quantize → train draft → batch and serve → fused kernels → cheaper tokens uv pip install vllm vllm download-kernels # precompiled kernels for faster startup W4A4 lands about 2.3 points below W4A16 on GSM8K Platinum (GPTQModel README). The trade is always accuracy. Not a bigger GPU. A tighter pipeline from compressed weights to fused attention ↓

2026.10.22 20:48

vLLM shipped a GPU speculator, and full-precision lost vLLM v0.32.0 on Oct 18, GPU n-gram speculator in Model Runner V2. speculators v0.9.0 the next day, 50% less hidden-state storage. 10 open-source GitHub repos that turn full-precision weights into cheap tokens: 167.8k combined stars. 9 Apache-2.0, 1 MIT. 8 of 10 pushed this week. Checked Oct 22. SHRINK THE WEIGHTS 01. llm-compressor ★3.9k -> Quantize and prune for vLLM 02. GPTQModel ★1.3k -> GPTQ and AWQ on any hardware 03. Model-Optimizer ★5.4k -> Quantize, prune, distill for vLLM 04. bitsandbytes ★8.5k -> 4-bit and 8-bit PyTorch kernels SPECULATE AHEAD 05. speculators ★914 -> Train draft models for vLLM 06. ArcticInference ★490 -> vLLM throughput and latency plugin BATCH AND SERVE 07. vllm ★94.9k -> Continuous batching, prefix caching, quantized serving 08. sglang ★37.6k -> Fast serving for agentic workloads 09. lmdeploy ★8.1k -> Compress, deploy, serve in one toolkit 10. flashinfer ★6.7k -> Attention, GEMM and MoE kernels the stack: full-precision model → quantize → train draft → batch and serve → fused kernels → cheaper tokens uv pip install vllm vllm download-kernels # precompiled kernels for faster startup W4A4 lands about 2.3 points below W4A16 on GSM8K Platinum (GPTQModel README). The trade is always accuracy. Not a bigger GPU. A tighter pipeline from compressed weights to fused attention ↓

Ridark@ridark_eth

2026.10.22 18:32

Opus 5.5 is the coldest $0 hire in AI. One builder's agent replaced a $200,000-a-year role overnight. Chief-of-staff agent running Opus 5.5 inside Viktor. Its own cloud computer. Owner's laptop shut. One rule: find $50,000 a month of new business, public data only, contact nobody without a yes. Builder's overnight log: → 9PM SPLIT the goal into 4 jobs, each with a day and a deadline → 11PM PULLED 240 dental clinics in Austin off Google Maps, cut every chain → 1AM READ every remaining site, kept 25 with one noticed detail each → 2AM WROTE an opener for all 25, sent zero → 3AM BUILT the offer on one page, price left blank for the owner → 8AM Owner typed $2,000. 25 × $2,000 = $50,000/month on the table $50,000 a month is potential revenue on a goal table, not signed clients. Nothing ships without the owner's yes. The blank price is the real gate. An agent that fills in your price loses the deal before you open your laptop. Every Monday it runs again. 100 new clinics a month. The funnel never stops.

커뮤니티로 전달