Joshua Guo(@jshguo):Got Qwen3.8 running locally and made a particle visualization for it. The model runs in LM Studio on my PC with a 20GB RX 7900 XT. I work from my Mac, and Pi Agent talks to it over the local network. Current setup: - unsloth/Qwen3.8-27B-UD-Q3_K_XL - 128K context length - 24 offloaded to GPU, automatically set by LM Studio - Flash Attention on - Q4_0 K/V cache on GPU - Batch size 256, one concurrent prediction - Default RoPE

Joshua Guo

@jshguo

Design Engineer / Building stuff with AI / / / Paid Partnerships ↓ partnerships@v2space.net /

参加 October 2022

922 フォロー中 26.8K ファン

Joshua Guo@jshguo

2026.08.27 07:04

Got Qwen3.8 running locally and made a particle visualization for it. The model runs in LM Studio on my PC with a 20GB RX 7900 XT. I work from my Mac, and Pi Agent talks to it over the local network. Current setup: - unsloth/Qwen3.8-27B-UD-Q3_K_XL - 128K context length - 24 offloaded to GPU, automatically set by LM Studio - Flash Attention on - Q4_0 K/V cache on GPU - Batch size 256, one concurrent prediction - Default RoPE - Speculative decoding and MTP off