Han Xiao(@hxiao):the charm of local/self-hosted AI is to run abliterated models: Qwen-3.8-27b-abliterated 110~120 tps on ½ nvidia rtx6k (½ bc cheaper and just enough size to run it). image+text, context 230K. possibly the best hill-climbed and actually usable abliterated model on a low budget gpu. the prefilling speed is slower than RadixArk's nvfp4 as their weights and the activations are 4-bit, so every matrix multiply runs natively on Blackwell tensor cores. but this gap shrinks in practice as most of the time cost in my long-horizon tasks is on decoding. https://t.co/3qCfN6GK2k if u haven't tried any abliterated models before, you definitely need it. also comparing their reasoning chain side by side vs the regular model, the difference is huge.

2026.10.16 17:21

the charm of local/self-hosted AI is to run abliterated models: Qwen-3.8-27b-abliterated 110~120 tps on ½ nvidia rtx6k (½ bc cheaper and just enough size to run it). image+text, context 230K. possibly the best hill-climbed and actually usable abliterated model on a low budget gpu. the prefilling speed is slower than RadixArk's nvfp4 as their weights and the activations are 4-bit, so every matrix multiply runs natively on Blackwell tensor cores. but this gap shrinks in practice as most of the time cost in my long-horizon tasks is on decoding. if u haven't tried any abliterated models before, you definitely need it. also comparing their reasoning chain side by side vs the regular model, the difference is huge.