A 27B model in 12 GB of VRAM. Full 262K context. 70 tok/s. And now it is UNCENSORED. I ran it on my own RTX 3090, held to 12 GB. This is Mirai S Qwen3.8-27B, 2.4 bits per weight, an 11 GB file. With my llama.cpp port the same card did 82K context at 38 tok/s inside 12 GB. Then @CaryPalmerr took the port and did the hard part. Same card, same file, his serve: - context: 82K -> 262,144 with q8 KV - decode: 38 -> 69.9 tok/s - VRAM peak: 11.7 GB - prefill: 1,124 tok/s He keeps part of the KV cache in VRAM and the rest in pinned RAM, and he drafts with MTP at every depth. He checked every kernel against my fork token for token and wrote down every experiment, the failed ones too. On his real 12 GB RTX 4070 it does 76.6. My part is small. I added abliteration. You cant edit these weights, they are compressed codes, so I measured the refusal direction on the quantized model and the engine removes it while the model runs. A 1.3 MB file and 1 flag: - harmful prompts refused: 64 of 64 -> 0 of 64 - held-out set: 78 of 82 -> 0 of 82 - decode: 69.9 -> 69.7 tok/s Counted by a regex, and with thinking on 1 of 24 still refuses. Deep in the context it gets slower on my card: 27.9 tok/s at 60K. So thats 12 GB of VRAM, 262K context, 70 tok/s, abliterated. Open weights, open code, runs on your desk. Cary did a genius amount of work and almost nobody has seen it. Go follow @CaryPalmerr right now, I mean it. serve: weights + vector:

Implemented batching support for footless-orcasaq-2-27B-nvidia-p100. Benchmarks show 40 tok/s at concurrency = 4 running OrcaSAQ-2-27B (@OrcaRouter) on a single NVIDIA Tesla P100. More updates to follow. • Repo: • Demo (pure prose):

Good news! OrcaSAQ-2-27B is reaching usable speeds on an Nvidia P100 🤩 • Decode: 24+ tps • Prefill: 200+ tps on long inputs Remember, P100 is a non tensors 2016 GPU 🤯 Wrapping up a few loose ends, organizing the project, and writing documentation now. Opening to the public soon!

Good news! OrcaSAQ-2-27B is reaching usable speeds on an Nvidia P100 🤩 • Decode: 24+ tps • Prefill: 200+ tps on long inputs Remember, P100 is a non tensors 2016 GPU 🤯 Wrapping up a few loose ends, organizing the project, and writing documentation now. Opening to the public soon!

Testing MiniMax-M3.1-Flash-Preview by @MiniMax_AI on kernel optimization. Promising results so far, comparable to DeepSeek V4.1 Flash, but in my experience it trades a little speed for slightly sharper outputs.

Alright, so I just got OrcaSAQ-2-27B (Qwen3.8-27B quantized by orcarouter) up and running on an Nvidia P100. Obviously, performance isn't the greatest, but I honestly didn't expect to run a modern-architecture model using trellis-based quantization on a 2016 GPU with no Tensor Cores that currently goes for around $100–$200 on the used market. Pretty incredible. Status: No MTP, fine-tuning required, PPL matches model documentation. - GPU: Nvidia P100 16GB - Model: OrcaSAQ-2-27B by @OrcaRouter - Engine: footless (In-house development, currently private) Kernels & runtime: Built with assistance from DeepSeek v4.1 Flash by @deepseek_ai .