Alok(@analogalok):Yesterday I trained my own Jev style decision model by finetuning Qwen3.5-4B on the LocalLLaMA/typed-decisions dataset with @UnslothAI added a decision head + trained with 4-bit QLoRA on a single A100 within 15 mins, 4 epochs -> 80% accuracy (2 Epochs are enough though) Merged the LoRA and quantized to Q4_K_M. Today its so satisfying to watch it play Flappy Bird locally on a single RTX 4060 via llama.cpp. 250ms latency (faster than most APIs I've tested) in just 4GB VRAM during inference. What should I finetune + test next?

Alok

@analogalok

Mechatronics Engineer AI belongs on your device. • Offline inference • No subscriptions. Teaching you to own your AI Intelligence Stack

参加 November 2013

241 フォロー中 5.9K ファン

Alok@analogalok

2026.10.11 11:26

Yesterday I trained my own Jev style decision model by finetuning Qwen3.5-4B on the LocalLLaMA/typed-decisions dataset with @UnslothAI added a decision head + trained with 4-bit QLoRA on a single A100 within 15 mins, 4 epochs -> 80% accuracy (2 Epochs are enough though) Merged the LoRA and quantized to Q4_K_M. Today its so satisfying to watch it play Flappy Bird locally on a single RTX 4060 via llama.cpp. 250ms latency (faster than most APIs I've tested) in just 4GB VRAM during inference. What should I finetune + test next?