Its time we focus on sub 24gb Mac’s - in this day and age, when someone goes and buys a $3000 Mac, while us local-pilled people understand that nearly all that matters is RAM and its speed, the main majority of people in the world consider and expect it to be a high end device - which when compared to the amount of raw compute you’d be able to get 5-10 years ago for that cost, it is. I will be focusing on our “mixture of models” theory; where a main orchestration/chat model spawns and delegates specifics tasks over to another model which specializes in that task and proceeds to act in a manner of; Main model 1 is unloaded -> task specific model 2 is loaded -> does task -> unload model 2 -> load model 1 back and resume I hope that over time this will be able to prove that its not a high parameter count that matters, but that having a handful of models which hyperfocus on specific topics being loaded and unloaded in an efficient manner can come to meet the needs of all automation.

Prefix caching now stores BY TOOL CALL instead of BY TURN allowing for dramatically speedier prefill speeds. Memory allocator bug has been fixed to 8gb, compiled decode for hybrid and other models enabled by default. Will be focusing heavily in more optimizations where I can in the next few days.

DeepSeek-V4-Flash CRACK'd (ablated/uncensored) - Mac's Only (Osaurus/vMLX) 100gb, 20token/s. JANGTQ ~85gb will be out shortly. This model has taught me so much. Most importantly, to actually read research papers that frontierlabs release 😅 Keep in mind the MMLU is honestly just for comparison to base (using logit answering, instruct mode for sake of speed) I went out of my way to make sure it has no issues with code formatting/syntax/etcetc. Max reasoning mode works.