Please stop using MLX 4bit affine quants, focused on something else. They are fast but not good enough, especially in larger contexts.
I learned the lesson the hard way using them and even thanks to
@ggerganov pointing me in the right direction a long time ago.