Run Ornith 1.5 9B and 35B A3B locally via Atomic Chat ๐ฆโ๐ฅ
We shipped the full 9B GGUF ladder on Hugging Face, from lossless BF16 (17.9 GB) down to 2-bit (2.8 GB) and measured all against stock quants
AD-Q4_K runs on a 16GB MacBook Air with 64k context and picks the same next token as the BF16 original 91.9% of the time