100 seconds TTFT with 256K cold cache will feel slooooooow to anyone used to OpenAI or Anthropic or Deepseek hosted speeds.
That said, for personal AI your cache hit rate should be high. Is it worth $10,000+? Depends on your use case.
You get privacy, security, offline access, and no guardrails. You also get a crazy fast personal desktop computer for AI development.
How deep are your pockets? ๐ธ๐ธ๐ธ
I reviewed the M5 Ultra Mac Studio with 256 GB of RAM.
Itโs the dream Mac for local AI agents. And Iโve gone local-only with Hermes.
My story, with LOTS of interactive charts: