Joshua Warren
@JoshuaSWarren
Omarchy M team. Making Apple Silicon fast under Linux: MLX, Mesa GPU, Neural Engine. Shipped an iPhone app with no Mac. Building Remnic.
6.4K Following 9K Followers
TensorFold now runs on Apple M1 GPUs under Linux via a native Vulkan backend.
Today's change sends each token's GPU work as one batch: 160 to 94 ms per token on an M1 Max, 41% faster, byte-identical output.
PR in progress.
Show more
Linux on the M6 Mac Mini, Day 5 - success!
Omarchy is installed on the Mac Mini's hard drive. It boots, it runs.
Still bringing up accessories, including Ethernet and GPU, so my work's not done, but 5 days from the macOS About screen to the Omarchy About screen isn't bad!
Show more
The last thing that sent me back to macOS is gone.
iPhone apps signed on Linux now get push notifications, Sign in with Apple and universal links in my newest apple-dev release.
No macOS. No Xcode.
Open source, built for @omarchy:
Show more
Apple locks fast local AI to macOS with Metal.
I picked the lock.
metal2vk translates Metal GPU code to Vulkan, unchanged.
598 of 610 real programs from @trymirai's uzu and @ashxhart's TensorFold so far.
Engines built for Macs now run on Linux.
Show more
Credit where it's due: MiniMax H3 is the model. The engine is @ashxhart's TensorFold, with H3 support by @u1tra_instinct and the MLX port from @pipenetwork and @AIBizarrothe.
My part: the Linux on Apple Silicon port. The int8 weights are on Hugging Face:
Show more
@ashxhart Amazing work, @u1tra_instinct - and thank you for open sourcing it! I've got something fun built on top of it and TensorFold for @OmarchyMac that I should be able to announce shortly, just waiting on a @huggingface upload to complete. 😀
Show more
Running AI on a Mac's GPU under Linux was painfully slow.
So I went into Mesa, the open source GPU driver, and made it 18x faster.
Decode: 1.5 to 27.4 tokens/sec.
Prompts: 55 to 237 tokens/sec.
On an M1 Max with Omarchy Linux.
It's all open source:
Show more
Omarchy on Apple Silicon makes local AI fast with my MLX and Apple Neural Engine drivers.
@dhh inspired me to believe computing should be fun, look good, and run fast.
So, I built Coreglass to make inference profiling fun, fast and beautiful.
Show more
Got an M1 Pro, M1 Ultra, base M2, M2 Pro or M2 Ultra? Or any M3 or newer Mac?
I'm bringing the Apple Neural Engine up on Omarchy and need data from your chip. Run our updated collector under macOS and Omarchy.
Show more
I need your help to expand the Linux driver for the Apple Neural Engine!
I am looking for volunteers with these specific chips, ideally running Omarchy already (but you can help some from macOS as well):
M1 Pro
M2
M2 Pro
M2 Ultra
No kernel knowledge needed. DM me if you can help.
Show more
mlx-omarchy v0.5.0 is up: Core ML model runs end to end on Linux for the first time ever.
Parakeet, encoder on the Apple Neural Engine, matches the macOS transcript byte for byte for on-device voice.
M1, M1 Max fully supported, M2+ work in progress.
Show more
I'm excited to be part of Omarchy!
With mlx-omarchy I've enabled users to run open source AI on M1 Macs with Omarchy. Now I'm setting my sights on the M5/M6 chips & combined Omarchy Mac/DGX Spark inference.
Helping you put that Mac Mini you bought this spring for your Claw to use!
Show more
An agent on @OmarchyMac on my M1 MacBook Pro just built an iPhone app, compiled it, pushed it to my iPhone and attached a live debugger to it.
No macOS device needed.
Building this into a package so you install apple-dev on Omarchy for everything you need for iOS & macOS dev.
Show more
mlx-omarchy now has 98.5% feature parity with mlx running natively on macOS.
Performance gap is closing, performance improved greatly today but there is still much to do before we have performance parity.
Hoping to be there by the end of the holiday weekend here in the US.
Show more
