I made a computer use agent that works without screenshots.
It uses the macOS accessibility API to interact with applications.
You can run it in the cloud with
@daytonaio, connect it to any LLM, and control any macOS application.
Here it is building a simple program in Xcode: