One of the changes made to improve Ghostty IO throughput was to introduce a new IO gather thread, bringing our total thread count to 3 per terminal instance. Under heavy IO load, this keeps our VT processing working at nearly full bandwidth without stalls.
Previously, Ghostty's IO thread (since 2023!) followed this general shape:
read_pty(); exit_if_quitting(); process_data()
The problem is that under heavy load, the pty kernel-side buffer becomes full, blocking the writer (e.g. TUI program). So while Ghostty was in `process_data`, the rest of the world just stalled. Then when we got back to `read_pty`, we'd read the next chunk (a syscall), and while we're paying the syscall cost, Ghostty isn't processing VT data.
Additionally, on macOS and Linux, the kernel only gives a max of 1024 bytes out of a `read()` on a pty (even if the buffer on either side is larger). So, when you `read_pty`, you get 1024 bytes and get back to blocking.
I tried reading from the pty until EAGAIN to fill a buffer but it actually didn't move the needle much cause the major slowdown issue were the stalls on eithe rside.
So the gather thread instead sits in a loop of reading from the pty and filling a set of preallocated buffers. If a read is less than 1024 bytes we assume we're not under any write pressure and dispatch immediately. This preserves latency.
But when we read a full 1024 byte we assume we are under write pressure and sit in a CPU spin loop (the context switch on a blocking read is higher than the time it takes the writer side to write to the pty). We do this until we get less than 1024 bytes or a timeout of 3 nanoseconds passes or the buffer is full (64KB). Then we dispatch.
As a result, under heavy load, Ghostty is effectively processing data through our VT processor at 100% efficiency so IO is fully bottlenecked there, for now.
This was all discovered in concert with LLM usage, which helped pull direct kernel source (XNU + Linux) to validate assumptions, write minimal harnesses in C to quickly verify, and rubber duck some of my approaches.