Hello. Quick intro post about myself: - 8 months ago I got tired of paying for AI credits. I thought I could build an inference server myself for cheaper. - I bought 2 RTX 3060s for $500 and it was a great disappointment. - The models back then were really bad and my hardware wasn't helping. I was able to run Qwen 3.5 27b at some ridiculously small quant (pre GSQ-RCO), at a great speed of 9 t/s and probably less than 100 t/s for prompt prefill. - I didn't give up. 3 months ago, I got exposed to Qwen 3.8 27b, and for the first time, it seemed like there was potential, and then I fell down the rabbit hole. - I decided to double down and buy 3 P100 Datacenter GPUs that were released in 2016 - when they came out, they were going for ~$6K; bought for $70 each. - Since then I've been working on optimizing these pre-tensor core GPUs for the best performance on modern models. - Today I'm at 50 t/s decode and 1500 t/s prompt prefill on Qwen3.8 Flash Next, serving 3 concurrent sessions at 200k context each. - Throughout the process, I’ve learned a ton about LLM inference, Linux, and AI hardware. That being said, this is still the beginning and I'm starting to hit some limitations. My overall mission is to replace or assist my cloud subscriptions with the cheapest hardware possible. If anyone has experience doing something similar, I'd love to learn more. Thanks @matt_feroz at getting me on Twitter.

I've been looking into public benchmarks and am astounded by the poor quality vs. how much we rely on them as an industry. @xeophon pointed me to this excellent site which is an independent third party which scrutinizes benchmarks and "looks at the data" 💪 For example, here are some popular benchmarks we all rely on. Before trusting a benchmark its worth running them and inspecting examples the grader failed carefully, to get a sense of the quality. I'm continuing to look more into this, and will write something up eventually once I've collected more data, but I think this is an important subject.

I took apart Claude Code, OpenCode, and Pi, then turned what I learned into a free course on harness engineering The fastest way to learn harness engineering? Build Claude Code from scratch. Why harness engineering? LangChain kept the same model and changed only the harness. Their coding agent went from 53% to 67% on Terminal-Bench, a top coding-agent benchmark. From outside the top 30 to the top 5. Same model. Better harness. Over the past few months, I've taken apart Claude Code, OpenCode, and Pi to see how they work under the hood. And you don't need to ship a coding agent for this to pay off. The same foundations power any agent, in e-commerce, finance, or sales. On top of that, people are increasingly realizing they need to build their own harness instead of relying on Claude Code or Codex. Ramp scaling its operations with a custom harness is a perfect example. So I turned what I learned into an in-depth course on harness engineering by building Claude Code from scratch. It's free and open source: 8 articles, 6 videos, and one codebase. It's perfect for people who learn by building. Everything in Python. For full transparency, @modal, Opik (by @Cometml), and Kitaru (by @zenml_io) sponsor the course, which lets me keep it 100% free as an independent AI engineer. Here are the 8 steps that I think are core to building and maintaining any harness: 1. Design the harness 2. Build the bare-bones agent loop 3. Sandbox it, locally and remotely 4. Apply 4 essential context engineering techniques 5. Add subagents as a context engineering technique 6. Deploy it headless to the cloud: CLI, webhooks, cron 7. Build benchmarks to measure it 8. Replay failed runs to debug agents Everything is on GitHub: Curious what others here think. Which of these steps has been the hardest to get right in your own agents?