Liquid AI(@liquidai):Can coding agents autonomously solve a production-grade problem from scratch? Last year, we tasked two agents to build a BPE tokenizer trainer with time limits. Both zero-shot a working toy trainer in 30 minutes. Neither scaled to production. We then changed the setup with loops. Loops allowed us to iterate against real production data at scale. The correctness was verified with two independent libraries whose results the agent could not manipulate. The result of this experiment is a tokenizer trainer called toktoktok. Read more: https://t.co/5DqakvXh7b Source:

2026.08.18 18:44

Can coding agents autonomously solve a production-grade problem from scratch? Last year, we tasked two agents to build a BPE tokenizer trainer with time limits. Both zero-shot a working toy trainer in 30 minutes. Neither scaled to production. We then changed the setup with loops. Loops allowed us to iterate against real production data at scale. The correctness was verified with two independent libraries whose results the agent could not manipulate. The result of this experiment is a tokenizer trainer called toktoktok. Read more: Source: