Very interesting paper. It is framed as: model intelligence is inversely proportional to # of human interventions However, I think that having a human might give a higher quality result, even if you could do it with less interventions. I used Solveit to implement & score myself on one of the tasks (very easy & reproducible paper setup, kudos for that! It was a relatively simple data anonymization task, my score was perfect (same as the AI). The resulting code was much simpler, almost half the LoC and methods. The median score of GPT 5.5 is 0.763. I would love to see more people's results! If you are interested in trying out a task, you can just use my dialog which is fully setup and just choose a different task! Share the results if you do