At $1 per coding task ARK’s model puts DeepSWE success at around 70% in July up from around 8% in March
As a developer I don’t buy tokens for their own sake, I want a working change. So if the same dollar gets a successful result much more often then more work becomes worth handing to an agent
I know I'm already seeing my own AI spend increasing as the tools get more capable. What do you think this graph will look like a year from now?