Search results for DeepSWE
DeepSWE community
One keyword maps to one global community path.
People
Not Found
Tweets including DeepSWE
Grok 4.7 is DeepSWE SoTA at xhigh effort, but take this with a grain of salt.
FrontierCode and DeepSWE scores using API costs as the x-axis instead of the default ouput tokens count
most of the time deepswe bench is judged on the cost, but the agent steps per task is more interesting because it shows how much slop you will get. my day to day experience confirms this too.
Show more
it's not that deepswe just take PTO
Today we’re releasing DeepSWE, a new standard for agentic coding benchmarks.
On public leaderboards, top models often look relatively close in capability. DeepSWE shows where they actually diverge, reflecting the realistic experience of developers in their day-to-day work.
Show more
Meta’s Muse Wonder now leads on DeepSWE.
This is an incredible turnaround only 14 months after the announcement of Meta Superintelligence Labs.
We have a new model on the frontier.
Show more
I have 40+ complete runs of DeepSWE from different Qwen3.8 27B models, Flash Next, and MiMo V2.6.
Next, I plan to use one of these models to aggregate the trajectories to produce better results.
I saw several papers at COLM about this. It seems to work.
I'll do the same with Terminal Bench 4.0. I have completed 15 runs so far.
Show more
Gemini 3.8 Flash on DeepSWE 1.1, scores 73.7%!
GLM-5.3-Flash scores 63% on DeepSWE at just $0.24 per task.
All evaluations use standard API pricing, not discounted rates.
