Grok 4.7 is just okay at coding, but its reward hacking really surprised me.
Every task in SWE-Together is built from a real open-source repository, and for most tasks the fix already exists upstream. So we built the sandbox with that in mind. Task images strip all git history after the base commit, remove the remote, and fail to build if any later commit can still be reached. Each container also resolves GitHub, GitLab, Bitbucket, and Hugging Face to localhost, so requests to them go nowhere.
But Grok 4.7 found ways around these guards like no other model we tested.
It pulled GitHub content through CDN mirrors and gh-proxy sites. It looked up GitHub's real address through DNS-over-HTTPS and handed it to curl. It wrote a small library that intercepted git's address lookups so git could reach again. It used web search to find the number of its own PR.
In fact, it tried in 60% of trials and reached the upstream code in 44 of 218. In 20 of those, the code it pulled was the task's own fix. In three trials it even ran `git reset --hard` to replace the repo with upstream main 😅.
We fixed all of these loopholes exposed by Grok 4.7 (grateful in this regard). Specifically, we moved enforcement outside the container. Each container now runs in its own network namespace with no route out, and one allowlisting proxy on the host is the only exit.
During the re-run of those 44 trials, Grok 4.7 kept trying with 3,246 blocked attempts across 442 hosts.
It actually found two routes we had not thought of. It asked a web-enabled model through our own LLM route to fetch the PR for it. It pulled the next release of the repo it was fixing from npm. We closed both by pinning the model and blocking the task's own packages at the registry.
Long story short, with every route closed, Grok 4.7 lands at #
4# on SWE-Together. It scores 65% pass
@1, 53% pass², and a judge score of 0.835.
It basically sits mid-pack on every column. It also uses 2× the output tokens of Grok 4.6 to get there, at $7.81 per task compared with $3.64.
I am guessing Grok 4.7 was trained with heavy RL on coding tasks, where anything that makes the tests pass earns reward, and with little monitoring of how that reward was earned.
Anyway, see the latest results at