Greg Pstrucha(@grichadev):i've been trying to run warden security benchmarks against 4.7 since yesterday and it hasn't been great. the last grok model that was absolutely crushing it in vulnerability scanning is 4.5 and we still use it as base driver. both 4.6 and 4.7 disappointed, were spinning for two long, missing or "disproving" some real results and costing way too much. original numbers are here https://t.co/eckofRYfYw (although we didn't add 4.7 yet).

Greg Pstrucha

@grichadev

ai @ sentry. undistinguished engineer. prev robinhood/facebook

Joined April 2010

521 Following 1.3K Followers

Greg Pstrucha@grichadev

2026.09.23 17:44

i've been trying to run warden security benchmarks against 4.7 since yesterday and it hasn't been great. the last grok model that was absolutely crushing it in vulnerability scanning is 4.5 and we still use it as base driver. both 4.6 and 4.7 disappointed, were spinning for two long, missing or "disproving" some real results and costing way too much. original numbers are here (although we didn't add 4.7 yet).