Philipp Schmid(@_philschmid):Gemini 3.7 Flash just took #1 on @ArtificialAnlys new AA-AnalystAgent. AA-AnalystAgent evaluates against 80 real-world quantitative analysis tasks across 14 business and scientific domains (finance, healthcare, hydrology, government appropriations). The Agent run inside an isolated Python 3.12 sandbox using AA's open-source Stirrup harness. They receive reference spreadsheets (.xlsx) and documents (.docx) alongside standard data libraries (pandas, polars, openpyxl, scipy, PyMuPDF) to inspect schemas, write scripts, handle edge cases, and calculate final figures. Gemini 3.7 Flash: • Accuracy: #1 with 60.0% pass^5 (70.5% pass@1, 77.5% pass@5) • Speed: 1.32s per task (fastest) • Cost: $0.54 avg per task (middle)

2026.08.19 13:10

Gemini 3.7 Flash just took #1# on @ArtificialAnlys new AA-AnalystAgent. AA-AnalystAgent evaluates against 80 real-world quantitative analysis tasks across 14 business and scientific domains (finance, healthcare, hydrology, government appropriations). The Agent run inside an isolated Python 3.12 sandbox using AA's open-source Stirrup harness. They receive reference spreadsheets (.xlsx) and documents (.docx) alongside standard data libraries (pandas, polars, openpyxl, scipy, PyMuPDF) to inspect schemas, write scripts, handle edge cases, and calculate final figures. Gemini 3.7 Flash: • Accuracy: #1# with 60.0% pass^5 (70.5% pass@1, 77.5% pass@5) • Speed: 1.32s per task (fastest) • Cost: $0.54 avg per task (middle)

显示更多