Automated harness optimization methods keep improving, but which scenarios you train on is usually fixed from the start. This paper makes that "curriculum" itself adaptive.
Title: ActiveSaddler: Automated Curriculum Learning for Agent Harness Optimization
URL:
📝 Overview
ActiveSaddler reframes which training scenarios feed agent harness optimization as an automated curriculum-learning problem, adapting that selection dynamically as optimization proceeds instead of fixing it upfront.
❗ Problem it solves
Fixed scenario ordering abandons unresolved weaknesses too early or wastes rollouts on already-fixed issues. The value of revisiting known weaknesses versus exploring new ones shifts during optimization, but static schedules can't track that.
🛠️ Method
Curriculum selection is modeled as a non-stationary bandit with dynamically instantiated arms — each arm a failure pattern, not an individual scenario. An LLM-based Arm Prioritizer scores learning potential every iteration across severity, fixability, breadth, and side-effect risk, while an Exploration Controller adaptively decides whether to exploit known weaknesses or explore unseen scenarios based on how many actionable targets remain.
📊 Results
Pass
@1 reached 59.8% on GAIA2 (+4.4 points over fixed random ordering) and 80.0% on Terminal-Bench 2.0 (+7.5 points). It's also far more cost-efficient — on GAIA2 it hits a target accuracy for $298 versus $1,360 for other methods.
#
AgentOptimization# #
CurriculumLearning#