🧭 Can an AI model really teach itself to improve when it's only handed a vague goal, with no task spec and no reward? A team from ByteDance Seed and collaborators built a benchmark to find out.
Title: Aspire: Can Models Self-Evolve from Vague Goals?
URL:
The benchmark hands the model nothing but a vague goal, lets it handle interpretation, training, and verification end-to-end, and then measures real capability gains against a hidden evaluation set.
🎯 Highlight 1: The cost of ambiguity
Simply rephrasing a task as vague dropped Claude Opus 4.8's score from 32.90% to 27.07% and GPT-5.6's from 36.23% to 29.58%. Agents burn time figuring out what to optimize for, leaving less time for actual training.
🔁 Highlight 2: Execution isn't the same as improvement
In self-evolution runs with Qwen3.5-4B/9B, most final checkpoints underperformed the base model, and 21 of 24 runs were rolled back to their starting state by the safety mechanism. Training against narrow self-evaluations produces local gains that fail to transfer to hidden evaluation.
🛠️ Highlight 3: Harness evolution can't beat the baseline either
Even when agents were allowed to rebuild their own execution harness, the best successor harness scored 27.22 versus 28.64 for the reference implementation — falling short due to narrow verification and gaps in output completeness checks.
💡 This research makes clear that deciding what to improve is the next big wall standing between agents and sustained self-improvement.
#
LLMAgents# #
SelfEvolution#