Training data for terminal agents often ships tasks where instruction, environment, solution, and verifier disagree — producing unsolvable tasks. A new synthesis framework cuts that at the root.
Title: FACET: Preserving Source Intent and Executable State in Terminal Task Synthesis
URL:
📌 Overview
Reconstructs related skills into rich scenarios, builds the environment first, and grounds instruction, solution, and verifier in that same executable state.
🧩 Problems it solves
・Multi-stage generation loses source dependencies and intermediate states
・Misaligned instruction/environment/solution/verifier yield unsolvable, unverifiable tasks
⚙️ Method
・Collects 71K+ skills, reconstructs into 5-dimensional scenarios
・Builds the environment in Docker, exposing its real state as a shared channel
・Generates instruction→solution→verifier sequentially; a router pinpoints and repairs only the failing part
📊 Results
・Synthesizes 6,078 validated tasks at 22.77 tests/task on average
・Fine-tuning Qwen3.5: 4B +40.5%, 9B +30.1%, 27B +16.5%
・The 27B (47.57) nears the ~15x larger 397B (49.06)
・70% end-to-end yield vs 15–28% for competitors
A clear case that high-quality executable tasks come from careful state grounding, not brute-force generation.
#
LLMAgents# #
TerminalBench#