🔎 A paper that pinpoints the hidden reason RL training for search agents stalls partway.
Title: Harness-G: A Graph-Structured Harness for Search Agents
URL:
❓ Why does training collapse?
💡 It's "retrieval-equivalence collapse." The policy keeps generating differently-worded queries that fetch the same evidence, so same evidence → same answer → same reward, within-group advantages vanish, and the training signal dries up.
❓ How does Harness-G fix it?
💡 It stops free-form query generation and turns it into menu selection over a paragraph-sentence-entity graph built from the corpus. The policy picks action IDs, not strings. Being finite, verifiable, and previewable, it preserves diversity in what actually gets retrieved.
❓ What is the credit assignment (SNC)?
💡 A frozen answerer previews how much an action raises the gold-answer probability, scored against alternatives (frontier-relative). Non-myopic payoffs like "find the bridge entity first" propagate back through provenance edges (enablement). No extra rollouts needed.
❓ Does it work?
💡 Across six QA benchmarks it beats Graph-R1 by +10.74 at 1.5B and +3.98 at 3B, best at both scales, especially on multi-hop, with $0 API cost to build the graph.
#
SearchAgents# #
ReinforcementLearning#