๐ A paper that pinpoints the hidden reason RL training for search agents stalls partway.
Title: Harness-G: A Graph-Structured Harness for Search Agents
URL:
โ Why does training collapse?
๐ก It's "retrieval-equivalence collapse." The policy keeps generating differently-worded queries that fetch the same evidence, so same evidence โ same answer โ same reward, within-group advantages vanish, and the training signal dries up.
โ How does Harness-G fix it?
๐ก It stops free-form query generation and turns it into menu selection over a paragraph-sentence-entity graph built from the corpus. The policy picks action IDs, not strings. Being finite, verifiable, and previewable, it preserves diversity in what actually gets retrieved.
โ What is the credit assignment (SNC)?
๐ก A frozen answerer previews how much an action raises the gold-answer probability, scored against alternatives (frontier-relative). Non-myopic payoffs like "find the bridge entity first" propagate back through provenance edges (enablement). No extra rollouts needed.
โ Does it work?
๐ก Across six QA benchmarks it beats Graph-R1 by +10.74 at 1.5B and +3.98 at 3B, best at both scales, especially on multi-hop, with $0 API cost to build the graph.
#
SearchAgents# #
ReinforcementLearning#