What if a model's endless repetition loops could be fixed with nothing but bit-level weight edits?
Tree-Ising: Repairing Language Models with Bit-Level Ising Edits
📌 Overview
A method that repairs Qwen3.5-0.8B's thinking-mode loops with sparse discrete bit edits to the weights. No gradients, no training.
⚠️ Problem
Prior work had an LLM agent read the weights and write a fix, taking about $312 and 15.6 hours.
🧩 Method
・Move only the sign and a 4-bit state in BF16 (shared exponent and mantissa untouched)
・Pick two layers along the loop direction and split into one 512-spin QUBO per output row
・Solve with simulated annealing and similar solvers
📊 Results
・MATH-500
@4: 54.6-55.1% vs. 51.7% for the agent edit
・GSM8K: 70.5-70.8% vs. 65.6%
・About $0.90 and 4 forward passes
・On the 2B model, GSM8K 21.6% to 69.4% (as reported)
Only loop repair on math benchmarks was tested, and the code is promised soon.
#
LLM# #
QuantumOptimization#