Looped Transformers throw away their "work in progress" — the intermediate states from earlier loops. This method puts that discarded signal to use at decode time, almost for free. Title: Decoding Looped Transformers Better for (Almost) Free URL: 📝 Overview LoopCD is a training-free contrastive decoding method for looped Transformers, which gain parameter efficiency by running a shared block through repeated recurrent cycles. No auxiliary model, no retraining required. ❗ Problem it solves Standard decoding discards the intermediate hidden states produced across loop iterations and uses only the final prediction, wasting the naturally aligned "weak-to-strong" prediction pairs that emerge as the loops refine. 🛠️ Method Two variants: LoopCD-Logits contrasts predictions after the output layers (one extra forward pass), while LoopCD-Hidden contrasts hidden states before the coda layers (essentially zero extra cost). Both extrapolate from the first loop toward the final one using a guidance strength ω, with an adaptive version that pushes harder where the model is less confident. 📊 Results Ouro-2.6B-Thinking's AIME 2024 pass@1 jumped from 61.88% to 73.33%, and Huginn's HumanEval climbed from 22.56% to 31.71%. Running at half the loop count with LoopCD still matches or beats the full-depth baseline, cutting forward FLOPs by 22.5–48.2%. 🏢 Use cases Drops directly into existing looped Transformers like Ouro, Huginn, and Parcae, boosting reasoning performance while cutting inference cost with no retraining. #Transformers# #Decoding#