Looped Transformers throw away their "work in progress" — the intermediate states from earlier loops. This method puts that discarded signal to use at decode time, almost for free. Title: Decoding Looped Transformers Better for (Almost) Free URL: 📝 Overview LoopCD is a training-free contrastive decoding method for looped Transformers, which gain parameter efficiency by running a shared block through repeated recurrent cycles. No auxiliary model, no retraining required. ❗ Problem it solves Standard decoding discards the intermediate hidden states produced across loop iterations and uses only the final prediction, wasting the naturally aligned "weak-to-strong" prediction pairs that emerge as the loops refine. 🛠️ Method Two variants: LoopCD-Logits contrasts predictions after the output layers (one extra forward pass), while LoopCD-Hidden contrasts hidden states before the coda layers (essentially zero extra cost). Both extrapolate from the first loop toward the final one using a guidance strength ω, with an adaptive version that pushes harder where the model is less confident. 📊 Results Ouro-2.6B-Thinking's AIME 2024 pass@1 jumped from 61.88% to 73.33%, and Huginn's HumanEval climbed from 22.56% to 31.71%. Running at half the loop count with LoopCD still matches or beats the full-depth baseline, cutting forward FLOPs by 22.5–48.2%. 🏢 Use cases Drops directly into existing looped Transformers like Ouro, Huginn, and Parcae, boosting reasoning performance while cutting inference cost with no retraining. #Transformers# #Decoding#

"Looped Diffusion Transformer" This new paper reuses the same Transformer blocks multiple times within each denoising step, repeatedly updating the hidden state to improve the prediction without increasing the model’s parameter count at inference-time. The found that a 260M model beats models up to 6.5x larger, and extra loops improve performance more than spending the same compute on extra denoising steps.

Loops for Robinhood Agents are here. Set your rules, and your agent runs them on your schedule, checking in before it goes live and logging decisions to your Activity Feed. Coming soon to Robinhood Agents.

Looped transformers are a popular architecture topic right now. This new technical report extends the loop across tokens. Recurrent Looped Transformer (RLT) makes the decoder recurrent over every token, prompt and response included. A causal encoder builds global KV memory. For each new token, the decoder combines the token's encoder representation with its own final hidden state from the previous token and a sliding-window cache of recent activations. With a 48-layer decoder, the computation path after t tokens runs through 48t decoder blocks, while each token still executes a fixed number of blocks. Depth grows with the sequence and per-token cost stays the same. The same state transition is used for pretraining, SFT, sampling and RL replay, and nothing resets at the prompt-response boundary. RL replay rebuilds states under the current weights instead of reusing stale rollout states. The report is a design proposal. The author states that reasoning gains, hardware speedups and RL scaling are goals that have not been measured yet. Paper: Chat with Paper:

Loop is now so good at detecting patterns that we're using it to optimize Brainstore. We pipe all of our query plans into Braintrust and found a bunch of low-level optimizations, eg a spot in our regex code where we were over-fetching duplicate ranges from object storage.

Loop engineering has emerged as a new skill for AI engineers But there is very little research measuring how effective it is. The best results on full tasks in a new benchmark is ~25%. LoopArena from AMAP evaluates the outer loop rather than the coding agent. A Controller model receives a structured summary after each round and instructs a separate fixed Worker agent on what to do or verify next, or decides to stop. Holding the Worker constant makes the result readable, since an end-to-end run cannot tell you whether success came from the guidance or from the agent carrying it out. The named failure modes will be familiar to anyone running long agent sessions: - Trusting a stale progress note - Skipping needed verification - Spending budget in the wrong direction - Stopping before the task is safe to submit Paper: Chat with Paper: