alphaXiv(@askalphaxiv):"LiFT: Loop Flow Transformers" Running more loops than they were trained on usually hurts generation quality for looped transformers. This paper LiFT fixes this by training each loop to progressively correct the model’s initial velocity estimate toward the flow-matching target, rather than predicting the final target immediately. This lets a model trained with just 2 loops improve with up to 16 loops at inference, outperforming a larger DiT on ImageNet with ~60% fewer parameters and 52% less inference compute.

2026.10.07 05:33

"LiFT: Loop Flow Transformers" Running more loops than they were trained on usually hurts generation quality for looped transformers. This paper LiFT fixes this by training each loop to progressively correct the model’s initial velocity estimate toward the flow-matching target, rather than predicting the final target immediately. This lets a model trained with just 2 loops improve with up to 16 loops at inference, outperforming a larger DiT on ImageNet with ~60% fewer parameters and 52% less inference compute.