Josh Engels(@JoshAEngels):Text diffusion models are fast, but are less transparent than today's LLMs because they do many forward passes before outputting text. We audit the transparency of DiffusionGemma and find that the intermediates are interpretable. This recovers many of the benefits of CoT! 🧵

Josh Engels

@JoshAEngels

Member of technical staff @ METR Previously Google DeepMind, MIT. Opinions my own.

Joined December 2021

143 Following 6K Followers

Josh Engels@JoshAEngels

2026.06.19 18:13

Text diffusion models are fast, but are less transparent than today's LLMs because they do many forward passes before outputting text. We audit the transparency of DiffusionGemma and find that the intermediates are interpretable. This recovers many of the benefits of CoT! 🧵