Ask for "a lizard wearing sunglasses" and you get either no sunglasses or a distorted lizard. Google just gave the control problem behind image generation a mathematical answer. Title: How Diffusion Controller unifies and simplifies AI image generation URL: 📝 Overview Diffusion Controller reframes image generation as a continuous control problem rather than a sequence of isolated steps. It keeps the base model frozen and adds a lightweight side network to steer its behavior. ❓ Problem Solved Inference-time guidance and heavy fine-tuning methods like LoRA sit in two disconnected camps with no unified mathematical principle, leaving engineers to guess their way through balancing preference alignment against image quality. 💡 Method & Approach The side network observes intermediate denoising states and injects tiny steering corrections without touching the model's internal weights. Two implementations — a policy-gradient (PPO) approach and a reward-weighted loss approach — both include guardrails against quality degradation. This design even lets you control closed-source models you can't otherwise access. 📊 Results On Stable Diffusion v1.4, Diffusion Controller outperformed LoRA across both SFT and RWL training tracks despite touching fewer layers, hit a 90% win rate over baseline in its fully unlocked version, and delivered the best subjective quality in human evaluation on complex prompts. #ImageGeneration# #DiffusionModels#