🔥 Awesome Controllable Generative Models
A curated and continuously updated collection of recent (2023–2025) research papers on controllable generative models, with a special focus on both UNet-based diffusion models and Transformer-based diffusion architectures.
This list emphasizes core advances in:
- 🧭 Control mechanisms – including condition injection, adapters, multi-modal control
- 👁️ Attention interpretation – revealing what diffusion models focus on
- 🎛️ Frequency-based control – using spectral domain knowledge to guide generation
- 🔁 Alignment & knowledge transfer – enabling more coherent, faithful, and data-efficient synthesis
- 🧑🎨 Image-to-image (I2I) editing – flexible, structure-preserving transformation across domains
💡 Our goal is not only to track the state-of-the-art in controllable generation, but also to offer a well-organized knowledge map for newcomers and researchers building on top of diffusion models.
🧭 Control Mechanism
| Paper | Venue | Links |
|---|
| OminiControl: Minimal and Universal Control for Diffusion Transformer | ICCV 2025 (Highlight) | Paper | Code |
| Rectified Diffusion Guidance for Conditional Generation | CVPR 2025 | Paper | Code |
| FlexControl: Computation-Aware Conditional Control with Differentiable Router | ICML 2025 (Poster) | Paper | Code |
| CTRL-Adapter: An Efficient and Versatile Framework for Adapting Diverse Controls to Any Diffusion Model | ICLR 2025 (Oral) | Paper | Code |
| Ctrl-U: Robust Conditional Image Generation via Uncertainty-aware Reward Modeling | ICLR 2025 | Paper | Code |
| ConceptCtrl: Concept Control of Zero-shot Personalized Image Generation | arXiv 2025 | Paper | Code |
| Is Noise Conditioning Necessary for Denoising Generative Models? | arXiv 2025 | Paper | Code |
| Ctrl‑X: Controlling Structure and Appearance Without Guidance | NeurIPS 2024 | Paper | Code |
| ControlNet++: Improving Conditional Controls with Efficient Consistency Feedback |
🧠 These papers push the boundary of how we guide generation, whether through minimal prompts, learned adapters, or uncertainty-aware mechanisms.
👁️ Attention & Interpretability
| Paper | Venue | Links |
|---|
| ConceptAttention: Diffusion Transformers Learn Highly Interpretable Features | ICML 2025 (Oral) | Paper | Code |
| ToMA: Token Merge with Attention for Diffusion Models | ICML 2025 (Poster) | Paper | Code |
| Attention in Diffusion Model: A Survey | arXiv 2025 | Paper | Code |
| Attention Distillation: A Unified Approach to Visual Characteristics Transfer | CVPR 2025 | Paper | Code |
| What the DAAM: Interpreting Stable Diffusion Using Cross Attention | ACL 2024 (Oral) | Paper | Code |
🔬 Interpretability is not just analysis — it's a step toward transparent and editable generative pipelines.
🎛️ Frequency Domain Control
| Paper | Venue | Links |
|---|
| DiffFNO: Diffusion Fourier Neural Operator for Arbitrary-Scale Super-Resolution | CVPR 2025 (Oral) | Paper | Code |
| PTDiffusion: Free Lunch for Generating Optical Illusion Hidden Pictures with Phase-Transferred Diffusion | CVPR 2025 (Poster) | Paper | Code |
| Diffusion-based Adversarial Purification from the Perspective of the Frequency Domain | ICML 2025 (Spotlight Poster) | Paper | Code |
| Frequency Autoregressive Image Generation with Continuous Tokens | arXiv 2025 | Paper | Code |
| FreeDiff: Progressive Frequency Truncation for Image Editing with Diffusion Models | ECCV 2024 (Poster) | Paper | Code |
| FreeU: Free Lunch in Diffusion U-Net | CVPR 2024 | Paper | Code |
| Frequency-Controlled Diffusion Model for Versatile Text-Guided Image-to-Image Translation | AAAI 2024 | Paper | Code |
| ResDiff: Combining CNN and Diffusion Model for Image Super-Resolution | AAAI 2023 | Paper | Code |
📡 Spectral and signal-level control provides low-level but powerful levers for generative consistency, resolution, and robustness.
🔁 Alignment & Knowledge Transfer
| Paper | Venue | Links |
|---|
| When Model Knowledge meets Diffusion: Data-free Synthesis with Domain-Class Alignment | ICML 2025 | Paper | Code |
🧬 These works align discrete symbolic knowledge with continuous generative priors, aiming for controllability in low-data or zero-shot regimes.
You may also consider including some notable image-to-image (I2I） editing methods in your collection.
Image Editing
| Paper | Venue | Links |
|---|
| In-Context Edit: Enabling Instructional Image Editing with In-Context Generation in Large Scale Diffusion Transformer | NeurIPS 2025 | Paper | Code |
| AnyEdit: Mastering Unified High-Quality Image Editing for Any Idea | CVPR 2025 (Oral) | Paper | Code |
| Stable Flow: Vital Layers for Training-Free Image Editing | CVPR 2025 | Paper | Code |
| UniReal: Universal Image Generation and Editing via Learning Real-world Dynamics | CVPR 2025 | Paper | Code |
| Taming Rectified Flow for Inversion and Editing | ICML 2025 | Paper | Code |
| Semantic Image Inversion and Editing using Rectified Stochastic Differential Equations | ICLR 2025 | Paper | Code |
| Diffusion Model-Based Image Editing: A Survey | TPAMI 2025 | Paper | Code |
| ChronoEdit: Towards Temporal Reasoning for Image Editing and World Simulation | arXiv 2025 | Paper | |