Research Paper#Diffusion Models, Reinforcement Learning, AI Alignment🔬 ResearchAnalyzed: Jan 3, 2026 16:47
Mitigating Preference Mode Collapse in Diffusion Models
Published:Dec 30, 2025 11:17
•1 min read
•ArXiv
Analysis
This paper addresses a critical issue in aligning text-to-image diffusion models with human preferences: Preference Mode Collapse (PMC). PMC leads to a loss of generative diversity, resulting in models producing narrow, repetitive outputs despite high reward scores. The authors introduce a new benchmark, DivGenBench, to quantify PMC and propose a novel method, Directional Decoupling Alignment (D^2-Align), to mitigate it. This work is significant because it tackles a practical problem that limits the usefulness of these models and offers a promising solution.
Key Takeaways
- •Identifies and quantifies Preference Mode Collapse (PMC) in text-to-image diffusion models.
- •Introduces DivGenBench, a new benchmark for measuring PMC.
- •Proposes Directional Decoupling Alignment (D^2-Align) to mitigate PMC.
- •D^2-Align improves alignment with human preference while maintaining diversity.
Reference
“D^2-Align achieves superior alignment with human preference.”