Mitigating Preference Mode Collapse in Diffusion Models

This paper addresses a critical issue in aligning text-to-image diffusion models with human preferences: Preference Mode Collapse (PMC). PMC leads to a loss of generative diversity, resulting in models producing narrow, repetitive outputs despite high reward scores. The authors introduce a new benchmark, DivGenBench, to quantify PMC and propose a novel method, Directional Decoupling Alignment (D^2-Align), to mitigate it. This work is significant because it tackles a practical problem that limits the usefulness of these models and offers a promising solution.

• •Identifies and quantifies Preference Mode Collapse (PMC) in text-to-image diffusion models.
• •Introduces DivGenBench, a new benchmark for measuring PMC.
• •Proposes Directional Decoupling Alignment (D^2-Align) to mitigate PMC.
• •D^2-Align improves alignment with human preference while maintaining diversity.