Generative Classifiers Outperform Discriminative Ones on Distribution Shift
Analysis
Key Takeaways
- •Discriminative classifiers often fail under distribution shift due to reliance on spurious correlations.
- •Generative classifiers, using class-conditional generative models, are proposed as a more robust alternative.
- •Diffusion-based and autoregressive generative classifiers achieve state-of-the-art performance on distribution shift benchmarks.
- •Generative classifiers reduce the impact of spurious correlations in realistic applications.
- •The paper provides analysis of generative classifier inductive biases and data properties for optimal performance.
“Generative classifiers...can avoid this issue by modeling all features, both core and spurious, instead of mainly spurious ones.”