Affine Divergence: Rethinking Activation Alignment in Neural Networks

This ArXiv paper explores a novel approach to aligning activation updates, potentially improving model performance. The research focuses on a concept called "Affine Divergence" to move beyond traditional normalization techniques.