Transformer Optimization Asymmetry Examined: A Stress Test Analysis

This ArXiv paper investigates directional optimization asymmetry in Transformers, a critical area for understanding and improving model training. The synthetic stress test provides valuable insights into how these models behave under specific conditions.