SAP: Pruning Transformer Attention for Efficiency

This research from SAP proposes Syntactic Attention Pruning (SAP) to improve the efficiency of Transformer-based language models. This method focuses on pruning attention heads, which may lead to faster inference and reduced computational costs.

• •SAP is a pruning technique for Transformer models.
• •The method aims to improve efficiency.
• •Research is published on ArXiv.