GLUE: Gradient-free Expert Unification
Published:Dec 27, 2025 04:59
•1 min read
•ArXiv
Analysis
This paper addresses the challenge of combining multiple pre-trained specialist models for new target domains. It proposes a novel method, GLUE, that avoids the computational cost of full backpropagation by using a gradient-free optimization technique (SPSA) to learn the mixture coefficients of expert models. This is significant because it allows for efficient adaptation to new domains without requiring extensive training. The results demonstrate improved accuracy compared to baseline methods, highlighting the practical value of the approach.
Key Takeaways
- •GLUE provides a gradient-free method for unifying expert models.
- •It uses SPSA for efficient learning of mixture coefficients.
- •GLUE outperforms baseline methods in terms of test accuracy.
- •It offers a computationally efficient alternative to full backpropagation.
Reference
“GLUE improves test accuracy by up to 8.5% over data-size weighting and by up to 9.1% over proxy-metric selection.”