GLUE: Gradient-free Expert Unification

This paper addresses the challenge of combining multiple pre-trained specialist models for new target domains. It proposes a novel method, GLUE, that avoids the computational cost of full backpropagation by using a gradient-free optimization technique (SPSA) to learn the mixture coefficients of expert models. This is significant because it allows for efficient adaptation to new domains without requiring extensive training. The results demonstrate improved accuracy compared to baseline methods, highlighting the practical value of the approach.

• •GLUE provides a gradient-free method for unifying expert models.
• •It uses SPSA for efficient learning of mixture coefficients.
• •GLUE outperforms baseline methods in terms of test accuracy.
• •It offers a computationally efficient alternative to full backpropagation.