Decomposing Task Vectors for Improved Model Editing
Research Paper#Model Editing, Task Vectors, AI🔬 Research|Analyzed: Jan 3, 2026 16:26•
Published: Dec 27, 2025 07:53
•1 min read
•ArXivAnalysis
This paper addresses a key limitation in using task vectors for model editing: the interference of overlapping concepts. By decomposing task vectors into shared and unique components, the authors enable more precise control over model behavior, leading to improved performance in multi-task merging, style mixing in diffusion models, and toxicity reduction in language models. This is a significant contribution because it provides a more nuanced and effective way to manipulate and combine model behaviors.
Key Takeaways
- •Proposes a decomposition method for task vectors to separate shared and unique knowledge.
- •Improves multi-task merging, style mixing, and toxicity reduction in different model types.
- •Addresses the problem of overlapping concepts in task vector arithmetic.
- •Offers a new framework for understanding and controlling task vector arithmetic.
Reference / Citation
View Original"By identifying invariant subspaces across projections, our approach enables more precise control over concept manipulation without unintended amplification or diminution of other behaviors."