Triangulation for Robust Mechanistic Interpretability in Multilingual LLMs

This paper addresses the challenge of understanding the inner workings of multilingual language models (LLMs). It proposes a novel method called 'triangulation' to validate mechanistic explanations. The core idea is to ensure that explanations are not just specific to a single language or environment but hold true across different variations while preserving meaning. This is crucial because LLMs can behave unpredictably across languages. The paper's significance lies in providing a more rigorous and falsifiable standard for mechanistic interpretability, moving beyond single-environment tests and addressing the issue of spurious circuits.