Unveiling 'Intention Collapse': A Novel Approach to Understanding Reasoning in Language Models

This paper introduces a novel concept, 'intention collapse,' and proposes metrics to quantify the information loss during language generation. The initial experiments, while small-scale, offer a promising direction for analyzing the internal reasoning processes of language models, potentially leading to improved model interpretability and performance. However, the limited scope of the experiment and the model-agnostic nature of the metrics require further validation across diverse models and tasks.

• •Introduces the concept of 'intention collapse' in language models.
• •Proposes three model-agnostic intention metrics: Hint, dimeff, and Recov.
• •Preliminary experiments show CoT reduces intention entropy and increases effective dimensionality.