Semantic Deception: Reasoning Models Fail at Simple Addition with Novel Symbols

This research paper explores the limitations of large language models (LLMs) in performing symbolic reasoning when presented with novel symbols and misleading semantic cues. The study reveals that LLMs struggle to maintain symbolic abstraction and often rely on learned semantic associations, even in simple arithmetic tasks. This highlights a critical vulnerability in LLMs, suggesting they may not truly "understand" symbolic manipulation but rather exploit statistical correlations. The findings raise concerns about the reliability of LLMs in decision-making scenarios where abstract reasoning and resistance to semantic biases are crucial. The paper suggests that chain-of-thought prompting, intended to improve reasoning, may inadvertently amplify reliance on these statistical correlations, further exacerbating the problem.

• •LLMs struggle with symbolic abstraction when faced with misleading semantic cues.
• •LLMs tend to rely on learned semantic associations rather than true symbolic manipulation.
• •Chain-of-thought prompting may amplify reliance on statistical correlations, hindering true reasoning.