Irrelevant facts about cats added to math problems increase LLM errors by 300%

The article highlights a significant vulnerability in Large Language Models (LLMs). Adding irrelevant information, specifically about cats, drastically increases error rates in math problems. This suggests that LLMs may struggle to filter out noise and focus on relevant information, impacting their ability to perform complex tasks. The 300% increase in errors is a substantial finding, indicating a critical area for improvement in LLM design and training.

• •LLMs are susceptible to irrelevant information.
• •Adding irrelevant details significantly degrades LLM performance in math.
• •The study highlights a need for improved noise filtering in LLMs.