New Benchmark Measures LLM Instruction Following Under Data Compression

This ArXiv paper introduces a novel benchmark that differentiates between compliance with constraints and semantic accuracy in instruction following for Large Language Models (LLMs). This is a crucial step towards understanding how LLMs perform when data is compressed, mirroring real-world scenarios where bandwidth is limited.

• •The research provides a new benchmark for evaluating LLMs.
• •The benchmark focuses on scenarios involving data compression.
• •It aims to separate constraint compliance from semantic accuracy.