Unlocking the Address Book: Dissecting the Sparse Semantic Structure of LLM Key-Value Caches via Sparse Autoencoders

This article, sourced from ArXiv, focuses on analyzing the internal workings of Large Language Models (LLMs). Specifically, it investigates the structure of key-value caches within LLMs using sparse autoencoders. The title suggests a focus on understanding and potentially improving the efficiency or interpretability of these caches.