Argus: Token-Aware LLM Inference Optimization

This paper addresses the critical challenge of optimizing LLM inference in dynamic and heterogeneous edge-cloud environments. The core contribution lies in its token-aware approach, which considers the variability in output token lengths and device capabilities. The Length-Aware Semantics (LAS) module and Lyapunov-guided Offloading Optimization (LOO) module, along with the Iterative Offloading Algorithm with Damping and Congestion Control (IODCC), represent a novel and comprehensive solution to improve efficiency and Quality-of-Experience in LLM inference. The focus on dynamic environments and heterogeneous systems is particularly relevant given the increasing deployment of LLMs in real-world applications.