Multi-Envelope DBF for LLM Quantization

This paper addresses the limitations of Double Binary Factorization (DBF) for extreme low-bit quantization of Large Language Models (LLMs). DBF, while efficient, suffers from performance saturation due to restrictive scaling parameters. The proposed Multi-envelope DBF (MDBF) improves upon DBF by introducing a rank-$l$ envelope, allowing for better magnitude expressiveness while maintaining a binary carrier and deployment-friendly inference. The paper demonstrates improved perplexity and accuracy on LLaMA and Qwen models.

• •Proposes Multi-envelope DBF (MDBF) to improve low-bit quantization of LLMs.
• •MDBF uses a rank-$l$ envelope for better magnitude expressiveness.
• •Maintains a binary carrier and deployment-friendly inference.
• •Demonstrates improved perplexity and zero-shot accuracy on LLaMA and Qwen models.