Tencent's Tiny AI: A Breakthrough in On-Device LLMs!

Tencent's new HY-1.8B-2Bit model marks a significant leap in on-device deployment of Generative AI, achieving impressive performance in a remarkably small package. By leveraging innovative 2-bit quantization, this model opens doors to more efficient and powerful AI experiences on mobile devices and other consumer hardware.

• •HY-1.8B-2Bit achieves performance close to full-precision models while being incredibly small (600MB).
• •The model significantly boosts speed, up to 3x faster, on edge devices.
• •Utilizes 2-bit quantization and quantization-aware training (QAT) for superior performance.