Search:
Match:
3 results
Research Paper#Transformer Architecture, Memory Compression, Long-Context LLMs🔬 ResearchAnalyzed: Jan 3, 2026 16:00

Trellis: Compressing KV Memory in Transformers

Published:Dec 29, 2025 20:32
1 min read
ArXiv

Analysis

This paper addresses the critical issue of quadratic complexity and memory constraints in Transformers, particularly in long-context applications. By introducing Trellis, a novel architecture that dynamically compresses the Key-Value cache, the authors propose a practical solution to improve efficiency and scalability. The use of a two-pass recurrent compression mechanism and online gradient descent with a forget gate is a key innovation. The demonstrated performance gains, especially with increasing sequence length, suggest significant potential for long-context tasks.
Reference

“Trellis replaces the standard KV cache with a fixed-size memory and train a two-pass recurrent compression mechanism to store new keys and values into memory.”

PermalinkArXiv
Paper#RAG, LLM, Information Retrieval🔬 ResearchAnalyzed: Jan 3, 2026 20:02

HiFi-RAG: Improved RAG for Open-Domain QA

Published:Dec 27, 2025 02:37
1 min read
ArXiv

Analysis

This paper presents HiFi-RAG, a novel Retrieval-Augmented Generation (RAG) system that won the MMU-RAGent NeurIPS 2025 competition. The core innovation lies in a hierarchical filtering approach and a two-pass generation strategy leveraging different Gemini 2.5 models for efficiency and performance. The paper highlights significant improvements over baselines, particularly on a custom dataset focusing on post-cutoff knowledge, demonstrating the system's ability to handle recent information.
Reference

“HiFi-RAG outperforms the parametric baseline by 57.4% in ROUGE-L and 14.9% in DeBERTaScore on Test2025.”

PermalinkArXiv
Research#llm📝 BlogAnalyzed: Dec 25, 2025 05:41

Suppressing Chat AI Hallucinations by Decomposing Questions into Four Categories and Tensorizing

Published:Dec 24, 2025 20:30
1 min read
Zenn LLM

Analysis

This article proposes a method to reduce hallucinations in chat AI by enriching the "truth" content of queries. It suggests a two-pass approach: first, decomposing the original question using the four-category distinction (四句分別), and then tensorizing it. The rationale is that this process amplifies the information content of the original single-pass question from a "point" to a "complex multidimensional manifold." The article outlines a simple method of replacing the content of a given 'question' with arbitrary content and then applying the decomposition and tensorization. While the concept is interesting, the article lacks concrete details on how the four-category distinction is applied and how tensorization is performed in practice. The effectiveness of this method would depend on the specific implementation and the nature of the questions being asked.
Reference

“The information content of the original single-pass question was a 'point,' but it is amplified to a 'complex multidimensional manifold.'”

PermalinkZenn LLM