FoldAct: Stable Context Folding for Long-Horizon RL
Research Paper#Reinforcement Learning, Large Language Models, Context Folding🔬 Research|Analyzed: Jan 3, 2026 19:41•
Published: Dec 28, 2025 00:24
•1 min read
•ArXivAnalysis
This paper addresses the scalability challenges of long-horizon reinforcement learning (RL) for large language models, specifically focusing on context folding methods. It identifies and tackles the issues arising from treating summary actions as standard actions, which leads to non-stationary observation distributions and training instability. The proposed FoldAct framework offers innovations to mitigate these problems, improving training efficiency and stability.
Key Takeaways
- •Addresses the non-stationary observation problem in context folding for long-horizon RL.
- •Introduces FoldAct framework with innovations to improve training stability and efficiency.
- •Achieves a 5.19x speedup in training.
- •Focuses on improving the training of long-horizon search agents.
Reference / Citation
View Original"FoldAct explicitly addresses challenges through three key innovations: separated loss computation, full context consistency loss, and selective segment training."