Deliberation Boosts LLM Forecasting Accuracy

This paper investigates a practical method to improve the accuracy of LLM-based forecasting by implementing a deliberation process, similar to how human forecasters improve. The study's focus on real-world forecasting questions and the comparison across different LLM configurations (diverse vs. homogeneous, shared vs. distributed information) provides valuable insights into the effectiveness of deliberation. The finding that deliberation improves accuracy in diverse model groups with shared information is significant and suggests a potential strategy for enhancing LLM performance in practical applications. The negative findings regarding contextual information are also important, as they highlight limitations in current LLM capabilities and suggest areas for future research.

• •Deliberation among diverse LLMs with shared information improves forecasting accuracy.
• •Homogeneous groups of LLMs did not benefit from deliberation.
• •Providing additional contextual information did not improve forecast accuracy.