TravelBench: A Real-World LLM Benchmark for Travel Planning

This paper introduces TravelBench, a new benchmark for evaluating LLMs in the complex task of travel planning. It addresses limitations in existing benchmarks by focusing on multi-turn interactions, real-world scenarios, and tool use. The controlled environment and deterministic tool outputs are crucial for reproducible evaluation, allowing for a more reliable assessment of LLM agent capabilities in this domain. The benchmark's focus on dynamic user-agent interaction and evolving constraints makes it a valuable contribution to the field.