OrchANN: I/O Orchestration for Fast Out-of-Core Vector Search
Analysis
This paper addresses the performance bottleneck of approximate nearest neighbor search (ANNS) at scale, specifically when data resides on SSDs (out-of-core). It identifies the challenges posed by skewed semantic embeddings, where existing systems struggle. The proposed solution, OrchANN, introduces an I/O orchestration framework to improve performance by optimizing the entire I/O pipeline, from routing to verification. The paper's significance lies in its potential to significantly improve the efficiency and speed of large-scale vector search, which is crucial for applications like recommendation systems and semantic search.
Key Takeaways
- •OrchANN is an out-of-core ANNS engine designed for skewed semantic embeddings.
- •It uses an I/O orchestration model for unified I/O governance.
- •Key features include heterogeneous local index selection, query-aware navigation graph, and multi-level pruning.
- •OrchANN outperforms existing systems in QPS, latency, and SSD access reduction.
- •Significant performance gains are achieved without sacrificing accuracy.
“OrchANN outperforms four baselines including DiskANN, Starling, SPANN, and PipeANN in both QPS and latency while reducing SSD accesses. Furthermore, OrchANN delivers up to 17.2x higher QPS and 25.0x lower latency than competing systems without sacrificing accuracy.”