OpenLLMetry: OpenTelemetry-based observability for LLMs

This article introduces OpenLLMetry, an open-source project built on OpenTelemetry for observing LLM applications. The key selling points are its open protocol, vendor neutrality (allowing integration with various monitoring platforms), and comprehensive instrumentation for LLM-specific components like prompts, token usage, and vector databases. The project aims to address the limitations of existing closed-protocol observability tools in the LLM space. The focus on OpenTelemetry allows for tracing the entire system execution, not just the LLM, and easy integration with existing monitoring infrastructure.