PolyInfer: Unified inference API across TensorRT, ONNX Runtime, OpenVINO, IREE

This submission on r/deeplearning discusses PolyInfer, a unified inference API designed to work across multiple popular inference engines like TensorRT, ONNX Runtime, OpenVINO, and IREE. The potential benefit is significant: developers could write inference code once and deploy it on various hardware platforms without significant modifications. This abstraction layer could simplify deployment, reduce vendor lock-in, and accelerate the adoption of optimized inference solutions. The discussion thread likely contains valuable insights into the project's architecture, performance benchmarks, and potential limitations. Further investigation is needed to assess the maturity and usability of PolyInfer.