CLIP-Joint-Detect: Enhancing Object Detection with Vision-Language Supervision

This paper introduces CLIP-Joint-Detect, a novel approach to object detection that leverages contrastive vision-language supervision, inspired by CLIP. The key innovation is integrating CLIP-style contrastive learning directly into the training process of object detectors. This is achieved by projecting region features into the CLIP embedding space and aligning them with learnable text embeddings. The paper demonstrates consistent performance improvements across different detector architectures and datasets, suggesting the effectiveness of this joint training strategy in addressing issues like class imbalance and label noise. The focus on maintaining real-time inference speed is also a significant practical consideration.