GIE-Bench: A Grounded Evaluation for Text-Guided Image Editing

This article introduces GIE-Bench, a new benchmark developed by Apple ML to improve the evaluation of text-guided image editing models. The current evaluation methods, which rely on image-text similarity metrics like CLIP, are considered imprecise. GIE-Bench aims to provide a more grounded evaluation by focusing on functional correctness. This is achieved through automatically generated multiple-choice questions that assess whether the intended changes were successfully implemented. This approach represents a significant step towards more accurate and reliable evaluation of AI models in image editing.