Unveiling Bias in Vision-Language Models: A Novel Multi-Modal Benchmark

The article proposes a benchmark to evaluate vision-language models beyond simple memorization, focusing on their susceptibility to popularity bias. This is a critical step towards understanding and mitigating biases in increasingly complex AI systems.

• •Focuses on a multi-modal ordinal regression benchmark.
• •Aims to expose popularity bias within vision-language models.
• •Contributes to the understanding of model limitations beyond memorization.