Generation Enhances Vision-Language Understanding at Scale

This paper investigates the impact of generative tasks on vision-language models, particularly at a large scale. It challenges the common assumption that adding generation always improves understanding, highlighting the importance of semantic-level generation over pixel-level generation. The findings suggest that unified generation-understanding models exhibit superior data scaling and utilization, and that autoregression on input embeddings is an effective method for capturing visual details.