Self-Bootstrapping Framework for Audio-Driven Visual Dubbing

This paper addresses the limitations of existing audio-driven visual dubbing methods, which often rely on inpainting and suffer from visual artifacts and identity drift. The authors propose a novel self-bootstrapping framework that reframes the problem as a video-to-video editing task. This approach leverages a Diffusion Transformer to generate synthetic training data, allowing the model to focus on precise lip modifications. The introduction of a timestep-adaptive multi-phase learning strategy and a new benchmark dataset further enhances the method's performance and evaluation.