Adaptive Weighting Improves Transfer Consistency in Adversarial Distillation

This research paper explores a novel method for improving the performance of knowledge distillation, particularly in adversarial settings. The core contribution lies in the sample-wise adaptive weighting strategy, which likely enhances the transfer of knowledge from a teacher model to a student model.