Title: Improving Corruption Robustness with Adversarial Feature Alignment Transformers. Abstract: Despite their success, vision transformers still remain vulnerable to image corruptions, such as noise or blur. Indeed, we find that the vulnerability mainly stems from the unstable self-attention mechanism, which is inherently built upon patch-based inputs and often becomes overly sensitive to the corruptions across patches. For example, when we only occlude a small number of patches with random noise (e.g., 10%), these patch-based corruptions would lead to severe accuracy drops and greatly mislead the intermediate features as well as the corresponding attentions over them. To alleviate this issue, we seek to explicitly reduce the sensitivity of attention layers to patch-based corruptions and improve the overall robustness of transformers. In this paper, we propose the Adversarial Feature Alignment Transformer (AFAT) that aligns the features between clean examples and patch-based corruptions. To construct these corrupted examples, we build a patch corruption model to identify and occlude the patches that could severely distract the intermediate attention layers. We highlight that the corruption model is trained adversarially to the following feature alignment process, which is essentially different from existing methods. In experiments, AFAT greatly improves the stability of attention layers and consistently yields better robustness on various benchmarks, including CIFAR-10/100-C, ImageNet-A, ImageNet-C, and ImageNet-P.