Transformers;self-attention;optimization;stability;spectral normalization;self-supervised learning;vision;speech;language;contrastive learning
$\sigma$Reparam: Stable Transformer Training with Spectral Reparametrization