University of Oxford; University of Toronto and Vector Institute; ETH Zurich
Lorenzo Noci, Chuning Li, Mufan Li, Bobby He, Thomas Hofmann, Chris Maddison, Dan Roy
Deep Learning Theory;Covariance SDE;Attention Mechanism;Infinite-Depth-and-Width;Scaling Limit
The Shaped Transformer: Attention Models in the Infinite Depth-and-Width Limit