Gatsby Unit, UCL; Google DeepMind; Gatsby Unit & SWC, UCL
Aaditya Singh, Stephanie Chan, Ted Moskovitz, Erin Grant, Andrew Saxe, Felix Hill
in-context learning;transformers;emergence;transience
The Transient Nature of Emergent In-Context Learning in Transformers