MBZUAI, Tohoku University; MBZUAI, Tohoku University, RIKEN; Tohoku University, RIKEN
Goro Kobayashi, Tatsuki Kuribayashi, Sho Yokoi, Kentaro Inui
Transformer;Attention map;Feed-forward;Contextualization;Interpretation;Analysis;Pre-trained models;Masked language models;Causal language models
Analyzing Feed-Forward Blocks in Transformers through the Lens of Attention Maps