Carnegie Mellon University; Tsinghua University
Kaiyue Wen, Yuchen Li, Bingbin Liu, Andrej Risteski
Transformer;Self Attention;Dyck Language;Context Free Grammar;Formal Language;Theory;Interpretability
Transformers are uninterpretable with myopic methods: a case study with bounded Dyck grammars