Harvard University; The Swiss AI Lab IDSIA, USI & SUPSI; AI Initiative, KAUST
recurrent neural networks;RNNs;transformers;computational power;automata;counter machines;formal languages;linear transformers;self-reference;self-referential weight matrix
Practical Computational Power of Linear Transformers and Their Recurrent and Self-Referential Extensions