Wroclaw U. of Science and Technology; Moves; Peking University; U. of Science and Technology of China; RuoxinTech; EleutherAI; Ohio State U.; Crisis24; Tsinghua University; National U. of Singapore; New York U.; Storyteller.io; U. of C., Santa Cruz; Purdue U.; U. of Electronic Science and Technology of China; Criteo AI Lab; Yale U.; Databaker Technology; U. of Oslo; U. of British Columbia; Nextremer; Generative AI Commons
large language model;scaling laws;open source;pretraining
RWKV: Reinventing RNNs for the Transformer Era