Seoul National University, Seoul, Republic of Korea; Hanyang University, Seoul, Republic of Korea and KT, Seoul, Republic of Korea; KT, Seoul, Republic of Korea; Hanyang University, Seoul, Republic of Korea
Minsoo Kim, Sihwa Lee, Janghwan Lee, Sukjin Hong, Du-Seong Chang, Wonyong Sung, Jungwook Choi
Generative Language Model;Quantization;QAT;Knowledge Distillation;Causal Attention;Language Modeling
Token-Scaled Logit Distillation for Ternary Weight Generative Language Models