Qualcomm AI Research; Qualcomm Technologies Inc.
Mingu Lee, Saurabh Pitre, Tianyu Jiang, Pierre-David Letourneau, Matthew Morse, Kanghwan Jang, Joseph Soriaga, Parham Noorzad, Hsin-Pai Cheng, Christopher Lott
transformer;efficient transformer;efficient attention
Composite Slice Transformer: An Efficient Transformer with Composition of Multi-Scale Multi-Range Attentions