. Strengths first:
The paper is one of the first to give an empirical study of quantization of MoE networks. It would be a good manual/starting point for practitioners in the field. Weaknessess:
Thoroughness: Despite having good results and having investigated several quantization options, one would still have questions of "what if?" style. There are many additional experiments and empirical evaluations that are needed to make it a stronger contribution, and to be certain of presented recommendations. For instance here are additional questions: 1) if inference happens in fp16, why to stick with uniform or log-uniform quantization schemes? how about non-inform quantization akin k-means? 2) why not to consider finer grouping for quantization instead of per-tensor and per-channel? 3) why PTQ calibration techniques are not discussed? are all calibrations work the same? 4) what is the tradeoff between # experts vs bit-width of compression? are there certain recommendation? and many other questions of this format
The paper would benefit from another proof-reading pass: there are many places where it is hard to understand what was exactly meant.
3) why PTQ calibration techniques are not discussed? are all calibrations work the same?