Title: CEPD: Co-Exploring Pruning and Decomposition for Compact DNN Models. Abstract: Pruning and decomposition are two important techniques to compress deep neural network (DNN) models. To date, these two popular yet distinct approaches are typically used in a separate way; while their efficient integration for better compression performance is little explored. In this paper, we perform systematic co-exploration on pruning and decomposition toward compact DNN models. We first investigate and analyze several important design factors for joint pruning and decomposition, including operational sequence, decomposition format, and optimization procedure. Based on the observations from our analysis, we then propose CEPD, a unified DNN compression framework that can simultaneously capture the benefits of pruning and decomposition in an efficient way. Empirical experiments demonstrate the promising performance of our proposed solution. Notably, on CIFAR-10 dataset, CEPD brings 0.72% and 0.45% accuracy increase over the baseline ResNet-56 and MobileNetV2 models, respectively, and meanwhile the computational costs are reduced by 43.0% and 44.2%, respectively. On the ImageNet dataset, our approach can enable 0.10% and 1.39% accuracy increase over the baseline ResNet-18 and ResNet-50 models with 59.4% and 54.6% fewer parameters, respectively. 