Title: Pruning with Output Error Minimization for Producing Efficient Neural Networks. Abstract: DNNs are known to have excessive parameters and thus are computationally expensive, which poses a challenge for implementations in various applications. Structured pruning is a technique of compressing a trained DNN model by removing redundant neurons (or channels). How well a pruned model maintains its accuracy depends on two factors. The first is compression ratio optimization, in other words, how many neurons are reduced in each layer. The other is layer-wise optimization, in other words, which neurons to be preserved in each layer. In this paper, we propose Pruning with Output Error Minimization (POEM), a layer-wise pruning method that conducts pruning and then performs reconstruction to compensate the error caused by pruning. The strength of POEM lies in its reconstruction using the weighted least squares method so as to minimize the output error of the activation function, while the previous methods minimize the error of the value before applying the activation function. The experiments with well-known DNN models and a large scale image recognition dataset show that POEM is better than the previous methods in maintaining the accuracy of those models.