been vast literature on pedestrian detection with lasting research activities. Due to space limit, in what follows we mainly focus on efforts that are closely related to our approach. As with many other computer vision tasks, hand-crafted features have played a critical role in attaining good performance. The histogram of oriented gradient or HOG descriptor by Dalal & Triggs [@DalTri:cvpr05] is perhaps the most well-known feature engineering technique constructed for pedestrian detection. This is extended to the integral channel features (ICF) by Dollar et al. [@DolEtAl:bmvc09] to efficiently extract features such as local sums, histograms, and Haar features using integral images, which is further improved in several ways by ACF [@DolEtAl:tpami14].

Recently, Convolutional Neural Network (CNN) based methods [@TiaEtAl:cvpr15; @TiaEtAl:iccv15; @HosEtAl:cvpr15; @CaiSabVas:iccv15; @ZhaEtAl:eccv16] have made significant progresses in pedestrian detection, among others. An unsupervised method based on convolutional sparse coding is used in Sermanet et al. [@SerEtAl:cvpr13] to pre-train CNN for pedestrian detection. Tian et al. [@TiaEtAl:cvpr15] jointly optimize pedestrian detection with other semantic tasks including pedestrian and scene attributes. Yang et al. [@YanChoLin:cvpr16] investigate the augmentation of CNN with scale-dependent pooling and layer-wise cascaded rejection classifiers to detect objects efficiently. Cai et al. [@CaiSabVas:iccv15] introduce complexity-aware cascaded detectors by 