Title: Pareto Rank-Preserving Supernetwork for HW-NAS. Abstract: In neural architecture search (NAS), training every sampled architecture is very time-consuming and should be avoided. 
Weight-sharing is a promising solution to speed up the evaluation process. 
However, a sampled subnetwork is not guaranteed to be estimated precisely unless a complete individual training process is done. 
Additionally, practical deep learning engineering processes require incorporating realistic hardware-performance metrics into the NAS evaluation process, also known as hardware-aware NAS (HW-NAS). 
HW-NAS results a Pareto front, a set of all architectures that optimize conflicting objectives, i.e. task-specific performance and hardware efficiency. 
This paper proposes a supernetwork training methodology that preserves the Pareto ranking between its different subnetworks resulting in more efficient and accurate neural networks for a variety of hardware platforms. The results show a 97% near Pareto front approximation in less than 2 GPU days of search, which provides x2 speed up compared to state-of-the-art methods. We validate our methodology on NAS-Bench-201, DARTS and ImageNet. Our optimal model achieves 77.2% accuracy (+1.7% compared to baseline) with an inference time of 3.68ms on Edge GPU for ImageNet.