Title: Understanding ReLU Network Robustness Through Test Set Certification Performance. Abstract: Neural networks might exhibit weak robustness against input perturbations within the learning distribution and become more severe for distributional shifts or data outside the distribution.
For their safer use, robustness certificates provide formal guarantees to the stability of the prediction in the vicinity of the input.
However, the relationship between correctness and robustness remains unclear.
In this work, we investigate the unexpected outcomes of verification methods applied to piecewise linear classifiers for clean, perturbed, in- and out-of-distribution samples.
In our experiments, we conduct a thorough analysis for image classification tasks and show that robustness certificates are strongly correlated with prediction correctness for in-distribution data.
In addition, we provide a theoretical demonstration that formal verification methods robustly certify samples sufficiently far from the training distribution.
These results are integrated with an experimental analysis and demonstrate their weakness compared to standard out-of-distribution detection methods.