Title: Capsa: A Unified Framework for Quantifying Risk in Deep Neural Networks. Abstract: The modern pervasiveness of large-scale deep neural networks (NNs) is driven by their extraordinary performance on complex problems but is also plagued by their sudden, unexpected, and often catastrophic failures, particularly on challenging scenarios. Unfortunately, existing algorithms to achieve risk-awareness of NNs are complex and ad-hoc.  Specifically, these methods require significant engineering changes, are often developed only for particular settings, and are not easily composable. Here we present Capsa, a flexible framework for extending models with risk-awareness. Capsa provides principled methodology for quantifying multiple forms of risk and composes different algorithms together to quantify different risk metrics in parallel. We validate Capsa by implementing state-of-the-art uncertainty estimation algorithms within the Capsa framework and benchmarking them on complex perception datasets. Furthermore, we demonstrate the ability of Capsa to easily compose aleatoric uncertainty, epistemic uncertainty, and bias estimation together in a single function set, and show how this integration provides a comprehensive awareness of NN risk. 
