Title: An Analysis of Information Bottlenecks. Abstract: Learning representations with information bottlenecks is a powerful information-theoretic approach for learning effective representations where unnecessary information is minimized while task-relevant information is maximized. Many machine learning algorithms have been derived based on information bottlenecks of representations. This study mathematically relates information bottlenecks of intermediate representations to the corresponding expected loss in general settings. We investigate the merit of our new mathematical findings with experiments across a range of architectures and learning settings. Through the theory and experiments, we provide a new foundation for understanding current and future methods for learning intermediate representations with information bottlenecks.