Successful training of convolutional neural networks is often associated with suffi-
 ciently deep architectures composed of high amounts of features. These networks
 typically rely on a variety of regularization and pruning techniques to converge
 to less redundant states. We introduce a novel bottom-up approach to expand
 representations in fixed-depth architectures. These architectures start from just a
 single feature per layer and greedily increase width of individual layers to attain
 effective representational capacities needed for a specific task. While network
 growth can rely on a family of metrics, we propose a computationally efficient
 version based on feature time evolution and demonstrate its potency in determin-
 ing feature importance and a networks’ effective capacity. We demonstrate how
 automatically expanded architectures converge to similar topologies that benefit
 from lesser amount of parameters or improved accuracy and exhibit systematic
 correspondence in representational complexity with the specified task. In contrast
 to conventional design patterns with a typical monotonic increase in the amount of
 features with increased depth, we observe that CNNs perform better when there is
 more learnable parameters in intermediate, with falloffs to earlier and later layers.
A bottom-up algorithm that expands CNNs starting with one feature per layer to architectures with sufficient representational capacity. Proposes to dynamically adjust the feature map depth of a fully convolutional neural network, formulating a measure of self-resemblance and boosting performance. Introduces a simple correlation-based metric to measure whether filters in neural networks are being used effectively, as a proxy for effective capacity. Aims to address the deep learning architecture search problem via incremental addition and removal of channels in intermediate layers of the network.