In  this  preliminary  work,  we  study  the  generalization  properties  of  infinite  ensembles  of infinitely-wide neural networks.   Amazingly, this model family admits tractable calculations for many information-theoretic quantities.   We report analytical and empirical investigations in the search for signals that correlate with generalization.
Infinite ensembles of infinitely wide neural networks are an interesting model family from an information theoretic perspective.