Neural Network Field Theories: Non-Gaussianity, Actions, and Locality
Both the path integral measure in field theory and ensembles of neural
networks describe distributions over functions. When the central limit theorem
can be applied in the infinite-width (infinite-$N$) limit, the ensemble of
networks corresponds to a free field theory. Although an expansion in $1/N$
corresponds to interactions in the field theory, others, such as in a small
breaking of the statistical independence of network parameters, can also lead
to interacting theories. These other expansions can be advantageous over the
$1/N$-expansion, for example by improved behavior with respect to the universal
approximation theorem. Given the connected correlators of a field theory, one
can systematically reconstruct the action order-by-order in the expansion
parameter, using a new Feynman diagram prescription whose vertices are the
connected correlators. This method is motivated by the Edgeworth expansion and
allows one to derive actions for neural network field theories. Conversely, the
correspondence allows one to engineer architectures realizing a given field
theory by representing action deformations as deformations of neural network
parameter densities. As an example, $\phi^4$ theory is realized as an
infinite-$N$ neural network field theory.
Mehmet Demirtas, James Halverson, Anindita Maiti, Matthew D. Schwartz, Keegan Stoner
# Neural Network Field Theories:

###### Abstract

Both the path integral measure in field theory and ensembles of neural networks describe distributions over functions. When the central limit theorem can be applied in the infinite-width (infinite-\(N\)) limit, the ensemble of networks corresponds to a free field theory. Although an expansion in \(1/N\) corresponds to interactions in the field theory, others, such as in a small breaking of the statistical independence of network parameters, can also lead to interacting theories. These other expansions can be advantageous over the \(1/N\)-expansion, for example by improved behavior with respect to the universal approximation theorem. Given the connected correlators of a field theory, one can systematically reconstruct the action order-by-order in the expansion parameter, using a new Feynman diagram prescription whose vertices are the connected correlators. This method is motivated by the Edgeworth expansion and allows one to derive actions for neural network field theories. Conversely, the correspondence allows one to engineer architectures realizing a given field theory by representing action deformations as deformations of neural network parameter densities. As an example, \(\phi^{4}\) theory is realized as an infinite-\(N\) neural network field theory.