of equilibrium statistical mechanics is the Boltzmann distribution, which describes the probability of finding a system in any one of its possible microscopic states. As we consider systems with larger and larger numbers of degrees of freedom, this probabilistic description converges onto a deterministic, thermodynamic description. In the emergence of thermodynamics from statistical mechanics, many microscopic details are lost, and many systems that differ in their microscopic constituents nonetheless exhibit quantitatively similar thermodynamic behavior. Perhaps the oldest example of this idea is the “law of corresponding states” [@guggenheim_45].

The power of statistical mechanics to describe collective, emergent phenomena in the inanimate world led many people to hope that it might also provide a natural language for describing networks of neurons [@hopfield_82; @amit+al_87; @amit_89; @hertz+al_91]. But if one takes the language of statistical mechanics seriously, then as we consider networks with larger and larger numbers of neurons, we should see the emergence of something like thermodynamics. If we can find this “thermodynamic limit” for real neurons, we can hope to find simpler universal behaviors for the network as a whole, independent of many microscopic details.

Theory {#looking_for}
======

At first sight, the notion of a thermodynamics for neural networks seems hopeless. 