Network of networks.  At the lowest level are the actual "neurons" - implemented in models - which form an interconnected decision-making matrix.  It's more a reflection of logic connections than specifying physical components.  It's not a network of computers or processors.  It is a network of neurons.  These neurons (the net) may indeed be spread out over the network or amongst CPUs or servers or workstations, but that's merely an implementation description, not how they work.