Protons and neutrons, being distinct types of particles, occupy different quantum states. One can think of two different "pools" of states, one for protons and one for neutrons. Now, for example, if there are significantly more neutrons than protons in a nucleus, some of the neutrons will be higher in energy than the available states in the proton pool. If we could move some particles from the neutron pool to the proton pool, in other words change some neutrons into protons, we would significantly decrease the energy. The imbalance between the number of protons and neutrons causes the energy to be higher than it needs to be, "for a given number of nucleons". This is the basis for the asymmetry term.