Many discussions about entropy and disorder use examples of decks of cards, pages of books thrown in the air, two gases being mixed in a container, even the state of a nursery at the end of the day compared to the beginning of the day in order to explain the idea of order (and disorder). In all these examples it is pointed out that the disorder of the system increases, or that the system is in an ordered state and finishes in a disordered state after something has happened. Take the case of throwing the pages of the book in the air. You start with the pages numbered in sequence (I didn't want to use the for "order"), you throw them in the air, they land on the floor, you collect them up and notice the pages are not in sequence anymore. ANd the point is "They are not in the sequence I call ordered. Nonetheless, they are in a new sequence." And, it appears to me that the probability to find the pages in this precise new sequence is equal to find them in the original sequence". In that sense, 'order' seems to be something that us humans define and it doesn't appear to be a property of the system. On the other hand, I can see that in the case of two gases mixing, empirically we find more states where the two types of molecules are occupying the entire volume of a container than one type of molecule in the left side and the other type in the right side of the container. Nonetheless, the precise state of each molecule, its position and therefore the entire state of the mixed up system is qualitative the same, isn't it? Isn't it equally difficult to make each molecule occupy that precise position in the mixed up state as in the unmixed state? Does this make sense?

  • 1
    $\begingroup$ "Order" is a rather individual notion. It also depends on what's immediately useful and what's not, so at different times faced with different problems you might call different arrangements ordered or disordered. $\endgroup$ – jjack Feb 25 '18 at 19:27
  • $\begingroup$ The same is when you're faced with frequency distributions. It depends on which distribution is achievable and most useful to you, so you might be tempted to call that one "order". $\endgroup$ – jjack Feb 25 '18 at 19:33
  • $\begingroup$ "Order" is a very subjective notion. You give the example of mixed gases which tend to intermingle, but of course examples elsewhere such as emulsions are notorious for separating into their constituent parts. The human-centricity of certain scientific concepts does not undermine their usefulness - it's simply necessarily to acknowledge that they are not a "hard science". $\endgroup$ – Steve Feb 25 '18 at 23:21

I think a key observation here is that entropy is used when you are trying to describe a system on a macroscopic scale, which means you want to make predictions about macroscopic quantities. Using your example of two kinds of gas in a box (call them red and blue), before we talk about entropy of various states, we should consider what kinds of quantities are meaningful at a macroscopic scale. One set of quantities that are not macroscopically relevant are the exact positions of every molecule in the box. That information is not accessible when the system is "coarse-grained," or viewed macroscopically.

To see what information is accessible, lets assume that your coarse-graining only allows you to divide the box into two cells, a left half and a right half. You are interested in knowing if the gas tends to be "mixed" or "separated." The individual positions of the molecules are not observable, but what is observable is the number of molecules of each type on each side of the box. We could define a quantity that measures the mixedness $M$ of the particles as something like $M=(N_B-N_R)/N_\text{tot}$, where $N_{B/R}$ is the number of blue/red molecules in the cell, and $N_\text{tot}$ is the total number of particles in the cell.

Now we can ask which state is more probable: the one with blue on one side and red on the other (unmixed), or the one with nearly equal numbers of red and blue molecules (mixed). There is only one unmixed state (or I guess two since you could have all the red on the right or all the red on the left), but many many mixed states--remember you don't get to measure positions of individual particles, but just how many of each type are in each cell.

So it boils down to identifying what states are meaningful on the macroscopic scale, and then counting how many different ways there are to produce the macroscopic state. Then you can assign an entropy to the state which is bigger for states that are more probable (i.e. can be made in many different ways).

The example of pages in a book being mixed up is a bit more difficult to see how entropy could enter, but the key point is to identify the macroscopic quantities which characterize the state. For this, we shouldn't ask about the precise position of page 1, page 2, and so on. Instead, a good macroscopic quantity for measuring disorder could be the number of pages which ended up in the correct location. Then we see that there is only one possibility where all the pages are in the correct position, but many more possibilities where none of them are in the correct position, and hence the latter option is a state of higher entropy. You could be more sophisticated about how you define a disordered state (maybe look at how many even numbers are next to another even number and odd next to odd), but the key is to focus on macroscopic properties of the system, and not on the individual position of each page.


The basic assumption of statistical physics is, as you correctly assume, that all states are equally probable (there may be external reasons why some are less probable than others, but in the simplest case this holds). The concept of order and disorder, then, arises only when we "squint" and look at the system at a more abstract, macroscopic, level.

Example: Imagine that you divide the box of gas into two (imaginary) containers. If you have $N$ particles in total, and $k$ of them are to the left (hence $N-k$ to the right), there are $N \choose k$ or $\frac{N!}{k!(N-k)!}$ ways of defining such a state. For even moderately large numbers of particles, this is a huge number. Compare this to the case where you have all particles on one side; there is exactly one state describing this situation.

As for the example with the pages, you are correct in you assessment that the concept of order is not unique, but it is well defined: there are $N!$ ways of ordering $N$ pages, but only one where they are in a predefined order, and the deviations from this order can be assigned different probabilities.

A high entropy state is then a state which does not surprise us when we find it, whereas a state (such as all molecules being in one corner, or the pages being in a steadily growing order, when there are vastly more states where this is not true) is one of low probability, and thus surprising.

  • $\begingroup$ Hi Skymandr, hit Enter too soon. What really confuses me is why we are surprised when we see "blue" molecules on the left side of the box and "red" molecules on the right side, and "we are not surprised when all the blue molecules (and red ones) are in the precise positions they are, in the mixed state. Like the "ordered" state, there is only one "mixed" state where the molecules occupy those precise positions. $\endgroup$ – user14188 Oct 19 '12 at 13:45
  • $\begingroup$ Yes, there is only one such case. The surprise comes first, when we look at the system from a distance, going from looking at individual particles to ensembles of particles. What would be a typical mixture of blue and red, for a particular set of imaginary boxes? If we make the boxes sufficiently small, the entropy does indeed vanish with out statistics. $\endgroup$ – skymandr Oct 19 '12 at 14:12
  • $\begingroup$ (I too pressed too quickly...) This is what is called effect of having to small boxes is in practical situations referred to as the "coarse graining" problem. This article has a nice discussion of it, and of what it means for the entropy/disorder concept: [link}(arxiv.org/abs/quant-ph/0403034). $\endgroup$ – skymandr Oct 19 '12 at 14:18
  • $\begingroup$ Thanks for the paper; I'll read it as soon as I can. Still, I think there is something wrong with my view of entropy and order/disorder. Without getting too philosophical, the confusion may be on "state of the system" or "number of states". I'm not comfortable with nature itself making a distinction between "separate blue and red" and "mixed blue and red". It is obvious that it "matters" to humans, but not to nature. $\endgroup$ – user14188 Oct 19 '12 at 17:46

Your Answer

