Unit 9: Part 5

Contents

• 5. Counting ways
  • An overall view of the argument
  • Counting ways of sharing quanta in an Einstein solid
  • Adding energy leads to more ways of sharing it
  • The factor (1 + N/q)
  • One oscillator competing with many others for energy
  • The biography of an oscillator: what happens often to one happens to many
  • A subtle and difficult argument
  • The flow of heat from hot to cold
  • The general meaning of hot and cold
  • Temperature and changes to number of ways
  • Kelvin temperature and the Boltzmann constant k
  • Measurment of k: the Einstein solid as a means of counting ways
  • The Boltzmann factor
  • Counting ways with heaters and thermometers: entropy

5. Counting ways

An overall view of the argument

This Part is wholly theoretical, devoted to developing further the understanding of the exchange and interchange of energy in a material, upon which a beginning was made in Part Four.

In earlier Parts we saw that what happens by chance is what happens in many ways; so to begin with, we inquire about the number of ways in which quanta of energy can be rearranged amongst the oscillators of the Einstein model described in Part Four. Because what matters in thermal equilibrium is the exchange of energy between two objects, we shall ask by how much the number of ways changes when energy is added or taken away. It turns out that the answer to this question is simpler than the answer to the question about how many ways there are of sharing quanta overall: indeed you have already met it as the factor (1 + N/q). in Part Four.

Then we argue that the exponential shape of the equilibrium distribution revealed in the computer experiments of Part Four can be explained in terms of this factor.

But the shape is related to the Kelvin temperature, and it is then possible to relate the temperature to the change in the number of ways of rearranging quanta when energy is added or removed, thus giving a fundamental relationship between temperature, energy, and counting of ways.

Finally, a new quantity, the entropy, is introduced. It is seen as a device for counting ways or changes in numbers of ways, without actually having to do the counting. It does this by exploiting the relationship between between temperature, energy, and numbers of ways referred to above.

As is appropriate to the more rigorous arguments given here, we shall refer to the sites in the Einstein model as oscillators. One atom, which can oscillate in three perpendicular directions, is equivalent to three such oscillator sites.

Counting ways of sharing quanta in an Einstein solid

When chance is the arbiter, what happens often is what happens in many ways. In Part Four, we have seen that when quanta are allowed to rearrange themselves at random amongst the oscillators of an Einstein solid, they find a steady, equilibrium distribution which has an exponential form, and also that quanta will sometimes go spontaneously from one such solid to another. We have seen that the meaning of hot and cold can be related to this random shuffling of quanta among atoms. To go further, it seems sensible to try to put these ideas together. To begin with, we try to see what counting the number of ways in which the quanta can arrange themselves among oscillators would be like.

Figure 69 shows an Einstein solid with just two oscillators - not a plausible thing in real life, but one that is easy to think about. It is shown with differing numbers of quanta shared amongst these oscillators.

Figure 69: A two-oscillator Einstein solid

Clearly, from figure 69, when the number of oscillators N = 2, the number of ways W of rearranging q quanta is simply q+ 1. The value of W is never so simple if there are more oscillators, but three features remain the same:

1 The number of ways can be counted by systematically rearranging the quanta in all possible arrangements, W being the number of such rearrangements.

2 The larger the number of quanta, the larger the number of rearrangements.

3 There is only one way of sharing no quanta.

Figure 70 illustrates the difference that is made by having three oscillators instead of two. For any given number of quanta, the number of ways W of sharing them is larger, but W still increases for every extra quantum, and its least value is still 1.

It is possible to work out a general result for W in terms of q and N. Doing this is not very hard, but is a distraction from our main purpose. The result is

W= (N+q-1)!/(N-1)!q!

For those who would like to see how the deduction goes, it is given in Appendix B in which the meaning of the symbol ! is given. But what really matters is that you understand what the job of finding W involves: simply counting out all the possible rearrangements of quanta.

Substituting N = 3 and q = 3 in the general result gives W = 10, as it must, as figure 70 shows. If there are many atoms and many quanta, W is very large; for example, for 900 oscillators and 900 quanta, as in several of the computer experiments of Part Four, W is about 10⁵⁴⁰. Obviously, such a large number is not to be discovered by actual counting, but must be deduced by argument.

You should also understand that adding more quanta increases W. This is pretty reasonable: when there are more things to share around, the number of ways of sharing them is likely to be bigger.

Figure 70: A three-oscillator Einstein solid

Adding energy leads to more ways of sharing it

Boiling a kettle means getting energy to go from the hot-plate to the kettle. Firing an explosive charge means getting energy to go violently into warming up the surroundings, at the expense of the chemicals in the charge. Making hydrogen and oxygen from water means getting energy from somewhere to break the hydrogen-oxygen bonds. Making a petrol engine work means getting energy from fuel combinations, and then from hot gases, to go into the kinetic energy of a moving car.

All the interesting and useful processes which involve energy involve getting it to go from one thing to another. It is not surprising, then, that what matters more than the number of ways of sharing energy out in a material, is the change in the number of ways when energy is added or removed.

As before, we continue to think about the special case of the Einstein solid, because it is reasonably easy to think about. Some, though not all, of the answers are applicable in general: this is what makes the exercise worth the bother.

We start by giving a result for the effect on the number of ways W of adding one quantum to an Einstein solid. This result, proved in Appendix B, comes from the result for W itself mentioned previously. As will be seen, the computer films also give evidence that this new result for the effect of adding one quantum is right.

If there are N oscillators sharing q quanta in W ways, and one more quantum is added, the new, larger number of ways W* is given by

W*/W= (N+q)/(q+1)

Figure 71, taken from figure 70, illustrates how this result works. It shows the case where N = 3 and q = 2, for which W = 6. When one quantum is added, the result just given predicts

W*/W= (3+2)/(2+1) = 5/3

so that W*=10

The ten different ways, when q = 2+ 1, are enumerated in figure 71.

Figure 71

The factor (1 + N/q)

Suppose there are 1 000 000 quanta shared among 1 000 000 oscillators. If one quantum is added, the number of ways goes from W to the larger value W*, where

W* /W = 2 000 000/1 000 001 ≈ 2 000 000/1 000 000 = 2

Now recall that N and q are, for decent-sized lumps of matter, going to be in excess of 10²⁰, and it is clear that the factor (N+q)/(q+1) can be written as (N+q)/q, that is (1 + N/q), without making any appreciable error.

This is our first main theoretical result. When one quantum is added to an Einstein solid consisting of N oscillators sharing q quanta, the number of ways W increases to W*, where

W*/W = (1 + N/q) ......Result 1

It contains the similar result for the effect of taking away one quantum, for this is like going from the system with W* arrangements to one with W arrangements. Taking one quantum away has the effect of dividing W by the factor (1 + N/q).

In Part Four, the factor (1 + N/q) was guessed to be the ratio of numbers of oscillators with energies different by one quantum, in the exponential equilibrium distribution of an Einstein solid (figure 72). We shall now inquire how it comes about that the distribution is exponential, having just this ratio. We shall argue that it follows from result 1 above, that the exponential distribution,

(number of oscillators having any particular number of quanta) / (number of oscillators having that number of quanta plus one more) = (1 + N/q), Exponential distribution

will describe the energy distribution in an Einstein solid.

Figure 72

One oscillator competing with many others for energy

We now think about one oscillator, exchanging quanta with a whole large collection of other oscillators, as illustrated in figure 73.

Figure 73

Suppose, to be definite, that this one oscillator has four quanta. If, at some other time, it has only three quanta, the spare quantum is now available to be shared among the others. If the collection of many oscillators contains a very large number, N, of oscillators, sharing a very large number, q, of quanta, adding one more quantum increases the number of ways in which quanta can be arranged amongst them from W to W*, where, using result 1,

W*/W= (1+N/q).

For the moment, let N = q, so that (1 + N/q) = 2. If the lone oscillator donates one quantum to be shared among all the others, there will be twice as many sharing possibilities for them to try out as there were when the lone oscillator had its fourth quantum, and the others did not have this extra quantum to share.

If the shuffling of quanta is random, what happens in twice as many ways happens twice as often. The system tries everything, without favour or prejudice. The result is that some states of affairs are in effect favoured: in particular, the lone oscillator will have three quanta twice as often as it will have four quanta, because there are twice as many arrangements with an extra quantum shared amongst the collection of many oscillators, as there are with an extra quantum not so shared.

The situation is similar to several of those discussed in Part Three. A card drawn from a pack is more often not an ace than it is an ace, because there are more ways to draw a card which is not an ace. Shaken boxes of marbles are more often found mixed than sorted, not because sorted arrangements are avoided, but because there are fewer of them than there are of unsorted arrangements.

In general, because, using Result 1,

(number of ways of sharing any particular number of quanta) / (number of ways of sharing one fewer quantum) = (1 + N/q), it follows that, for any one oscillator sharing quanta with many others, (number of occasions when an oscillator has any particular number of quanta) / (number of occasions when it has that number of quanta plus one more ) = (1 + N/q) .... Result 2

Result 2 is nearly, but not quite, what we require. It says how often just one oscillator has each of two different energies. To explain the exponential distribution, we must be able to say how many oscillators have each of two energies, at one moment. An argument linking these two questions follows.

The biography of an oscillator: what happens often to one happens to many

In the fourth film in the series, Change and chance; a model of thermal equilibrium in a solid, illustrated in figures 51 to 56, the fate of just one oscillator was followed over a long period of time. Instead of looking at the behaviour of many oscillators at one time, the biography of one of them was examined.

A subtle and difficult argument

No one can blame you if you find the foregoing argument hard to follow. It is subtle, and it is difficult. It is, we believe, right in essence, but a full, careful, and complete argument would be even harder to follow. Such an argument is to be found in the most advanced books of statistical mechanics, under the title, the canonical ensemble.

If you find it too much to cope with, it may be best to take the following more general view. The experiments in the computer films suggest that the equilibrium distribution for an Einstein solid has the ratio (1 + N/q), in that the suggestion that this is the ratio can be tested against the results of the experiments. The computer is, in effect, being used to show what happens, as an alternative to having to calculate it. It is also easy to see that the ratio (1 + N/q) has a simple meaning. It is the ratio we have called W*/W, the ratio of numbers of ways of arranging quanta after and before adding an extra quantum. The exponential distribution having this ratio arises because any one atom has to share quanta with the rest, and the average behaviour will be decided by the relative numbers of ways there are of sharing or of not sharing extra quanta.

The flow of heat from hot to cold

Why does heat go from hot to cold? If the answer were not rather interesting, the question would seem a silly one. The interest of the answer lies not so much in its content as in what kind of answer it is. It is not an answer of the kind, Because it is pushed ..., or of the kind, It must because this causes that ..., such as one might give to a question about why a spacecraft moves more quickly as it comes nearer to the Earth. The answer is of the kind we have given before in this Unit, but not elsewhere in the course. (However, in Unit 1, it could have been argued that a rubber band has the length it has because that is the most likely length.) Heat flow from hot to cold happens by chance; it happens because it is likely to happen. It always happens because it is so very, very likely to happen that the chance of its not happening is too remote to bother about.

The sixth film in the series, Change and chance: a model of thermal equilibrium in a solid, illustrated in figures 61 to 65, gives an example of what this answer means, and the idea is summarized in Part Four.

It is now possible to discuss this example more quantitatively. The lefthand half of figure 61 has 900 oscillators and 900 quanta. (For atoms in the captions, now read oscillators) If it were to lose one quantum, the number of ways of sharing quanta within that half would be divided by two (the ratio 1 + N/q, with N = q). In fact, W for this half goes from about 10⁵⁴⁰ to half that number.

The model solid on the righthand side of figure 61 has only 300 quanta shared among 900 oscillators, so that (1 + N/q) = 4. The 300 quanta can be rearranged in some 10³⁰⁰ ways, as it happens, but if an extra quantum is added to their number. W is multiplied by four.

When the two parts are separate, as in figure 61, any of the 10⁵⁴⁰ ways of arranging quanta in the lefthand part can co-exist with every one of the 10³⁰⁰ ways for the righthand part. The total number of ways for both at once is the product of these two. or 10⁸⁴⁰.

Now suppose that they are put in contact, as in figure 63, and that one quantum goes from left to right. The new total number of ways is the product of the two new numbers of ways for each part, given above, and is thus given by

new number of ways = ½ × 10⁵⁴⁰ × 4 × 10³⁰⁰
= 2 × 10⁸⁴⁰ compared with 10⁸⁴⁰ ways beforehand.

The transfer of a quantum makes more difference to the number of ways for the cold part than it does for the hot part. The net effect of a transfer from left to right is an increase in the number of ways, so such transfers will happen more often than not. Further quanta will be likely to follow the first, until a further transfer does not any longer increase the number of ways.

Certainly, quanta do go the wrong way. Such a move reduces the number of ways, in this example by a factor 2 (W for the righthand part being divided by 4, W for the lefthand part being multiplied by 2). So to start with, the impartial effect of chance will be to make transfers from left to right four times as often as from right to left. But this is another way of saying that quanta will, on average, go the proper way. Would 50 quanta go the wrong way? This would lead to a reduction in the number of ways of 2⁵⁰ times, compared with an increase of 2⁵⁰ times for flow the right way. So the first is 2¹⁰⁰ times more likely than the second. Similarly, that 50 quanta should flow the right way is about 2⁵⁰ times more likely, and that they should flow the wrong way is about 2⁵⁰ times less likely, than is no net flow. It is this one-way transfer that makes us call the one half hot and the other cold.

The general meaning of hot and cold

Looking at figures 63 and 64 you might well feel that we have been to a great deal of trouble to discuss a simple matter: that energy goes, like scent in a room, from where it is more concentrated to where it is not. This is, in fact, a quite proper objection: the effect shown in figures 63 and 64 is little different from diffusion. But heat flow is not always just like diffusion, for in general, materials exchanging heat will not be as simple as the Einstein solid model. Their energy levels need not be equally spaced, and they need not be identical materials. We have tried to cast the argument in a form which illustrates a general rule about heat flow, whose value is not limited to special simple models.

For any pair of objects whatever, if the thermal transfer of energy raises the number of ways in which the atoms can be distributed over energy levels (more crudely, the number of ways of sharing the energy out amongst the atoms), then such a transfer Will happen. A cold object is always one for which there is a large effect on the number of ways, when energy flows in or out. A hot object is always one for which the corresponding effect is small. Two objects are in thermal equilibrium if taking energy from either and giving it to the other reduces the number of ways for the first by the same factor as it increases the number of ways for the second.

Hot means good energy giver because fewer quanta lead to not too many fewer ways.

Cold means good energy taker because more quanta lead to very msany more ways.

Equally hot means exchanging quanta has the same effect on the number of ways.

Temperature and changes to number of ways

The temperature can be given a quite precise expression in terms of changes to numbers of ways.

For the Einstein solid, we saw that the addition of one quantum multiplied the number of ways by a factor, and we have now seen that this factor depends only on how hot the solid is. Let the factor have any value, say 1.5.

Then W*/W = 1.5

if one quantum is added. If another quantum is added, and W* now means the number of ways after adding both quanta,

W*/W = 1.5 × 1.5.

If many quanta are added, the result is the long product of factors

W*/W = 1.5 × 1.5 × 1.5 × 1.5 ....

It is convenient to take logarithms, and it is conventional to use natural logarithms, to base e. Then if W* is the number of ways after adding many quanta,

In W*/W = ln W* - ln W = ln 1.5 + ln 1.5 + ln 1.5 + ....

so long as the added quanta amount to too little energy to make a significant difference to the ratio (that is, to how hot the solid is).

On the righthand side of this equation giving the change in ln W, there are just as many terms equal to ln 1.5 as there are quanta added, so the righthand side is proportional to the change ΔU in the energy U shared among the atoms of the solid, since ΔU is the number of added quanta multiplied by the energy of each quantum. (The energy U could be changed in other ways, for example, by squashing the solid, but we are restricting ourselves to changes in U brought about by adding quanta, that is, by promoting some atoms to higher energies while keeping the same set of levels.)

Writing Δ ln W for the difference ln W* - ln W, we obtain

Δ ln W ∝ ΔU (constant volume, almost constant temperature)

The interest of this result is that it is not restricted to the simple case of the Einstein solid, but can be applied to most sorts of matter, solid, liquid, or gas.

Previously, we saw that high temperatures T go with small changes in numbers of ways. Clearly, Δ ln W cannot be proportional to T, for that would have the reverse effect. In order to settle on a way of relating temperature to numbers of ways, the conventional choice is to put

Δ ln W ∝ ΔU/T (constant volume, nearly constant temperature)

This is a matter of choice; we are trying to find a fundamental way of expressing an idea which so far has been given no quantitative meaning (apart from arbitrary practical scales) - the idea of temperature. To have 1 /T² in place of 1/T would be permissible, but foolish, since it would only complicate matters for no good reason.

The proportionality of Δ ln W to ΔU/T can be made into an equality by inserting a constant k, giving

Δ ln W = ΔU/kT

or kT = ΔU/Δ ln W (both at constant volume, and for small changes)

This is our second main result; a fundamental definition of temperature in terms of energy and numbers of ways. The name Boltzmann constant has been given to k.

Kelvin temperature and the Boltzmann constant k

The next step is to consider what meaning and value to give to k. If we choose its value arbitrarily, then the scale of temperature is settled, as the following example shows. The simplest arbitrary value to give to k is 1. Think now of a typical Einstein solid, to which one quantum is added. In the instances so far, the ratio W*/W = (1 + N/q) has been between 1 and 10, so Δ ln W is a number of the order 1. ΔU is here the energy of one quantum. If the frequency f of atomic oscillations is of the order of 10¹² Hz, which is about right, the energy of a quantum (hf) is of the order of 10^-21 joule per quantum, h being the Planck constant. Then kT is also of the order of 10^-21 joule per particle, and if k = 1, T is of the order 10^-21 joule per particle.

It may seem odd to have temperatures whose values are always minute and are in joules per particle. From an atomic point of view, this could be thought to be very natural, since T is, simply, roughly equal to the average energy one atom possesses.

Despite the naturalness of this way of looking at temperature, the custom is to have temperatures measured on another scale, on which values of T are larger and more convenient. Instead of choosing the value of k, the usual thing to do is to choose to measure T in units, and on a scale, of its own, and to find what value of k that choice implies. It should already be clear that if T is a number greater than 1, k must be very small, because kT is very small.

The convention is to decide, quite arbitrarily, that the temperature of melting ice is just 273 degrees above absolute zero. This scale is called the Kelvin scale, and temperatures are measured in kelvins, symbol K. Room temperature is about 300 K, for example, while helium is a liquid at about 4 K. Because kT is in joules, k will now have the unit joules per kelvin, symbol J K^-1.

In order to measure the value of k which follows from choosing the Kelvin scale for T, one has to find some physical system - anything will do - for which the change Δln W can be calculated when energy ΔU is added. There are many ways of doing this job. One way, using a gas as the material whose change in numbers of ways is calculated, appears in Part Six. Because we already know how to calculate Δln W for an Einstein solid, it is convenient at this stage to use that model again.

Measurment of k: the Einstein solid as a means of counting ways

For an Einstein solid, with q quanta shared among N oscillators, the ratio W*/W = (1 +N/q), for one quantum added, so that

Δln W = ln(1 +N/q) (addition of one quantum)

If one quantum of energy ε is added, the energy U of the solid increases by ΔU = ε.

Then the equation

kT = ΔU / Δ ln W

which defines the temperature T, becomes

kT = ε/ln(1 +N/q).

In Part Four, this same equation was produced, though T was there defined in a less fundamental way, as relating to the slope of an exponential distribution.

In Part Five, we have seen how the slope of the distribution is related to changes in numbers of ways.

Starting in Part Four, at The heat capacity of an Einstein solid, there was given an argument which led from the equation above - relating temperature T to the energy levels and numbers of atoms and quanta - to the result

molar heat capacity of an Einstein solid ≈ 3 kL

where L is the Avogadro constant. This result was seen to be an approximation which might be expected to be good at high temperatures, but not at low temperatures. If you omitted the argument in Part Four, you can go back to it now.

Table 6 in Part Four gives values of molar heat capacities. The table makes it clear that, as predicted there is a group of solid monatomic elements whose heat capacities are all much the same. The value of this common heat capacity seems to be about 25 J mol^-1 K^-1.

Assuming that 3KL = 25 J mol^-1 K^-1

and since L = 6 × 10²³ mol^-1

then k= 1.39 × 10^-23 J K^-1

compared with k = 1.38 × 10^-23 J K^-1, taken from tables of physical constants.

You will notice that k is small, as we suggested it must be when introducing it in the previous section. k is of the order 10^-23 J K^-1 because at temperatures of the order 300 K, the energy one atom is likely to have is about kT. Since this energy is a few times 10^-21 J, k = kT/T is of order 10^-23 J K^-1. The Boltzmann constant k can be regarded as a scale constant, ensuring that the numbers Δ ln W , or the small magnitudes of quantum energies, do not lead to very small numerical values of the temperature.

The Boltsmann factor

We can now bring out more of the meaning of the Boltzmann factor, first introduced in Part Four. This factor gives the ratio of the number of atoms to be expected in a high energy level to the number to be expected in a low energy level, if the levels differ by energy E.

It has its origin in the equation

Δ ln W = ΔU/kT (constant volume)

Just as in figure 73, and in the argument that goes with it, to promote one atom to a level having energy E above its present level, energy E must be taken from all the other atoms of the material and its surroundings, of which the one atom forms a part. This reduces the number of ways in which the energy can be shared amongst these atoms, by an amount given by

Where -E has been written for ΔU, since U is to be reduced by an amount E.

If W* is the number of ways after energy E is removed, and W is the number of ways before, then

ln W*/W= -E/kT

and W*/W= e^-E/kT

But what happens in few ways W*, will happen rarely. The number n_high of atoms which happen to acquire energy E will compare with the number n_low which do not, in the ratio

n_high/n_low = W*/W so that

n_high/n_low = e^-E/kT

This is the result we reached in Part Four, arguing only in terms of the simple Einstein model. It has a wide range of applications, in physics and in chemistry, some of which are looked at in Part Six.

Counting ways with heaters and thermometers: entropy

Suppose you warm a kettle of water by 10 K, starting at room temperature (about 300 K). By what factor does the number of ways of spreading the energy over molecules rise, as a result of the added energy? The answer can be found from

Δ ln W = ΔU/kT(constant volume)

One kilogramme of water needs 4200 J to warm it by 1 K, so the kettle might have been supplied with 42 000 J, if it contained 1 kg of water. k ≈ 1 .4 × 10^-23 J K^-1, so we have

Δ ln W ≈ 42000/(1.4 × 10^-23 × 300) ≈ 10²⁵.

And this is only the logarithm of the number of times there are more ways after supplying the energy, compared with the number before. Since

ln(W_warm/W_cool) ≈ 10²⁵

W_warm/W_cool ≈ e¹⁰²⁵ .

Are we serious? Indeed we are: knowing Boltzmann's constant, it is possible to calculate changes in numbers of ways simply from measurements made with heaters and thermometers. Given such information, it becomes possible to begin to tackle some of the problems raised earlier in this Unit: problems about which way processes will go. Some of these applications are touched on in Part Six.

In the equation

k Δ ln W = ΔU/T (constant volume)

the quantity k Δ ln W on the left can be measured by means of measurements of the quantities ΔU and T, on the right. All that one requires is a heater and a thermometer which measures T in kelvins. In the above example,

k Δ ln W

has the value 42000/300= 140 J K^-1.

The quantity k Δ ln W, which, unlike W itself, can be related directly to simple laboratory measurements in the way just described, has been given a special name. the change of entropy, written ΔS. Above, ΔS was 140 J K^-1. In general,

ΔS = k Δ ln W

and, with a proper choice of the zero of S,

S = k ln W

This last equation seems only to be giving a new name to something introduced before. Yet it is the cornerstone of thermodynamics, from the statistical point of view. It says what entropy is. What is entropy? It is just the logarithm of the number W of distinct states a thing can be in, scaled down by the constant k to make the value more manageable. The result is important because entropy is a remarkably useful quantity to know about.

The meaning and use of entropy are discussed further in Part Six, where we see how entropy is measured, and how it is used by physicists, chemists, and engineers. Here we have merely introduced its definition in terms of W; a definition Boltzmann thought so important that he asked for it to be carved on his tombstone. Because every system tries out all the ways open to it, if more ways become accessible, it explores them all. W therefore tends to increase. If W increases, so does the entropy, k ln W. The rule that the direction of a process will be such as to increase the total entropy is called the Second Law of Thermodynamics.