3.8: Convergence in Distribution
This section is concenred with the convergence of probability distributions, a topic of basic importance in probability theory. Since we will be almost exclusively concerned with the convergences of sequences of various kinds, it's helpful to introduce the notation \(\N_+^* = \N_+ \cup \{\infty\} = \{1, 2, \ldots\} \cup \{\infty\}\).
Distributions on \((\R, \mathscr R)\)
Definition
We start with the most important and basic setting, the measurable space \((\R, \mathscr R)\), where \(\R\) is the set of real numbers of course, and \(\mathscr R\) is the Borel \(\sigma\)-algebra of subsets of \(\R\). Recall that if \(P\) is a probability measure on \((\R, \mathscr R)\), then the function \(F: \R \to [0, 1]\) defined by \(F(x) = P(-\infty, x]\) for \(x \in \R\) is the (cumulative) distribution function of \(P\). Recall also that \(F\) completely determines \(P\). Here is the definition for convergence of probability measures in this setting:
Suppose \(P_n\) is a probability measure on \((\R, \mathscr R)\) with distribution function \(F_n\) for each \(n \in \N_+^*\). Then \(P_n\) converges (weakly) to \(P_\infty\) as \(n \to \infty\) if \(F_n(x) \to F_\infty(x)\) as \(n \to \infty\) for every \(x \in \R\) where \(F_\infty\) is continuous. We write \(P_n \Rightarrow P_\infty\) as \(n \to \infty\).
Recall that a distribution function \(F\) is continuous at \(x \in \R\) if and only if \(\P(X = x) = 0\), so that \(x\) is not an atom of the distribution (a point of positive probability). We will see shortly why this condition on \(F_\infty\) is appropriate. Of course, a probability measure on \((\R, \mathscr R)\) is usually associated with a real-valued random variable for some random experiment that is modeled by a probability space \((\Omega, \mathscr F, \P)\). So to review, \(\Omega\) is the set of outcomes, \(\mathscr F\) is the \(\sigma\)-algebra of events, and \(\P\) is the probability measure on the sample space \((\Omega, \mathscr F)\). If \(X\) is a real-valued random variable defined on the probability space, then the distribution of \(X\) is the probability measure \(P\) on \((\R, \mathscr R)\) defined by \(P(A) = \P(X \in A)\) for \(A \in \mathscr R\), and then of course, the distribution function of \(X\) is the function \(F\) defined by \(F(x) = \P(X \le x)\) for \(x \in \R\). Here is the convergence terminology used in this setting:
Suppose that \(X_n\) is a real-valued random variable with distribution \(P_n\) for each \(n \in \N_+^*\). If \(P_n \Rightarrow P_\infty\) as \(n \to \infty\) then we say that \(X_n\) converges in distribution to \(X_\infty\) as \(n \to \infty\). We write \(X_n \to X_\infty\) as \(n \to \infty\) in distribution.
So if \(F_n\) is the distribution function of \(X_n\) for \(n \in \N_+^*\), then \(X_n \to X_\infty\) as \(n \to \infty\) in distribution if \(F_n(x) \to F_\infty(x)\) at every point \(x \in \R\) where \(F_\infty\) is continuous. On the one hand, the terminology and notation are helpful, since again most probability measures are associated with random variables (and every probability measure can be). On the other hand, the terminology and notation can be a bit misleading since the random variables, as functions, do not converge in any sense, and indeed the random variables need not be defined on the same probability spaces. It is only the distributions that converge. However, often the random variables are defined on the same probability space \((\Omega, \mathscr F, \P)\), in which case we can compare convergence in distribution with the other modes of convergence we have or will study:
- Convergence with probability 1
- Convergence in probability
- Convergence in mean
We will show, in fact, that convergence in distribution is the weakest of all of these modes of convergence. However, strength of convergence should not be confused with importance. Convergence in distribution is one of the most important modes of convergence; the central limit theorem, one of the two fundamental theorems of probability, is a theorem about convergence in distribution.
Preliminary Examples
The examples below show why the definition is given in terms of distribution functions, rather than probability density functions, and why convergence is only required at the points of continuity of the limiting distribution function. Note that the distributions considered are probability measures on \((\R, \mathscr R)\), even though the support of the distribution may be a much smaller subset. For the first example, note that if a deterministic sequence converges in the ordinary calculus sense, then naturally we want the sequence (thought of as random variables) to converge in distribution. Expand the proof to understand the example fully.
Suppose that \(x_n \in \R\) for \(n \in \N_+^*\). Define random variable \(X_n = x_n\) with probability 1 for each \(n \in \N_+^*\). Then \(x_n \to x_\infty\) as \(n \to \infty\) if and only if \(X_n \to X_\infty\) as \(n \to \infty\) in distribution.
Proof
For \(n \in \N_+^*\), the CDF \(F_n\) of \(X_n\) is given by \(F_n(x) = 0\) for \(x \lt x_n\) and \(F_n(x) = 1\) for \(x \ge x_n\).
- Suppose that \(x_n \to x_\infty\) as \(n \to \infty\). If \(x \lt x_\infty\) then \(x \lt x_n\), and hence \(F_n(x) = 0\), for all but finitely many \(n \in \N_+\), and so \(F_n(x) \to 0\) as \(n \to \infty\). If \(x \gt x_\infty\) then \(x \gt x_n\),and hence \(F_n(x) = 1\), for all but finitely many \(n \in \N_+\), and so \(F_n(x) \to 1\) as \(n \to \infty\). Nothing can be said about the limiting behavior of \(F_n(x_\infty)\) as \(n \to \infty\) without more information. For example, if \(x_n \le x_\infty\) for all but finitely many \(n \in \N_+\) then \(F_n(x_\infty) \to 1\) as \(n \to \infty\). If \(x_n \gt x_\infty\) for all but finitely many \(n \in \N_+\) then \(F_n(x_\infty) \to 0\) as \(n \to \infty\). If \(x_n \lt x_\infty\) for infinitely many \(n \in \N_+\) and \(x_n \gt x_\infty\) for infinitely many \(n \in \N_+\) then \(F_n(x_\infty)\) does not have a limit as \(n \to \infty\). But regardless, we have \(F_n(x) \to F_\infty(x)\) as \(n \to \infty\) for every \(x \in \R\) except perhaps \(x_\infty\), the one point of discontinuity of \(F_\infty\). Hence \(X_n \to X_\infty\) as \(n \to \infty\) in distribution.
- Conversely, suppose that \(X_n \to X_\infty\) as \(n \to \infty\) in distribution. If \(x \lt x_\infty\) then \(F_n(x) \to 0\) as \(n \to \infty\) and hence \(x \lt x_n\) for all but finitely many \(n \in \N_+\). If \(x \gt x_\infty\) then \(F_n(x) \to 1\) as \(n \to \infty\) and hence \(x \ge x_n\) for all but finitely many \(n \in \N_+\). So, for every \(\epsilon \gt 0\), \(x_n \in (x_\infty - \epsilon, x_\infty + \epsilon)\) for all but finitely many \(n \in \N_+\), and hence \(x_n \to x_\infty\) as \(n \to \infty\).
The proof is finished, but let's look at the probability density functions to see that these are not the proper objects of study. For \(n \in \N_+^*\), the PDF \(f_n\) of \(X_n\) is given by \(f_n(x_n) = 1\) and \(f_n(x) = 0\) for \(x \in \R \setminus \{x_n\}\). Only when \(x_n = x_\infty\) for all but finitely many \(n \in \N_+\) do we have \(f_n(x) \to f(x)\) for \(x \in \R\).
For the example below, recall that \( \Q \) denotes the set of rational numbers. Once again, expand the proof to understand the example fully
For \(n \in \N_+\), let \(P_n\) denote the discrete uniform distribution on \(\left\{\frac{1}{n}, \frac{2}{n}, \ldots \frac{n-1}{n}, 1\right\}\) and let \(P_\infty\) denote the continuous uniform distribution on the interval \([0, 1]\). Then
- \(P_n \Rightarrow P_\infty\) as \(n \to \infty\)
- \(P_n(\Q) = 1\) for each \(n \in \N_+\) but \(P_\infty(\Q) = 0\).
Proof
As usual, let \(F_n\) denote the CDF of \(P_n\) for \(n \in \N_+^*\).
- For \(n \in \N_+\) note that \( F_n \) is given by \( F_n(x) = \lfloor n \, x \rfloor / n \) for \( x \in [0, 1] \). But \( n \, x - 1 \le \lfloor n \, x \rfloor \le n \, x \) so \( \lfloor n \, x \rfloor / n \to x \) as \( n \to \infty \) for \(x \in [0, 1]\). Of course, \(F_n(x) = 0\) for \(x \lt 0\) and \(F_n(x) = 1\) for \(x \gt 1\). So \(F_n(x) \to F_\infty(x)\) as \(n \to \infty\) for all \(x \in \R\).
- Note that by definition, so \(P_n(\Q) = 1\) for \(n \in \N_+\). On the other hand, \( P_\infty \) is a continuous distribution and \( \Q \) is countable, so \(P_\infty(\Q) = 0\).
The proof is finished, but let's look at the probability density functions. For \(n \in \N_+\), the PDF \(f_n\) of \(P_n\) is given by \(f_n(x) = \frac{1}{n}\) for \(x \in \left\{\frac{1}{n}, \frac{2}{n}, \ldots \frac{n-1}{n}, 1\right\}\) and \(f_n(x) = 0\) otherwise. Hence \( 0 \le f_n(x) \le \frac{1}{n} \) for \(n \in \N_+\) and \(x \in \R\), so \(f_n(x) \to 0\) as \(n \to \infty\) for every \( x \in \R \).
The point of the example is that it's reasonable for the discrete uniform distribution on \(\left\{\frac{1}{n}, \frac{2}{n}, \ldots \frac{n-1}{n}, 1\right\}\) to converge to the continuous uniform distribution on \([0, 1]\), but once again, the probability density functions are evidently not the correct objects of study.
Probability Density Functions
As the previous example shows, it is quite possible to have a sequence of discrete distributions converge to a continuous distribution (or the other way around). Recall that probability density functions have very different meanings in the discrete and continuous cases: density with respect to counting measure in the first case, and density with respect to Lebesgue measure in the second case. This is another indication that distribution functions, rather than density functions, are the correct objects of study. However, if probability density functions of a fixed type converge then the distributions converge. Recall again that we are thinnking of our probability distributions as measures on \((\R, \mathscr R)\) even when supported on a smaller subset.
Convergence in distribution in terms of probability density functions.
- Suppose that \(f_n\) is a probability density function for a discrete distribution \(P_n\) on a countable set \(S \subseteq \R\) for each \(n \in \N_+^*\). If \(f_n(x) \to f_\infty(x)\) as \(n \to \infty\) for each \(x \in S\) then \(P_n \Rightarrow P_\infty\) as \(n \to \infty\).
- Suppose that \(f_n\) is a probability density function for a continuous distribution \(P_n\) on \(\R\) for each \(n \in \N_+^*\) If \(f_n(x) \to f(x)\) as \(n \to \infty\) for all \(x \in \R\) (except perhaps on a set with Lebesgue measure 0) then \(P_n \Rightarrow P_\infty\) as \(n \to \infty\).
Proof
- Fix \(x \in \R\). Then \(P_n(-\infty, x] = \sum_{y \in S, \, y \le x} f(y)\) for \(n \in \N_+\) and \(P(-\infty, x] = \sum_{y \in S, \, y \le x} f(y)\). It follows from Scheffé's theorem with the measure space \((S, \mathscr P(S), \#)\) that \(P_n(-\infty, x] \to P(-\infty, x]\) as \(n \to \infty\).
- Fix \(x \in \R\). Then \(P_n(-\infty, x] = \int_{-\infty}^x f(y) \, dy\) for \(n \in \N_+\) and \(P(-\infty, x] = \int_{-\infty}^x f(y) \, dy\). It follows from Scheffé's theorem with the measure space \((\R, \mathscr R, \lambda)\) that \(P_n(-\infty, x] \to P(-\infty, x]\) as \(n \to \infty\).
Convergence in Probability
Naturally, we would like to compare convergence in distribution with other modes of convergence we have studied.
Suppose that \(X_n\) is a real-valued random variable for each \(n \in \N_+^*\), all defined on the same probability space. If \(X_n \to X_\infty\) as \(n \to \infty\) in probability then \(X_n \to X_\infty\) as \(n \to \infty\) in distribution.
Proof
Let \(F_n\) denote the distribution function of \(X_n\) for \(n \in \N_+^*\). Fix \(\epsilon \gt 0\). Note first that \(\P(X_n \le x) = \P(X_n \le x, X_\infty \le x + \epsilon) + \P(X_n \le x, X_\infty \gt x + \epsilon) \). Hence \(F_n(x) \le F_\infty(x + \epsilon) + \P\left(\left|X_n - X_\infty\right| \gt \epsilon\right)\). Next, note that \(\P(X_\infty \le x - \epsilon) = \P(X_\infty \le x - \epsilon, X_n \le x) + \P(X_\infty \le x - \epsilon, X_n \gt x)\). Hence \(F_\infty(x - \epsilon) \le F_n(x) + \P\left(\left|X_n - X_\infty\right|\right) \gt \epsilon\). From the last two results it follows that \[ F_\infty(x - \epsilon) - \P\left(\left|X_n - X_\infty\right| \gt \epsilon\right) \le F_n(x) \le F_\infty(x + \epsilon) + \P\left(\left|X_n - X_\infty\right| \gt \epsilon\right) \] Letting \(n \to \infty\) and using convergence in probability gives \[ F_\infty(x - \epsilon) \le \liminf_{n \to \infty} F_n(x) \le \limsup_{n \to \infty} F_n(x) \le F_\infty(x + \epsilon) \] Finally, letting \(\epsilon \downarrow 0\) we see that if \(F_\infty\) is continuous at \(x\) then \(F_n(x) \to F_\infty(x)\) as \(n \to \infty\).
Our next example shows that even when the variables are defined on the same probability space, a sequence can converge in distribution, but not in any other way.
Let \(X\) be an indicator variable with \(\P(X = 0) = \P(X = 1) = \frac{1}{2}\), so that \(X\) is the result of tossing a fair coin. Let \(X_n = 1 - X \) for \(n \in \N_+\). Then
- \(X_n \to X\) as \(n \to \infty\) in distribution.
- \(\P(X_n \text{ does not converge to } X \text{ as } n \to \infty) = 1\).
- \(X_n \) does not converge to \( X \) as \(n \to \infty\) in probability.
- \(X_n\) does not converge to \(X\) as \(n \to \infty\) in mean.
Proof
- This trivially holds since \(1 - X\) has the same distribution as \(X\).
- This follows since \(\left|X_n - X\right| = 1\) for every \(n \in \N_+\).
- This follows since \(\P\left(\left|X_n - X\right| \gt \frac{1}{2}\right) = 1\) for each \(n \in \N_+\).
- This follows since \(\E\left(\left|X_n - X\right|\right) = 1\) for each \(n \in \N_+\).
The critical fact that makes this counterexample work is that \(1 - X\) has the same distribution as \(X\). Any random variable with this property would work just as well, so if you prefer a counterexample with continuous distributions, let \(X\) have probability density function \(f\) given by \(f(x) = 6 x (1 - x)\) for \(0 \le x \le 1\). The distribution of \(X\) is an example of a beta distribution.
The following summary gives the implications for the various modes of convergence; no other implications hold in general.
Suppose that \(X_n\) is a real-valued random variable for each \(n \in \N_+^*\), all defined on a common probability space.
- If \(X_n \to X_\infty\) as \(n \to \infty\) with probability 1 then \(X_n \to X_\infty\) as \(n \to \infty\) in probability.
- If \(X_n \to X_\infty\) as \(n \to \infty\) in mean then \(X_n \to X_\infty\) as \(n \to \infty\) in probability.
- If \(X_n \to X_\infty\) as \(n \to \infty\) in probability then \(X_n \to X_\infty\) as \(n \to \infty\) in distribtion.
It follows that convergence with probability 1, convergence in probability, and convergence in mean all imply convergence in distribution, so the latter mode of convergence is indeed the weakest. However, our next theorem gives an important converse to part (c) in (7) , when the limiting variable is a constant. Of course, a constant can be viewed as a random variable defined on any probability space.
Suppose that \(X_n\) is a real-valued random variable for each \(n \in \N_+\), defined on the same probability space, and that \(c \in \R\). If \(X_n \to c\) as \(n \to \infty\) in distribution then \(X_n \to c\) as \(n \to \infty\) in probability.
Proof
Assume that the probability space is \((\Omega, \mathscr F, \P)\). Note first that \(\P(X_n \le x) \to 0\) as \(n \to \infty\) if \(x \lt c\) and \(\P(X_n \le x) \to 1\) as \(n \to \infty\) if \(x \gt c\). It follows that \(\P\left(\left|X_n - c\right| \le \epsilon\right) \to 1\) as \(n \to \infty\) for every \(\epsilon \gt 0\).
The Skorohod Representation
As noted in the summary above , convergence in distribution does not imply convergence with probability 1, even when the random variables are defined on the same probability space. However, the next theorem, known as the Skorohod representation theorem , gives an important partial result in this direction.
Suppose that \(P_n\) is a probability measure on \((\R, \mathscr R)\) for each \(n \in \N_+^*\) and that \(P_n \Rightarrow P_\infty\) as \(n \to \infty\). Then there exist real-valued random variables \(X_n\) for \(n \in \N_+^*\), defined on the same probability space, such that
- \(X_n\) has distribution \(P_n\) for \(n \in \N_+^*\).
- \(X_n \to X_\infty\) as \(n \to \infty\) with probability 1.
Proof
Let \((\Omega, \mathscr F, \P)\) be a probability space and \(U\) a random variable defined on this space that is uniformly distributed on the interval \((0, 1)\). For a specific construction, we could take \(\Omega = (0, 1)\), \(\mathscr F\) the \(\sigma\)-algebra of Borel measurable subsets of \((0, 1)\), and \(\P\) Lebesgue measure on \((\Omega, \mathscr F)\) (the uniform distribution on \((0, 1)\)). Then let \(U\) be the identity function on \(\Omega\) so that \(U(\omega) = \omega\) for \(\omega \in \Omega\), so that \(U\) has probability distribution \(\P\). We have seen this construction many times before.
- For \(n \in \N_+^*\), let \(F_n\) denote the distribution function of \(P_n\) and define \(X_n = F_n^{-1}(U)\) where \(F_n^{-1}\) is the quantile functions of \(F_n\). Recall that \(X_n\) has distribution function \(F_n\) and therefore \(X_n\) has distribution \(P_n\) for \(n \in \N_+^*\). Of course, these random variables are also defined on \((\Omega, \mathscr F, \P)\).
- Let \(\epsilon \gt 0\) and let \(u \in (0, 1)\). Pick a continuity point \(x\) of \(F_\infty\) such that \(F_\infty^{-1}(u) - \epsilon \lt x \lt F_\infty^{-1}(u)\). Then \(F_\infty(x) \lt u\) and hence \(F_n(x) \lt u\) for all but finitely many \(n \in \N_+\). It follows that \(F_\infty^{-1}(u) - \epsilon \lt x \lt F_n^{-1}(u)\) for all but finitely many \(n \in \N_+\). Let \(n \to \infty\) and \(u \downarrow 0\) to conclude that \(F_\infty^{-1}(u) \le \liminf_{n \to \infty} F_n^{-1}(u)\). Next, let \(v\) satisfy \(0 \lt u \lt v \lt 1\) and let \(\epsilon \gt 0\). Pick a continuity point \(x\) of \(F_\infty\) such that \(F_\infty^{-1}(v) \lt x \lt F_\infty^{-1}(v) + \epsilon\). Then \(u \lt v \lt F_\infty(x)\) and hence \(u \lt F_n(x)\) for all but finitely many \(n \in \N_+\). It follows that \(F_n^{-1}(u) \le x \lt F_\infty^{-1}(v) + \epsilon\) for all but finitely many \(n \in \N_+\). Let \(n \to \infty\) and \(\epsilon \downarrow 0\) to conclude that \(\limsup_{n \to \infty} F_n^{-1}(u) \le F_\infty^{-1}(v)\). Letting \(v \downarrow u\) it follows that \(\limsup_{n \to \infty} F_n^{-1}(u) \le F_\infty^{-1}(u)\) if \(u\) is a point of continuity of \(F_\infty^{-1}\). Therefore \(F_n^{-1}(u) \to F_\infty^{-1}(u)\) as \(n \to \infty\) if \(u\) is a point of continuity of \(F_\infty^{-1}\). Recall from analysis that since \(F_\infty^{-1}(u)\) is increasing, the set \(D \subseteq (0, 1)\) of discontinuities of \(F_\infty^{-1}\) is countable. Since \( U \) has a continuous distribution, \(\P(U \in D) = 0\). Finally, it follows that \(\P(X_n \to X_\infty \text{ as } n \to \infty) = 1\).
The following theorem illustrates the value of the Skorohod representation and the usefulness of random variable notation for convergence in distribution. The theorem is also quite intuitive, since a basic idea is that continuity should preserve convergence .
Suppose that \(X_n\) is a real-valued random variable for each \(n \in \N_+^*\) (not necessarily defined on the same probability space). Suppose also that \(g: \R \to \R\) is measurable, and let \(D_g\) denote the set of discontinuities of \(g\), and \(P_\infty\) the distribution of \(X_\infty\). If \(X_n \to X_\infty\) as \(n \to \infty\) in distribution and \(P_\infty(D_g) = 0\), then \(g(X_n) \to g(X_\infty)\) as \(n \to \infty\) in distribution.
Proof
By Skorohod's theorem, there exists random variables \(Y_n\) for \(n \in \N_+^*\), defined on the same probability space \((\Omega, \mathscr F, \P)\), such that \(Y_n\) has the same distribution as \(X_n\) for \(n \in \N_+^*\), and \(Y_n \to Y_\infty\) as \(n \to \infty\) with probability 1. Since \(\P(Y_\infty \in D_g) = P_\infty(D_g) = 0\) it follows that \(g(Y_n) \to g(Y_\infty)\) as \(n \to \infty\) with probability 1. Hence by the theorem above , \(g(Y_n) \to g(Y_\infty)\) as \(n \to \infty\) in distribution. But \(g(Y_n)\) has the same distribution as \(g(X_n)\) for each \(n \in \N_+^*\).
As a simple corollary, if \(X_n\) converges \(X_\infty\) as \(n \to \infty\) in distribution, and if \(a, \, b \in \R\) then \(a + b X_n\) converges to \(a + b X\) as \(n \to \infty\) in distribution. But we can do a little better:
Suppose that \(X_n\) is a real-valued random variable and that \(a_n, \, b_n \in \R\) for each \(n \in \N_+^*\). If \(X_n \to X_\infty\) as \(n \to \infty\) in distribution and if \(a_n \to a_\infty\) and \(b_n \to b_\infty\) as \(n \to \infty\), then \(a_n + b_n X_n \to a + b X_\infty\) as \(n \to \infty\) in distribution.
Proof
Again by Skorohod's theorem , there exist random variables \(Y_n\) for \(n \in \N_+^*\), defined on the same probability space \((\Omega, \mathscr F, \P)\) such that \(Y_n\) has the same distribution as \(X_n\) for \(n \in \N_+^*\) and \(Y_n \to Y_\infty\) as \(n \to \infty\) with probability 1. Hence also \(a_n + b_n Y_n \to a_\infty + b_\infty Y_\infty\) as \(n \to \infty\) with probability 1. By the result above , \(a_n + b_n Y_n \to a_\infty + b_\infty Y_\infty\) as \(n \to \infty\) in distribution. But \(a_n + b_n Y_n\) has the same distribution as \(a_n + b_n X_n\) for \(n \in \N_+^*\).
The definition of convergence in distribution requires that the sequence of probability measures converge on sets of the form \((-\infty, x]\) for \(x \in \R\) when the limiting distrbution has probability 0 at \(x\). It turns out that the probability measures will converge on lots of other sets as well, and this result points the way to extending convergence in distribution to more general spaces. To state the result, recall that if \(A\) is a subset of a topological space, then the boundary of \(A\) is \(\partial A = \cl(A) \setminus \interior(A)\) where \(\cl(A)\) is the closure of \(A\) (the smallest closed set that contains \(A\)) and \(\interior(A)\) is the interior of \(A\) (the largest open set contained in \(A\)).
Suppose that \(P_n\) is a probability measure on \((\R, \mathscr R)\) for \(n \in \N_+^*\). Then \(P_n \Rightarrow P_\infty\) as \(n \to \infty\) if and only if \(P_n(A) \to P_\infty(A)\) as \(n \to \infty\) for every \(A \in \mathscr R\) with \(P(\partial A) = 0\).
Proof
Suppose that \(P_n \Rightarrow P_\infty\) as \(n \to \infty\). Let \(X_n\) be a random variable with distribution \(P_n\) for \(n \in \N_+^*\). (We don't care about the underlying probability spaces.) If \(A \in \mathscr R\) then the set of discontinuities of \(\bs 1_A\), the indicator function of \(A\), is \(\partial A\). So, suppose \(\P_\infty(\partial A) = 0\). By the continuity theorem above , \(\bs 1_A(X_n) \to \bs 1_A(X_\infty)\) as \(n \to \infty\) in distribution. Let \(G_n\) denote the CDF of \(\bs 1_A(X_n)\) for \(n \in \N_+^*\). The only possible points of discontinuity of \(G_\infty\) are 0 and 1. Hence \(G_n\left(\frac 1 2\right) \to G_\infty\left(\frac 1 2\right) \) as \(n \to \infty\). But \(G_n\left(\frac 1 2\right) = P_n(A^c)\) for \(n \in \N_+^*\). Hence \(P_n(A^c) \to \P_\infty(A^c)\) and so also \(P_n(A) \to P_\infty(A)\) as \(n \to \infty\).
Conversely, suppose that the condition in the theorem holds. If \(x \in \R\), then the boundary of \((-\infty, x]\) is \(\{x\}\), so if \(P_\infty\{x\} = 0\) then \(P_n(-\infty, x] \to P_\infty(-\infty, x]\) as \(n \to \infty\). So by definition, \(P_n \Rightarrow P_\infty\) as \(n \to \infty\).
In the context of this result, suppose that \(a, \, b \in \R\) with \(a \lt b\). If \(P\{a\} = P\{b\} = 0\), then as \(n \to \infty\) we have \(P_n(a, b) \to P(a, b)\), \(P_n[a, b) \to P[a, b)\), \(P_n(a, b] \to P(a, b]\), and \(P_n[a, b] \to P[a, b]\). Of course, the limiting values are all the same.
Examples and Applications
Next we will explore several interesting examples of the convergence of distributions on \((\R, \mathscr R)\). There are several important cases where a special distribution converges to another special distribution as a parameter approaches a limiting value. Indeed, such convergence results are part of the reason why such distributions are special in the first place.
The Hypergeometric Distribution
Recall that the hypergeometric distribution with parameters \(m\), \(r\), and \(n\) is the distribution that governs the number of type 1 objects in a sample of size \(n\), drawn without replacement from a population of \(m\) objects with \(r\) objects of type 1. It has discrete probability density function \(f\) given by \[ f(k) = \frac{\binom{r}{k} \binom{m - r}{n - k}}{\binom{m}{n}}, \quad k \in \{0, 1, \ldots, n\} \] The pramaters \(m\), \(r\), and \(n\) are positive integers with \(n \le m\) and \(r \le m\). The hypergeometric distribution is studied in more detail in the chapter on Finite Sampling Models
Recall next that Bernoulli trials are independent trials, each with two possible outcomes, generically called success and failure . The probability of success \(p \in [0, 1]\) is the same for each trial. The binomial distribution with parameters \(n \in \N_+\) and \(p\) is the distribution of the number successes in \(n\) Bernoulli trials. This distribution has probability density function \(g\) given by \[ g(k) = \binom{n}{k} p^k (1 - p)^{n - k}, \quad k \in \{0, 1, \ldots, n\} \] The binomial distribution is studied in more detail in the chapter on Bernoulli Trials. Note that the binomial distribution with parameters \(n\) and \(p = r / m\) is the distribution that governs the number of type 1 objects in a sample of size \(n\), drawn with replacement from a population of \(m\) objects with \(r\) objects of type 1. This fact is motivation for the following result:
Suppose that \(r_m \in \{0, 1, \ldots, m\}\) for each \(m \in \N_+\) and that \(r_m / m \to p\) as \(m \to \infty\). For fixed \(n \in \N_+\), the hypergeometric distribution with parameters \(m\), \(r_m\), and \(n\) converges to the binomial distribution with parameters \(n\) and \(p\) as \(m \to \infty\).
Proof
Recall that for \( a \in \R \) and \( j \in \N \), we let \( a^{(j)} = a \, (a - 1) \cdots [a - (j - 1)] \) denote the falling power of \( a \) of order \( j \). The hypergeometric PDF can be written as \[ f_m(k) = \binom{n}{k} \frac{r_m^{(k)} (m - r_m)^{(n - k)}}{m^{(n)}}, \quad k \in \{0, 1, \ldots, n\} \] In the fraction above, the numerator and denominator both have \( n \) fractors. Suppose that we group the \( k \) factors in \( r_m^{(k)} \) with the first \( k \) factors of \( m^{(n)} \) and the \( n - k \) factors of \( (m - r_m)^{(n-k)} \) with the last \( n - k \) factors of \( m^{(n)} \) to form a product of \( n \) fractions. The first \( k \) fractions have the form \( (r_m - j) \big/ (m - j) \) for some \( j \) that does not depend on \( m \). Each of these converges to \( p \) as \( m \to \infty \). The last \( n - k \) fractions have the form \( (m - r_m - j) \big/ (m - k - j) \) for some \( j \) that does not depend on \( m \). Each of these converges to \( 1 - p \) as \( m \to \infty \). Hence \[f_m(k) \to \binom{n}{k} p^k (1 - p)^{n-k} \text{ as } m \to \infty \text{ for each } k \in \{0, 1, \ldots, n\}\] The result now follows from the theorem above on density functions.
From a practical point of view, the last result means that if the population size \(m\) is
large
compared to sample size \(n\), then the hypergeometric distribution with parameters \(m\), \(r\), and \(n\) (which corresponds to sampling without replacement) is well approximated by the binomial distribution with parameters \(n\) and \(p = r / m\) (which corresponds to sampling with replacement). This is often a useful result, not computationally, but rather because the binomial distribution has fewer parameters than the hypergeometric distribution (and often in real problems, the parameters may only be known approximately). Specifically, in the limiting binomial distribution, we do not need to know the population size \(m\) and the number of type 1 objects \(r\)
individually
, but only in the
ratio
\(r / m\).
In the ball and urn experiment, set \(m = 100\) and \(r = 30\). For each of the following values of \(n\) (the sample size), switch between sampling without replacement (the hypergeometric distribution) and sampling with replacement (the binomial distribution). Note the difference in the probability density functions. Run the simulation 1000 times for each sampling mode and compare the relative frequency function to the probability density function.
- 10
- 20
- 30
- 40
- 50
The Binomial Distribution
Recall again that the binomial distribution with parameters \(n \in \N_+\) and \(p \in [0, 1]\) is the distribution of the number successes in \(n\) Bernoulli trials, when \(p\) is the probability of success on a trial. This distribution has probability density function \(f\) given by \[ f(k) = \binom{n}{k} p^k (1 - p)^{n - k}, \quad k \in \{0, 1, \ldots, n\} \] Recall also that the
Poisson distribution
with parameter \(r \in (0, \infty)\) has probability density function \(g\) given by \[g(k) = e^{-r} \frac{r^k}{k!}, \quad k \in \N\] The distribution is named for Simeon Poisson and governs the number of
random points
in a region of time or space, under certain ideal conditions. The parameter \(r\) is proportional to the size of the region of time or space. The Poisson distribution is studied in more detail in the chapter on the Poisson Process.
Suppose that \(p_n \in [0, 1]\) for \(n \in \N_+\) and that \(n p_n \to r \in (0, \infty)\) as \(n \to \infty\). Then the binomial distribution with parameters \(n\) and \(p_n\) converges to the Poisson distribution with parameter \(r\) as \(n \to \infty\).
Proof
For \( k, \, n \in \N \) with \( k \le n \), the binomial PDF can be written as \[ f_n(k) = \frac{n^{(k)}}{k!} p_n^k (1 - p_n)^{n - k} = \frac{1}{k!} (n p_n) \left[(n - 1) p_n\right] \cdots \left[(n - k + 1) p_n\right] (1 - p_n)^{n - k} \] First, \( (n - j) p_n \to r \) as \(n \to \infty\) for \(j \in \{0, 1, \ldots, n - 1\}\). Next, by a famous limit from calculus, \( (1 - p_n)^n = (1 - n p_n / n)^n \to e^{-r} \) as \( n \to \infty \). Hence also \((1 - p_n)^{n-k} \to e^{-r}\) as \(n \to \infty\) for fixed \(k \in \N_+\). Therefore \(f_n(k) \to e^{-r} r^k / k!\) as \(n \to \infty\) for each \(k \in \N_+\). The result now follows from the theorem above on density functions.
From a practical point of view, the convergence of the binomial distribution to the Poisson means that if the number of trials \(n\) is
large
and the probability of success \(p\)
small
, so that \(n p^2\) is small, then the binomial distribution with parameters \(n\) and \(p\) is well approximated by the Poisson distribution with parameter \(r = n p\). This is often a useful result, again not computationally, but rather because the Poisson distribution has fewer parameters than the binomial distribution (and often in real problems, the parameters may only be known approximately). Specifically, in the approximating Poisson distribution, we do not need to know the number of trials \(n\) and the probability of success \(p\)
individually
, but only in the
product
\(n p\). As we will see in the next chapter, the condition that \(n p^2\) be small means that the variance of the binomial distribution, namely \(n p (1 - p) = n p - n p^2\) is approximately \(r = n p\), which is the variance of the approximating Poisson distribution.
In the binomial timeline experiment, set the parameter values as follows, and observe the graph of the probability density function. (Note that \(n p = 5\) in each case.) Run the experiment 1000 times in each case and compare the relative frequency function and the probability density function. Note also the successes represented as
random points
in discrete time.
- \(n = 10\), \(p = 0.5\)
- \(n = 20\), \(p = 0.25\)
- \(n = 100\), \(p = 0.05\)
In the Poisson experiment, set \(r = 5\) and \(t = 1\), to get the Poisson distribution with parameter 5. Note the shape of the probability density function. Run the experiment 1000 times and compare the relative frequency function to the probability density function. Note the similarity between this experiment and the one in the previous exercise.
The Geometric Distribution
Recall that the geometric distribution on \(\N_+\) with success parameter \(p \in (0, 1]\) has probability density function \(f\) given by \[ f(k) = p (1 - p)^{k-1}, \quad k \in \N_+\] The geometric distribution governs the trial number of the first success in a sequence of Bernoulli trials.
Suppose that \(U\) has the geometric distribution on \(\N_+\) with success parameter \(p \in (0, 1]\). For \( n \in \N_+ \), the conditional distribution of \( U \) given \( U \le n \) converges to the uniform distribution on \(\{1, 2, \ldots, n\}\) as \(p \downarrow 0\).
Proof
The CDF \(F\) of \( U \) is given by \( F(k) = 1 - (1 - p)^k \) for \(k \in \N_+\). Hence for \(n \in \N_+\), the conditional CDF of \( U \) given \( U \le n \) is \[ F_n(k) = \P(U \le k \mid U \le n) = \frac{\P(U \le k)}{\P(U \le n)} = \frac{1 - (1 - p)^k}{1 - (1 - p)^n}, \quad k \in \{1, 2, \ldots n\} \] Using L'Hospital's rule, gives \( F_n(k) \to k / n \) as \( p \downarrow 0 \) for \(k \in \{1, 2, \ldots, n\}\). As a function of \(k\) this is the CDF of the uniform distribution on \( \{1, 2, \ldots, n\} \).
Next, recall that the
exponential distribution
with rate parameter \(r \in (0, \infty)\) has distribution function \(G\) given by \[ G(t) = 1 - e^{-r t}, \quad 0 \le t \lt \infty \] The exponential distribution governs the time between
arrivals
in the Poisson model of random points in time.
Suppose that \(U_n\) has the geometric distribution on \(\N_+\) with success parameter \(p_n \in (0, 1]\) for \(n \in \N_+\), and that \(n p_n \to r \in (0, \infty)\) as \(n \to \infty\). The distribution of \(U_n / n\) converges to the exponential distribution with parameter \(r\) as \(n \to \infty\).
Proof
Let \( F_n \) denote the CDF of \( U_n / n \). Then for \( x \in [0, \infty) \) \[ F_n(x) = \P\left(\frac{U_n}{n} \le x\right) = \P(U_n \le n x) = \P\left(U_n \le \lfloor n x \rfloor\right) = 1 - \left(1 - p_n\right)^{\lfloor n x \rfloor} \] We showed in the proof of the convergence of the binomial distribution that \( (1 - p_n)^n \to e^{-r} \) as \( n \to \infty \), and hence \( \left(1 - p_n\right)^{n x} \to e^{-r x} \) as \( n \to \infty \). But by definition, \( \lfloor n x \rfloor \le n x \lt \lfloor n x \rfloor + 1\) or equivalently, \( n x - 1 \lt \lfloor n x \rfloor \le n x \) so it follows from the squeeze theorem that \( \left(1 - p_n \right)^{\lfloor n x \rfloor} \to e^{- r x} \) as \( n \to \infty \). Hence \( F_n(x) \to 1 - e^{-r x} \) as \( n \to \infty \). As a function of \(x \in [0, \infty), this is the CDF of the exponential distribution with parameter \(r\).
Note that the limiting condition on \(n\) and \(p\) in the last result is precisely the same as the condition for the convergence of the binomial distribution to the Poisson distribution. For a deeper interpretation of both of these results, see the section on the Poisson distribution.
In the negative binomial experiment, set \(k = 1\) to get the geometric distribution. Then decrease the value of \(p\) and note the shape of the probability density function. With \(p = 0.5\) run the experiment 1000 times and compare the relative frequency function to the probability density function.
In the gamma experiment, set \(k = 1\) to get the exponential distribution, and set \(r = 5\). Note the shape of the probability density function. Run the experiment 1000 times and compare the empirical density function and the probability density function. Compare this experiment with the one in the previous exercise, and note the similarity, up to a change in scale.
The Matching Distribution
For \(n \in \N_+\), consider a random permutation \((X_1, X_2, \ldots, X_n)\) of the elements in the set \(\{1, 2, \ldots, n\}\). We say that a match occurs at position \(i\) if \(X_i = i\).
\(\P\left(X_i = i\right) = \frac{1}{n}\) for each \(i \in \{1, 2, \ldots, n\}\).
Proof
The number of permutations of \(\{1, 2, \ldots, n\}\) is \(n!\). For \(i \in \{1, 2, \ldots, n\}\), the number of such permutations with \(i\) in position \(i\) is \((n - 1)!\). Hence \(\P(X_i = i) = (n - 1)! / n! = 1 / n\). A more direct argument is that \(i\) is no more or less likely to end up in position \(i\) as any other number.
So the matching events all have the same probability, which varies inversely with the number of trials.
\(\P\left(X_i = i, X_j = j\right) = \frac{1}{n (n - 1)}\) for \(i, \, j \in \{1, 2, \ldots, n\}\) with \(i \ne j\).
Proof
Again, the number of permutations of \(\{1, 2, \ldots, n\}\) is \(n!\). For distinct \(i, \, j \in \{1, 2, \ldots, n\}\), the number of such permutations with \(i\) in position \(i\) and \(j\) in position \(j\) is \((n - 2)!\). Hence \(\P(X_i = i, X_j = j) = (n - 2)! / n! = 1 / n (n - 1)\).
So the matching events are dependent, and in fact are positively correlated. In particular, the matching events do not form a sequence of Bernoulli trials. The matching problem is studied in detail in the chapter on Finite Sampling Models. In that section we show that the number of matches \(N_n\) has probability density function \(f_n\) given by: \[ f_n(k) = \frac{1}{k!} \sum_{j=0}^{n-k} \frac{(-1)^j}{j!}, \quad k \in \{0, 1, \ldots, n\} \]
The distribution of \(N_n\) converges to the Poisson distribution with parameter 1 as \(n \to \infty\).
Proof
For \( k \in \N \), \[ f_n(k) = \frac{1}{k!} \sum_{j=0}^{n-k} \frac{(-1)^j}{j!} \to \frac{1}{k!} \sum_{j=0}^\infty \frac{(-1)^j}{j!} = \frac{1}{k!} e^{-1} \] As a function of \(k \in \N\), this is the PDF of the Poisson distribution with parameter 1. So the result follows from the theorem above on density functions.
In the matching experiment, increase \(n\) and note the apparent convergence of the probability density function for the number of matches. With selected values of \(n\), run the experiment 1000 times and compare the relative frequency function and the probability density function.
The Extreme Value Distribution
Suppose that \((X_1, X_2, \ldots)\) is a sequence of independent random variables, each with the standard exponential distribution (parameter 1). Thus, recall that the common distribution function \(G\) is given by \[ G(x) = 1 - e^{-x}, \quad 0 \le x \lt \infty \]
As \(n \to \infty\), the distribution of \(Y_n = \max\{X_1, X_2, \ldots, X_n\} - \ln n \) converges to the distribution with distribution function \(F\) given by \[ F(x) = e^{-e^{-x}}, \quad x \in \R\]
Proof
Let \( X_{(n)} = \max\{X_1, X_2, \ldots, X_n\} \) and recall that \( X_{(n)} \) has CDF \( G^n \). Let \( F_n \) denote the CDF of \( Y_n \). For \( x \in \R \) \[ F_n(x) = \P(Y_n \le x) = \P\left(X_{(n)} \le x + \ln n \right) = G^n(x + \ln n) = \left[1 - e^{-(x + \ln n) }\right]^n = \left(1 - \frac{e^{-x}}{n} \right)^n \] By our famous limit from calculus again, \( F_n(x) \to e^{-e^{-x}} \) as \( n \to \infty \).
The limiting distribution in Exercise (27) is the standard extreme value distribution , also known as the standard Gumbel distribution in honor of Emil Gumbel. Extreme value distributions are studied in detail in the chapter on Special Distributions.
The Pareto Distribution
Recall that the Pareto distribution with shape parameter \(a \in (0, \infty)\) has distribution function \(F\) given by \[F(x) = 1 - \frac{1}{x^a}, \quad 1 \le x \lt \infty\] The Pareto distribution, named for Vilfredo Pareto, is a heavy-tailed distribution sometimes used to model financial variables. It is studied in more detail in the chapter on Special Distributions.
Suppose that \(X_n\) has the Pareto distribution with parameter \(n\) for each \(n \in \N_+\). Then
- \(X_n \to 1\) as \(n \to \infty\) in distribution (and hence also in probability).
- The distribution of \(Y_n = nX_n - n\) converges to the standard exponential distribution as \(n \to \infty\).
Proof
- The CDF of \( X_n \) is \( F_n(x) = 1 - 1 / x^n \) for \( x \ge 1 \). Hence \( F_n(x) = 0 \) for \( n \in \N_+ \) and \( x \le 1 \) while \( F_n(x) \to 1 \) as \( n \to \infty \) for \( x \gt 1 \). Thus the limit of \( F_n \) agrees with the CDF of the constant 1, except at \(x = 1\), the point of discontinuity.
- Let \( G_n \) denote the CDF of \( Y_n \). For \( x \ge 0 \), \[ G_n(x) = \P(Y_n \le x) = \P(X_n \le 1 + x / n) = 1 - \frac{1}{(1 + x / n)^n} \] By our famous theorem from calculus again, it follows that \( G_n(x) \to 1 - 1 / e^x = 1 - e^{-x} \) as \( n \to \infty \). As a function of \(x \in [0, \infty\), this is the CDF of the standard exponential distribution.
Fundamental Theorems
The two fundamental theorems of basic probability theory, the law of large numbers and the central limit theorem, are studied in detail in the chapter on Random Samples. For this reason we will simply state the results in this section. So suppose that \((X_1, X_2, \ldots)\) is a sequence of independent, identically distributed, real-valued random variables (defined on the same probability space) with mean \(\mu \in (-\infty. \infty)\) and standard deviation \(\sigma \in (0, \infty)\). For \(n \in \N_+\), let \( Y_n = \sum_{i=1}^n X_i \) denote the sum of the first \(n\) variables, \( M_n = Y_n \big/n \) the average of the first \( n \) variables, and \( Z_n = (Y_n - n \mu) \big/ \sqrt{n} \sigma \) the standard score of \( Y_n \).
The fundamental theorems of probability
- \( M_n \to \mu \) as \( n \to \infty \) with probability 1 (and hence also in probability and in distribution). This is the law of large numbers .
- The distribution of \( Z_n \) converges to the standard normal distribution as \( n \to \infty \). This is the central limit theorem .
In part (a), convergence with probability 1 is the strong law of large numbers while convergence in probability and in distribution are the weak laws of large numbers .
General Spaces
Our next goal is to define convergence of probability distributions on more general measurable spaces. For this discussion, you may need to refer to other sections in this chapter: the integral with respect to a positive measure, properties of the integral, and density functions. In turn, these sections depend on measure theory developed in the chapters on Foundations and Probability Measures.
Definition and Basic Properties
First we need to define the type of measurable spaces that we will use in this subsection.
We assume that \((S, d)\) is a complete, separable metric space and let \(\mathscr S\) denote the Borel \(\sigma\)-algebra of subsets of \(S\), that is, the \(\sigma\)-algebra generated by the topology. The standard spaces that we often use are special cases of the measurable space \((S, \mathscr S)\):
- Discrete : \(S\) is countable and is given the discrete metric so \(\mathscr S\) is the collection of all subsets of \(S\).
- Euclidean : \(\R^n\) is given the standard Euclidean metric so \(\mathscr R_n\) is the usual \(\sigma\)-algebra of Borel measurable subsets of \(\R^n\).
Additional details
Recall that the metric space \((S, d)\) is complete if every Cauchy sequence in \(S\) converges to a point in \(S\). The space is separable if there exists a coutable subset that is dense. A complete, separable metric space is sometimes called a Polish space because such spaces were extensively studied by a group of Polish mathematicians in the 1930s, including Kazimierz Kuratowski.
As suggested by our setup, the definition for convergence in distribution involves both measure theory and topology. The motivation is the theorem above for the one-dimensional Euclidean space \((\R, \mathscr R)\).
Convergence in distribution :
- Suppose that \(P_n\) is a probability measure on \((S, \mathscr S)\) for each \(n \in \N_+^*\). Then \(P_n\) converges (weakly) to \(P_\infty\) as \(n \to \infty\) if \(P_n(A) \to P_\infty(A)\) as \(n \to \infty\) for every \(A \in \mathscr S\) with \(P_\infty(\partial A) = 0\). We write \(P_n \Rightarrow P_\infty\) as \(n \to \infty\).
- Suppose that \(X_n\) is a random variable with distribution \(P_n\) on \((S, \mathscr S)\) for each \(n \in \N_+^*\). Then \(X_n\) converges in distribution to \(X_\infty\) as \(n \to \infty\) if \(P_n \Rightarrow P_\infty\) as \(n \to \infty\). We write \(X_n \to X_\infty\) as \(n \to \infty\) in distribution.
Notes
- The definition makes sense since \(A \in \mathscr S\) implies \(\partial A \in \mathscr S\). Specifically, \(\cl(A) \in \mathscr S\) because \(\cl(A)\) is closed, and \(\interior(A) \in \mathscr S\) because \(\interior(A)\) is open.
- The random variables need not be defined on the same probability space.
Let's consider our two special cases. In the discrete case, as usual, the measure theory and topology are not really necessary.
Suppose that \(P_n\) is a probability measures on a discrete space \((S, \mathscr S)\) for each \(n \in \N_+^*\). Then \(P_n \Rightarrow P_\infty\) as \(n \to \infty\) if and only if \(P_n(A) \to P_\infty(A)\) as \(n \to \infty\) for every \(A \subseteq S\).
Proof
This follows from the definition. Every subset is both open and closed so \(\partial A = \emptyset\) for every \(A \subseteq S\).
In the Euclidean case, it suffices to consider distribution functions, as in the one-dimensional case. If \(P\) is a probability measure on \((\R^n, \mathscr R_n)\), recall that the distribution function \(F\) of \(P\) is given by \[F(x_1, x_2, \ldots, x_n) = P\left((-\infty, x_1] \times (-\infty, x_2] \times \cdots \times (-\infty, x_n]\right), \quad (x_1, x_2, \ldots, x_n) \in \R^n\]
Suppose that \(P_n\) is a probability measures on \((\R^n, \mathscr R_n)\) with distribution function \(F_n\) for each \(n \in \N_+^*\). Then \(P_n \Rightarrow P_\infty\) as \(n \to \infty\) if and only if \(F_n(\bs x) \to F_\infty(\bs x)\) as \(n \to \infty\) for every \(\bs x \in \R^n\) where \(F_\infty\) is continuous.
Convergence in Probability
As in the case of \((\R, \mathscr R)\), convergence in probability implies convergence in distribution.
Suppose that \(X_n\) is a random variable with values in \(S\) for each \(n \in \N_+^*\), all defined on the same probability space. If \(X_n \to X_\infty\) as \(n \to \infty\) in probability then \(X_n \to X_\infty\) as \(n \to \infty\) in distribution.
Notes
Assume that the common probability space is \((\Omega, \mathscr F, \P)\). Recall that convergence in probability means that \(\P[d(X_n, X_\infty) \gt \epsilon] \to 0\) as \(n \to \infty\) for every \(\epsilon \gt 0\),
So as before, convergence with probability 1 implies convergence in probability which in turn implies convergence in distribution.
Skorohod's Representation Theorem
As you might guess, Skorohod's theorem for the one-dimensional Euclidean space \((\R, \mathscr R)\) can be extended to the more general spaces. However the proof is not nearly as straightforward, because we no longer have the quantile function for constructing random variables on a common probability space.
Suppose that \(P_n\) is a probability measures on \((S, \mathscr S)\) for each \(n \in \N_+^*\) and that \(P_n \Rightarrow P_\infty\) as \(n \to \infty\). Then there exists a random variable \(X_n\) with values in \(S\) for each \(n \in \N_+^*\), defined on a common probability space, such that
- \(X_n\) has distribution \(P_n\) for \(n \in \N_+^*\)
- \(X_n \to X_\infty\) as \(n \to \infty\) with probability 1.
One of the main consequences of Skorohod's representation, the preservation of convergence in distribution under continuous functions, is still true and has essentially the same proof. For the general setup, suppose that \((S, d, \mathscr S)\) and \((T, e, \mathscr T)\) are spaces of the type described above .
Suppose that \(X_n\) is a random variable with values in \(S\) for each \(n \in \N_+^*\) (not necessarily defined on the same probability space). Suppose also that \(g: S \to T\) is measurable, and let \(D_g\) denote the set of discontinuities of \(g\), and \(P_\infty\) the distribution of \(X_\infty\). If \(X_n \to X_\infty\) as \(n \to \infty\) in distribution and \(P_\infty(D_g) = 0\), then \(g(X_n) \to g(X_\infty)\) as \(n \to \infty\) in distribution.
Proof
By Skorohod's theorem, there exists random variables \(Y_n\) with values in \(S\) for \(n \in \N_+^*\), defined on the same probability space \((\Omega, \mathscr F, \P)\), such that \(Y_n\) has the same distribution as \(X_n\) for \(n \in \N_+^*\), and \(Y_n \to Y_\infty\) as \(n \to \infty\) with probability 1. Since \(\P(Y_\infty \in D_g) = P_\infty(D_g) = 0\) it follows that \(g(Y_n) \to g(Y_\infty)\) as \(n \to \infty\) with probability 1. Hence \(g(Y_n) \to g(Y_\infty)\) as \(n \to \infty\) in distribution. But \(g(Y_n)\) has the same distribution as \(g(X_n)\) for each \(n \in \N_+^*\).
A simple consequence of the continuity theorem is that if a sequence of random vectors in \(\R^n\) converge in distribution, then the sequence of each coordinate also converges in distribution. Let's just consider the two-dimensional case to keep the notation simple.
Suppose that \((X_n, Y_n)\) is a random variable with values in \(\R^2\) for \(n \in \N_+^*\) and that \((X_n, Y_n) \to (X_\infty, Y_\infty)\) as \(n \to \infty\) in distribution. Then
- \(X_n \to X_\infty\) as \(n \to \infty\) in distribution.
- \(Y_n \to Y_\infty\) as \(n \to \infty\) in distribution.
Scheffé's Theorem
Our next discussion concerns an important result known as Scheffé's theorem , named after Henry Scheffé. To state our theorem, suppose that \( (S, \mathscr S, \mu) \) is a measure space, so that \( S \) is a set, \( \mathscr S \) is a \( \sigma \)-algebra of subsets of \( S \), and \( \mu \) is a positive measure on \( (S, \mathscr S) \). Further, suppose that \( P_n \) is a probability measure on \( (S, \mathscr S) \) that has density function \( f_n \) with respect to \( \mu \) for each \( n \in \N_+ \), and that \( P \) is a probability measure on \( (S, \mathscr S) \) that has density function \( f \) with respect to \( \mu \).
If \(f_n(x) \to f(x)\) as \(n \to \infty\) for almost all \( x \in S \) (with respect to \( \mu \)) then \(P_n(A) \to P(A)\) as \(n \to \infty\) uniformly in \(A \in \mathscr S\).
Proof
From basic properties of the integral it follows that for \( A \in \mathscr S \), \[\left|P(A) - P_n(A)\right| = \left|\int_A f \, d\mu - \int_A f_n \, d\mu \right| = \left| \int_A (f - f_n) \, d\mu\right| \le \int_A \left|f - f_n\right| \, d\mu \le \int_S \left|f - f_n\right| \, d\mu\] Let \(g_n = f - f_n\), and let \(g_n^+\) denote the positive part of \(g_n\) and \(g_n^-\) the negative part of \(g_n\). Note that \(g_n^+ \le f\) and \(g_n^+ \to 0\) as \(n \to \infty\) almost everywhere on \( S \).  But \(\int_\R g_n \, d\mu = 0\) so \(\int_\R g_n^+ \, d\mu = \int_\R g_n^- \, d\mu\). Therefore \(\int_S \left|g_n\right| \, d\mu = 2 \int_S g_n^+ d\mu \to 0\) as \(n \to \infty\). Hence \(P_n(A) \to P(A)\) as \(n \to \infty\) uniformly in \(A \in \mathscr S\).
Of course, the most important special cases of Scheffé's theorem are to discrete distributions and to continuous distributions on a subset of \( \R^n \), as in the theorem above on density functions.
Expected Value
Generating functions are studied in the chapter on Expected Value. In part, the importance of generating functions stems from the fact that ordinary (pointwise) convergence of a sequence of generating functions corresponds to the convergence of the distributions in the sense of this section. Often it is easier to show convergence in distribution using generating functions than directly from the definition.
In addition, converence in distribution has elegant characterizations in terms of the convergence of the expected values of certain types of functions of the underlying random variables.


1.1: First Aid and Personal Protective Equipment
First Things First
Job site accidents and injuries as a result of tools and equipment being misused or failing are quite common. Cuts and punctures from sharp objects, contusions from blunt objects or impacts, burns from open flame torches and hot pipes, splashing of chemicals or debris to the eyes, and electrical shock are just a few of the common injuries associated with the building maintenance trade.
As construction and maintenance processes offer the potential for many types of traumatic and life threatening injuries, workers in skilled trades should be aware of the hazards and be prepared to respond in the event of an injury. While many industry tasks are performed by a single person, often isolated from others, it is recommended that industry workers receive First Aid and Cardiopulmonary Resuscitation (CPR), and Occupational Safety and Health Administration (OSHA) 10 or 30 Hour Training for Construction certificates. Training will enable industry trades-persons to better assess workplace hazards and respond to them appropriately, whether an incident involves yourself, a teammate, or others on the job site. In person, hands-on First Aid/CPR training can be found through local health and welfare organizations, educational institutions (credit or non-credit), and medical providers. OSHA in person courses can be found at local educational institutions (credit or non-credit), and in online formats through various educational institutions and commercial providers.
First Aid Kit
Although we all hope that we never need one, a first aid kit should be kept on the service vehicle or on the job site at all times. Be sure your first aid kit has you prepared for the type of injuries connected to your field of work. The size of the first aid kit should reflect the number of employees kit is intended to service. Most commercial first aid kits are rated by the amount of people or employees to be served.


4.2: Timbre
- A short introduction to the basic element of music known as "color" or "timbre".
One of the basic elements of music is called color, or timbre (pronounced "TAM-ber"). Timbre describes all of the aspects of a musical sound that do not have anything to do with the sound's pitch, loudness, or length. In other words, if a flute plays a note, and then an oboe plays the same note, for the same length of time, at the same loudness, you can still easily distinguish between the two sounds, because a flute sounds different from an oboe. This difference is in the timbre of the sounds.
Timbre is caused by the fact that each note from a musical instrument is a complex wave containing more than one frequency. For instruments that produce notes with a clear and specific pitch, the frequencies involved are part of a harmonic series. For other instruments (such as drums), the sound wave may have an even greater variety of frequencies. We hear each mixture of frequencies not as separate sounds, but as the color of the sound. Small differences in the balance of the frequencies - how many you can hear, their relationship to the fundamental pitch, and how loud they are compared to each other - create the many different musical colors.
The harmonics at the beginning of each note - the attack - are especially important for timbre, so it is actually easier to identify instruments that are playing short notes with strong articulations than it is to identify instruments playing long, smooth notes.
The human ear and brain are capable of hearing and appreciating very small variations in timbre. A listener can hear not only the difference between an oboe and a flute, but also the difference between two different oboes. The general sound that one would expect of a type of instrument - a trombone for example - is usually called its timbre or color. Variations in timbre between specific instruments - two different trombones, for example, or two different trombone players, or the same trombone player using different types of sound in different pieces - may be called differences in timbre or color, or may be called differences in tone or in tone quality. Tone quality may refer specifically to "quality", as when a young trombonist is encouraged to have a "fuller" or "more focussed" tone quality, or it can refer neutrally to differences in sound, as when an orchestral trombonist is asked to play with a "brassy" tone quality in one passage and a "mellow" tone quality in another.
Many words are used to describe timbre. Some are somewhat interchangeable, and some may have slightly different meanings for different musicians, so no attempt will be made to provide definitions. Here are a few words commonly used to describe either timbre or tone quality.
- Reedy
- Brassy
- Clear
- Focussed or unfocussed
- Breathy (pronounced "BRETH-ee")
- Rounded
- Piercing
- Strident
- Harsh
- Warm
- Mellow
- Resonant
- Dark or Bright
- Heavy or Light
- Flat
- Having much, little, or no vibrato (a controlled wavering in the sound); or narrow or wide, or slow or fast, vibrato
For more information on what causes timbre, please see Harmonic Series I, Standing Waves and Musical Instruments, and Standing Waves and Wind Instruments.) For activities that introduce children to the concept of timbre, please see Timbre Activities


By considering the difference between grammatical roles and thematic roles, we saw that the syntax and the semantics of sentences are represented differently from each other in our minds. We also have evidence from neural imaging that syntactic knowledge and semantic knowledge are processed differently in the brain.
A YouTube element has been excluded from this version of the text. You can view it online here: ecampusontario.pressbooks.pub/essentialsoflinguistics/?p=203
Check Yourself
1. What kind of ERP is likely to be observed in response to this sentence?
The children’s feet have grown so I bought them some new
spoons
.
2. What kind of ERP is likely to be observed in response to this sentence?
Elias told the coach that he wanted to learn to
swam
.
3. What kind of ERP is likely to be observed in response to this sentence?
Before the dinner party, Isla spent a long time in the
cooked
.
Answers
Video Script
When we started talking about semantics, we observed that a sentence’s syntax influences its semantics, because of the principle of
compositionality
. For example, we saw that a given string of words can have two different meanings if it has two different grammatical syntactic structures. And yet, we also observed that syntax is independent of syntax. A noun phrase that has the semantic thematic role of Agent often occupies the syntactic position of Subject, but not all Agents are Subjects, and not all Subjects are Agents!
The division of labour between thematic roles and grammatical roles is some evidence that syntax and semantics are represented differently in our minds. There’s also evidence from
neural imaging
to show that our brains process semantic information differently from syntactic information. This evidence comes from
electroencephalography
or
EEG
. Electroencephalography uses electrodes to measure electrical activity on a person’s scalp from which scientists can draw conclusions about the person’s neural activity. The particular EEG technique that gets used in neurolinguistics is
ERPs
or
event-related potentials
, which measure the timing of the neural response to a particular event, like a sound or a word.
When we’re observing ERPs, we always do so by comparing responses to different kinds of events, and the usual comparison is between events that are expected and events that are unexpected. For example, a sentence like, “She takes her coffee with cream and …” sets up a very strong expectation in your mind of what the next word will be. If the next word that arrives in the sentence matches your mind’s expectation, then the electrical response at your scalp will look something like this: the baseline condition. But if the next word that shows up violates your mind’s expectation, then compare your brain’s response: We observe a spike in negative voltage about 400 milliseconds after that unexpected word appears.
This response is called an
N400
. The N in N400 stands for a negative voltage, and the 400 indicates that this spike in negative voltage shows up, on average, about 400 milliseconds after the event. The N400 was first observed in 1980 by Kutas & Hillyard and has been replicated hundreds of times since then. It’s clear from all these studies that the particular kind of event that leads to an N400 response is a word that is unexpected in the semantic context.
The N400 is the brain’s response to an unexpected or surprising event, but not every kind of surprise will produce an N400. In other words, we have expectations about things besides the meanings of sentences. Think about a simple sentence like, “The bread was…” If that sentence finishes with “eaten”, that fits our mind’s expectation, and this is the baseline brain response. Now, what expectation do you have for this sentence, “The ice cream was in the..”? You probably expect a noun to come next, to follow the preposition and determiner. But if what comes next is not a noun but a verb participle, this violates your mind’s expectation. Notice that the word
eaten
is semantically consistent with ice cream, but is not consistent with the syntax of the sentence: determiners are followed by nouns, not verbs. So the brain’s response is a positive voltage about 600 milliseconds after that unexpected word: a
P600
.
When we’re using language in real time — either reading or listening — our mind sets up expectations about what’s going to happen next. If what happens next violates our semantic expectations, the brain’s response is an N400. And if what happens next violates our syntactic expectations, the brain’s response is a P600.
These two different brain responses give us further evidence that syntax is independent of semantics in our brains!


2.8D: Synovial Joints
-
- Last updated
- Save as PDF
Learning Objectives
- Describe the structural features of a synovial joint
- Discuss the function of additional structures associated with synovial joints
- List the six types of synovial joints and give an example of each
Synovial joints are the most common type of joint in the body (Figure \(\PageIndex{1}\)). A key structural characteristic for a synovial joint that is not seen at fibrous or cartilaginous joints is the presence of a joint cavity. This fluid-filled space is the site at which the articulating surfaces of the bones contact each other. Also unlike fibrous or cartilaginous joints, the articulating bone surfaces at a synovial joint are not directly connected to each other with fibrous connective tissue or cartilage. This gives the bones of a synovial joint the ability to move smoothly against each other, allowing for increased joint mobility.
Structural Features of Synovial Joints
Synovial joints are characterized by the presence of a joint cavity. The walls of this space are formed by the articular capsule , a fibrous connective tissue structure that is attached to each bone just outside the area of the bone’s articulating surface. The bones of the joint articulate with each other within the joint cavity.
Friction between the bones at a synovial joint is prevented by the presence of the articular cartilage , a thin layer of hyaline cartilage that covers the entire articulating surface of each bone. However, unlike at a cartilaginous joint, the articular cartilages of each bone are not continuous with each other. Instead, the articular cartilage acts like a Teflon ® coating over the bone surface, allowing the articulating bones to move smoothly against each other without damaging the underlying bone tissue. Lining the inner surface of the articular capsule is a thin synovial membrane . The cells of this membrane secrete synovial fluid (synovia = “a thick fluid”), a thick, slimy fluid that provides lubrication to further reduce friction between the bones of the joint. This fluid also provides nourishment to the articular cartilage, which does not contain blood vessels. The ability of the bones to move smoothly against each other within the joint cavity, and the freedom of joint movement this provides, means that each synovial joint is functionally classified as a diarthrosis.
Outside of their articulating surfaces, the bones are connected together by ligaments, which are strong bands of fibrous connective tissue. These strengthen and support the joint by anchoring the bones together and preventing their separation. Ligaments allow for normal movements at a joint, but limit the range of these motions, thus preventing excessive or abnormal joint movements. Ligaments are classified based on their relationship to the fibrous articular capsule. An extrinsic ligament is located outside of the articular capsule, an intrinsic ligament is fused to or incorporated into the wall of the articular capsule, and an intracapsular ligament is located inside of the articular capsule.
At many synovial joints, additional support is provided by the muscles and their tendons that act across the joint. A tendon is the dense connective tissue structure that attaches a muscle to bone. As forces acting on a joint increase, the body will automatically increase the overall strength of contraction of the muscles crossing that joint, thus allowing the muscle and its tendon to serve as a “dynamic ligament” to resist forces and support the joint. This type of indirect support by muscles is very important at the shoulder joint, for example, where the ligaments are relatively weak.
Additional Structures Associated with Synovial Joints
A few synovial joints of the body have a fibrocartilage structure located between the articulating bones. This is called an articular disc , which is generally small and oval-shaped, or a meniscus , which is larger and C-shaped. These structures can serve several functions, depending on the specific joint. In some places, an articular disc may act to strongly unite the bones of the joint to each other. Examples of this include the articular discs found at the sternoclavicular joint or between the distal ends of the radius and ulna bones. At other synovial joints, the disc can provide shock absorption and cushioning between the bones, which is the function of each meniscus within the knee joint. Finally, an articular disc can serve to smooth the movements between the articulating bones, as seen at the temporomandibular joint. Some synovial joints also have a fat pad, which can serve as a cushion between the bones.
Additional structures located outside of a synovial joint serve to prevent friction between the bones of the joint and the overlying muscle tendons or skin. A bursa (plural = bursae) is a thin connective tissue sac filled with lubricating liquid. They are located in regions where skin, ligaments, muscles, or muscle tendons can rub against each other, usually near a body joint (Figure \(\PageIndex{2}\)). Bursae reduce friction by separating the adjacent structures, preventing them from rubbing directly against each other. Bursae are classified by their location. A subcutaneous bursa is located between the skin and an underlying bone. It allows skin to move smoothly over the bone. Examples include the prepatellar bursa located over the kneecap and the olecranon bursa at the tip of the elbow. A submuscular bursa is found between a muscle and an underlying bone, or between adjacent muscles. These prevent rubbing of the muscle during movements. A large submuscular bursa, the trochanteric bursa, is found at the lateral hip, between the greater trochanter of the femur and the overlying gluteus maximus muscle. A subtendinous bursa is found between a tendon and a bone. Examples include the subacromial bursa that protects the tendon of shoulder muscle as it passes under the acromion of the scapula, and the suprapatellar bursa that separates the tendon of the large anterior thigh muscle from the distal femur just above the knee.
Figure \(\PageIndex{2}\): Bursae. Bursae are fluid-filled sacs that serve to prevent friction between skin, muscle, or tendon and an underlying bone. Three major bursae and a fat pad are part of the complex joint that unites the femur and tibia of the leg.
A tendon sheath is similar in structure to a bursa, but smaller. It is a connective tissue sac that surrounds a muscle tendon at places where the tendon crosses a joint. It contains a lubricating fluid that allows for smooth motions of the tendon during muscle contraction and joint movements.
HOMEOSTATIC IMBALANCES
Bursitis
Bursitis is the inflammation of a bursa near a joint. This will cause pain, swelling, or tenderness of the bursa and surrounding area, and may also result in joint stiffness. Bursitis is most commonly associated with the bursae found at or near the shoulder, hip, knee, or elbow joints. At the shoulder, subacromial bursitis may occur in the bursa that separates the acromion of the scapula from the tendon of a shoulder muscle as it passes deep to the acromion. In the hip region, trochanteric bursitis can occur in the bursa that overlies the greater trochanter of the femur, just below the lateral side of the hip. Ischial bursitis occurs in the bursa that separates the skin from the ischial tuberosity of the pelvis, the bony structure that is weight bearing when sitting. At the knee, inflammation and swelling of the bursa located between the skin and patella bone is prepatellar bursitis (“housemaid’s knee”), a condition more commonly seen today in roofers or floor and carpet installers who do not use knee pads. At the elbow, olecranon bursitis is inflammation of the bursa between the skin and olecranon process of the ulna. The olecranon forms the bony tip of the elbow, and bursitis here is also known as “student’s elbow.”
Bursitis can be either acute (lasting only a few days) or chronic. It can arise from muscle overuse, trauma, excessive or prolonged pressure on the skin, rheumatoid arthritis, gout, or infection of the joint. Repeated acute episodes of bursitis can result in a chronic condition. Treatments for the disorder include antibiotics if the bursitis is caused by an infection, or anti-inflammatory agents, such as nonsteroidal anti-inflammatory drugs (NSAIDs) or corticosteroids if the bursitis is due to trauma or overuse. Chronic bursitis may require that fluid be drained, but additional surgery is usually not required.
Types of Synovial Joints
Synovial joints are subdivided based on the shapes of the articulating surfaces of the bones that form each joint. The six types of synovial joints are pivot, hinge, condyloid, saddle, plane, and ball-and socket-joints (Figure \(\PageIndex{3}\)).
Pivot Joint
At a pivot joint , a rounded portion of a bone is enclosed within a ring formed partially by the articulation with another bone and partially by a ligament (see Figure \(\PageIndex{3}\).a). The bone rotates within this ring. Since the rotation is around a single axis, pivot joints are functionally classified as a uniaxial diarthrosis type of joint. An example of a pivot joint is the atlantoaxial joint, found between the C1 (atlas) and C2 (axis) vertebrae. Here, the upward projecting dens of the axis articulates with the inner aspect of the atlas, where it is held in place by a ligament. Rotation at this joint allows you to turn your head from side to side. A second pivot joint is found at the proximal radioulnar joint . Here, the head of the radius is largely encircled by a ligament that holds it in place as it articulates with the radial notch of the ulna. Rotation of the radius allows for forearm movements.
Hinge Joint
In a hinge joint , the convex end of one bone articulates with the concave end of the adjoining bone (see Figure9.4.3.b). This type of joint allows only for bending and straightening motions along a single axis, and thus hinge joints are functionally classified as uniaxial joints. A good example is the elbow joint, with the articulation between the trochlea of the humerus and the trochlear notch of the ulna. Other hinge joints of the body include the knee, ankle, and interphalangeal joints between the phalanx bones of the fingers and toes.
Condyloid Joint
At a condyloid joint (ellipsoid joint), the shallow depression at the end of one bone articulates with a rounded structure from an adjacent bone or bones (see Figure9.4.3.e). The knuckle (metacarpophalangeal) joints of the hand between the distal end of a metacarpal bone and the proximal phalanx bone are condyloid joints. Another example is the radiocarpal joint of the wrist, between the shallow depression at the distal end of the radius bone and the rounded scaphoid, lunate, and triquetrum carpal bones. In this case, the articulation area has a more oval (elliptical) shape. Functionally, condyloid joints are biaxial joints that allow for two planes of movement. One movement involves the bending and straightening of the fingers or the anterior-posterior movements of the hand. The second movement is a side-to-side movement, which allows you to spread your fingers apart and bring them together, or to move your hand in a medial-going or lateral-going direction.
Saddle Joint
At a saddle joint , both of the articulating surfaces for the bones have a saddle shape, which is concave in one direction and convex in the other (see Figure9.4.3.c). This allows the two bones to fit together like a rider sitting on a saddle. Saddle joints are functionally classified as biaxial joints. The primary example is the first carpometacarpal joint, between the trapezium (a carpal bone) and the first metacarpal bone at the base of the thumb. This joint provides the thumb the ability to move away from the palm of the hand along two planes. Thus, the thumb can move within the same plane as the palm of the hand, or it can jut out anteriorly, perpendicular to the palm. This movement of the first carpometacarpal joint is what gives humans their distinctive “opposable” thumbs. The sternoclavicular joint is also classified as a saddle joint.
Plane Joint
At a plane joint (gliding joint), the articulating surfaces of the bones are flat or slightly curved and of approximately the same size, which allows the bones to slide against each other (see Figure9.4.3.d). The motion at this type of joint is usually small and tightly constrained by surrounding ligaments. Based only on their shape, plane joints can allow multiple movements, including rotation. Thus plane joints can be functionally classified as a multiaxial joint. However, not all of these movements are available to every plane joint due to limitations placed on it by ligaments or neighboring bones. Thus, depending upon the specific joint of the body, a plane joint may exhibit only a single type of movement or several movements. Plane joints are found between the carpal bones (intercarpal joints) of the wrist or tarsal bones (intertarsal joints) of the foot, between the clavicle and acromion of the scapula (acromioclavicular joint), and between the superior and inferior articular processes of adjacent vertebrae (zygapophysial joints).
Ball-and-Socket Joint
The joint with the greatest range of motion is the ball-and-socket joint . At these joints, the rounded head of one bone (the ball) fits into the concave articulation (the socket) of the adjacent bone (see Figure9.4.3.f). The hip joint and the glenohumeral (shoulder) joint are the only ball-and-socket joints of the body. At the hip joint, the head of the femur articulates with the acetabulum of the hip bone, and at the shoulder joint, the head of the humerus articulates with the glenoid cavity of the scapula.
Ball-and-socket joints are classified functionally as multiaxial joints. The femur and the humerus are able to move in both anterior-posterior and medial-lateral directions and they can also rotate around their long axis. The shallow socket formed by the glenoid cavity allows the shoulder joint an extensive range of motion. In contrast, the deep socket of the acetabulum and the strong supporting ligaments of the hip joint serve to constrain movements of the femur, reflecting the need for stability and weight-bearing ability at the hip.
Watch this video to see an animation of synovial joints in action. Synovial joints are places where bones articulate with each other inside of a joint cavity. The different types of synovial joints are the ball-and-socket joint (shoulder joint), hinge joint (knee), pivot joint (atlantoaxial joint, between C1 and C2 vertebrae of the neck), condyloid joint (radiocarpal joint of the wrist), saddle joint (first carpometacarpal joint, between the trapezium carpal bone and the first metacarpal bone, at the base of the thumb), and plane joint (facet joints of vertebral column, between superior and inferior articular processes). Which type of synovial joint allows for the widest range of motion?
Joints
Arthritis is a common disorder of synovial joints that involves inflammation of the joint. This often results in significant joint pain, along with swelling, stiffness, and reduced joint mobility. There are more than 100 different forms of arthritis. Arthritis may arise from aging, damage to the articular cartilage, autoimmune diseases, bacterial or viral infections, or unknown (probably genetic) causes.
The most common type of arthritis is osteoarthritis, which is associated with aging and “wear and tear” of the articular cartilage (Figure \(\PageIndex{4}\)). Risk factors that may lead to osteoarthritis later in life include injury to a joint; jobs that involve physical labor; sports with running, twisting, or throwing actions; and being overweight. These factors put stress on the articular cartilage that covers the surfaces of bones at synovial joints, causing the cartilage to gradually become thinner. As the articular cartilage layer wears down, more pressure is placed on the bones. The joint responds by increasing production of the lubricating synovial fluid, but this can lead to swelling of the joint cavity, causing pain and joint stiffness as the articular capsule is stretched. The bone tissue underlying the damaged articular cartilage also responds by thickening, producing irregularities and causing the articulating surface of the bone to become rough or bumpy. Joint movement then results in pain and inflammation. In its early stages, symptoms of osteoarthritis may be reduced by mild activity that “warms up” the joint, but the symptoms may worsen following exercise. In individuals with more advanced osteoarthritis, the affected joints can become more painful and therefore are difficult to use effectively, resulting in increased immobility. There is no cure for osteoarthritis, but several treatments can help alleviate the pain. Treatments may include lifestyle changes, such as weight loss and low-impact exercise, and over-the-counter or prescription medications that help to alleviate the pain and inflammation. For severe cases, joint replacement surgery (arthroplasty) may be required.
Joint replacement is a very invasive procedure, so other treatments are always tried before surgery. However arthroplasty can provide relief from chronic pain and can enhance mobility within a few months following the surgery. This type of surgery involves replacing the articular surfaces of the bones with prosthesis (artificial components). For example, in hip arthroplasty, the worn or damaged parts of the hip joint, including the head and neck of the femur and the acetabulum of the pelvis, are removed and replaced with artificial joint components. The replacement head for the femur consists of a rounded ball attached to the end of a shaft that is inserted inside the diaphysis of the femur. The acetabulum of the pelvis is reshaped and a replacement socket is fitted into its place. The parts, which are always built in advance of the surgery, are sometimes custom made to produce the best possible fit for a patient.
Gout is a form of arthritis that results from the deposition of uric acid crystals within a body joint. Usually only one or a few joints are affected, such as the big toe, knee, or ankle. The attack may only last a few days, but may return to the same or another joint. Gout occurs when the body makes too much uric acid or the kidneys do not properly excrete it. A diet with excessive fructose has been implicated in raising the chances of a susceptible individual developing gout.
Other forms of arthritis are associated with various autoimmune diseases, bacterial infections of the joint, or unknown genetic causes. Autoimmune diseases, including rheumatoid arthritis, scleroderma, or systemic lupus erythematosus, produce arthritis because the immune system of the body attacks the body joints. In rheumatoid arthritis, the joint capsule and synovial membrane become inflamed. As the disease progresses, the articular cartilage is severely damaged or destroyed, resulting in joint deformation, loss of movement, and severe disability. The most commonly involved joints are the hands, feet, and cervical spine, with corresponding joints on both sides of the body usually affected, though not always to the same extent. Rheumatoid arthritis is also associated with lung fibrosis, vasculitis (inflammation of blood vessels), coronary heart disease, and premature mortality. With no known cure, treatments are aimed at alleviating symptoms. Exercise, anti-inflammatory and pain medications, various specific disease-modifying anti-rheumatic drugs, or surgery are used to treat rheumatoid arthritis.
Visit this website to learn about a patient who arrives at the hospital with joint pain and weakness in his legs. What caused this patient’s weakness?
Watch this animation to observe hip replacement surgery (total hip arthroplasty), which can be used to alleviate the pain and loss of joint mobility associated with osteoarthritis of the hip joint. What is the most common cause of hip disability?
Watch this video to learn about the symptoms and treatments for rheumatoid arthritis. Which system of the body malfunctions in rheumatoid arthritis and what does this cause?
Chapter Review
Synovial joints are the most common type of joints in the body. They are characterized by the presence of a joint cavity, inside of which the bones of the joint articulate with each other. The articulating surfaces of the bones at a synovial joint are not directly connected to each other by connective tissue or cartilage, which allows the bones to move freely against each other. The walls of the joint cavity are formed by the articular capsule. Friction between the bones is reduced by a thin layer of articular cartilage covering the surfaces of the bones, and by a lubricating synovial fluid, which is secreted by the synovial membrane.
Synovial joints are strengthened by the presence of ligaments, which hold the bones together and resist excessive or abnormal movements of the joint. Ligaments are classified as extrinsic ligaments if they are located outside of the articular capsule, intrinsic ligaments if they are fused to the wall of the articular capsule, or intracapsular ligaments if they are located inside the articular capsule. Some synovial joints also have an articular disc (meniscus), which can provide padding between the bones, smooth their movements, or strongly join the bones together to strengthen the joint. Muscles and their tendons acting across a joint can also increase their contractile strength when needed, thus providing indirect support for the joint.
Bursae contain a lubricating fluid that serves to reduce friction between structures. Subcutaneous bursae prevent friction between the skin and an underlying bone, submuscular bursae protect muscles from rubbing against a bone or another muscle, and a subtendinous bursa prevents friction between bone and a muscle tendon. Tendon sheaths contain a lubricating fluid and surround tendons to allow for smooth movement of the tendon as it crosses a joint.
Based on the shape of the articulating bone surfaces and the types of movement allowed, synovial joints are classified into six types. At a pivot joint, one bone is held within a ring by a ligament and its articulation with a second bone. Pivot joints only allow for rotation around a single axis. These are found at the articulation between the C1 (atlas) and the dens of the C2 (axis) vertebrae, which provides the side-to-side rotation of the head, or at the proximal radioulnar joint between the head of the radius and the radial notch of the ulna, which allows for rotation of the radius during forearm movements. Hinge joints, such as at the elbow, knee, ankle, or interphalangeal joints between phalanx bones of the fingers and toes, allow only for bending and straightening of the joint. Pivot and hinge joints are functionally classified as uniaxial joints.
Condyloid joints are found where the shallow depression of one bone receives a rounded bony area formed by one or two bones. Condyloid joints are found at the base of the fingers (metacarpophalangeal joints) and at the wrist (radiocarpal joint). At a saddle joint, the articulating bones fit together like a rider and a saddle. An example is the first carpometacarpal joint located at the base of the thumb. Both condyloid and saddle joints are functionally classified as biaxial joints.
Plane joints are formed between the small, flattened surfaces of adjacent bones. These joints allow the bones to slide or rotate against each other, but the range of motion is usually slight and tightly limited by ligaments or surrounding bones. This type of joint is found between the articular processes of adjacent vertebrae, at the acromioclavicular joint, or at the intercarpal joints of the hand and intertarsal joints of the foot. Ball-and-socket joints, in which the rounded head of a bone fits into a large depression or socket, are found at the shoulder and hip joints. Both plane and ball-and-sockets joints are classified functionally as multiaxial joints. However, ball-and-socket joints allow for large movements, while the motions between bones at a plane joint are small.
Interactive Link Questions
Watch this video to see an animation of synovial joints in action. Synovial joints are places where bones articulate with each other inside of a joint cavity. The different types of synovial joints are the ball-and-socket joint (shoulder joint), hinge joint (knee), pivot joint (atlantoaxial joint, between C1 and C2 vertebrae of the neck), condyloid joint (radiocarpal joint of the wrist), saddle joint (first carpometacarpal joint, between the trapezium carpal bone and the first metacarpal bone, at the base of the thumb), and plane joint (facet joints of vertebral column, between superior and inferior articular processes). Which type of synovial joint allows for the widest ranges of motion?
Answer: Ball-and-socket joint.
Visit this website to read about a patient who arrives at the hospital with joint pain and weakness in his legs. What caused this patient’s weakness?
Answer: Gout is due to the accumulation of uric acid crystals in the body. Usually these accumulate within joints, causing joint pain. This patient also had crystals that accumulated in the space next to his spinal cord, thus compressing the spinal cord and causing muscle weakness.
Watch this animation to observe hip replacement surgery (total hip arthroplasty), which can be used to alleviate the pain and loss of joint mobility associated with osteoarthritis of the hip joint. What is the most common cause of hip disability?
Answer: The most common cause of hip disability is osteoarthritis, a chronic disease in which the articular cartilage of the joint wears away, resulting in severe hip pain and stiffness.
Watch this video to learn about the symptoms and treatments for rheumatoid arthritis. Which system of the body malfunctions in rheumatoid arthritis and what does this cause?
Answer: The immune system malfunctions and attacks healthy cells in the lining of your joints. This causes inflammation and pain in the joints and surrounding tissues.
Review Questions
Q. Which type of joint provides the greatest range of motion?
A. ball-and-socket
B. hinge
C. condyloid
D. plane
Answer: A
Q. Which type of joint allows for only uniaxial movement?
A. saddle joint
B. hinge joint
C. condyloid joint
D. ball-and-socket joint
Answer: B
Q. Which of the following is a type of synovial joint?
A. a synostosis
B. a suture
C. a plane joint
D. a synchondrosis
Answer: C
Q. A bursa ________.
A. surrounds a tendon at the point where the tendon crosses a joint
B. secretes the lubricating fluid for a synovial joint
C. prevents friction between skin and bone, or a muscle tendon and bone
D. is the strong band of connective tissue that holds bones together at a synovial joint
Answer: C
Q. At synovial joints, ________.
A. the articulating ends of the bones are directly connected by fibrous connective tissue
B. the ends of the bones are enclosed within a space called a subcutaneous bursa
C. intrinsic ligaments are located entirely inside of the articular capsule
D. the joint cavity is filled with a thick, lubricating fluid
Answer: D
Q. At a synovial joint, the synovial membrane ________.
A. forms the fibrous connective walls of the joint cavity
B. is the layer of cartilage that covers the articulating surfaces of the bones
C. forms the intracapsular ligaments
D. secretes the lubricating synovial fluid
Answer: D
Q. Condyloid joints ________.
A. are a type of ball-and-socket joint
B. include the radiocarpal joint
C. are a uniaxial diarthrosis joint
D. are found at the proximal radioulnar joint
Answer: B
Q. A meniscus is ________.
A. a fibrocartilage pad that provides padding between bones
B. a fluid-filled space that prevents friction between a muscle tendon and underlying bone
C. the articular cartilage that covers the ends of a bone at a synovial joint
D. the lubricating fluid within a synovial joint
Answer: A
Critical Thinking Questions
Q. Describe the characteristic structures found at all synovial joints.
A. All synovial joints have a joint cavity filled with synovial fluid that is the site at which the bones of the joint articulate with each other. The articulating surfaces of the bones are covered by articular cartilage, a thin layer of hyaline cartilage. The walls of the joint cavity are formed by the connective tissue of the articular capsule. The synovial membrane lines the interior surface of the joint cavity and secretes the synovial fluid. Synovial joints are directly supported by ligaments, which span between the bones of the joint. These may be located outside of the articular capsule (extrinsic ligaments), incorporated or fused to the wall of the articular capsule (intrinsic ligaments), or found inside of the articular capsule (intracapsular ligaments). Ligaments hold the bones together and also serve to resist or prevent excessive or abnormal movements of the joint.
Q. Describe the structures that provide direct and indirect support for a synovial joint.
A. Direct support for a synovial joint is provided by ligaments that strongly unite the bones of the joint and serve to resist excessive or abnormal movements. Some joints, such as the sternoclavicular joint, have an articular disc that is attached to both bones, where it provides direct support by holding the bones together. Indirect joint support is provided by the muscles and their tendons that act across a joint. Muscles will increase their contractile force to help support the joint by resisting forces acting on it.
Glossary
- articular capsule
- connective tissue structure that encloses the joint cavity of a synovial joint
- articular cartilage
- thin layer of hyaline cartilage that covers the articulating surfaces of bones at a synovial joint
- articular disc
- meniscus; a fibrocartilage structure found between the bones of some synovial joints; provides padding or smooths movements between the bones; strongly unites the bones together
- ball-and-socket joint
- synovial joint formed between the spherical end of one bone (the ball) that fits into the depression of a second bone (the socket); found at the hip and shoulder joints; functionally classified as a multiaxial joint
- bursa
- connective tissue sac containing lubricating fluid that prevents friction between adjacent structures, such as skin and bone, tendons and bone, or between muscles
- condyloid joint
- synovial joint in which the shallow depression at the end of one bone receives a rounded end from a second bone or a rounded structure formed by two bones; found at the metacarpophalangeal joints of the fingers or the radiocarpal joint of the wrist; functionally classified as a biaxial joint
- extrinsic ligament
- ligament located outside of the articular capsule of a synovial joint
- hinge joint
- synovial joint at which the convex surface of one bone articulates with the concave surface of a second bone; includes the elbow, knee, ankle, and interphalangeal joints; functionally classified as a uniaxial joint
- intracapsular ligament
- ligament that is located within the articular capsule of a synovial joint
- intrinsic ligament
- ligament that is fused to or incorporated into the wall of the articular capsule of a synovial joint
- meniscus
- articular disc
- pivot joint
- synovial joint at which the rounded portion of a bone rotates within a ring formed by a ligament and an articulating bone; functionally classified as uniaxial joint
- plane joint
- synovial joint formed between the flattened articulating surfaces of adjacent bones; functionally classified as a multiaxial joint
- proximal radioulnar joint
- articulation between head of radius and radial notch of ulna; uniaxial pivot joint that allows for rotation of radius during pronation/supination of forearm
- saddle joint
- synovial joint in which the articulating ends of both bones are convex and concave in shape, such as at the first carpometacarpal joint at the base of the thumb; functionally classified as a biaxial joint
- subcutaneous bursa
- bursa that prevents friction between skin and an underlying bone
- submuscular bursa
- bursa that prevents friction between bone and a muscle or between adjacent muscles
- subtendinous bursa
- bursa that prevents friction between bone and a muscle tendon
- synovial fluid
- thick, lubricating fluid that fills the interior of a synovial joint
- synovial membrane
- thin layer that lines the inner surface of the joint cavity at a synovial joint; produces the synovial fluid
- tendon
- dense connective tissue structure that anchors a muscle to bone
- tendon sheath
- connective tissue that surrounds a tendon at places where the tendon crosses a joint; contains a lubricating fluid to prevent friction and allow smooth movements of the tendon


24.5: Biological Amines and the Henderson-Hasselbalch Equation
-
- Last updated
- Save as PDF
- Steven Farmer, Dietmar Kennepohl, Tim Soderberg, & William Reusch
- University of Illinois Springfield
Objectives
- identify the form that amine bases take within living cells.
- use the Henderson‑Hasselbalch equation to calculate the percentage of a base that is protonated in a solution, given the p K a value of the associated ion and the pH of the solution.
- explain why organic chemists write cellular amines in their protonated form and amino acids in their ammonium carboxylate form.
The Henderson-Hasselbalch equation is very useful relating the pKa of a buffered solution to the relative amounts of an acid and its conjugate base. In Section 20-3 , we used the Henderson-Hasselbalch equation to show that under physiological pH, carboxylic acids are almost completely dissociated into their carboxylate ions.
\[\mathrm{pH}=\mathrm{pK}_{\mathrm{a}}+\log \left(\frac{\text { concentration of conjugate base }}{\text { concentration of weak acid }}\right) \tag{Henderson-Hasselbalch equation}\]
So, what does the side chain of a lysine amino acid residue look like if it is on the surface of a protein in an aqueous solution buffered pH 7.0? Is it protonated or deprotonated? The values in the Henderson-Hasselbalch can be used for an amine with the ammonium salt written as, HA = RNH 3 + , and the amine as being, A - = RNH 2 . With an approximate pK a of 10.8 for the protonated amine HA, it should be >99% protonated, in the positively-charged, ammonium form:
\[\begin{align*}7.0 &= 10.8 + \log \left(\dfrac{[\ce{RNH2}]}{[\ce{RNH3^{+}}]}\right) \\[4pt] \dfrac{[\ce{RNH2}]}{[\ce{RNH3^{+}}]} &= 1.6 \times 10^{-4}\end{align*}\]
So, \(\ce{[RNH3^{+}]} \gg \ce{[RNH2]}\) at this pH. Consequently, in an aqueous solution buffered at pH 7, carboxylic acid groups can be expected to be essentially 100% deprotonated and negatively charged (ie. in the carboxylate form), and amine groups essentially 100% protonated and positively charged (i.e., in the ammonium form).
Alcohols are fully protonated and neutral at pH 7, as are thiols. The imidazole group on the histidine side chain has a pK a near 7, and thus exists in physiological solutions as mixture of both protonated and deprotonated forms.
Exercise \(\PageIndex{1}\)
Would you expect an aromatic hetererocycle, pyrrole, to be protonated at pH = 7.3? Use the Henderson-Hasselbalch equation to determine your answer. pKa of protonated pyrrole is 0.4.
- Answer
-
\[\begin{align*}7.3 &= 0.4 + \log \left(\dfrac{[\ce{RNH2}]}{[\ce{RNH3^{+}}]}\right) \\[4pt] \dfrac{[\ce{RNH2}]}{[\ce{RNH3^{+}}]} &= 7.9 \times 10^{6}\end{align*}\]
So, \(\ce{[RNH2]} \gg \ce{[RNH3^{+}]}\). Thus pyrrole would be almost completely unprotonated at pH = 7.


14.0: Introduction
Pain is one of the most common reasons individuals seek medical treatment, and it can arise from a number of sources, including trauma, cancer, and autoimmune and other diseases. When pain and its underlying causes are not managed well, chronic problems such as disability, lost wages, and mental health issues can result. In addition to nonpharmacologic options, pain is often managed with nonopioid analgesics, which include several medications that clients can acquire without a prescription, and opioid agonists, which are reserved for more severe forms of pain and require a prescription. This chapter will review the definition of pain, explain how pain occurs in the body, and describe some ways to treat pain with pharmacologic therapies.