4.8: Expected Value and Covariance Matrices
The main purpose of this section is a discussion of expected value and covariance for random matrices and vectors. These topics are somewhat specialized, but are particularly important in multivariate statistical models and for the multivariate normal distribution. This section requires some prerequisite knowledge of linear algebra.
We assume that the various indices \( m, \, n, p, k \) that occur in this section are positive integers. Also we assume that expected values of real-valued random variables that we reference exist as real numbers, although extensions to cases where expected values are \(\infty\) or \(-\infty\) are straightforward, as long as we avoid the dreaded indeterminate form \(\infty - \infty\).
Basic Theory
Linear Algebra
We will follow our usual convention of denoting random variables by upper case letters and nonrandom variables and constants by lower case letters. In this section, that convention leads to notation that is a bit nonstandard, since the objects that we will be dealing with are vectors and matrices. On the other hand, the notation we will use works well for illustrating the similarities between results for random matrices and the corresponding results in the one-dimensional case. Also, we will try to be careful to explicitly point out the underlying spaces where various objects live.
Let \(\R^{m \times n}\) denote the space of all \(m \times n\) matrices of real numbers. The \( (i, j) \) entry of \( \bs{a} \in \R^{m \times n} \) is denoted \( a_{i j} \) for \( i \in \{1, 2, \ldots, m\} \) and \( j \in \{1, 2, \ldots, n\} \). We will identify \(\R^n\) with \(\R^{n \times 1}\), so that an ordered \(n\)-tuple can also be thought of as an \(n \times 1\) column vector. The transpose of a matrix \(\bs{a} \in \R^{m \times n}\) is denoted \(\bs{a}^T\)—the \( n \times m \) matrix whose \( (i, j) \) entry is the \( (j, i) \) entry of \( \bs{a} \). Recall the definitions of matrix addition, scalar multiplication, and matrix multiplication. Recall also the standard inner product (or dot product ) of \( \bs{x}, \, \bs{y} \in \R^n \): \[ \langle \bs{x}, \bs{y} \rangle = \bs{x} \cdot \bs{y} = \bs{x}^T \bs{y} = \sum_{i=1}^n x_i y_i \] The outer product of \( \bs{x} \) and \(\bs{y}\) is \( \bs{x} \bs{y}^T \), the \( n \times n \) matrix whose \( (i, j) \) entry is \( x_i y_j \). Note that the inner product is the trace (sum of the diagonal entries) of the outer product. Finally recall the standard norm on \( \R^n \), given by \[ \|\bs{x}\| = \sqrt{\langle \bs{x}, \bs{x}\rangle} = \sqrt{x_1^2 + x_2^2 + \cdots + x_n^2}\] Recall that inner product is bilinear , that is, linear (preserving addition and scalar multiplication) in each argument separately. As a consequence, for \( \bs{x}, \, \bs{y} \in \R^n \), \[ \|\bs{x} + \bs{y}\|^2 = \|\bs{x}\|^2 + \|\bs{y}\|^2 + 2 \langle \bs{x}, \bs{y} \rangle \]
Expected Value of a Random Matrix
As usual, our starting point is a random experiment modeled by a probability space \((\Omega, \mathscr F, \P)\). So to review, \( \Omega \) is the set of outcomes, \( \mathscr F \) the collection of events, and \( \P \) the probability measure on the sample space \( (\Omega, \mathscr F) \). It's natural to define the expected value of a random matrix in a component-wise manner.
Suppose that \(\bs{X}\) is an \(m \times n\) matrix of real-valued random variables, whose \((i, j)\) entry is denoted \(X_{i j}\). Equivalently, \(\bs{X}\) is as a random \(m \times n\) matrix, that is, a random variable with values in \( \R^{m \times n} \). The expected value \(\E(\bs{X})\) is defined to be the \(m \times n\) matrix whose \((i, j)\) entry is \(\E\left(X_{i j}\right)\), the expected value of \(X_{i j}\).
Many of the basic properties of expected value of random variables have analogous results for expected value of random matrices, with matrix operation replacing the ordinary ones. Our first two properties are the critically important linearity properties . The first part is the additive property —the expected value of a sum is the sum of the expected values.
\(\E(\bs{X} + \bs{Y}) = \E(\bs{X}) + \E(\bs{Y})\) if \(\bs{X}\) and \(\bs{Y}\) are random \(m \times n\) matrices.
Proof
This is true by definition of the matrix expected value and the ordinary additive property. Note that \( \E\left(X_{i j} + Y_{i j}\right) = \E\left(X_{i j}\right) + \E\left(Y_{i j}\right) \). The left side is the \( (i, j) \) entry of \( \E(\bs{X} + \bs{Y}) \) and the right side is the \( (i, j) \) entry of \( \E(\bs{X}) + \E(\bs{Y}) \).
The next part of the linearity properties is the scaling property —a nonrandom matrix factor can be pulled out of the expected value.
Suppose that \(\bs{X}\) is a random \(n \times p\) matrix.
- \(\E(\bs{a} \bs{X}) = \bs{a} \E(\bs{X})\) if \(\bs{a} \in \R^{m \times n}\).
- \( \E(\bs{X} \bs{a}) = \E(\bs{X}) \bs{a}\) if \( \bs{a} \in \R^{p \times n} \).
Proof
- By the ordinary linearity and scaling properties, \( \E\left(\sum_{j=1}^n a_{i j} X_{j k}\right) = \sum_{j=1}^n a_{i j} \E\left(X_{j k}\right) \). The left side is the \( (i, k) \) entry of \( \E(\bs{a} \bs{X}) \) and the right side is the \( (i, k) \) entry of \( \bs{a} \E(\bs{X}) \).
- The proof is similar to (a).
Recall that for independent, real-valued variables, the expected value of the product is the product of the expected values. Here is the analogous result for random matrices.
\(\E(\bs{X} \bs{Y}) = \E(\bs{X}) \E(\bs{Y})\) if \(\bs{X}\) is a random \(m \times n\) matrix, \(\bs{Y}\) is a random \(n \times p\) matrix, and \(\bs{X}\) and \(\bs{Y}\) are independent.
Proof
By the ordinary linearity properties and by the independence assumption, \[ \E\left(\sum_{j=1}^n X_{i j} Y_{j k}\right) = \sum_{j=1}^n \E\left(X_{i j} Y_{j k}\right) = \sum_{j=1}^n \E\left(X_{i j}\right) \E\left(Y_{j k}\right)\] The left side is the \( (i, k) \) entry of \( \E(\bs{X} \bs{Y}) \) and the right side is the \( (i, k) \) entry of \( \E(\bs{X}) \E(\bs{Y}) \).
Actually the previous result holds if \( \bs{X} \) and \( \bs{Y} \) are simply uncorrelated in the sense that \( X_{i j} \) and \( Y_{j k} \) are uncorrelated for each \( i \in \{1, \ldots, m\} \), \( j \in \{1, 2, \ldots, n\} \) and \( k \in \{1, 2, \ldots p\} \). We will study covariance of random vectors in the next subsection.
Covariance Matrices
Our next goal is to define and study the covariance of two random vectors.
Suppose that \(\bs{X}\) is a random vector in \(\R^m\) and \(\bs{Y}\) is a random vector in \(\R^n\).
- The covariance matrix of \(\bs{X}\) and \(\bs{Y}\) is the \(m \times n\) matrix \(\cov(\bs{X}, \bs{Y})\) whose \((i,j)\) entry is \(\cov\left(X_i, Y_j\right)\) the ordinary covariance of \(X_i\) and \(Y_j\).
- Assuming that the coordinates of \( \bs{X} \) and \(\bs{Y}\) have positive variance, the correlation matrix of \( \bs{X} \) and \( \bs{Y} \) is the \( m \times n \) matrix \( \cor(\bs{X}, \bs{Y}) \) whose \( (i, j) \) entry is \( \cor\left(X_i, Y_j\right)\), the ordinary correlation of \( X_i \) and \( Y_j \)
Many of the standard properties of covariance and correlation for real-valued random variables have extensions to random vectors. For the following three results, \( \bs X \) is a random vector in \( \R^m \) and \( \bs Y \) is a random vector in \( \R^n \).
\(\cov(\bs{X}, \bs{Y}) = \E\left(\left[\bs{X} - \E(\bs{X})\right]\left[\bs{Y} - \E(\bs{Y})\right]^T\right)\)
Proof
By the definition of the expected value of a random vector and by the defintion of matrix multiplication, the \( (i, j) \) entry of \( \left[\bs{X} - \E(\bs{X})\right]\left[\bs{Y} - \E(\bs{Y})\right]^T \) is simply \( \left[X_i - \E\left(X_i\right)\right] \left[Y_j - \E\left(Y_j\right)\right] \). The expected value of this entry is \( \cov\left(X_i, Y_j\right) \), which in turn, is the \( (i, j) \) entry of \( \cov(\bs{X}, \bs{Y}) \)
Thus, the covariance of \( \bs{X} \) and \( \bs{Y} \) is the expected value of the outer product of \( \bs{X} - \E(\bs{X}) \) and \( \bs{Y} - \E(\bs{Y}) \). Our next result is the computational formula for covariance: the expected value of the outer product of \( \bs{X} \) and \( \bs{Y} \) minus the outer product of the expected values.
\(\cov(\bs{X},\bs{Y}) = \E\left(\bs{X} \bs{Y}^T\right) - \E(\bs{X}) \left[\E(\bs{Y})\right]^T\).
Proof
The \( (i, j) \) entry of \( \E\left(\bs{X} \bs{Y}^T\right) - \E(\bs{X}) \left[\E(\bs{Y})\right]^T\) is \( \E\left(X_i, Y_j\right) - \E\left(X_i\right) \E\left(Y_j\right) \), which by the standard computational formula, is \( \cov\left(X_i, Y_j\right) \), which in turn is the \( (i, j) \) entry of \( \cov(\bs{X}, \bs{Y}) \).
The next result is the matrix version of the symmetry property.
\(\cov(\bs{Y}, \bs{X}) = \left[\cov(\bs{X}, \bs{Y})\right]^T\).
Proof
The \( (i, j) \) entry of \( \cov(\bs{X}, \bs{Y}) \) is \( \cov\left(X_i, Y_j\right) \), which is the \((j, i) \) entry of \( \cov(\bs{Y}, \bs{X}) \).
In the following result, \( \bs{0} \) denotes the \( m \times n \) zero matrix.
\(\cov(\bs{X}, \bs{Y}) = \bs{0}\) if and only if \(\cov\left(X_i, Y_j\right) = 0\) for each \(i\) and \(j\), so that each coordinate of \(\bs{X}\) is uncorrelated with each coordinate of \(\bs{Y}\).
Proof
This follows immediately from the definition of \( \cov(\bs{X}, \bs{Y}) \).
Naturally, when \( \cov(\bs{X}, \bs{Y}) = \bs{0} \), we say that the random vectors \( \bs{X} \) and \(\bs{Y}\) are uncorrelated . In particular, if the random vectors are independent, then they are uncorrelated. The following results establish the bi-linear properties of covariance.
The additive properties.
- \(\cov(\bs{X} + \bs{Y}, \bs{Z}) = \cov(\bs{X}, \bs{Z}) + \cov(\bs{Y}, \bs{Z})\) if \(\bs{X}\) and \(\bs{Y}\) are random vectors in \(\R^m\) and \(\bs{Z}\) is a random vector in \(\R^n\).
- \(\cov(\bs{X}, \bs{Y} + \bs{Z}) = \cov(\bs{X}, \bs{Y}) + \cov(\bs{X}, \bs{Z})\) if \(\bs{X}\) is a random vector in \(\R^m\), and \(\bs{Y}\) and \(\bs{Z}\) are random vectors in \(\R^n\).
Proof
- From the ordinary additive property of covariance, \( \cov\left(X_i + Y_i, Z_j\right) = \cov\left(X_i, Z_j\right) + \cov\left(Y_i, Z_j\right) \). The left side is the \( (i, j) \) entry of \( \cov(\bs{X} + \bs{Y}, \bs{Z}) \) and the right side is the \( (i, j) \) entry of \( \cov(\bs{X}, \bs{Z}) + \cov(\bs{Y}, \bs{Z}) \).
- The proof is similar to (a), using the additivity of covariance in the second argument.
The scaling properties
- \(\cov(\bs{a} \bs{X}, \bs{Y}) = \bs{a} \cov(\bs{X}, \bs{Y})\) if \(\bs{X}\) is a random vector in \(\R^n\), \(\bs{Y}\) is a random vector in \(\R^p\), and \(\bs{a} \in \R^{m \times n}\).
- \(\cov(\bs{X}, \bs{a} \bs{Y}) = \cov(\bs{X}, \bs{Y}) \bs{a}^T\) if \(\bs{X}\) is a random vector in \(\R^m\), \(\bs{Y}\) is a random vector in \(\R^n\), and \(\bs{a} \in \R^{k \times n}\).
Proof
- Using the ordinary linearity properties of covariance in the first argument, we have \[ \cov\left(\sum_{j=1}^n a_{i j} X_j, Y_k\right) = \sum_{j=1}^n a_{i j} \cov\left(X_j, Y_k\right) \] The left side is the \( (i, k) \) entry of \( \cov(\bs{a} \bs{X}, \bs{Y}) \) and the right side is the \( (i, k) \) entry of \( \bs{a} \cov(\bs{X}, \bs{Y}) \).
- The proof is similar to (a), using the linearity of covariance in the second argument.
Variance-Covariance Matrices
Suppose that \(\bs{X}\) is a random vector in \(\R^n\). The covariance matrix of \(\bs{X}\) with itself is called the variance-covariance matrix of \(\bs{X}\): \[ \vc(\bs{X}) = \cov(\bs{X}, \bs{X}) = \E\left(\left[\bs{X} - \E(\bs{X})\right]\left[\bs{X} - \E(\bs{X})\right]^T\right)\]
Recall that for an ordinary real-valued random variable \( X \), \( \var(X) = \cov(X, X) \). Thus the variance-covariance matrix of a random vector in some sense plays the same role that variance does for a random variable.
\(\vc(\bs{X})\) is a symmetric \(n \times n\) matrix with \(\left(\var(X_1), \var(X_2), \ldots, \var(X_n)\right)\) on the diagonal.
Proof
Recall that \( \cov\left(X_i, X_j\right) = \cov\left(X_j, X_i\right) \). Also, the \( (i, i) \) entry of \( \vc(\bs{X}) \) is \( \cov\left(X_i, X_i\right) = \var\left(X_i\right) \).
The following result is the formula for the variance-covariance matrix of a sum, analogous to the formula for the variance of a sum of real-valued variables.
\(\vc(\bs{X} + \bs{Y}) = \vc(\bs{X}) + \cov(\bs{X}, \bs{Y}) + \cov(\bs{Y}, \bs{X}) + \vc(\bs{Y})\) if \(\bs{X}\) and \(\bs{Y}\) are random vectors in \(\R^n\).
Proof
This follows from the additive property of covariance: \[ \vc(\bs{X} + \bs{Y}) = \cov(\bs{X} + \bs{Y}, \bs{X} + \bs{Y}) = \cov(\bs{X}, \bs{X}) + \cov(\bs{X}, \bs{Y}) + \cov(\bs{Y}, \bs{X}) + \cov(\bs{Y}, \bs{Y}) \]
Recall that \( \var(a X) = a^2 \var(X) \) if \( X \) is a real-valued random variable and \( a \in \R \). Here is the analogous result for the variance-covariance matrix of a random vector.
\(\vc(\bs{a} \bs{X}) = \bs{a} \vc(\bs{X}) \bs{a}^T\) if \(\bs{X}\) is a random vector in \(\R^n\) and \(\bs{a} \in \R^{m \times n}\).
Proof
This follows from the scaling property of covariance: \[ \vc(\bs{a} \bs{X}) = \cov(\bs{a} \bs{X}, \bs{a} \bs{X}) = \bs{a} \cov(\bs{X}, \bs{X}) \bs{a}^T \]
Recall that if \( X \) is a random variable, then \( \var(X) \ge 0 \), and \( \var(X) = 0 \) if and only if \( X \) is a constant (with probability 1). Here is the analogous result for a random vector:
Suppose that \( \bs{X} \) is a random vector in \( \R^n \).
- \( \vc(\bs{X}) \) is either positive semi-definite or positive definite.
- \(\vc(\bs{X})\) is positive semi-definite but not positive definite if and only if there exists \(\bs{a} \in \R^n\) and \(c \in \R\) such that, with probability 1, \(\bs{a}^T \bs{X} = \sum_{i=1}^n a_i X_i = c\)
Proof
- From the previous result, \(0 \le \var\left(\bs{a}^T \bs{X}\right) = \vc\left(\bs{a}^T \bs{X}\right) = \bs{a}^T \vc(\bs{X}) \bs{a} \) for every \( \bs{a} \in \R^n \). Thus, by definition, \( \vc(\bs{X}) \) is either positive semi-definite or positive definite.
- In light of (a), \( \vc(\bs{X}) \) is positive semi-definite but not positive definite if and only if there exists \( \bs{a} \in \R^n \) such that \( \bs{a}^T \vc(\bs{X}) \bs{a} = \var\left(\bs{a}^T \bs{X}\right) = 0 \). But in turn, this is true if and only if \( \bs{a}^T \bs{X} \) is constant with probability 1.
Recall that since \(\vc(\bs{X})\) is either positive semi-definite or positive definite, the eigenvalues and the determinant of \(\vc(\bs{X})\) are nonnegative. Moreover, if \(\vc(\bs{X})\) is positive semi-definite but not positive definite, then one of the coordinates of \(\bs{X}\) can be written as a linear transformation of the other coordinates (and hence can usually be eliminated in the underlying model). By contrast, if \(\vc(\bs{X})\) is positive definite, then this cannot happen; \(\vc(\bs{X})\) has positive eigenvalues and determinant and is invertible.
Best Linear Predictor
Suppose that \(\bs{X}\) is a random vector in \(\R^m\) and that \(\bs{Y}\) is a random vector in \(\R^n\). We are interested in finding the function of \(\bs{X}\) of the form \(\bs{a} + \bs{b} \bs{X}\), where \(\bs{a} \in \R^n\) and \(\bs{b} \in \R^{n \times m}\), that is closest to \(\bs{Y}\) in the mean square sense. Functions of this form are analogous to linear functions in the single variable case. However, unless \( \bs{a} = \bs{0} \), such functions are not linear transformations in the sense of linear algebra, so the correct term is affine function of \( \bs{X} \). This problem is of fundamental importance in statistics when random vector \(\bs{X}\), the predictor vector is observable, but not random vector \(\bs{Y}\), the response vector . Our discussion here generalizes the one-dimensional case, when \(X\) and \(Y\) are random variables. That problem was solved in the section on Covariance and Correlation. We will assume that \(\vc(\bs{X})\) is positive definite, so that \( \vc(\bs{X}) \) is invertible, and none of the coordinates of \(\bs{X}\) can be written as an affine function of the other coordinates. We write \( \vc^{-1}(\bs{X}) \) for the inverse instead of the clunkier \( \left[\vc(\bs{X})\right]^{-1} \).
As with the single variable case, the solution turns out to be the affine function that has the same expected value as \( \bs{Y} \), and whose covariance with \( \bs{X} \) is the same as that of \( \bs{Y} \).
Define \( L(\bs{Y} \mid \bs{X}) = \E(\bs{Y}) + \cov(\bs{Y},\bs{X}) \vc^{-1}(\bs{X}) \left[\bs{X} - \E(\bs{X})\right] \). Then \( L(\bs{Y} \mid \bs{X}) \) is the only affine function of \( \bs{X} \) in \( \R^n \) satisfying
- \( \E\left[L(\bs{Y} \mid \bs{X})\right] = \E(\bs{Y}) \)
- \( \cov\left[L(\bs{Y} \mid \bs{X}), \bs{X}\right] = \cov(\bs{Y}, \bs{X}) \)
Proof
From linearity, \[ \E\left[L(\bs{Y} \mid \bs{X})\right] = E(\bs{Y}) + \cov(\bs{Y}, \bs{X}) \vc^{-1}(\bs{X})\left[\E(\bs{X}) - \E(\bs{X})\right] = 0\] From linearity and the fact that a constant vector is independent (and hence uncorrelated) with any random vector, \[ \cov\left[L(\bs{Y} \mid \bs{X}), \bs{X}\right] = \cov(\bs{Y}, \bs{X}) \vc^{-1}(\bs{X}) \cov(\bs{X}, \bs{X}) = \cov(\bs{Y}, \bs{X}) \vc^{-1}(\bs{X}) \vc(\bs{X}) = \cov(\bs{Y}, \bs{X}) \] Conversely, suppose that \( \bs{U} = \bs{a} + \bs{b} \bs{X} \) for some \( \bs{a} \in \R^n \) and \( \bs{b} \in \R^{m \times n} \), and that \( \E(\bs{U}) = \E(\bs{Y}) \) and \( \cov(\bs{U}, \bs{X}) = \cov(\bs{Y}, \bs{X}) \). From the second equation, again using linearity and the uncorrelated property of constant vectors, we get \( \bs{b} \cov(\bs{X}, \bs{X}) = \cov(\bs{Y}, \bs{X}) \) and therefore \( \bs{b} = \cov(\bs{Y}, \bs{X}) \vc^{-1}(\bs{X}) \). Then from the first equation, \( \bs{a} + \bs{b} \E(\bs{X}) = \bs{Y} \) so \( \bs{a} = \E(\bs{Y}) - \bs{b} \E(\bs{X}) \).
A simple corollary is the \( \bs{Y} - L(\bs{Y} \mid \bs{X}) \) is uncorrelated with any affine function of \( \bs{X} \):
If \( \bs{U} \) is an affine function of \( \bs{X} \) then
- \( \cov\left[\bs{Y} - L(\bs{Y} \mid \bs{X}), \bs{U}\right] = \bs{0} \)
- \( \E\left(\langle \bs{Y} - L(\bs{Y} \mid \bs{X}), \bs{U}\rangle\right) = 0\)
Proof
Suppose that \( \bs{U} = \bs{a} + \bs{b} \bs{X} \) where \( \bs{a} \in \R^n \) and \( \bs{b} \in \R^{m \times n} \). For simplicity, let \( \bs{L} = L(\bs{Y} \mid \bs{X}) \)
- From the previous result, \( \cov(\bs{Y}, \bs{X}) = \cov(\bs{L}, \bs{X}) \). Hence using linearity, \[ \cov\left(\bs{Y} - \bs{L}, \bs{U}\right) = \cov(\bs{Y} - \bs{L}, \bs{a}) + \cov(\bs{Y} - \bs{L}, \bs{X}) \bs{b}^T = \bs{0} + \left[\cov(\bs{Y}, \bs{X}) - \cov(\bs{L}, \bs{X})\right] = \bs{0} \]
- Recall that \(\langle \bs{Y} - \bs{L}, \bs{U}\rangle\) is the trace of \( \cov(\bs{Y} - \bs{L}, \bs{U}) \) and hence has expected value 0 by part (a).
The variance-covariance matrix of \( L(\bs{Y} \mid \bs{X}) \), and its covariance matrix with \( \bs{Y} \) turn out to be the same, again analogous to the single variable case.
Additional properties of \( L(\bs{Y} \mid \bs{X}) \):
- \( \cov\left[\bs{Y}, L(\bs{Y} \mid \bs{X})\right] = \cov(\bs{Y}, \bs{X}) \vc^{-1}(\bs{X}) \cov(\bs{X}, \bs{Y}) \)
- \( \vc\left[L(\bs{Y} \mid \bs{X})\right] = \cov(\bs{Y}, \bs{X}) \vc^{-1}(\bs{X}) \cov(\bs{X}, \bs{Y}) \)
Proof
Recall that \( L(\bs{Y} \mid \bs{X}) = \E(\bs{Y}) + \cov(\bs{Y},\bs{X}) \vc^{-1}(\bs{X}) \left[\bs{X} - \E(\bs{X})\right] \)
- Using basic properties of covariance, \[ \cov\left[Y, L(\bs{Y} \mid \bs{X})\right] = \cov\left[\bs{Y}, \bs{X} - \E(\bs{X})\right] \left[\cov(\bs{Y}, \bs{X}) \vc^{-1}(\bs{X})\right]^T = \cov(\bs{Y}, \bs{X}) \vc^{-1}(\bs{X}) \cov(\bs{X}, \bs{Y}) \]
- Using basic properties of variance-covariance, \[ \vc\left[L(\bs{Y} \mid \bs{X})\right] = \vc\left[\cov(\bs{Y}, \bs{X}) \vc^{-1}(\bs{X}) \bs{X} \right] = \cov(\bs{Y}, \bs{X}) \vc^{-1}(\bs{X}) \vc(\bs{X}) \left[\cov(\bs{Y}, \bs{X}) \vc^{-1}(\bs{X})\right]^T = \cov(\bs{Y}, \bs{X}) \vc^{-1}(\bs{X}) \cov(\bs{X}, \bs{Y})\]
Next is the fundamental result that \( L(\bs{Y} \mid \bs{X}) \) is the affine function of \( \bs{X} \) that is closest to \( \bs{Y} \) in the mean square sense.
Suppose that \( \bs{U} \in \R^n \) is an affine function of \( \bs{X} \). Then
- \( \E\left(\|\bs{Y} - L(\bs{Y} \mid \bs{X})\|^2\right) \le \E\left(\|\bs{Y} - \bs{U}\|^2\right) \)
- Equality holds in (a) if and only if \( \bs{U} = L(\bs{Y} \mid \bs{X}) \) with probability 1.
Proof
Again, let \( \bs{L} = L(\bs{Y} \mid \bs{X}) \) for simplicity and let \( \bs{U} \in \R^n \) be an affine function of \( \bs{X}\).
- Using the linearity of expected value, note that \[ \E\left(\|\bs{Y} - \bs{U}\|^2\right) = \E\left[\|(\bs{Y} - \bs{L}) + (\bs{L} - \bs{U})\|^2\right] = \E\left(\|\bs{Y} - \bs{L}\|^2\right) + 2 \E(\langle \bs{Y} - \bs{L}, \bs{L} - \bs{U}\rangle) + \E\left(\|\bs{L} - \bs{U}\|^2\right) \] But \( \bs{L} - \bs{U} \) is an affine function of \( \bs{X} \) and hence the middle term is 0 by our previous corollary. Hence \( \E\left(\|\bs{Y} - \bs{U}\|^2\right) = \E\left(\|\bs{L} - \bs{Y}\|^2\right) + \E\left(\|\bs{L} - \bs{U}\|^2\right) \ge \E\left(\|\bs{L} - \bs{Y}\|^2\right) \)
- From (a), equality holds in the inequality if and only if \( \E\left(\|\bs{L} - \bs{U}\|^2\right) = 0 \) if and only if \( \P(\bs{L} = \bs{U}) = 1 \).
The variance-covariance matrix of the difference between \( \bs{Y} \) and the best affine approximation is given in the next theorem.
\( \vc\left[\bs{Y} - L(\bs{Y} \mid \bs{X})\right] = \vc(\bs{Y}) - \cov(\bs{Y}, \bs{X}) \vc^{-1}(\bs{X}) \cov(\bs{X}, \bs{Y}) \)
Proof
Again, we abbreviate \( L(\bs{Y} \mid \bs{X}) \) by \(\bs{L}\). Using basic properties of variance-covariance matrices, \[ \vc(\bs{Y} - \bs{L}) = \vc(\bs{Y}) - \cov(\bs{Y}, \bs{L}) - \cov(\bs{L}, \bs{Y}) + \vc(\bs{L}) \] But \( \cov(\bs{Y}, \bs{L}) = \cov(\bs{L}, \bs{Y}) = \vc(\bs{L}) = \cov(\bs{Y}, \bs{X}) \vc^{-1}(\bs{X}) \cov(\bs{Y}, \bs{X}) \). Substituting gives the result.
The actual mean square error when we use \( L(\bs{Y} \mid \bs{X}) \) to approximate \( \bs{Y} \), namely \( \E\left(\left\|\bs{Y} - L(\bs{Y} \mid \bs{X})\right\|^2\right) \), is the trace (sum of the diagonal entries) of the variance-covariance matrix above. The function of \(\bs{x}\) given by \[ L(\bs{Y} \mid \bs{X} = \bs{x}) = \E(\bs{Y}) + \cov(\bs{Y},\bs{X}) \vc^{-1}(\bs{X}) \left[\bs{x} - \E(\bs{X})\right] \] is known as the (distribution) linear regression function . If we observe \(\bs{x}\) then \(L(\bs{Y} \mid \bs{X} = \bs{x})\) is our best affine prediction of \(\bs{Y}\).
Multiple linear regression is more powerful than it may at first appear, because it can be applied to non-linear transformations of the random vectors. That is, if \( g: \R^m \to \R^j \) and \( h: \R^n \to \R^k \) then \( L\left[h(\bs{Y}) \mid g(\bs{X})\right] \) is the affine function of \( g(\bs{X}) \) that is closest to \( h(\bs{Y}) \) in the mean square sense. Of course, we must be able to compute the appropriate means, variances, and covariances.
Moreover, Non-linear regression with a single, real-valued predictor variable can be thought of as a special case of multiple linear regression. Thus, suppose that \(X\) is the predictor variable, \(Y\) is the response variable, and that \((g_1, g_2, \ldots, g_n)\) is a sequence of real-valued functions. We can apply the results of this section to find the linear function of \(\left(g_1(X), g_2(X), \ldots, g_n(X)\right)\) that is closest to \(Y\) in the mean square sense. We just replace \(X_i\) with \(g_i(X)\) for each \(i\). Again, we must be able to compute the appropriate means, variances, and covariances to do this.
Examples and Applications
Suppose that \((X, Y)\) has probability density function \(f\) defined by \(f(x, y) = x + y\) for \(0 \le x \le 1\), \(0 \le y \le 1\). Find each of the following:
- \(\E(X, Y)\)
- \(\vc(X, Y)\)
Answer
- \(\left(\frac{7}{12}, \frac{7}{12}\right)\)
- \(\left[\begin{matrix} \frac{11}{144} & -\frac{1}{144} \\ -\frac{1}{144} & \frac{11}{144}\end{matrix}\right]\)
Suppose that \((X, Y)\) has probability density function \(f\) defined by \(f(x, y) = 2 (x + y)\) for \(0 \le x \le y \le 1\). Find each of the following:
- \(\E(X, Y)\)
- \(\vc(X, Y)\)
Answer
- \(\left(\frac{5}{12}, \frac{3}{4}\right)\)
- \(\left[\begin{matrix} \frac{43}{720} & \frac{1}{48} \\ \frac{1}{48} & \frac{3}{80} \end{matrix} \right]\)
Suppose that \((X, Y)\) has probability density function \(f\) defined by \(f(x, y) = 6 x^2 y\) for \(0 \le x \le 1\), \(0 \le y \le 1\). Find each of the following:
- \(\E(X, Y)\)
- \(\vc(X, Y)\)
Answer
Note that \(X\) and \(Y\) are independent.
- \(\left(\frac{3}{4}, \frac{2}{3}\right)\)
- \(\left[\begin{matrix} \frac{3}{80} & 0 \\ 0 & \frac{1}{18} \end{matrix} \right]\)
Suppose that \((X, Y)\) has probability density function \(f\) defined by \(f(x, y) = 15 x^2 y\) for \(0 \le x \le y \le 1\). Find each of the following:
- \(\E(X, Y)\)
- \(\vc(X, Y)\)
- \(L(Y \mid X)\)
- \(L\left[Y \mid \left(X, X^2\right)\right]\)
- Sketch the regression curves on the same set of axes.
Answer
- \(\left( \frac{5}{8}, \frac{5}{6} \right)\)
- \(\left[ \begin{matrix} \frac{17}{448} & \frac{5}{336} \\ \frac{5}{336} & \frac{5}{252} \end{matrix} \right]\)
- \(\frac{10}{17} + \frac{20}{51} X\)
- \(\frac{49}{76} + \frac{10}{57} X + \frac{7}{38} X^2\)
Suppose that \((X, Y, Z)\) is uniformly distributed on the region \(\left\{(x, y, z) \in \R^3: 0 \le x \le y \le z \le 1\right\}\). Find each of the following:
- \(\E(X, Y, Z)\)
- \(\vc(X, Y, Z)\)
- \(L\left[Z \mid (X, Y)\right]\)
- \(L\left[Y \mid (X, Z)\right]\)
- \(L\left[X \mid (Y, Z)\right]\)
- \( L\left[(Y, Z) \mid X\right] \)
Answer
- \(\left(\frac{1}{4}, \frac{1}{2}, \frac{3}{4}\right)\)
- \(\left[\begin{matrix} \frac{3}{80} & \frac{1}{40} & \frac{1}{80} \\ \frac{1}{40} & \frac{1}{20} & \frac{1}{40} \\ \frac{1}{80} & \frac{1}{40} & \frac{3}{80} \end{matrix}\right]\)
- \(\frac{1}{2} + \frac{1}{2} Y\). Note that there is no \(X\) term.
- \(\frac{1}{2} X + \frac{1}{2} Z\). Note that this is the midpoint of the interval \([X, Z]\).
- \(\frac{1}{2} Y\). Note that there is no \(Z\) term.
- \( \left[\begin{matrix} \frac{1}{3} + \frac{2}{3} X \\ \frac{2}{3} + \frac{1}{3} X \end{matrix}\right] \)
Suppose that \(X\) is uniformly distributed on \((0, 1)\), and that given \(X\), random variable \(Y\) is uniformly distributed on \((0, X)\). Find each of the following:
- \(\E(X, Y)\)
- \(\vc(X, Y)\)
Answer
- \(\left(\frac{1}{2}, \frac{1}{4}\right)\)
- \(\left[\begin{matrix} \frac{1}{12} & \frac{1}{24} \\ \frac{1}{24} & \frac{7}{144} \end{matrix} \right]\)


2.3.4: Tableaux
While many derivation systems operate with arrangements of sentences, tableaux operate with signed formulas. A signed formula is a pair consisting of a truth value sign ( \(\True\) or \(\False\) ) and a sentence \[\sFmla{\True}{A} \text{ or } \sFmla{\False}{A}.\nonumber\] A tableau consists of signed formulas arranged in a downward-branching tree. It begins with a number of assumptions and continues with signed formulas which result from one of the signed formulas above it by applying one of the rules of inference. Each rule allows us to add one or more signed formulas to the end of a branch, or two signed formulas side by side—in this case a branch splits into two, with the two added signed formulas forming the ends of the two branches.
A rule applied to a complex signed formula results in the addition of signed formulas which are immediate sub-formulas. They come in pairs, one rule for each of the two signs. For instance, the \(\TRule{\True}{\land}\) rule applies to \(\sFmla{\True}{A \land B}\) , and allows the addition of both the two signed formulas \(\sFmla{\True}{A}\) and \(\sFmla{\True}{B}\) to the end of any branch containing \(\sFmla{\True}{A \land B}\) , and the rule \(\TRule{\False}{A \land B}\) allows a branch to be split by adding \(\sFmla{\False}{A}\) and \(\sFmla{\False}{B}\) side-by-side. A tableau is closed if every one of its branches contains a matching pair of signed formulas \(\sFmla{\True}{A}\) and \(\sFmla{\False}{A}\) .
The \(\Proves\) relation based on tableaux is defined as follows: \(\Gamma \Proves A\) iff there is some finite set \(\Gamma_0 = \{B_1, \dots, B_n\} \subseteq \Gamma\) such that there is a closed tableau for the assumptions \[\{\sFmla{\False}{A}, \sFmla{\True}{B_1}, \dots, \sFmla{\True}{B_n}\}\nonumber\] For instance, here is a closed tableau that shows that \(\Proves (A \land B) \lif A\) :
\[\begin{array}{lcl} 1. & \False\,(A \land B) \lif A & \text { Assumption } \\ 2 . & \True\, A \land B & \lif \False\, 1 \\ 3 . & \False\, A & \lif \False\, 1 \\ 4 . & \True\, A & \lif \True\, 2 \\ 5 . & \True\, B & \lif \True\, 2 \\ & \otimes & \end{array}\nonumber\]
A set \(\Gamma\) is inconsistent in the tableau calculus if there is a closed tableau for assumptions \[\{\sFmla{\True}{B_1}, \dots, \sFmla{\True}{B_n}\}\nonumber\] for some \(B_i \in \Gamma\) .
Tableaux were invented in the 1950s independently by Evert Beth and Jaakko Hintikka, and simplified and popularized by Raymond Smullyan. They are very easy to use, since constructing a tableau is a very systematic procedure. Because of the systematic nature of tableaux, they also lend themselves to implementation by computer. However, a tableau is often hard to read and their connection to proofs are sometimes not easy to see. The approach is also quite general, and many different logics have tableau systems. Tableaux also help us to find structures that satisfy given (sets of) sentences: if the set is satisfiable, it won’t have a closed tableau, i.e., any tableau will have an open branch. The satisfying structure can be “read off” an open branch, provided every rule it is possible to apply has been applied on that branch. There is also a very close connection to the sequent calculus: essentially, a closed tableau is a condensed derivation in the sequent calculus, written upside-down.


4.3: Assignment- Research Question Task
First, review the 3 resources noted below.
- This infographic about fantasy cover art illustrates one way of reporting data. From the accompanying notes, you will get a good idea of the research method used, and you should be able to infer (that is, make an informed guess based on relevant evidence) the research question.
- This research uses the same pool of sources (cover art from fantasy titles published by Orbit in 2008 and 2009) but collects different data (details of the visual representations of heroines) to answer a different research question. Again, you should be able to infer the question from the answer as represented in the infographic.
- Meta-Analysis and Review of the LiteratureA meta-analysis is a form of research in which the researcher examines all of a certain type of artifact, in this case all the scientific articles on climate change published between November 2012 and December 2013 in peer-reviewed journals. The purpose of meta-analysis is to identify trends in the available data that would not be visible from any one article alone. A meta-analysis is similar to another type of research that begins by examining all the artifacts in a certain category: the review of the literature (aka literature review).
Then, complete this writing task.
Pick ONE of the above examples of research (“Fantasy Book Covers,” “Heroines,” or “Scientific Consensus on Anthropogenic Global Warming: A Pie Chart”).
For the one you choose, state what you feel the research question was of the author in a single sentence. Then briefly explain what the research method used was. You may need to infer (that is, use evidence and logical reasoning to draw a conclusion about) the research method.
Contributors and Attributions
CC licensed content, Shared previously
- Composition II. Authored by : Janet Zepernick. Provided by : Pittsburg State University. Located at : http://www.pittstate.edu/ . Project : Kaleidoscope Open Course Project. License : CC BY: Attribution


7.25C: Northern Blots
- Evaluate the applications of Northern Blots
The Northern blot is a technique used in molecular biology research to study gene expression in a sample, through detection of RNA (or isolated messenger RNA ). With Northern blotting it is possible to observe cellular control over structure and function by determining the particular gene expression levels during differentiation, morphogenesis, as well as abnormal or diseased conditions. Northern blotting involves the use of electrophoresis to separate RNA samples by size and detection with a hybridization probe complementary to part of or the entire target sequence.
The term ‘Northern blot’ actually refers specifically to the capillary transfer of RNA from the electrophoresis gel to the blotting membrane. However, the entire process is commonly referred to as Northern blotting. The northern blot technique was developed in 1977 by James Alwine, David Kemp, and George Stark at Stanford University. Northern blotting takes its name from its similarity to the first blotting technique, the Southern blot, named for biologist Edwin Southern. The major difference is that RNA, rather than DNA, is analyzed in the Northern blot.
A general blotting procedure starts with extraction of total RNA from a homogenized tissue sample or from cells. Eukaryotic mRNA can then be isolated through the use of oligo (dT) cellulose chromatography to isolate only those RNAs with a poly(A) tail. RNA samples are then separated by gel electrophoresis. Since the gels are fragile and the probes are unable to enter the matrix, the RNA samples, now separated by size, are transferred to a nylon membrane through a capillary or vacuum blotting system.
Key Points
- RNA (either total RNA or just mRNA) is separated by gel electrophoresis, usually an agarose gel. Because there are so many different RNA molecules on the gel, it usually appears as a smear rather than discrete bands.
- The RNA is transfered to a sheet of special blotting paper called nitrocellulose, though other types of paper, or membranes, can be used. The RNA molecules retain the same pattern of separation they had on the gel.
- The blot is incubated with a probe which is single-stranded DNA. This probe will form base pairs with its complementary RNA sequence and bind to form a double-stranded RNA-DNA molecule. The probe is either radioactive or has an enzyme bound to it.
Key Terms
- hybridization : The act of hybridizing, or the state of being hybridized.


3.3: Immigration and Urbanization
-
- Last updated
- Save as PDF
Industry pulled ever more Americans into cities. Manufacturing needed the labor pool and the infrastructure. America’s urban population increased sevenfold in the half century after the Civil War. Soon the United States had more large cities than any country in the world. The 1920 U.S. census revealed that, for the first time, a majority of Americans lived in urban areas. Much of that urban growth came from the millions of immigrants pouring into the nation. Between 1870 and 1920, over twenty-five million immigrants arrived in the United States.
By the turn of the twentieth century, new immigrant groups such as Italians, Poles, and Eastern European Jews made up a larger percentage of arrivals than the Irish and Germans. The specific reasons that immigrants left their particular countries and the reasons they came to the United States (what historians call push and pull factors ) varied. For example, a young husband and wife living in Sweden in the 1880s and unable to purchase farmland might read an advertisement for inexpensive land in the American Midwest and immigrate to the United States to begin a new life. A young Italian man might simply hope to labor in a steel factory long enough to save up enough money to return home and purchase land for a family. A Russian Jewish family persecuted in European pogroms might look to the United States as a sanctuary. Or perhaps a Japanese migrant might hear of fertile farming land on the West Coast and choose to sail for California. But if many factors pushed people away from their home countries, by far the most important factor drawing immigrants was economics. Immigrants came to the United States looking for work.
Industrial capitalism was the most important factor that drew immigrants to the United States between 1880 and 1920. Immigrant workers labored in large industrial complexes producing goods such as steel, textiles, and food products, replacing smaller and more local workshops. The influx of immigrants, alongside a large movement of Americans from the countryside to the city, helped propel the rapid growth of cities like New York, Pittsburgh, Cleveland, Milwaukee, and St. Louis. By 1890, immigrants and their children accounted for roughly 60 percent of the population in most large northern cities (and sometimes as high as 80 or 90 percent). Many immigrants, especially from Italy and the Balkans, always intended to return home with enough money to purchase land. But what about those who stayed? Did the new arrivals assimilate together in the American melting pot—becoming just like those already in the United States—or did they retain, and sometimes even strengthen, their traditional ethnic identities? The answer lies somewhere in between. Immigrants from specific countries—and often even specific communities—often clustered together in ethnic neighborhoods. They formed vibrant organizations and societies, such as Italian workmen’s clubs, Eastern European Jewish mutual aid societies, and Polish Catholic churches, to ease the transition to their new American home. Immigrant communities published newspapers in dozens of languages and purchased spaces to maintain their arts, languages, and traditions alive. And from these foundations they facilitated even more immigration: after staking out a claim to some corner of American life, they wrote home and encouraged others to follow them (historians call this chain migration ).
Many cities’ politics adapted to immigrant populations. The infamous urban political machines often operated as a kind of mutual aid society. New York City’s Democratic Party machine, popularly known as Tammany Hall , drew the greatest ire from critics and seemed to embody all of the worst of city machines, but it also responded to immigrant needs. In 1903, journalist William Riordon published a book, Plunkitt of Tammany Hall , which chronicled the activities of ward heeler George Washington Plunkitt. Plunkitt elaborately explained to Riordon the difference between “honest graft” and “dishonest graft”: “I made my pile in politics, but, at the same time, I served the organization and got more big improvements for New York City than any other livin’ man.” While exposing corruption, Riordon also revealed the hard work Plunkitt undertook on behalf of his largely immigrant constituency. On a typical day, Riordon wrote, Plunkitt was awakened at two a.m. to bail out a saloonkeeper who stayed open too late, was awakened again at six a.m. because of a fire in the neighborhood and spent time finding lodgings for the families displaced by the fire, and, after spending the rest of the morning in court to secure the release of several of his constituents, found jobs for four unemployed men, attended an Italian funeral, visited a church social, and dropped in on a Jewish wedding. He returned home at midnight. 7
Tammany Hall’s corruption, especially under the reign of William “Boss” Tweed, was legendary, but the public works projects that funded Tammany Hall’s graft also provided essential infrastructure and public services for the city’s rapidly expanding population. Water, sewer, and gas lines; schools, hospitals, civic buildings, and museums; police and fire departments; roads, parks (notably Central Park), and bridges (notably the Brooklyn Bridge): all could, in whole or in part, be credited to Tammany’s reign. Still, machine politics could never be enough. As the urban population exploded, many immigrants found themselves trapped in crowded, crime-ridden slums. Americans eventually took notice of this urban crisis and proposed municipal reforms but also grew concerned about the declining quality of life in rural areas.
While cities boomed, rural worlds languished. Some Americans scoffed at rural backwardness and reveled in the countryside’s decay, but many romanticized the countryside, celebrated rural life, and wondered what had been lost in the cities. Sociologist Kenyon Butterfield, concerned by the sprawling nature of industrial cities and suburbs, regretted the eroding social position of rural citizens and farmers: “Agriculture does not hold the same relative rank among our industries that it did in former years.” Butterfield saw “the farm problem” as part of “the whole question of democratic civilization.” 8 He and many others thought the rise of the cities and the fall of the countryside threatened traditional American values. Many proposed conservation. Liberty Hyde Bailey, a botanist and rural scholar selected by Theodore Roosevelt to chair a federal Commission on Country Life in 1907, believed that rural places and industrial cities were linked: “Every agricultural question is a city question.” 9
Many longed for a middle path between the cities and the country. New suburban communities on the outskirts of American cities defined themselves in opposition to urban crowding. Americans contemplated the complicated relationships between rural places, suburban living, and urban spaces. Los Angeles became a model for the suburban development of rural places. Dana Barlett, a social reformer in Los Angeles, noted that the city, stretching across dozens of small towns, was “a better city” because of its residential identity as a “city of homes.” 10 This language was seized upon by many suburbs that hoped to avoid both urban sprawl and rural decay. In Glendora, one of these small towns on the outskirts of Los Angeles, local leaders were “loath as anyone to see it become cosmopolitan.” Instead, in order to have Glendora “grow along the lines necessary to have it remain an enjoyable city of homes,” they needed to “bestir ourselves to direct its growth” by encouraging not industry or agriculture but residential development. 11


3.1: Functions
Terminology
The idea of mapping one set of objects to another is basic to mathematics. Mapping and correspondence are undefined terms. We call the set mapped from the domain and the set mapped to the codomain . The range is the subset of the codomain to which a function is mapped.
A mapping is a function if and only if each input is mapped to exactly one output.
A mapping is onto if and only if for each element of the codomain there exists some element of the domain mapped to it.
Onto functions are also referred to as surjections .
A mapping is one-to-one if and only if \(f(x)=f(a)\) implies \(x=a\).
One-to-one functions are also referred to as injections .
A bijection is a function that is one-to-one (injective) and onto (surjective).
The inverse mapping for a mapping \(f:D \to C\) is the mapping \(f^{-1}:C \to D\) such that \(f^{-1}(y)=x\) if and only if \(f(x)=y\).
Note that the inverse mapping of a function is also a function if and only if the function is one-to-one.
Consider the examples in Table \(\PageIndex{6}\) \(D=\{-3,-2,-1,0,1,2,3\}, C=\{1,1/2,1/3,1/4,1/5,1/6,1/7\}\).
| Name | Function | Onto | 1-1 | |
| \(f\) |
\(f(1)=1, f(2)=1/2, f(3)=1/3, f(4)=1/4, f(5)=1/5\) |
Yes | No | Yes |
| \(g\) |
\(g(1)=1, g(2)=1/2, g(3)=1/3, g(4)=1/2, g(5)=1/3\) |
Yes | No | No |
| \(h\) |
\(h(1)=1, h(2)=1/2, h(3)=1/3, h(3)=1/6, h(4)=1/4, h(5)=1/5, h(5)=1/7h\) |
No | Yes | No |
Because many functions will have numbers as their domains and/or codomains we use shorthand notation for many common sets of numbers. We denote the integers with \(Z\). We denote the real numbers with \(R\). If we want to refer to only positive numbers we use a + superscript. For example \(Z^+\) refers to the positive integers (also known as the counting numbers). Note as well that 0 is neither negative nor positive. If we want all the positive integers and zero we use \(Z^+ \cup \{0\}\).
Practice
Checkpoint \(\PageIndex{7}\)
Write a function from \(D=\{−3,−2,−1,0,1,2,3\}\) to \(C=\{0,1,2,3,4,5,6,7,8,9,10\}\).
Checkpoint \(\PageIndex{8}\)
Is your function onto? 1-1?
Checkpoint \(\PageIndex{9}\)
Write a 1-1 function from \(D\) to \(C\).
Checkpoint \(\PageIndex{10}\)
Can you write a function from \(D\) onto \(C\)?
Checkpoint \(\PageIndex{11}\)
Write a function from \(Z\) onto \(\{0,1\}\).


3: Transformations
Transformations will be the focus of this chapter. They are functions first and foremost, often used to push objects from one place in a space to a more convenient place, but transformations do much more. They will be used to define different geometries, and we will think of a transformation in terms of the sorts of objects (and functions) that are unaffected by it.
-
- 3.3: The Extended Plane
- Consider again inversion about the circle C given by |z−z_0|=r, and observe that points close to z_0 get mapped to points in the plane far away from z_0. In fact, a sequence of points in complex numbers whose limit is z_0 will be inverted to a sequence of points whose magnitudes go to ∞. Conversely, any sequence of points in complex numbers having magnitudes marching off to ∞ will be inverted to a sequence of points whose limit is z_0.
-
- 3.4: Möbius Transformations
- If we compose two Möbius transformations, the result is another Möbius transformation. Since Möbius transformations are composed of inversions, they will embrace the finer qualities of inversions. For instance, since inversion preserves clines, so do Möbius transformations, and since inversion preserves angle magnitudes, Möbius transformations preserve angles (as an even number of inversions).


12.2: The Leadership Process
-
- Last updated
- Save as PDF
2. What are the processes associated with people coming to leadership positions?
Leadership is a process, a complex and dynamic exchange relationship built over time between leader and follower and between leader and the group of followers who depend on each other to attain a mutually desired goal. 10 There are several key components to this "working relationship": the leader, the followers, the context (situation), the leadership process per se, and the consequences (outcomes) (see Exhibit 12.3 ). 11 Across time, each component interacts with and influences the other components, and whatever consequences (such as leader-follower trust) are created influence future interactions. As any one of the components changes, so too will leadership. 12
The Leader
Leaders are people who take charge of or guide the activities of others. They are often seen as the focus or orchestrater of group activity, the people who set the tone of the group so that it can move forward to attain its goals. Leaders provide the group with what is required to fulfill its maintenance and task-related needs. (Later in the chapter, we will return to the “leader as a person” as part of our discussion of the trait approach to leadership.)
The Follower
The follower is not a passive player in the leadership process. Edwin Hollander, after many years of studying leadership, suggested that the follower is the most critical factor in any leadership event. 13 It is, after all, the follower who perceives the situation and comes to define the needs that the leader must fulfill. In addition, it is the follower who either rejects leadership or accepts acts of leadership by surrendering his power to the leader to diminish task uncertainty, to define and manage the meaning of the situation to the follower, and to orchestrate the follower’s action in pursuit of goal attainment.
The follower’s personality and readiness to follow determine the style of leadership that will be most effective. For example, individuals with an internal locus of control are much more responsive to participative styles of leadership than individuals with an external locus of control. 14 Individuals with an authoritarian personality are highly receptive to the effectiveness of directive acts of leadership. 15 It is the followers' expectations, as well as their performance-based needs, that determine what a leader must do in order to be effective.
The strength of the follower’s self-concept has also been linked to the leadership process. High-self-esteem individuals tend to have a strong sense of self-efficacy, that is, a generalized belief they can be successful in difficult situations. They therefore tend to be strongly motivated to perform and persist in the face of adversity. 16 The high-self-esteem follower tends to be responsive to participative styles of leadership. Low-self-esteem individuals, who doubt their competence and worthiness and their ability to succeed in difficult situations, function better with supportive forms of leadership. This helps them deal with the stress, frustration, and anxiety that often emerge with difficult tasks. Followers without a readiness to follow, limited by their inability to perform and lack of motivation and commitment, usually need more directive forms of leadership. 17
Follower behavior plays a major role in determining what behaviors leaders engage in. For example, followers who perform at high levels tend to cause their leaders to be considerate in their treatment and to play a less directive role. Followers who are poor performers, on the other hand, tend to cause their leaders to be less warm toward them and to be more directive and controlling in their leadership style. 18
The Context
Situations make demands on a group and its members, and not all situations are the same. Context refers to the situation that surrounds the leader and the followers. Situations are multidimensional. We discuss the context as it pertains to leadership in greater detail later in this chapter, but for now let’s look at it in terms of the task and task environment that confront the group. Is the task structured or unstructured? Are the goals of the group clear or ambiguous? Is there agreement or disagreement about goals? Is there a body of knowledge that can guide task performance? Is the task boring? Frustrating? Intrinsically satisfying? Is the environment complex or simple, stable or unstable? These factors create different contexts within which leadership unfolds, and each factor places a different set of needs and demands on the leader and on the followers.
The Process
The process of leadership is separate and distinct from the leader (the person who occupies a central role in the group). The process is a complex, interactive, and dynamic working relationship between leader and followers. This working relationship, built over time, is directed toward fulfilling the group’s maintenance and task needs. Part of the process consists of an exchange relationship between the leader and follower. The leader provides a resource directed toward fulfilling the group’s needs, and the group gives compliance, recognition, and esteem to the leader. To the extent that leadership is the exercise of influence, part of the leadership process is captured by the surrender of power by the followers and the exercise of influence over the followers by the leader. 19 Thus, the leader influences the followers and the followers influence the leader, the context influences the leader and the followers, and both leader and followers influence the context.
The Consequences
A number of outcomes or consequences of the leadership process unfold between leader, follower, and situation. At the group level, two outcomes are important:
- Have the group’s maintenance needs been fulfilled? That is, do members of the group like and get along with one another, do they have a shared set of norms and values, and have they developed a good working relationship? Have individuals’ needs been fulfilled as reflected in attendance, motivation, performance, satisfaction, citizenship, trust, and maintenance of the group membership?
- Have the group’s task needs been met? That is, there are also important consequences of the leadership process for individuals: attendance, motivation, performance, satisfaction, citizenship, trust, and maintenance of their group membership.
The leader-member exchange (LMX) theory of the leadership process focuses attention on consequences associated with the leadership process. The theory views leadership as consisting of a number of dyadic relationships linking the leader with a follower. A leader-follower relationship tends to develop quickly and remains relatively stable over time. The quality of the relationship is reflected by the degree of mutual trust, loyalty, support, respect, and obligation. High- and low-quality relationships between a leader and each of his followers produce in and out groups among the followers. Members of the in group come to be key players, and high-quality exchange relationships tend to be associated with higher levels of performance, commitment, and satisfaction than are low-quality exchange relationships. 20 Attitudinal similarity and extroversion appear to be associated with a high-quality leader-member relationship. 21
The nature of the leadership process varies substantially depending on the leader, the followers, and the situation and context. Thus, leadership is the function of an interaction between the leader, the follower, and the context.
The leadership context for the leader of a group of assembly line production workers differs from the context for the leader of a self-managing production team and from the context confronted by the lead scientists in a research laboratory. The leadership tactics that work in the first context might fail miserably in the latter two.
catching the entrepreneurial spirit
How a Start-Up Finds the Right Leader
Start-ups, by their very nature, require innovation to bring new products and services to market. Along with establishing a new brand or product, the leader has to develop the relationships and processes that make a company succeed, or risk its early demise. While leading an established firm has its challenges, a start-up requires even more from a leader.
How critical is leadership to a start-up? Ask the four cofounders of the now-defunct PYP (Pretty Young Professionals), a website founded as a source of information for young professional women. What began as four young professional women working on a new start-up ended with hurt feelings and threats of legal action. In 2010, Kathryn Minshew, Amanda Pouchot, Caroline Ghosn, and Alex Cavoulacos decided to create the website and Minshew was named CEO (Cohan 2011a). Lines blurred about Minshew’s authority and the ultimate look, feel, and direction of the website. Ideals about shared leadership, where the company was going, and how it was going to get there ultimately got lost in the power shuffle. By June 2011, passwords were changed and legal actions began, and in August Minshew and Cavoulacos left altogether (Cohan 2011b).
When the legal haggling from PYP was over, Alex Cavoulacos and Kathryn Minshew, joined by Melissa McCreery, tried again. But this time, rather than hoping for the best, they put a leadership plan in place. Minshew was named CEO of the new start-up, The Daily Muse, with Cavoulacos as chief operating officer and McCreery as editor in chief. Rather than trusting to luck, the three cofounders based their team positions on strengths and personalities. Cavoulacos and McCreery agreed that Minshew’s outgoing personality and confidence made her the proper choice as CEO (Casserly 2013).
No single trait will guarantee that a person can lead a start-up from idea to greatness, but a survey of successful entrepreneurs does show some common traits. According to David Barbash, a partner at Boston-based law firm Posternak Blankstein & Lund LLP, personality is paramount: “You can have great technology but if you’re not a great communicator it may die in the lab” (Casserly 2013 n.p.). A start-up needs a leader who is confident and willing, if not eager, to face the future. According to Michelle Randall, a principal of Enriching Leadership International, start-up CEOs have to be willing to fundraise and not be too proud to beg (Casserly 2013). Peter Shankman, an entrepreneur and angel investor, says leaders have to be willing to make the hard decisions, even risking being the bad guy (Casserly 2013).
Gary Vaynerchuk credits his success to six factors. Angel investor, social media marketer, and early social media adopter, Vaynerchuk leveraged YouTube in its early years to market wine from the family’s liquor store, eventually increasing sales from $3 million to $60 million a year (Clifford 2017). Gary believes good leaders recognize that they don’t dictate to the market, but rather respond to where it is going. They have respect for and believe in other people, and have a strong work ethic, what Vaynerchuk called a “lunch pail work ethic”: they are willing to put in long hours because they love the work, not the perks. He also stresses that he loves technology and doesn’t fear it, is obsessed with the youth of today, and is optimistic about people and the future of humanity (Vaynerchuk 2017).
Leading a startup requires more than simple management. It requires the right leader for the right company at the right time, which means matching the right management skills with the proper flexibility and drive to keep it all together and moving in the right direction.
- Why would start-up leaders need different leadership qualities than someone managing an established firm?
- Vaynerchuk has been quoted as saying that if you live for Friday, get a different job. How does this apply to successful entrepreneurs?
Sources:
Casserly, Meghan. 2013. “Rocks, Paper, CEO: Finding The Best Leader For Your Startup.”
Forbes.
www.forbes.com/sites/meghanc...ding-the-best- leader-for-your-startup/#16b520cd20a5
Clifford, Catherine. 2017. “Self-made millionaire Gary Vaynerchuk: This is the real secret to success.” CNBC. www.cnbc.com/2017/03/13/self...k-shares-real- secret-to-success.html
Cohan, Peter. 2011a. “A Cautionary Tale: Friendship, Business Ethics, and Bad Breakups (Acts I and II).
Forbes.
August 9, 2011. www.forbes.com/sites/peterco.../#256d318b2735
Cohan, Peter. 2011b. “A Cautionary Tale: Friendship, Business Ethics, and Bad Breakups (Acts III and IV).
Forbes.
August 9, 2011. www.forbes.com/sites/peterco...utionary-tale- friendship-business-ethics-and-bad-breakups-acts-iii-and-iv/3/#66b22dd4a4f6
Vaynerchuk, Gary. 2017. “What Makes Me a Great CEO.”
https://www.garyvaynerchuk.com/makes-
great-ceo/
concept check
- What are the processes associated with people coming to leadership positions?


1.13.7: Equilibrium - Liquid-Solids - Hildebrand Rules
A given homogeneous liquid system (at pressure \(\mathrm{p}\)) contains two chemical substances \(\mathrm{i}\) and \(\mathrm{j}\) at temperature \(\mathrm{T}\). Chemical substance \(\mathrm{j}\) at temperature \(\mathrm{T}\) and \(\mathrm{p}\) is a liquid which being in vast excess in this system is the solvent. The system is cooled and pure solid substance \(\mathrm{i}\) separates out leaving the system less concentrated in the solute \(\mathrm{i}\). The solution is dilute and we assume that the thermodynamic properties of the solution are ideal. Then, from the Schroeder–van Laar Equation
\[-\ln \left[\mathrm{x}_{\mathrm{i}}(\mathrm{s} \ln )\right]=\frac{\left[\Delta_{\mathrm{fius}} \mathrm{H}_{\mathrm{i}}^{0}(\mathrm{~T}, \mathrm{p})\right]}{\mathrm{R}} \,\left[\frac{1}{\mathrm{~T}}-\frac{1}{\mathrm{~T}_{\mathrm{i}}^{0}}\right] \label{a} \]
\(\Delta_{\text {fus }} \mathrm{H}_{\mathrm{i}}^{0}\) is the molar enthalpy of fusion of chemical substance \(\mathrm{i}\), melting point \(\mathrm{T}_{\mathrm{i}}^{0}\). Mole fraction \(\mathrm{x}_{\mathrm{i}}(\mathrm{s} \ln )\) is the composition of the saturated solution at temperature \(\mathrm{T}\); i.e. the solubility of substance \(\mathrm{i}\). From Equation \ref{a},
\[\ln \left[\frac{1}{\mathrm{x}_{\mathrm{i}}(\mathrm{s} \ln )}\right]^{\mathrm{eq}}=\frac{\left[\Delta_{\mathrm{fus}} \mathrm{H}_{\mathrm{i}}^{0}(\mathrm{~T}, \mathrm{p})\right]}{\mathrm{R}} \,\left[\frac{\mathrm{T}_{\mathrm{i}}^{0}-\mathrm{T}}{\mathrm{T} \, \mathrm{T}_{\mathrm{i}}^{0}}\right] \label{b} \]
Equation \ref{b} forms the background to several generalisations concerning solubilities; i.e. Hildebrand Rules [1]. We note that \(\Delta_{\text {fus }} \mathrm{H}_{\mathrm{i}}^{0}(\mathrm{~T}, \mathrm{p})\) and \(\mathrm{T}_{\mathrm{i}}^{0}\) characterise the solute .
- Solubilities increase with increase in temperature.
- For two solutes with equal \(\Delta_{\text {fus }} \mathrm{H}_{\mathrm{i}}^{0}(\mathrm{~T}, \mathrm{p})\), the solute with lower \(\mathrm{T}_{\mathrm{i}}^{0}\) will be more soluble at a common temperature \(\mathrm{T}\).
- For two solutes with the same \(\mathrm{T}_{\mathrm{i}}^{0}\), the solid with lower \(\Delta_{\text {fus }} \mathrm{H}_{\mathrm{i}}^{0}(\mathrm{~T}, \mathrm{p})\) will be more soluble.
Footnote
[1] see I. Prigogine and R. Defay, Chemical Thermodynamics, transl. D. H. Everett, Longmans Greeen, London, 1953.


4.9: Database Management Systems
selected template will load here
This action is not available.
A database looks like one or more files. For the data in the database to be read, changed, added, or removed, a software program must access it. The software creates a database by building tables, forms, reports, and other important variables. Many software applications have this ability: iTunes can read its database to give you a listing of its songs (and play the songs); your mobile-phone software can interact with your list of contacts. Companies of all sizes use this software to enable themselves to streamline the data they have collected to be useful for multiple purposes such as marketing, customer service, and sales. Database management systems help businesses to collect complex data and customize it for their own use. When selecting Database Management Software (DBMS,) the company needs to know what they want to utilize and establish goals. Questions that need to be answered are; What software can you use to create a database, change a database’s structure, or analyze? For example, Apache OpenOffice.org Base can create, modify, and analyze databases in open-database (ODB) format. Microsoft’s Access DBMS is used to work with databases in its own Microsoft Access Database format. Both Access and Base have the ability to read and write to other database formats as well.
Microsoft Access and Open Office Base are examples of personal database-management systems. These systems are primarily used to develop and analyze single-user databases. These databases are not meant to be shared across a network or the Internet but are instead installed on a particular device and work with a single user at a time.


5.4: Introduction to Quadratic Residues and Nonresidues
The question that we need to answer in this section is the following. If \(p\) is an odd prime and \(a\) is an integer relatively prime to \(p\). Is \(a\) a perfect square modulo \(p\).
Let \(m\) be a positive integer. An integer \(a\) is a quadratic residue of \(m\) if \((a,m)=1\) and the congruence \(x^2\equiv a (mod \ m)\) is solvable. If the congruence \(x^2\equiv a (mod \ m)\) has no solution, then \(a\) is a quadratic nonresidue of \(m\).
Notice that \(1^2=6^2\equiv 1(mod \ 7)\), \(3^2=4^2\equiv 2(mod \ 7)\) and \(2^2=5^2\equiv 4(mod \ 7)\). Thus \(1,2,4\) are quadratic residues modulo 7 while \(3,5,6\) are quadratic nonresidues modulo 7.
Let \(p\neq 2\) be a prime number and \(a\) is an integer such that \(p\nmid a\). Then either a is quadratic nonresidue modulo \(p\) or \[x^2\equiv a(mod \ p)\] has exactly two incongruent solutions modulo \(p\).
If \(x^2\equiv a(mod \ p)\) has a solution, say \(x=x'\), then \(-x'\) is a solution as well. Notice that \(-x' \not\equiv x'(mod \ p)\) because then \(p\mid 2x'\) and hence \(p\nmid x_0\).
We now show that there are no more than two incongruent solutions. Assume that \(x=x'\) and \(x=x''\) are both solutions of \(x^2\equiv a(mod \ p)\). Then we have \[(x')^2- (x'')^2=(x'+x'')(x'-x'')\equiv 0(mod \ p).\] Hence \[x'\equiv x''(mod \ p)\ \ \mbox{or} \ \ x'\equiv -x''(mod \ p).\]
The following theorem determines the number of integers that are quadratic residues modulo an odd prime.
If \(p\neq 2\) is a prime, then there are exactly \((p-1)/2\) quadratic residues modulo \(p\) and \((p-1)/2\) quadratic nonresidues modulo \(p\) in the set of integers \(1,2...,p-1\).
To find all the quadratic residues of \(p\) among all the integers \(1,2,...,p-1\), we determine the least positive residue modulo \(p\) of \(1^2,2^2,...,(p-1)^2\). Considering the \(p-1\) congruences and because each congruence has either no solution or two incongruent solutions, there must be exactly \((p-1)/2\) quadratic residues of \(p\) among \(1,2,...,p-1\). Thus the remaining are \((p-1)/2\) quadratic nonresidues of \(p\).
Exercises
- Find all the quadratic residues of 3.
- Find all the quadratic residues of 13.
- find all the quadratic residues of 18.
- Show that if \(p\) is prime and \(p\geq 7\), then there are always two consecutive quadratic residues of \(p\). Hint: Show that at least one of \(2,5\) or 10 is a quadratic residue of \(p\).
- Show that if \(p\) is prime and \(p\geq 7\), then there are always two quadratic residues of \(p\) that differ by 3.
Contributors and Attributions
-
Dr. Wissam Raji, Ph.D., of the American University in Beirut. His work was selected by the Saylor Foundation’s Open Textbook Challenge for public release under a Creative Commons Attribution ( CC BY ) license.


12.2F: Neutralization Reaction
- Describe how neutralizing antibodies serve to block viral attachment to cells thus inhibiting viral replication
A neutralizing antibody defends a cell from an antigen or infectious body by inhibiting or neutralizing any effect it has biologically. The antibody response is crucial for preventing many viral infections and may also contribute to the resolution of an infection. When a vertebrate is infected with a virus, antibodies are produced against many epitopes of multiple virus proteins. A subset of these antibodies can block viral infection by a process called neutralization. This usually involves the formation of a virus-antibody complex.
This virus-antibody complex can prevent viral infections in many ways. It may interfere with virion binding to receptors, block uptake into cells, prevent uncoating of the genomes in endosomes, or cause aggregation of virus particles. Many enveloped viruses are lysed when antiviral antibodies and serum complement disrupt membranes. Antibodies can also neutralize viral infectivity by binding to cell surface receptors.
Neutralizing antibodies have shown potential in the treatment of retroviral infections. Medical professionals and researchers have shown how the encoding of genes which influence the production of this particular type of antibody could help in the treatment of infections that attack the immune system. Experts in the field have used HIV treatment as an example of infections these antibodies can treat. Recently, potent and broadly neutralizing human antibodies against influenza have been reported, and have suggested possible strategies to generate an improved vaccine that would confer long-lasting immunity. Another disease which has been linked to the production of neutralizing antibodies is multiple sclerosis.
In diagnostic immunology and virology laboratories, the evaluation of neutralizing antibodies, which destroy the infectivity of viruses, can be measured by the neutralization method. In this procedure, patient serum is mixed with a suspension of infectious virus particles of the same type as those suspected of causing disease in the patient. A control suspension of virus is mixed with normal serum and is then inoculated into an appropriate cell culture. If the patient serum contains antibody to the virus, the antibody will bind to the virus particles and prevent them from invading the cells in culture, thereby neutralizing the infectivity of the virus. This technique is labor-intensive, demanding, and time consuming. It application is restricted to laboratories that perform routine viral cultures and related diagnosis.
Key Points
- When a vertebrate is infected with a virus, antibodies are produced against it. Some of the antibodies can block viral infection by neutralization which is usually the result of a formation of a virus-antibody complex. This complex can prevent viral infections in many ways.
- Neutralizing antibodies have shown potential in the treatment of retroviral infections such as HIV. Recently, potent and broadly neutralizing human antibodies against influenza have been reported.
- In diagnostic immunology and virology laboratories, the evaluation of neutralizing antibodies, which destroy the infectivity of viruses, can be measured by the neutralization method.
Key Terms
- neutralization : In the immunological sense refers to the ability of antibodies to block the site(s) on bacteria or viruses that they use to enter their target cell. One example of this within biology is a neutralizing antibody.
- virion : A single individual particle of a virus (the viral equivalent of a cell).
- endosomes : membrane-bound compartments inside eukaryotic cells.