Modes of variation

In statistics, modes of variation are a continuously indexed set of vectors or functions that are centered at a mean and are used to depict the variation in a population or sample. Typically, variation patterns in the data can be decomposed in descending order of eigenvalues with the directions represented by the corresponding eigenvectors or eigenfunctions. Modes of variation provide a visualization of this decomposition and an efficient description of variation around the mean. Both in principal component analysis (PCA) and in functional principal component analysis (FPCA), modes of variation play an important role in visualizing and describing the variation in the data contributed by each eigencomponent. In real-world applications, the eigencomponents and associated modes of variation aid to interpret complex data, especially in exploratory data analysis (EDA).
Formulation.
Modes of variation are a natural extension of PCA and FPCA. 
Modes of variation in PCA.
If a random vector formula_1 has the mean vector formula_2, and the covariance matrix formula_3 with eigenvalues formula_4 and corresponding orthonormal eigenvectors formula_5, by eigendecomposition of a real symmetric matrix, the covariance matrix formula_6 can be decomposed as
where formula_8 is an orthogonal matrix whose columns are the eigenvectors of formula_6, and formula_10 is a diagonal matrix whose entries are the eigenvalues of formula_6. By the Karhunen–Loève expansion for random vectors, one can express the centered random vector in the eigenbasis 
where formula_13 is the principal component associated with the formula_14-th eigenvector formula_15, with the properties
Then the formula_18-th mode of variation of formula_19 is the set of vectors, indexed by formula_20, 
where formula_22 is typically selected as formula_23.
Modes of variation in FPCA.
For a square-integrable random function formula_24, where typically formula_25 and formula_26 is an interval, denote the mean function by formula_27, and the covariance function by 
where formula_29 are the eigenvalues and formula_30 are the orthonormal eigenfunctions of the linear Hilbert–Schmidt operator
By the Karhunen–Loève theorem, one can express the centered function in the eigenbasis, 
where 
is the formula_18-th principal component with the properties
Then the formula_18-th mode of variation of formula_38 is the set of functions, indexed by formula_20,
that are viewed simultaneously over the range of formula_20, usually for formula_42.
Estimation.
The formulation above is derived from properties of the population. Estimation is needed in real-world applications. The key idea is to estimate mean and covariance.
Modes of variation in PCA.
Suppose the data formula_43 represent formula_44 independent drawings from some formula_45-dimensional population formula_19 with mean vector formula_47 and covariance matrix formula_6. These data yield the sample mean vector formula_49, and the sample covariance matrix formula_50 with eigenvalue-eigenvector pairs formula_51. Then the formula_18-th mode of variation of formula_19 can be estimated by
Modes of variation in FPCA.
Consider formula_44 realizations formula_56 of a square-integrable random function formula_57 with the mean function formula_27 and the covariance function formula_59. Functional principal component analysis provides methods for the estimation of formula_60 and formula_61 in detail, often involving point wise estimate and interpolation. Substituting estimates for the unknown quantities, the formula_18-th mode of variation of formula_38 can be estimated by
Applications.
Modes of variation are useful to visualize and describe the variation patterns in the data sorted by the eigenvalues. In real-world applications, modes of variation associated with eigencomponents allow to interpret complex data, such as the evolution of function traits and other infinite-dimensional data. To illustrate how modes of variation work in practice, two examples are shown in the graphs to the right, which display the first two modes of variation. The solid curve represents the sample mean function. The dashed, dot-dashed, and dotted curves correspond to modes of variation with formula_65 and formula_66, respectively.
The first graph displays the first two modes of variation of female mortality data from 41 countries in 2003. The object of interest is log hazard function between ages 0 and 100 years. The first mode of variation suggests that the variation of female mortality is smaller for ages around 0 or 100, and larger for ages around 25. An appropriate and intuitive interpretation is that mortality around 25 is driven by accidental death, while around 0 or 100, mortality is related to congenital disease or natural death. 
Compared to female mortality data, modes of variation of male mortality data shows higher mortality after around age 20, possibly related to the fact that life expectancy for women is higher than that for men.