Sliced inverse regression

Sliced inverse regression (or SIR) is a tool for dimensionality reduction in the field of multivariate statistics.
In statistics, regression analysis is a method of studying the relationship between a response variable "y" and its input variable formula_1, which is a "p"-dimensional vector. There are several approaches in the category of regression. For example, parametric methods include multiple linear regression, and non-parametric methods include local smoothing.
As the number of observations needed to use local smoothing methods scales exponentially with high-dimensional data (as "p" grows), reducing the number of dimensions can make the operation computable. Dimensionality reduction aims to achieve this by showing only the most important dimension of the data. SIR uses the inverse regression curve, formula_2, to perform a weighted principal component analysis.
Model.
Given a response variable formula_3 and a (random) vector formula_4 of explanatory variables, SIR is based on the model
where formula_6 are unknown projection vectors, formula_7 is an unknown number smaller than formula_8, formula_9 is an unknown function on formula_10as it only depends onformula_7 arguments, and formula_12 is a random variable representing error with formula_13 and a finite variance of formula_14. The model describes an ideal solution, where formula_3 depends on formula_4 only through aformula_7 dimensional subspace; i.e., one can reduce the dimension of the explanatory variables fromformula_8 to a smaller numberformula_7 without losing any information.
An equivalent version of formula_20 is: the conditional distribution of formula_21 given formula_22 depends on formula_22 only through the formula_7 dimensional random vector formula_25. It is assumed that this reduced vector is as informative as the original formula_26 in explaining formula_27.
The unknown formula_28 are called the "effective dimension reducing directions" (EDR-directions). The space that is spanned by these vectors is denoted by the "effective dimension reducing space" (EDR-space).
Relevant linear algebra background.
Given formula_29, then formula_30, the set of all linear combinations of these vectors is called a linear subspace and is therefore a vector space. The equation says that vectors formula_31 span formula_32, but the vectors that span space formula_32 are not unique.
The dimension of formula_34 is equal to the maximum number of linearly independent vectors in formula_32. A set of formula_36 linear independent vectors of formula_37 makes up a basis of formula_37. The dimension of a vector space is unique, but the basis itself is not. Several bases can span the same space. Dependent vectors can still span a space, but the linear combinations of the latter are only suitable to a set of vectors lying on a straight line.
Inverse regression.
Computing the inverse regression curve (IR) means instead of looking for 
it is actually
The center of the inverse regression curve is located at formula_44. Therefore, the centered inverse regression curve is
which is a formula_8 dimensional curve in formula_40.
Inverse regression versus dimension reduction.
The centered inverse regression curve lies on a formula_7-dimensional subspace spanned by formula_49. This is a connection between the model and inverse regression.
Given this condition and formula_20, the centered inverse regression curve formula_45 is contained in the linear subspace spanned by formula_52, where formula_53.
Estimation of the EDR-directions.
After having had a look at all the theoretical properties, the aim now is to estimate the EDR-directions. For that purpose, weighted principal component analyses are needed. If the sample means formula_54, formula_55 would have been standardized to formula_56. Corresponding to the theorem above, the IR-curve formula_57 lies in the space spanned by formula_58, where formula_59. As a consequence, the covariance matrix formula_60 is degenerate in any direction orthogonal to the formula_61. Therefore, the eigenvectors formula_62 associated with the largestformula_63 eigenvalues are the standardized EDR-directions.
Algorithm.
The algorithm to estimate the EDR-directions via SIR is as follows.
1. Let formula_64 be the covariance matrix of formula_55. Standardize formula_55 to
(formula_20 can also be rewritten as
where formula_70.)
2. Divide the range of formula_71 into formula_72 non-overlapping slices formula_73 is the number of observations within each slice and formula_74 is the indicator function for the slice:
3. Compute the mean of formula_76 over all slices, which is a crude estimate formula_77 of the inverse regression curve formula_78:
4. Calculate the estimate for formula_80:
5. Identify the eigenvalues formula_82 and the eigenvectors formula_83 of formula_84, which are the standardized EDR-directions.
6. Transform the standardized EDR-directions back to the original scale. The estimates for the EDR-directions are given by: