# Causal Inference with Corrupted Data:
Measurement Error, Missing Values,
Discretization, and Differential Privacy

Anish Agarwal
Amazon Core AI  Rahul Singh
MIT Economics We thank Alberto Abadie, Isaiah Andrews, Josh Angrist, Dmitry
Arkhangelsky, David Autor, Abhijit Banerjee, David Bruns-Smith, Victor
Chernozhukov, Jon Cohen, Rachel Cummings, Peng Ding, Avi Feller, Bryan Graham,
Florian Gunsilius, Chris Harshaw, Skip Hirshberg, Guido Imbens, Simon Jaeger,
Michael Jansson, Michael Jordan, Patrick Kline, Lihua Lei, Lexin Li, Anna
Mikusheva, Ismael Mourifie, Sendhil Mullainathan, Whitney Newey, Betsy Ogburn,
Demian Pouzo, Jim Powell, Ashesh Rambachan, James Robins, Jesse Rothstein,
Andrea Rotnitzky, Frank Schilbach, Vira Semenova, Devavrat Shah, Garima
Sharma, Dennis Shen, Nakul Singh, Jann Spiess, Liyang Sun, Tavneet Suri,
Vasilis Syrgkanis, Panos Toulis, Suhas Vijaykumar, and Bin Yu for helpful
comments. We thank Leon Deng, Ajinkya Gundaria, and Caleb Rollins for their
contributions through the MIT Undergraduate Research Opportunity Program.
Rahul Singh thanks the Jerry Hausman Dissertation Fellowship. Part of this
work was done while both authors were visiting the Simons Institute for the
Theory of Computing.

(Original draft: July 6, 2021. This draft: November 9, 2022)

###### Abstract

The US Census Bureau will deliberately corrupt data sets derived from the 2020
US Census in an effort to maintain privacy, suggesting a painful trade-off
between the privacy of respondents and the precision of economic analysis. To
investigate whether this trade-off is inevitable, we formulate a
semiparametric model of causal inference with high dimensional corrupted data.
We propose a procedure for data cleaning, estimation, and inference with data
cleaning-adjusted confidence intervals. We prove consistency, Gaussian
approximation, and semiparametric efficiency by finite sample arguments, with
a rate of $n^{-1/2}$ for semiparametric estimands that degrades gracefully for
nonparametric estimands. Our key assumption is that the true covariates are
approximately low rank, which we interpret as approximate repeated
measurements and validate in the Census. In our analysis, we provide
nonasymptotic theoretical contributions to matrix completion, statistical
learning, and semiparametric statistics. Calibrated simulations verify the
coverage of our data cleaning-adjusted confidence intervals and demonstrate
the relevance of our results for 2020 Census data.

Keywords: disclosure avoidance, heterogeneous treatment effect, matrix
completion, semiparametric efficiency

###  1 Introduction

####  1.1 Motivation and research question

The 2010 US Census inadvertently revealed too much information. In a simulated
hack, researchers at the Census Bureau were able to re-identify between 52 and
179 million respondents from ostensibly anonymous summary tables [Hawes,
2021]. To protect privacy, the Bureau will inject synthetic noise into
forthcoming summary tables of the 2020 Census [Jarmin, 2019] and coarsen wage
microdata in the Current Population Survey (CPS) [Benedetto et al., 2022].
Techniques like these, called privacy mechanisms in computer science,
guarantee a particular notion of privacy called differential privacy via
deliberate data corruption [Dwork et al., 2006]. Differential privacy is
widely implemented in the technology sector, e.g. Apple iOS and Google Chrome
data. Due to its recent adoption in the government sector, researchers in
economics and statistics have warned of a looming trade-off: the privacy of
respondents versus the precision of economic analysis [Duchi et al., 2018,
Abowd and Schmutte, 2019, Hotz et al., 2022].

We study differential privacy and discretization as modern challenges for
causal inference. Economic data continue to suffer from classical types of
data corruption in the form of missing values and measurement error
[Griliches, 1986]. Therefore we analyze a class of data corruptions that
encompasses both modern and classical issues simultaneously, while remaining
agnostic about their relative magnitudes. Our research question is not only
how to conduct causal inference, but if it is even possible, with high
dimensional economic data that suffer from measurement error, missing values,
discretization, and differential privacy.

####  1.2 Contributions

We study a broad class of causal parameters, including semiparametric
estimands such as the average treatment effect, the local average treatment
effect, and the average elasticity, as well as nonparametric estimands such as
heterogeneous treatment effects, in a nonlinear and high dimensional setting.
Our main contribution is a procedure for automatic data cleaning, causal
estimation, and finally inference with confidence intervals that account for
the bias and variance consequences of data cleaning. Our key assumption is
that the true covariates are approximately low rank, which we validate for US
Census data and interpret from a causal perspective. In particular, we argue
that covariates collected from the Census include approximate repeated
measurements—e.g. disability benefits, medical benefits, and unemployment
benefits—which implies that they are approximately low rank. This phenomenon
powers our entire analysis. There are three key aspects of our contribution.

First, our relatively simple procedure adapts to the type and level of data
corruption. The same code works in a variety of settings, allowing for
classical corruptions such as measurement error and missing values as well as
modern corruptions such as discretization and differential privacy. Crucially,
the researcher does not need to know in advance the corruption distribution,
e.g. its parametric form or covariance structure, and in this way we depart
from the error-in-variable Lasso and Dantzig literatures [Loh and Wainwright,
2012, Rosenbaum and Tsybakov, 2013, Datta and Zou, 2017]. We depart from
previous work on principal component regression [Agarwal et al., 2021, Agarwal
et al., 2020a] by proposing new variants for causal inference. We propose an
error-in-variable balancing weight that adapts to the causal parameter of
interest—a natural yet original solution. In particular, the error-in-variable
balancing weight appears to be the first of its kind. In the way our procedure
handles missing values, it shares principles with multiple imputation [Rubin,
1976]. In the appendix, we extend our method to handle outcome attrition
[Heckman, 1979, Hausman and Wise, 1979, Das et al., 2003, Huber, 2014, Bia et
al., 2020, Singh, 2021].

Second, our theoretical analysis allows the rate of data cleaning to be slower
than the rate of causal inference, so an analyst can use matrix completion
[Candès and Recht, 2009, Candès and Tao, 2010, Keshavan et al., 2009, Hastie
et al., 2015, Chatterjee, 2015] for automatic data cleaning of covariates.
This key result extends the classic semiparametric framework, where the goal
is to obtain $n^{-1/2}$ convergence for the causal parameter despite a
possibly slow rate of convergence for a nonparametric regression. Our goal is
to obtain $n^{-1/2}$ convergence for the causal parameter despite a possibly
slow rate of convergence for high dimensional data cleaning. We build on both
the classic semiparametric literature for low dimensional domains [Hasminskii
and Ibragimov, 1979, Klaassen, 1987, Robinson, 1988, Bickel et al., 1993,
Newey, 1994, Andrews, 1994, Robins et al., 1995, Robins and Rotnitzky, 1995,
Ai and Chen, 2003, van der Laan and Rubin, 2006] and more recent developments
for high dimensional domains [Zheng and van der Laan, 2011, Chernozhukov et
al., 2016, Chernozhukov et al., 2018a, Chernozhukov et al., 2022a,
Chernozhukov et al., 2021]. En route, we extend semiparametric and
nonparametric debiased machine learning theory to i.n.i.d. corrupted data,
with new results on nominal and conservative variance estimation that are of
independent interest. We also develop an original theory of implicit data
cleaning. Altogether, our framework translates slow, on-average data cleaning
guarantees into fast causal estimation and inference guarantees.

Third, our empirical results suggest that there exist scenarios in which the
trade-off between privacy and precision can be overcome, and others in which
it cannot. We replicate and extend [Autor et al., 2013]’s seminal paper about
the effect of import competition on US labor markets. To begin, we demonstrate
the plausibility of our key assumption: Census data products contain many
variables that are approximate repeated measurements. Next, we deliberately
corrupt the data, injecting synthetic noise calibrated to the privacy level
mandated for the 2020 US Census. We implement differential privacy and
discretization in a way that belongs to our class of data corruptions, which
can therefore be cleaned and adjusted for in the confidence interval. We find
that the main results of [Autor et al., 2013] can be recovered without losing
statistical precision. In this representative setting for economic research,
it appears to be possible to achieve both privacy at the individual level and
precision at the population level.

The structure of this paper is as follows. Section 2 situates our
contributions within the context of related work. Section 3 formalizes our
class of data corruptions and our key assumption. Section 4 proposes our
procedure and demonstrates its performance in simulations. Section 5
theoretically justifies our procedure, and verifies the key assumption for
nonlinear factor models. Section 6 presents the semi-synthetic exercise and
discusses limitations. Section 7 concludes. For readability, we reserve
certain details for the appendix: additional examples in Appendix A, further
empirical results in Appendix B, and nonlinear basis functions in Appendix C.

###  2 Related work

Semiparametric statistics. We use two insights from classic semiparametric
theory. First, a causal parameter typically has regression and balancing
weight representations, and both appear in the semiparametrically efficient
asymptotic variance [Newey, 1994]. We directly build on this insight: an
error-in-variable regression and an error-in-variable balancing weight appear
in our data cleaning-adjusted confidence intervals. Both quantities also
appear in our doubly robust estimating equation [Robins et al., 1995, Robins
and Rotnitzky, 1995, van der Laan and Rubin, 2006, Chernozhukov et al., 2016,
Chernozhukov et al., 2018a, Foster and Syrgkanis, 2019, Chernozhukov et al.,
2022a]. Second, sample splitting [Klaassen, 1987] eliminates the restrictive
condition that function classes used in estimation must be simple [Zheng and
van der Laan, 2011, Chernozhukov et al., 2016, Chernozhukov et al., 2018a,
Chernozhukov et al., 2021]. We combine these two classic ideas with implicit
data cleaning, which appears to be a new idea.

Low rank models in econometrics. A vast literature studies the identification,
estimation, and inference of latent factors $(\lambda_{i},\mu_{j})$ in models
of the form

$Z_{i,\cdot}=X_{i,\cdot}+H_{i,\cdot},\quad X_{ij}=\lambda_{i}^{T}\mu_{j}$ (1)

where $Z_{i,\cdot}$ is observed, the ambient dimension $dim(X_{i,\cdot})$ is
high and growing, and the latent dimension $dim(\lambda_{i})$ is low and fixed
[Bai and Ng, 2002, Onatski, 2009, Bai and Ng, 2013, Bai and Wang, 2014].
Rather than imposing a fixed linear factor model, we require that the
approximate rank of $X_{i,\cdot}$ diverges much more slowly than
$dim(X_{i,\cdot})$. The nonlinear factor model
$X_{ij}=g(\lambda_{i},\mu_{j})$, where $dim(\lambda_{j})$ may be diverging, is
sufficient but unnecessary for our analysis. Like the factor model literature,
we allow for weak correlation and heteroscedastity of measurement errors
within units [Bai, 2003, Bai, 2009]. We further allow for missingness and do
not need to identify the latent factors for downstream causal inference.

Whereas we study treatment effects, policy effects, and elasticities in cross
sectional data, a rich literature studies panel data settings where the
analyst observes many units over time. One strand of this literature considers
inference on the latent factors themselves in factor augmented regressions
[Stock and Watson, 2002, Bai and Ng, 2006, Wang and Fan, 2017]. Another strand
considers inference for average treatment on the treated via synthetic
control. In particular, works in this strand posit a low rank [Chernozhukov et
al., 2018b, Bai and Ng, 2019, Xiong and Pelger, 2019, Fernández-Val et al.,
2020, Agarwal et al., 2020b, Feng, 2020, Athey et al., 2021] or approximately
low rank [Arkhangelsky et al., 2019, Agarwal et al., 2021] factor model for
potential outcomes over units and times. Observed outcomes are interpreted as
corrupted potential outcomes with idiosyncratic noise. By contrast, we study
general nonseparable causal models, and we interpret observed covariates as
corrupted observations of true covariates that are approximately low rank.

The only previous work to consider both measurement error and missingness in
cross sectional treatment effects appears to be [Kallus et al., 2018]. The
authors consider average treatment effect and prove consistency, without
inference, for a parametric linear model. In [Kallus et al., 2018], the true
covariates $X_{i,\cdot}$ are low dimensional and Gaussian; each corrupted
covariate $Z_{ij}$ is drawn independently from a known exponential family
distribution; and each missing value is drawn i.i.d. Consistency requires
$dim(X_{i,\cdot})\ll n$ and correct specification of the measurement error
distribution. By contrast, we consider a broad class of semiparametric and
nonparametric causal parameters. Moreover, we prove Gaussian approximation and
semiparametric efficiency with data cleaning-adjusted confidence intervals. We
do not assume knowledge of the distributions of $X_{i,\cdot}$ and
$Z_{i,\cdot}$; we allow $X_{i,\cdot}$ to be high dimensional; and we allow for
dependent and i.n.i.d. data corruption.

Error-in-variable regression. We provide a framework to repurpose error-in-
variable regression estimators for downstream causal inference. Error-in-
variables regression has a vast literature spanning econometrics, statistics,
and computer science studying the model

$Y_{i}=\gamma_{0}(X_{i,\cdot})+\varepsilon_{i},\quad
Z_{i,\cdot}=X_{i,\cdot}+H_{i,\cdot}$ (2)

where $(X_{i,\cdot},\varepsilon_{i},H_{i,\cdot})$ are mutually independent and
$(\varepsilon_{i},H_{i,\cdot})$ are mean zero [Schennach and Hu, 2013]. We
consider a generalization of this setting with missingness, and we define our
causal parameter as a scalar summary of $\gamma_{0}$.

Methods in econometrics typically assume auxiliary information: repeated
measurements [Li and Vuong, 1998, Li, 2002, Delaigle et al., 2008],
instrumental variables [Newey, 2001, Schennach, 2007, Hu and Schennach, 2008,
Wang and Hsiao, 2011, Chen et al., 2011, Singh et al., 2019], and negative
controls [Miao et al., 2018, Miao and Tchetgen, 2018, Deaner, 2018, Tchetgen
Tchetgen et al., 2020, Singh, 2020]. We do not require explicit auxiliary
information, though there is a deep connection to the repeated measurement
model, which we describe in Section 3.

An important class of methods in statistics extends Lasso and the Dantzig
selector [Loh and Wainwright, 2012, Rosenbaum and Tsybakov, 2013, Datta and
Zou, 2017]. The model is high dimensional, linear, and sparse:

$Y_{i}=X_{i,\cdot}\beta+\varepsilon_{i},\quad
Z_{i,\cdot}=(X_{i,\cdot}+H_{i,\cdot})\odot\pi_{i,\cdot}$ (3)

where $dim(X_{i,\cdot})$ may exceed $n$, $\beta$ has $s\ll n$ nonzero
coefficients, and $\pi_{i,\cdot}\in\\{\texttt{NA},1\\}^{dim(X_{i,\cdot})}$
encodes missingness. Analysis places three strong assumptions: exact sparsity
of $\beta$, a restricted eigenvalue bounded away from zero, and knowledge of
the covariance of measurement error $H_{i,\cdot}$. By contrast, we assume
$X_{i,\cdot}$ are approximately low rank and $H_{i,\cdot}$ are subexponential;
the analyst does not need to know the measurement error covariance.

We propose new variants of principal component regression (PCR) for the error-
in-variable regression and balancing weight. Previous work studies PCR for
error-in-variable regression only, considering models as in (3). [Agarwal et
al., 2021] perform data cleaning on the training set and test set together,
while [Agarwal et al., 2020a] perform data cleaning on the training set and
test set separately. Explicit data cleaning on the test set induces
correlation across observations, and therefore poses a challenge for
downstream statistical inference. We use implicit data cleaning on the test
set to preserve independence, and we prove fast rates of generalization.
Unlike previous work, we simultaneously allow $X_{i,\cdot}$ to be
approximately low rank, $\gamma_{0}$ to be nonlinear, and
$(\pi_{ij},\pi_{ik})$ to be dependent.

Missing values. The literature on missing covariates considers a rich variety
of assumptions. We focus on the simple case that covariates are missing
completely at random (MCAR). We do, however, allow for dependent missingness
within a unit. For the richer setting where covariates are missing at random
(MAR), popular methods involve multiple imputation [Rubin, 1976], generalized
propensity scores [Rosenbaum and Rubin, 1984, D’Agostino Jr and Rubin, 2000],
hot deck imputation [Abadie and Imbens, 2012], and doubly robust extensions
thereof [Mayer et al., 2020].

###  3 Model overview

We define the key aspects of the model: the causal parameter, class of data
corruptions, and key approximation assumption.

####  3.1 Causal parameter

For readability, we focus on one causal parameter in the main text: the
average treatment effect with i.n.i.d. data, which we denote by

$\theta_{0}=\frac{1}{n}\sum_{i=1}^{n}\theta_{i},\quad\theta_{i}=\mathbb{E}[Y_{i}^{(1)}-Y_{i}^{(0)}],$

where $Y_{i}^{(d)}$ is the potential outcome for unit $i$ under intervention
$D=d$. $\theta_{0}$ is a sample average because different units may be drawn
from different distributions. In Appendix A, we consider a general class of
semiparametric and nonparametric causal parameters including the local average
treatment effect, the average elasticity, and heterogeneous treatment effects.

We denote the actual outcome by $Y_{i}\in\mathbb{R}$, the assigned treatment
by $D_{i}\in\\{0,1\\}$, and the vector of covariates that determine treatment
selection by $X_{i,\cdot}\in\mathbb{R}^{p}$. In order to express $\theta_{0}$
in terms of $(Y_{i},D_{i},X_{i,\cdot})$, we impose some additional structure
on the problem. Generalizing a classic assumption in the literature on
distribution shift, we assume that the conditional distributions
$\mathbb{P}(Y_{i}|D_{i},X_{i,\cdot})$ and $\mathbb{P}(D_{i}|X_{i,\cdot})$ are
common across units; distribution shift is only in the marginal distributions
of covariates $\mathbb{P}_{i}(X_{i,\cdot})$.

Imposing these conditions as well as selection on $X_{i,\cdot}$, we recover
two classic formulations of the treatment effect. The outcome formulation is
in terms of the outcome mechanism $\gamma_{0}$, also called the regression,
which is common across units:

$\theta_{i}=\mathbb{E}[\gamma_{0}(1,X_{i,\cdot})-\gamma_{0}(0,X_{i,\cdot})],\quad\gamma_{0}(D_{i},X_{i,\cdot})=\mathbb{E}[Y_{i}|D_{i},X_{i,\cdot}].$

The treatment formulation is in terms of the treatment mechanism
$\mathbb{E}[D_{i}|X_{i,\cdot}]$, which is also common across units, and which
appears in the denominator of the balancing weight $\alpha_{0}$:

$\theta_{i}=\mathbb{E}[Y_{i}\cdot\alpha_{0}(D_{i},X_{i,\cdot})],\quad\alpha_{0}(D_{i},X_{i,\cdot})=\frac{D_{i}}{\mathbb{E}[D_{i}|X_{i,\cdot}]}-\frac{1-D_{i}}{1-\mathbb{E}[D_{i}|X_{i,\cdot}]}.$

Our estimation and analysis combine both classic formulations.

####  3.2 Data corruption

The crux of our problem is that, instead of observing $(Y,D_{i},X_{i,\cdot})$,
we observe $(Y_{i},D_{i},Z_{i,\cdot})$ where

$Y_{i}=\gamma_{0}(D_{i},X_{i,\cdot})+\varepsilon_{i},\quad
Z_{i,\cdot}=(X_{i,\cdot}+H_{i,\cdot})\odot\pi_{i,\cdot}.$ (4)

Though the outcome $Y_{i}$ is generated from treatment $D_{i}$ and true
covariates $X_{i,\cdot}$, we do not observe the true covariates; instead, we
observe the corrupted covariates $Z_{i,\cdot}$, which are the true covariates
$X_{i,\cdot}$ plus additive corruption $H_{i,\cdot}$, multiplied by a masking
vector $\pi_{i,\cdot}\in\\{\texttt{NA},1\\}^{p}$. Our concise model (4)
generalizes the models of previous work (1), (2), (3), and it encompasses all
four types of corruption.111It may also be viewed as a concise nonlinear
LISREL model [Jöreskog and Sörbom, 1996]. For example, to encode classical
measurement error [Schennach, 2016], one could let $Z_{i,\cdot}$ equal
$X_{i,\cdot}$ plus a vector of Gaussian noise. To encode missing values, let
$Z_{i,\cdot}=X_{i,\cdot}\odot\pi_{i,\cdot}$.

Discretization is a process by which a continuous vector $X_{i,\cdot}$ is
mapped to a discrete vector $Z_{i,\cdot}$, and our class encodes several
variants. For example, the covariate of interest may be a vector of
probabilities $X_{i,\cdot}$, yet we observe actual occurrences
$Z_{i,\cdot}\sim\text{Bernoulli}(X_{i,\cdot})$. Another example is randomized
rounding, where continuous values are randomly rounded to nearby integers,
e.g. $Z_{i,\cdot}=sign(X_{i,\cdot})\text{Poisson}(|X_{i,\cdot}|)$. While our
class does not include rounding in the familiar sense, it provides guidance on
which types of rounding can be handled in downstream causal inference. As
such, it suggests alternative types of discretization for wage data in the CPS
which are more favorable for economic research.

Finally, to encode differential privacy, let $Z_{i,\cdot}$ equal $X_{i,\cdot}$
plus a vector of Laplacian noise. What is the connection between differential
privacy and Laplacian noise? Differential privacy is a concept from computer
science which means plausible deniability that any individual contributed
their data to tabular summaries. The canonical mechanism that ensures
differential privacy is to add Laplacian noise, calibrating the variance of
the Laplacian to the variability of the true values and other properties of
the tabular summary statistics [Dwork et al., 2006]. In the context of the
Census, we consider adding Laplacian noise to data on aggregate units, which
we formalize in Section 6. Injecting synthetic noise in this way helps to
prevent the kind of attack simulated on the 2010 Census.

Across examples, $H_{i,\cdot}$ is sub-exponential, i.e. its tails are no worse
than the tails of an exponential distribution. So are compositions of various
types of data corruption since the class of sub-exponential distributions is
closed under addition. Therefore our class of data corruptions includes
classical and modern issues simultaneously. In particular, it allows us to
address the trade-off between privacy and precision in the context of
measurement error—a major aspect of the problem that is often overlooked
[Steed et al., 2022]. In Appendix A, we extend the model to accommodate
attrition and privacy of the outcome.

####  3.3 Key assumption: Approximate repeated measurements

Our key assumption is that the true covariates $X_{i,\cdot}$ are approximately
low rank: the rank of the matrix $\boldsymbol{X}\in\mathbb{R}^{n\times p}$ is
approximately $r\ll(n,p)$. Among the $n$ units in the data set, there are
approximately only $r$ latent types of unit, i.e., each unit can be
approximated as a linear combination of $r$ latent types. Equivalently, among
the $p$ covariates in the data set, there are approximately only $r$ latent
types of covariate. To interpret this assumption, we propose the intuition of
repeated measurements. In the classic repeated measurement model, we have
access to two noisy measurements $(Z_{i1},Z_{i2})$ of one signal; in our
model, we have access to $p$ noisy measurements $(Z_{i1},...,Z_{ip})$ that are
approximately repeated measurements of only $r$ signals, where both $(r,p)$
grow with sample size $n$, yet $r\ll(n,p)$.

We place this assumption because it holds in Census data. Previewing our
empirical application, consider the commuting zone (CZ) level data set of
[Autor et al., 2013]. The mainland US consists of 722 CZs interpretable as
local economies, each of which has a vector of covariates
$X_{i,\cdot}\in\mathbb{R}^{13}$ used in the authors’ main specification. The
variables include percent employment in manufacturing, percent college
educated, and percent employment among women—variables that are not precisely
repeated measurements but approximately so. We compute the singular value
decomposition of $\boldsymbol{X}$ then visualize its singular values, also
called its principal components, in Figure 1(a). We see that only about five
principal components are significantly positive; $r=5$. In Figure 1(b), we
confirm similar results with an augmented specification
$X_{i,\cdot}\in\mathbb{R}^{30}$ using additional variables from [Autor et al.,
2013]’s appendix such as average disability, unemployment, and medical
benefits. Again, these variables admit interpretation as approximate repeated
measurements with $r=5$. In Appendix B, we verify our key assumption on a
broad variety of economic data sets.

(a) Original specification

(b) Augmented specification

Figure 1:  The key assumption holds in Census data

Our key assumption is more than statistically convenient; it is causally
meaningful. Consider the special case in which the true covariates are exactly
low rank, i.e. $r=rank(\boldsymbol{X})$. The singular value decomposition is
$\boldsymbol{X}=\boldsymbol{U}\boldsymbol{\Sigma}\boldsymbol{V}^{T}$ where
$\boldsymbol{U}\in\mathbb{R}^{n\times r}$,
$\boldsymbol{\Sigma}\in\mathbb{R}^{r\times r}$, and
$\boldsymbol{V}\in\mathbb{R}^{p\times r}$. $\boldsymbol{V}$ consists of $r$
vectors in $\mathbb{R}^{p}$, called the right singular vectors of
$\boldsymbol{X}$, which are also the eigenvectors of the empirical covariance
$n^{-1}\boldsymbol{X}^{T}\boldsymbol{X}$. The span of these vectors is an $r$
dimensional subspace of $\mathbb{R}^{p}$, i.e. a low dimensional subset of a
high dimensional ambient space. In this scenario, we are assuming that
treatment assignment is determined by the subspace. Equivalently, the
treatment assignment for unit $i$ depends on the projection of $X_{i,\cdot}$
onto this subspace. More generally, when covariates are approximately low
rank,
$\boldsymbol{X}=\boldsymbol{X}^{\textsc{(lr)}}+\boldsymbol{E}^{\textsc{(lr)}}$,
where
$\boldsymbol{X}^{\textsc{(lr)}}=\boldsymbol{U}\boldsymbol{\Sigma}\boldsymbol{V}^{T}$
is a rank $r$ approximation to $\boldsymbol{X}$ and
$\boldsymbol{E}^{\textsc{(lr)}}$ is the approximation residual. We can either
assume (i) selection is determined by $\boldsymbol{X}^{\textsc{(lr)}}$ only,
i.e. the subspace spanned by $\boldsymbol{V}$; or (ii) selection is determined
by both $\boldsymbol{X}^{\textsc{(lr)}}$ and $\boldsymbol{E}^{\textsc{(lr)}}$.
To handle the latter case, which is the most general, we keep track of
$\Delta_{E}=\|\boldsymbol{E}^{\textsc{(lr)}}\|_{\max}$ in our theoretical
analysis. In this sense, we provide analysis that is robust to violations of
the exactly low rank assumption from both a statistical and causal
perspective.

###  4 Data cleaning-adjusted confidence interval

We would like a procedure that estimates causal parameters as if data were
uncorrupted, yet adjusts for data cleaning in the confidence interval.
Moreover, we would like a procedure that does not require knowledge of the
corruption covariance structure in advance, departing from previous work. If
such a procedure were to exist, it would in some sense preempt the looming
trade-off between privacy and precision.

####  4.1 Why is inference hard?

We illustrate key concepts with an average treatment effect simulation. By
construction, the treatment effect is $\theta_{0}=2.2$. We consider a data
generating process, detailed in Appendix I, which satisfies our key
assumption: one sample involves a matrix of covariates
$\boldsymbol{X}\in\mathbb{R}^{100\times 100}$ with rank $r=5$. See Appendix B
for similar results using alternative dimensions of $\boldsymbol{X}$. To make
the problem interesting, we allow for nonlinear outcome and treatment
mechanisms. Figure 2 plots the principal components of true covariates
$\boldsymbol{X}$ in red. As expected, five principal components are nonzero
and the rest are zero since $rank(\boldsymbol{X})=5$.

Figure 2: The key assumption holds in simulated data

As a first pass, we implement ordinary least squares (OLS) with robust
standard errors in Stata: regress Y D Z, vce(robust). Running OLS on clean
data 1000 times, the point estimates $\hat{\theta}$ (Figure 3(a)) are centered
around the true value of 2.2, and appear Gaussian. To evaluate the quality of
OLS standard errors, we visualize how the studentized point estimates
$(\hat{\theta}-\theta_{0})/\hat{\sigma}$ compare the the standard normal
density (Figure 3(b)). OLS works well in the absence of data corruption; there
is nothing hidden in the data generating process for clean data. We repeat
this exercise introducing measurement error with variance that is 20% of the
variance of the covariates. Inversion of the empirical covariance matrix
$n^{-1}\boldsymbol{Z}^{T}\boldsymbol{Z}$ becomes numerically unstable,
manifesting in point estimates that are erratic (Figure 4) and standard errors
that are typically NA’s. OLS is not well-suited to the combination of high
dimensional covariates, (approximate) low rank, and measurement error. We
further investigate this phenomenon is Appendix B.

(a) Estimation

(b) Inference

Figure 3:  OLS succeeds on clean data

Data corruption can derail causal inference, which motivates filling the NA’s,
reigning in the extremes, and otherwise de-noising the values in
$\boldsymbol{Z}$ in hopes of recovering $\boldsymbol{X}$. These are precisely
the goals of matrix completion applied to the matrix $\boldsymbol{Z}$. In this
work, we turn to the vast literature on matrix completion [Candès and Recht,
2009, Candès and Tao, 2010, Keshavan et al., 2009, Hastie et al., 2015,
Chatterjee, 2015] for automated data cleaning. To select an appropriate matrix
completion method, we return to Figure 2 to visualize the principal components
of the corrupted covariates $\boldsymbol{Z}=\boldsymbol{X}+\boldsymbol{H}$ for
various noise-to-signal ratios (defined as the noise variance divided by the
signal variance). Figure 2 shows that the initial five principal components
remain virtually unchanged, while the lower principal components are
amplified; signal remains spectrally concentrated while noise is spectrally
diffuse. Therefore a natural way to clean the covariates would be to discard
the lower principal components—in essence, to perform principal component
analysis (PCA), also called hard singular value thresholding.222Alternative
choices include canonical correlation analysis and partial least squares,
which clean $\boldsymbol{Z}$ using $Y$ [Wold, 1982, Wold et al., 1984]. We
leave these directions to future research.

Figure 4: OLS fails on corrupted data

Why is inference hard after data cleaning? Several challenges arise. First,
manual and automated data cleaning may induce strong dependence among
observations; which central limit theorem could we use to prove Gaussian
approximation? Our answer is to introduce sample splitting, which is a classic
idea [Klaassen, 1987], and implicit data cleaning, which is a new idea.
Second, if we turn to automated data cleaning, the best rates of convergence
to the true matrix $\boldsymbol{X}$ are slower than $n^{-1/2}$; how will we
possibly obtain a standard error of order $\hat{\sigma}n^{-1/2}$? Our answer
is to aim for double rate robustness by introducing a doubly robust estimating
equation [Chernozhukov et al., 2018a, van der Laan and Rose, 2018, Rotnitzky
et al., 2021]. The third issue is a theoretical one to which we will return in
Section 5: the best rates of matrix completion are not for recovering specific
matrix entries but rather averages across matrix entries; how is this even
related to downstream semiparametric inference? Our answer is to develop an
algorithmic and analytic framework that forges the connection.

####  4.2 Overview of the procedure

Split the observations $(Y_{i},D_{i},Z_{i,\cdot})$ into equally sized train
and test sets, each with $m=n/2$ observations. Our procedure consists of four
steps, which we state at a high level before filling in the details.

  1. 1.

Data cleaning: $\hat{\boldsymbol{X}}$ using train.

  2. 2.

Error-in-variable regression: $\hat{\gamma}$ using train.

  3. 3.

Error-in-variable balancing: $\hat{\alpha}$ using train.

  4. 4.

Causal parameter: $\hat{\theta}\pm 1.96\hat{\sigma}n^{-1/2}$ using test.

We opt for simplicity at each step, essentially combining PCA and OLS (albeit
in new ways). We view these high level steps a template for more complex
procedures in future work.

Step 1: Data cleaning. The automated data cleaning procedure is extremely
simple: fill in missing values as zeros, scale appropriately, then perform
PCA. Importantly, we import the scalings from train to test.

For any mathematical operations to be well defined, the NA’s must be filled in
somehow. To begin, we tally the likelihood of non-missingness for each
covariate $j\in[p]$ in train:

$\hat{\rho}_{j}=\max\left\\{\frac{1}{m}\sum_{i\in\textsc{train}}\operatorname{\mathbbm{1}}(Z_{ij}\neq\texttt{NA}),\frac{1}{m}\right\\},\quad\operatorname{\hat{\boldsymbol{\rho}}}=diag(\hat{\rho}_{1},...,\hat{\rho}_{p})\in\mathbb{R}^{p\times
p}.$

Next, we fill in missing values with a fill operator defined such that

$\textsc{fill}(Z_{ij})=\begin{cases}\frac{Z_{ij}}{\hat{\rho}_{j}}&\text{ if
}Z_{ij}\neq\texttt{NA}\\\ 0&\text{ if }Z_{ij}=\texttt{NA}.\end{cases}$

The fill operator may act on either $\boldsymbol{Z}^{\textsc{train}}$ or
$\boldsymbol{Z}^{\textsc{test}}$, but it always uses the likelihoods
$\operatorname{\hat{\boldsymbol{\rho}}}$ calculated from
$\boldsymbol{Z}^{\textsc{train}}$. After filling train, we project it onto its
own principal subspace to calculate the cleaned training covariates
$\hat{\boldsymbol{X}}$:

$\textsc{fill}(\boldsymbol{Z}^{\textsc{train}})=\hat{\boldsymbol{U}}\hat{\boldsymbol{\Sigma}}\hat{\boldsymbol{V}}^{T},\quad\hat{\boldsymbol{X}}=\hat{\boldsymbol{U}}_{k}\hat{\boldsymbol{\Sigma}}_{k}\hat{\boldsymbol{V}}_{k}^{T}.$

Effectively, we are taking the SVD of
$\textsc{fill}(\boldsymbol{Z}^{\textsc{train}})$ and truncating it to include
only the top $k$ principal components, where $k$ is a chosen by the analyst.
Figure 2 suggests a choice of $k$, though other popular options include cross-
validation and theoretical rules [Chatterjee, 2015, Gavish and Donoho, 2014].
Below, we empirically verify that our results are robust to different choices
of $k$. This implementation of PCA preserves the ambient dimension $p$.

Step 2: Error-in-variable regression. Our error-in-variables regression is
also simple: after cleaning train, perform ordinary least squares (OLS) on
train, then use this OLS coefficient on the filled test for prediction. We
only fill, and do not clean, the test set.

Nonlinearity can be introduced into the regression to allow for treatment
effect heterogeneity. See Appendix C for a characterization of what
nonlinearity is allowed, and how nonlinearity manifests in the theoretical
guarantees. For the main text, we focus on the interacted dictionary, which
allows for heterogeneity and segments the OLS coefficient:

$b(D_{i},\hat{X}_{i,\cdot})=(D_{i}\hat{X}_{i,\cdot},(1-D_{i})\hat{X}_{i,\cdot}),\quad\hat{\beta}=\begin{bmatrix}\hat{\beta}^{\textsc{treat}}\\\
\hat{\beta}^{\textsc{untreat}}\end{bmatrix}.$

Then the OLS coefficient is

$\hat{\beta}=[\\{b(D^{\textsc{train}},\hat{\boldsymbol{X}})\\}^{T}b(D^{\textsc{train}},\hat{\boldsymbol{X}})]^{\dagger}[\\{b(D^{\textsc{train}},\hat{\boldsymbol{X}})\\}^{T}Y^{\textsc{train}}],$

where $\dagger$ means pseudoinverse. The subtlety is in how predictions are
constructed from $\hat{\beta}$. Out of sample prediction does not involve
cleaning the test set: for $i\in\textsc{test}$,

$\hat{\gamma}(D_{i},Z_{i,\cdot})=b\\{D_{i},\textsc{fill}(Z_{i,\cdot})\\}\hat{\beta}.$

Step 3: Error-in-variable balancing. Our error-in-variable balancing weight
generalizes our error-in-variable regression. It avoids the estimation and
inversion of propensity scores, which may be numerically unstable in high
dimensions. Pleasingly, it achieves exact balance for any finite sample size,
in a sense that we formalize below. Moreover, it adapts to the causal
parameter of interest, as we explain in Appendix F.

The only difference from the error-in-variable regression is that we replace
the sufficient statistic
$[\\{b(D^{\textsc{train}},\hat{\boldsymbol{X}})\\}^{T}Y^{\textsc{train}}]\in\mathbb{R}^{p^{\prime}}$
with another sufficient statistic that we call the counterfactual moment
$\hat{M}\in\mathbb{R}^{p^{\prime}}$. The counterfactual moment resembles the
expression
$\theta_{i}=\mathbb{E}[\gamma_{0}(1,X_{i,\cdot})-\gamma_{0}(0,X_{i,\cdot})]$,
and it is the only aspect of the algorithm that changes for different causal
parameters. Formally,

$\hat{\eta}=[\\{b(D^{\textsc{train}},\hat{\boldsymbol{X}})\\}^{T}b(D^{\textsc{train}},\hat{\boldsymbol{X}})]^{\dagger}\hat{M},\quad\hat{M}=[\\{b(1,\hat{\boldsymbol{X}})\\}^{T}-\\{b(0,\hat{\boldsymbol{X}})\\}^{T}]\operatorname{\mathbbm{1}}_{m}$

where $\operatorname{\mathbbm{1}}_{m}\in\mathbb{R}^{m}$ is a vector of ones.
As before we do not clean the test set: for $i\in\textsc{test}$,

$\hat{\alpha}(D_{i},Z_{i,\cdot})=b\\{D_{i},\textsc{fill}(Z_{i,\cdot})\\}\hat{\eta}.$

Step 4: Causal estimation and inference. The final step uses the error-in-
variable regression $\hat{\gamma}$ and error-in-variable balancing weight
$\hat{\alpha}$ learned from train, and evaluates them on test according to the
doubly robust estimating equation: for $i\in\textsc{test}$,

$\hat{\psi}_{i}=\hat{\gamma}(1,Z_{i,\cdot})-\hat{\gamma}(1,Z_{i,\cdot})+\hat{\alpha}(D_{i},Z_{i,\cdot})\\{Y_{i}-\hat{\gamma}(D_{i},Z_{i,\cdot})\\}.$

This process generates a vector $\hat{\psi}\in\mathbb{R}^{m}$, with
$\hat{\psi}_{i}$ corresponding to the empirical influence of observation
$i\in\textsc{test}$. Reversing the roles of train and test, we generate
another such vector. Slightly abusing notation, we concatenate the two to
obtain a vector $\hat{\psi}\in\mathbb{R}^{n}$. Our estimator of the causal
parameter $\hat{\theta}$, its variance $\hat{\sigma}^{2}$, and its data
cleaning-adjusted confidence interval are

$\hat{\theta}=\textsc{mean}(\hat{\psi}),\quad\hat{\sigma}^{2}=\textsc{var}(\hat{\psi}),\quad\textsc{CI}=\hat{\theta}\pm
1.96\hat{\sigma}n^{-1/2}.$

####  4.3 Properties of the procedure

Step 1: Data cleaning. We fill missing values by zeroes, and then scale
appropriately, for two reasons: (i) the procedure is asymptotically unbiased,
and (ii) the procedure avoids unnecessary correlations. We compare our
proposal to another that fills missing values with means, which we call fill-
as-means.

######  Proposition 4.1 (Filling with zeros is unbiased and simple).

For our proposal,

$\mathbb{E}[\textsc{fill}(Z^{\textsc{test}}_{ij})|X^{\textsc{test}}_{ij},\textsc{train}]=X^{\textsc{test}}_{ij}\frac{\rho_{j}}{\hat{\rho}_{j}}.$

The alternative procedure of filling missing values with means from train
gives

$\mathbb{E}[\textsc{fill-as-
means}(Z^{\textsc{test}}_{ij})|X^{\textsc{test}}_{ij},\textsc{train}]=X^{\textsc{test}}_{ij}\rho_{j}+\bar{Z}_{j}^{\textsc{train}}(1-\rho_{j}).$

fill-as-means gives a convex combination of the signal we wish to recover and
the noisy average from train. The noisy average introduces additional
correlations that we avoid with our simpler approach.333Yet another procedure
is called hot-deck imputation, which introduces correlations with a martingale
structure [Abadie and Imbens, 2012]. Our simple approach avoids these
correlations as well.

After filling missing values, we clean the data by PCA with hyperparameter
$k$. Below, we verify the robustness of the procedure to different choices of
$k$. In practice, we recommend that an analyst should examine a plot of
principal components, e.g. Figure 1, and select $k$ that is just after the
“elbow” so that it is close to $r$.

Step 2: Error-in-variable regression. Rather than explicitly cleaning the test
covariates, we implicitly clean them, to avoid correlations among predictions.
The following result formalizes this implicit data cleaning and what it
achieves.

######  Proposition 4.2 (Implicit data cleaning preserves independence).

For $i\in\textsc{test}$

$\hat{\gamma}(D_{i},Z_{i,\cdot})=b(D_{i},Z_{i,\cdot})\tilde{\beta},\quad\tilde{\beta}=\begin{bmatrix}\operatorname{\hat{\boldsymbol{\rho}}}^{-1}\hat{\beta}^{\textsc{treat}}\\\
\operatorname{\hat{\boldsymbol{\rho}}}^{-1}\hat{\beta}^{\textsc{untreat}}\end{bmatrix}$

where we replace NA with $0$ in $Z_{i,\cdot}$. Therefore for
$(i,j)\in\textsc{test}$,

$\hat{\gamma}(D_{i},Z_{i,\cdot})\raisebox{0.50003pt}{\rotatebox[origin={c}]{90.0}{$\models$}}\hat{\gamma}(D_{j},Z_{j,\cdot})|\textsc{train}.$

Remarkably, post-multiplying $b(D_{i},Z_{i,\cdot})$ by $\tilde{\beta}$ handles
the measurement error, missingness, discretization, and differential privacy
of $Z_{i,\cdot}$ while also producing high quality predictions of $Y_{i}$.
Moreover, since $\tilde{\beta}$ is learned exclusively from train, it is
deterministic conditional on train, so predictions for observations
$(i,j)\in\textsc{test}$ preserve their independence. This property will be
essential for our inferential theory.

Our new variant of PCR has broader use outside of causal inference. For
example, in online learning, a corrupted test observation $Z_{i,\cdot}$ does
not need to be explicitly cleaned with respect to test or even train. Instead,
it is sufficient to implicitly clean $Z_{i,\cdot}$ by post multiplying it with
the coefficient $\tilde{\beta}$. For test observations, data cleaning and
prediction can be combined into one step.

Step 3: Error-in-variable balancing. The error-in-variable balancing weight
shares the same implicit data cleaning and therefore preserves independence in
the same way. In addition, it provides a strong guarantee of finite sample
balance between the treated and untreated subpopulations.

######  Proposition 4.3 (The balancing weight exactly balances covariates).

For any finite sample,

$\frac{1}{m}\sum_{i\in\textsc{train}}\hat{X}_{i,\cdot}=\frac{1}{m}\sum_{i\in\textsc{train}}D_{i}\hat{X}_{i,\cdot}\cdot\hat{\omega}_{i}^{\textsc{train}}=\frac{1}{m}\sum_{i\in\textsc{train}}(1-D_{i})\hat{X}_{i,\cdot}\cdot\hat{\omega}_{i}^{\textsc{untreat}}$

where
$(\hat{\omega}_{i}^{\textsc{treat}},\hat{\omega}_{i}^{\textsc{untreat}})\in\mathbb{R}$
are balancing weights computed from $\hat{\eta}$: for each
$i\in\textsc{train}$,

$\hat{\omega}_{i}^{\textsc{treat}}=\hat{X}_{i,\cdot}\hat{\eta}^{\textsc{treat}},\quad\hat{\omega}_{i}^{\textsc{untreat}}=-\hat{X}_{i,\cdot}\hat{\eta}^{\textsc{untreat}}.$

Deterministically, the error-in-variable balancing weight exactly balances the
full population, the treated subpopulation, and the untreated subpopulation
with respect to their cleaned covariates. It is precisely the reweighting that
would ensure comparability of treated and untreated groups in a stratified
sampling design. We articulate a more general balancing property for generic
causal parameters in Appendix F. We also clarify the sense in which the error-
in-variable regression and balancing weight coincide on train but not test.

Step 4: Causal estimation and inference. In our estimation procedure, we deal
with measurement error bias by cleaning the data. For some special cases, the
measurement error bias actually has a closed form solution, which depends on
the regression, propensity score, covariate density, and derivatives thereof
[Battistin and Chesher, 2014]. Our approach avoids estimation of the
propensity scores, covariate density, and derivatives, which would be
challenging in high dimensions. Instead, we simply combine PCA and OLS.

Multiple imputation [Rubin, 1976] is a popular procedure for handling missing
values, and it has important similarities and differences compared to our
approach. Both share the goal of estimating and quantifying uncertainty for a
downstream scalar parameter. In multiple imputation, the analyst makes, say,
two copies of the original data set, then imputes missing values (with some
randomness so each imputation may be different). Estimates and standard errors
from each copy are then averaged. In our procedure, we split the sample into
two folds: train and test. We clean train and compute estimates and standard
errors with test, then reverse the roles and take the average. We opt for
sample splitting, rather than copying, and we additionally consider
measurement error.

####  4.4 Adapting to the type and level of corruption

Next, we demonstrate that our four step procedure performs well in simulations
with a broad variety of data corruptions. We run the same code in every
setting; the procedure adapts to the type and level of data corruption,
without prior knowledge of the corruption covariance structure.

To begin, we consider measurement error $Z_{i,\cdot}=X_{i,\cdot}+H_{i,\cdot}$,
where $H_{i,\cdot}$ is Gaussian noise, in the average treatment effect
simulation described above. Recall that $\theta_{0}=2.2$,
$\boldsymbol{X}\in\mathbb{R}^{100\times 100}$, and $r=5$. We implement our
procedure on corrupted data 1000 times, collecting 1000 point estimates
$\hat{\theta}$ and 1000 standard errors $\hat{\sigma}$. For a 20% noise-to-
signal ratio, we visualize the studentized point estimates
$(\hat{\theta}-\theta_{0})/\hat{\sigma}$ in Figure 5(a). Qualitatively, the
histogram closely resembles the standard normal density.

(a) Inference

(b) Coverage

Figure 5:  Our approach adapts to measurement error

We quantify performance in coverage tables. In Table 5(b), different rows
correspond to different noise-to-signal ratios. Initially, we consider the
oracle tuning of the PCA hyperparameter $k=r$. For each noise-to-signal ratio,
we record the average point estimates, which are close to $\theta_{0}=2.2$.
Next, we record the average standard errors, which adaptively increase in
length to higher noise levels. Impressively, a 100% noise-to-signal ratio
setting corresponds to a confidence interval that is only about 10% longer.
These confidence intervals are the correct length, since about 950 of them
include the true value $\theta_{0}=2.2$.

(a) Inference

(b) Coverage

Figure 6:  Our approach adapts to missing values

Table 5(b) revisits the issue of tuning the hyperparameter $k$. This time, we
fix the noise-to-signal ratio to 20%. Different rows correspond to different
tunings: $k=r$, $k=r+2$, and $k=r+5$. Point estimates remain close to the true
value $\theta_{0}=2.2$. The standard errors adaptively increase in length when
$k$ deviates from $r$, though the length only increases about 10%. The
confidence intervals are again the correct length, attaining nominal coverage.

(a) Inference

(b) Coverage

Figure 7:  Our approach adapts to discretization

We repeat this exercise with other types of data corruption: missing values
(Figure 6), discretization (Figure 7), and differential privacy (Figure 8).
For missing values $Z_{i,\cdot}=X_{i,\cdot}\odot\pi_{i,\cdot}$, we consider
non-response of 10%, 30%, and 50% of all covariate entries. Key variables such
as income in Census Bureau surveys are missing 40% of the time. Fortunately,
our procedure performs well even with this high level of missingness. For
discretization, we consider randomized rounding
$Z_{i,\cdot}=sign(X_{i,\cdot})\text{Poisson}(|X_{i,\cdot}|)$, which
corresponds to a 33% noise-to-signal ratio. Finally, for differential privacy
$Z_{i,\cdot}=X_{i,\cdot}+H_{i,\cdot}$, where $H_{i,\cdot}$ is Laplacian noise,
we obtain results that are nearly identical to measurement error. Across
settings, our results are robust to hyperparameter tuning.

(a) Inference

(b) Coverage

Figure 8:  Our approach adapts to differential privacy

###  5 Finite sample analysis

In the previous section, we articulate three reasons why inference after data
cleaning is hard. First, data cleaning may induce a great deal of dependence.
We introduce implicit data cleaning as an algorithmic solution, yet we still
need to provide a theory of implicit data cleaning: why is it okay to not
clean the test covariates? Second, the best rates of data cleaning are slower
than $n^{-1/2}$. We incorporate the doubly robust estimating equation in the
hope of achieving double rate robustness, yet we still need to prove that it
works: how is causal inference still possible with standard errors of order
$\hat{\sigma}n^{-1/2}$? Third, data cleaning recovers averages across matrix
entries. How can we translate guarantees about recovering averages into
guarantees about the coverage of data cleaning-adjusted confidence intervals?
In this section, we answer these three theoretical questions with finite
sample analysis.

We prove four theorems, each corresponding to a step in the procedure.

  1. 1.

Data cleaning: $\hat{\boldsymbol{X}}$ converges to
$\boldsymbol{X}^{\textsc{train}}$.

  2. 2.

Error-in-variable regression: $\hat{\gamma}$ converges to $\gamma_{0}$.

  3. 3.

Error-in-variable balancing weight: $\hat{\alpha}$ converges to $\alpha_{0}$.

  4. 4.

Causal parameter: $\mathbb{P}\\{\theta_{0}\in(\hat{\theta}\pm
1.96\hat{\sigma}n^{-1/2})\\}$ converges to $0.95$.

We have already verified that our key assumption, that covariates are
approximately low rank, is reasonable in practice for US Census data. In a
corollary, we verify that it is reasonable in theory: it holds for a broad
class of linear and nonlinear factor models.

####  5.1 Step 1: Data cleaning

For the data cleaning guarantee, we place four assumptions on the corrupted
data. To lighten notation, we suppress indexing by train.

######  Assumption 5.1 (Bounded signal).

There exists an absolute constant $\bar{A}<\infty$ such that for all
$i\in[m],j\in[p]$, $|X_{ij}|\leq\bar{A}$.

Assumption 5.1 is a standard boundedness assumption imposed on true values in
the matrix completion literature.

######  Assumption 5.2 (Measurement error).

Each row of measurement error $H_{i,\cdot}$ is mean zero and subexponential,
i.e. $\mathbb{E}[H_{i,\cdot}|X_{i,\cdot}]=0$ and there exists $a\geq 1$ and
$K_{a}<\infty$ such that $\|H_{i,\cdot}|X_{i,\cdot}\|_{\psi_{a}}\leq K_{a}$.
Hence there exists $\kappa^{2}>0$ such that
$\|\mathbb{E}[H_{i,\cdot}^{T}H_{i,\cdot}|X_{i,\cdot}]\|_{op}\leq\kappa^{2}$.
We assume measurement error is independent across rows.

Measurement error is independent across rows, but it may be dependent within a
given row. If in a given row of $H_{i,\cdot}\in\mathbb{R}^{p}$ each coordinate
is independent, then $K_{a}$ and $\kappa^{2}$ are constants (i.e. they do not
scale with $p$) [Vershynin, 2018, Lemma 3.4.2]. More generally,
$(K_{a},\kappa)$ quantify the level of dependence among the entries of
$H_{i,\cdot}$ within a row. Our model allows for a great deal of
heteroscedasticity. In particular, the results to follow are conditional on
$\boldsymbol{X}$ so, for example, the shape of the distribution of $H_{ij}$
may depend on $X_{ij}$ as long as it is mean zero and has tails no wider than
those of an exponential distribution. As such, it encompasses discretization
and differential privacy.

######  Assumption 5.3 (Missing values).

In each row of missingness $\pi_{i,\cdot}$, $\pi_{ij}$ is 1 with probability
$\rho_{j}$ and NA otherwise. Identifying NA with $0$, we assume there exists
$\bar{K}<\infty$ such that
$\|\pi_{i,\cdot}-(\rho_{1},...,\rho_{p})|X_{i,\cdot}\|_{\psi_{2}}\leq\bar{K}$.
We assume missingness is independent across rows.

Our missingness model generalizes the standard missingness model in the PCR
error-in-variables literature in two ways: (i) the missingness of one variable
may depend on the missingness of another, and (ii) different variables may be
missing with different probabilities. These differences are easily visualized
by the contrast

$\mathbb{E}[\pi_{i,\cdot}^{T}\pi_{i,\cdot}]=\begin{bmatrix}\rho_{1}&\rho_{12}&...&\rho_{1p}\\\
\rho_{12}&\rho_{2}&...&\rho_{2p}\\\ \vdots&\vdots&\ddots&\vdots\\\
\rho_{1p}&\rho_{2p}&...&\rho_{p}\end{bmatrix}\quad\text{versus}\quad\mathbb{E}[\pi_{i,\cdot}^{T}\pi_{i,\cdot}]=\begin{bmatrix}\rho&\rho^{2}&...&\rho^{2}\\\
\rho^{2}&\rho&...&\rho^{2}\\\ \vdots&\vdots&\ddots&\vdots\\\
\rho^{2}&\rho^{2}&...&\rho\end{bmatrix}$

where $\rho_{jj^{\prime}}=\mathbb{E}[\pi_{ij}\pi_{ij^{\prime}}]$ does not
necessarily equal
$\rho_{j}\rho_{j^{\prime}}=\mathbb{E}[\pi_{ij}]\mathbb{E}[\pi_{ij^{\prime}}]$.
These additional degrees of flexibility are crucial for Census data, where
non-responses for different variables are often correlated and where non-
response rates of different variables can be vastly different. As with
measurement error, missingness is independent across rows, but it may be
dependent within a given row. If in a given row of
$\pi_{i,\cdot}\in\mathbb{R}^{p}$ each coordinate is independent, then
$\rho_{jj^{\prime}}=\rho_{j}\rho_{j^{\prime}}$ and $\bar{K}$ is constant. More
generally, $\bar{K}$ quantifies the level of dependence among the entries of
$\pi_{i,\cdot}$ within a row. Our model allows for different probabilities of
missingness for different variables in a way that can depend on the signal.
The results to follow are conditional on $\boldsymbol{X}$ so, for example, the
probability $\rho_{j}$ may depend on $X_{\cdot,j}$. For our theoretical
results, we define the additional notation

$\rho_{\min}\coloneqq\min_{j\in[p]}\rho_{j},\quad\boldsymbol{\rho}=diag(\rho_{1},...,\rho_{p})\in\mathbb{R}^{p\times
p}.$

######  Assumption 5.4 (Concentrated signal).

Consider the approximation $\boldsymbol{X}^{\textsc{(lr)}}$ to
$\boldsymbol{X}$, with singular values $s_{1},...,s_{r}$. Assume that
$s_{1},...,s_{r}\geq C\sqrt{\frac{mp}{r}}$, where $C$ is an absolute constant.

Assumption 5.4 is analogous to incoherence-style conditions in econometrics
[Bai and Ng, 2019, Agarwal et al., 2021, Agarwal et al., 2020a, Agarwal et
al., 2020b] and the notion of pervasiveness in matrix completion [Fan et al.,
2018]. Similar to a strong factor assumption [Onatski, 2012, Anatolyev and
Mikusheva, 2021], it ensures that the explanatory power of
$\boldsymbol{X}^{\textsc{(lr)}}$ dominates the explanatory power of various
error terms. Specifically, it ensures that signal is spectrally concentrated.
A natural setting in which Assumption 5.4 holds is if
$X^{\textsc{(lr)}}_{ij}=\Theta(1)$ and $s_{1},...s_{r}=\Theta(\tau)$, i.e.,
they are well-balanced. Then, for absolute constants
$C,C^{\prime},C^{\prime\prime}>0$,

$C\cdot
r\cdot\tau^{2}=\sum_{k}s_{k}^{2}=\|\boldsymbol{X}^{\textsc{(lr)}}\|^{2}_{Fr}=C^{\prime}\cdot
mp$

which implies $\tau=C^{\prime\prime}\sqrt{\frac{mp}{r}}$. We leave to future
work the extension of our results to settings with different spectral
assumptions on $\boldsymbol{X}^{\textsc{(lr)}}$.

######  Remark 5.1 (Approximately low rank signal).

We parametrize our rates by the quality of low rank approximation.

Without loss of generality,
$\boldsymbol{X}=\boldsymbol{X}^{\textsc{(lr)}}+\boldsymbol{E}^{\textsc{(lr)}}$,
where $\boldsymbol{X}^{\textsc{(lr)}}$ is a low rank approximation to
$\boldsymbol{X}$ and $\boldsymbol{E}^{\textsc{(lr)}}$ is the approximation
residual. The two key quantities are
$r=rank\\{\boldsymbol{X}^{\textsc{(lr)}}\\}$ and
$\Delta_{E}=\|\boldsymbol{E}^{\textsc{(lr)}}\|_{\max}$. It is with loss of
generality that $r$ and $\Delta_{E}$ are simultaneously well behaved.
Intuitively, as $r$ decreases $\Delta_{E}$ increases (and vice-versa). Indeed,
if $\boldsymbol{X}^{\textsc{(lr)}}=\boldsymbol{X}$ then trivially $r\leq(m,p)$
and $\Delta_{E}=0$; conversely, if $\boldsymbol{X}^{\textsc{(lr)}}=0$, then
$r=0$ but $\Delta_{E}=\bar{A}$. As we show in the corollary, under a nonlinear
factor model, both $r$ and $\Delta_{E}$ are well behaved: $r\ll(m,p)$ and
$\Delta_{E}\rightarrow 0$. Until that corollary, our rates are parametrized by
$(r,\Delta_{E})$.

######  Theorem 5.1 (Finite sample data cleaning rate).

Suppose Assumptions 5.1, 5.2, 5.3, and 5.4 hold. Furthermore, suppose that
$k=r$ and $\rho_{\min}>\frac{23\log(mp)}{m}$. Then for an absolute constant
$C>0$,

$\frac{1}{m}\mathbb{E}\|\hat{\boldsymbol{X}}-\boldsymbol{X}\|^{2}_{2,\infty}\leq
C_{1}\cdot\frac{r\ln^{5}(mp)}{\rho_{\min}^{4}}\left(\frac{1}{m}+\frac{1}{p}+\Delta_{E}^{2}\right)$

where $C_{1}=C\cdot\bar{A}^{4}(K_{a}+\bar{K})^{2}(\kappa+K_{a}+\bar{K})^{2}$.

The norm of convergence is the so-called $(2,\infty)$ norm:

$\frac{1}{m}\|\hat{\boldsymbol{X}}-\boldsymbol{X}\|^{2}_{2,\infty}=\max_{j\in[p]}\frac{1}{m}\|\hat{X}_{i,\cdot}-X_{\cdot,j}\|_{2}^{2}=\max_{j\in[p]}\frac{1}{m}\sum_{i=1}^{m}(\hat{X}_{ij}-X_{ij})^{2}$

i.e. a maximum across columns and an average across rows. For any given
variable $j\in[p]$, Theorem 5.1 guarantees that data cleaning performs well on
average across observations $i\in[m]$. Our rate requires both $m$ and $p$ to
increase. Rather than a curse, there is blessing of dimensionality: more
repeated measurements improve the quality of data cleaning. For the bound to
be meaningful, $(r,\Delta_{E})$ must be simultaneously well behaved, which is
our key assumption. Recall that $(K_{a},\kappa,\bar{K})$ quantify the level of
corruption dependence within a row. As long as the dependence is weak, e.g.
$(K_{a},\kappa,\bar{K})$ scale as some power of $\ln(mp)$, this dependence in
negligible. Our downstream results for the error-in-variable regression and
balancing weight are predicated on this data cleaning guarantee. As such, the
geometry used in data cleaning is the crux of the entire framework: signal is
spectrally concentrated, while noise is spectrally diffuse, so we can
concentrate out the noise.

####  5.2 Step 2: Error-in-variable regression

We place three additional assumptions for the error-in-variable regression
guarantee.

######  Assumption 5.5 (Response noise).

The response noise satisfies $\mathbb{E}[\varepsilon_{i}]=0$ and
$\mathbb{V}[\varepsilon_{i}]\leq\bar{\sigma}^{2}$. It is independent across
rows and independent of $Z_{i,\cdot}$.

This weak condition permits measurement error and differential privacy of the
outcome $Y_{i}$. See Appendix A for outcome attrition. Next we assume train
and test each contains a sufficient variety of observations. For a matrix
$\boldsymbol{M}\in\mathbb{R}^{m\times p}$, we define its row space as
$\textsc{row}(\boldsymbol{M})=span\\{M_{i,\cdot}\\}$.

######  Assumption 5.6 (Row space inclusion).

Assume
$\textsc{row}[b\\{\boldsymbol{X}^{\textsc{(lr)},\textsc{train}}\\}]=\textsc{row}[b\\{\boldsymbol{X}^{\textsc{(lr)},\textsc{test}}\\}]$.

This property permits
$\boldsymbol{X}^{\textsc{(lr)},\textsc{train}}\neq\boldsymbol{X}^{\textsc{(lr)},\textsc{test}}$,
and also permits the two matrices to have different SVDs. In Appendix E, we
verify that Assumption 5.6 holds with high probability under weak auxiliary
conditions. Finally, we place a weak technical condition.

######  Assumption 5.7 (Well conditioned estimators).

Let $\hat{s}^{\prime}_{k^{\prime}}$ be the smallest non-zero singular value of
$b(D^{\textsc{train}},\hat{\boldsymbol{X}})$. Assume that
$\hat{s}^{\prime}_{k^{\prime}}\gtrsim\frac{\bar{\varepsilon}}{\text{polynomial}(m,p)}$
where $\mathbb{E}[\varepsilon_{i}^{8}]\leq\bar{\varepsilon}^{8}$.

For $(\hat{\beta},\hat{\eta})$ to be well conditioned, the singular value
$\hat{s}^{\prime}_{k^{\prime}}$ should not be too small. In particular, it
must be bounded below by an arbitrary negative power of $m$ and $p$. We view
the Assumption 5.7 as a diagnostic tool for empirical practice: choose the PCA
hyperparameter $k$ such that the $k^{\prime}$-th singular value
$\hat{s}^{\prime}_{k^{\prime}}$ is not too close to zero.

Before stating the result, we introduce a theoretical device $\beta^{*}$ as
the coefficient of the best low rank nonlinear approximation to $\gamma_{0}$.
In particular, we approximate

$\gamma_{0}(D_{i},X_{i,\cdot})=b(D_{i},X^{\textsc{(lr)}}_{i,\cdot})\beta^{*}+\phi_{i}^{\textsc{(lr)}}$

where $\phi_{i}^{\textsc{(lr)}}$ is the approximation error. It turns out to
be convenient to keep track of this approximation error by defining
$\phi_{i}:=\gamma_{0}(D_{i},X_{i,\cdot})-b(D_{i},X_{i,\cdot})\beta^{*}$. There
will be a trade-off: a richer dictionary $b$ leads to a smaller approximation
error in terms of $\|\phi\|^{2}_{2}$, but more compounding of measurement
error and missingness. The following result helps to characterize how the
compounded data corruption magnifies $(\rho_{\min}^{-1},r,\Delta_{E})$ but
nothing else.

######  Remark 5.2 (General dictionaries).

In the main text, we state results that hold for a broad class of
dictionaries, with the dictionary specific constant $C_{b}^{\prime}$ and the
concise notation $(\rho^{\prime}_{\min},r^{\prime},\Delta_{E}^{\prime})$ in
Theorems 5.2 and 5.3. In Appendix C we prove that

$C_{b}^{\prime}\leq
2^{d_{\max}}\cdot\bar{A}^{2d_{\max}}_{\max}\cdot\|\hat{\boldsymbol{X}}\|_{\max}^{2d_{\max}}\cdot,\quad\frac{1}{\rho^{\prime}_{\min}}\leq\frac{d_{\max}\bar{A}^{d_{\max}}}{\rho_{\min}},\quad
r^{\prime}\leq r^{d_{\max}},\quad\Delta_{E}^{\prime}\leq
C\bar{A}^{d_{\max}}\cdot d_{\max}\Delta_{E}$

where $d_{\max}$ is the degree of the polynomial dictionary. We articulate
restrictions on the class of dictionaries in Appendix C. For the interacted
dictionary, $d_{\max}=2$.

######  Remark 5.3 (Bound on $\|\hat{\boldsymbol{X}}\|_{\max}$).

Under further incoherence style assumptions, we bound
$\|\hat{\boldsymbol{X}}\|_{\max}\leq C\sqrt{r}$ in Appendix D. Alternatively,
one can bound

$\|\hat{\boldsymbol{X}}\|_{\max}\leq\|\hat{\boldsymbol{X}}-\boldsymbol{X}\|_{\max}+\|\boldsymbol{X}\|_{\max}\leq\|\hat{\boldsymbol{X}}-\boldsymbol{X}\|^{2}_{2,\infty}+\bar{A}$

then appeal to Theorem 5.1 with high probability. Doing so does not affect the
powers of $(m,p)$ in the main results, but does increase the complexity of the
pre-factors.

######  Theorem 5.2 (Finite sample error-in-variable regression rate).

Suppose the conditions of Theorem 5.1 hold, as well as Assumptions 5.5, 5.6,
and 5.7.

$\text{If}\quad\rho^{\prime}_{\min}\gg\tilde{C}\sqrt{r^{\prime}}\ln^{\frac{3}{2}}(mp)\left\\{\frac{1}{\sqrt{p}}\vee\frac{1}{\sqrt{m}}\vee\Delta_{E}\right\\},\quad\tilde{C}:=C\bar{A}\Big{(}\kappa+\bar{K}+K_{a}\Big{)}\quad\text{then}$
(5) $\displaystyle\mathcal{R}(\hat{\gamma})$ $\displaystyle\leq
C_{b}^{\prime}C_{1}C_{2}\cdot\bar{\sigma}^{2}\cdot\frac{(r^{\prime})^{3}\ln^{8}(mp)}{(\rho^{\prime}_{\min})^{6}}\|\beta^{*}\|^{2}_{1}\left(\frac{1}{m}+\frac{p}{m^{2}}+\frac{1}{p}+\left(1+\frac{p}{m}\right)(\Delta^{\prime}_{E})^{2}+p(\Delta^{\prime}_{E})^{4}\right)$
$\displaystyle\quad+C_{2}\cdot\frac{(r^{\prime})^{2}\ln^{3}(mp)}{(\rho^{\prime}_{\min})^{2}}\Delta_{\phi}\left(1+(\Delta^{\prime}_{E})^{2}\right)$

where
$\Delta_{\phi}=\frac{1}{m}\|\phi^{\textsc{train}}\|_{2}^{2}\vee\frac{1}{m}\|\phi^{\textsc{test}}\|_{2}^{2}$
and

$\displaystyle C_{1}$
$\displaystyle=C\bar{A}^{4}(K_{a}+\bar{K})^{2}(\kappa+\bar{K}+K_{a})^{2},\quad
C_{2}=C\cdot\bar{A}^{4}(\kappa+\bar{K}+K_{a})^{2}.$

######  Corollary 5.1 (Simplified regression rate).

Suppose the conditions of Theorem 5.2 hold. Further suppose $\gamma_{0}$ is
exactly linear in signal, which is exactly low rank. Then

$\displaystyle\mathcal{R}(\hat{\gamma})$ $\displaystyle\leq
C_{1}C_{2}\cdot\bar{\sigma}^{2}\cdot\frac{r^{3}\ln^{8}(mp)}{\rho_{\min}^{6}}\|\beta^{*}\|^{2}_{1}\left(\frac{1}{m}+\frac{p}{m^{2}}+\frac{1}{p}\right).$

The norm of convergence is a generalized mean square error

$\mathcal{R}(\hat{\gamma})=\mathbb{E}\left[\frac{1}{m}\sum_{i\in\textsc{test}}\\{\hat{\gamma}(D_{i},Z_{i,\cdot})-\gamma_{0}(D_{i},X_{i,\cdot})\\}^{2}\right],$

which is a relaxation of mean square error, where the expectation is over
randomness in train and test. Two aspects of our problem necessitate this
norm: (i) given the $(2,\infty)$ data cleaning guarantee in Theorem 5.1, this
is the best we can do; and (ii) for i.n.i.d. data, a population risk is
otherwise not well defined.444Interestingly, even with i.i.d. data, (i)
necessitates this norm. Since the estimator $\hat{\gamma}$ does not involve
cleaning test, this result is our desired theory of implicit data cleaning.
The bound requires both $m$ and $p$ to increase, $p\ll m^{2}$, and
$\rho_{\min}\gg p^{-1/2}\vee m^{-1/2}\vee\Delta_{E}$. For the bound to be
meaningful, $(r,\Delta_{E})$ must be simultaneously well behaved and the
corruption dependence must be weak. Finally, the bound includes the nonlinear
approximation error $\Delta_{\phi}$ and the size of the theoretical device
$\|\beta^{*}\|_{1}$. $\|\beta^{*}\|_{1}$ is well behaved if $\beta^{*}$ is
approximately sparse. In summary, we keep track of the low rank approximation
error $\Delta_{E}$ and the nonlinear sparse approximation error
$\Delta_{\phi}$. To deal with $\Delta_{E}$, we demonstrate that nonlinear
factor models admit low rank approximation below. Due to our doubly robust
approach, inference for the causal parameter $\theta_{0}$ is robust to non-
vanishing $\Delta_{\phi}$—a discussion we revisit later.

We make several innovations relative to previous work in the PCR error-in-
variables literature. First, we propose an error-in-variable regression
estimator that does not clean the test covariates, so we must develop a new
theory of implicit data cleaning. Second, we define and analyze a new norm of
convergence which we will subsequently use in causal inference. See Appendix E
for a comparison of our norm with the norms in previous work. Third, we allow
for dependence of missingness across variables and for different probabilities
of missingness across variables. This flexibility is realistic for Census
data. Fourth, we consider a nonlinear regression function $\gamma_{0}$ that is
approximated by a nonlinear dictionary of basis functions $b$. The dictionary
of basis functions is important for causal inference, since it allows for
treatment effect heterogeneity, and it requires a novel characterization of
which nonlinearities do not compound data corruption too much.

####  5.3 Step 3: Error-in-variable balancing

We place one final assumptions for the error-in-variable balancing weight.

######  Assumption 5.8 (Row space inclusion).

Assume
$\hat{M}\in\textsc{row}\\{b(D^{\textsc{train}},\hat{\boldsymbol{X}})\\}.$

Whereas Assumption 5.6 is about the low rank approximation of the signal
across train and test, Assumption 5.8 is about the counterfactual moment in
relation to train after cleaning. With
$\hat{M}=[\\{b(D^{\textsc{train}},\hat{\boldsymbol{X}})\\}^{T}Y^{\textsc{train}}]$,
which reverts to error-in-variable regression, Assumption 5.8 immediately
holds. In other cases, it limits the counterfactual queries that an analyst
may ask. Because it is about empirical quantities, we view it as a diagnostic
tool that an analyst should use to determine whether the counterfactual can be
extrapolated.

As before, we introduce a theoretical device $\eta^{*}$ as the coefficient of
the best low rank nonlinear approximation to $\alpha_{0}$. In particular, we
approximate

$\alpha_{0}(D_{i},Z_{i,\cdot})=b(D_{i},X^{\textsc{(lr)}}_{i,\cdot})\eta^{*}+\zeta_{i}^{\textsc{(lr)}}$

where $\zeta_{i}^{\textsc{(lr)}}$ is the approximation error.555A further
assumption that the treatment mechanism only depends on signal, i.e.
$\mathbb{E}[D_{i}|X_{i,\cdot},H_{i,\cdot},\pi_{i,\cdot}]=\mathbb{E}[D_{i}|X_{i,\cdot}]$,
implies
$\alpha_{0}(D_{i},Z_{i,\cdot})=\alpha_{0}(D_{i},X_{i,\cdot})=b(D_{i},X^{\textsc{(lr)}}_{i,\cdot})\eta^{*}+\zeta_{i}^{\textsc{(lr)}}$.
As before, we study this approximation error by defining
$\zeta_{i}:=\alpha_{0}(D_{i},Z_{i,\cdot})-b(D_{i},X_{i,\cdot})\eta^{*}$. There
will be a trade-off: a richer dictionary $b$ leads to a smaller approximation
error in terms of $\|\zeta\|^{2}_{2}$, but amplification of
$(\rho^{-1}_{\min},r,\Delta_{E})$.

######  Remark 5.4 (General causal parameters).

In the main text, we state results that hold for a broad class of causal
parameters, with parameter specific constants
$(C_{m}^{\prime},C_{m}^{\prime\prime})$ in Theorem 5.3. In Appendix F, we
characterize $(C_{m}^{\prime},C_{m}^{\prime\prime})$ for several examples. For
ATE with the interacted dictionary, $C_{m}^{\prime}=1$ and
$C_{m}^{\prime\prime}=\bar{A}.$

######  Theorem 5.3 (Finite sample error-in-variable balancing weight rate).

Suppose the conditions of Theorem 5.1 hold, as well as Assumptions 5.6, 5.7,
and 5.8. If (5) holds and $\|\alpha_{0}\|_{\infty}\leq\bar{\alpha}$,

$\displaystyle\mathcal{R}(\hat{\alpha})$ $\displaystyle\leq
C_{3}\cdot\frac{(r^{\prime})^{5}\ln^{13}(mp)}{(\rho^{\prime}_{\min})^{10}}\|\eta^{*}\|^{2}_{1}$
$\displaystyle\quad\cdot\left\\{\frac{1}{m}+\frac{1}{p}+\frac{p}{m^{2}}+\frac{m}{p^{2}}+\left(1+\frac{p}{m}+\frac{m}{p}\right)(\Delta^{\prime}_{E})^{2}+(m+p)(\Delta^{\prime}_{E})^{4}+mp(\Delta^{\prime}_{E})^{6}\right\\}+2\Delta_{\zeta}$

where
$\Delta_{\zeta}=\frac{1}{m}\|\zeta^{\textsc{train}}\|_{2}^{2}\vee\frac{1}{m}\|\zeta^{\textsc{test}}\|_{2}^{2}$
and

$\displaystyle C_{3}$
$\displaystyle=C\bar{A}^{14}(C_{b}^{\prime}+\sqrt{C_{m}^{\prime}}+C_{m}^{\prime\prime}+\bar{\alpha}+\bar{A})^{2}(K_{a}+\bar{K})^{4}(\kappa+\bar{K}+K_{a})^{6}.$

######  Corollary 5.2 (Simplified balancing weight rate).

Suppose the conditions of Theorem 5.3 hold. Further suppose $\alpha_{0}$ is
exactly linear in signal, which is exactly low rank. Then

$\displaystyle\mathcal{R}(\hat{\alpha})$ $\displaystyle\leq
C_{3}\cdot\frac{r^{5}\ln^{13}(mp)}{\rho^{10}_{\min}}\|\eta^{*}\|^{2}_{1}\cdot\left\\{\frac{1}{m}+\frac{1}{p}+\frac{p}{m^{2}}+\frac{m}{p^{2}}\right\\}$

replacing $C_{b}^{\prime}$ with $1$ in the definition of $C_{3}$.

The norm of convergence is a generalized mean square error

$\mathcal{R}(\hat{\alpha})=\mathbb{E}\left[\frac{1}{m}\sum_{i\in\textsc{test}}\\{\hat{\alpha}(D_{i},Z_{i,\cdot})-\alpha_{0}(D_{i},Z_{i,\cdot})\\}^{2}\right],$

which is the same relaxation of mean square error as before. The bound
requires both $m$ and $p$ to increase, $m^{1/2}\ll p\ll m^{2}$, and
$\rho_{\min}\gg p^{-1/2}\vee m^{-1/2}\vee\Delta_{E}$. For the bound to be
meaningful, $(r,\Delta_{E})$ must be simultaneously well behaved and the
corruption dependence must be weak. Finally, the bound includes the nonlinear
approximation error $\Delta_{\zeta}$ and the size of theoretical device
$\|\eta^{*}\|_{1}$. In summary, we keep track of the low rank approximation
error $\Delta_{E}$ and the nonlinear sparse approximation error
$\Delta_{\zeta}$. Nonlinear factor models imply bounds on $\Delta_{E}$. Due to
our doubly robust approach, inference for the causal parameter $\theta_{0}$ is
robust to non-vanishing $\Delta_{\zeta}$—a discussion we revisit below.

Theorem 5.3 innovates in all of the ways that Theorem 5.2 does and more. Most
importantly, it analyzes a new PCR estimator for a new estimand: the error-in-
variable balancing weight. A rich literature proposes balancing weight
estimators for causal inference with clean data, but to our knowledge, ours is
the first error-in-variable balancing weight estimator for causal inference
with corrupted data. As developed in Appendix F, Theorem 5.3 actually holds
for a broad class of counterfactual moments and therefore a broad class of
causal parameters. Moreover, the counterfactual moment
$\hat{M}=[\\{b(D^{\textsc{train}},\hat{\boldsymbol{X}})\\}^{T}Y^{\textsc{train}}]$
recovers error-in-variable regression. We choose not to simply subsume Theorem
5.2 by Theorem 5.3 for two reasons. First, doing so would require that $Y_{i}$
and $\varepsilon_{i}$ are bounded, which rules out differential privacy for
the outcome; see the Appendix A discussion. Second, Theorem 5.2 has lower
powers of $(r,\rho_{\min}^{-1})$ and avoids the term $\frac{m}{p^{2}}$ so it
is typically a tighter bound.

####  5.4 Step 4: Causal estimation and inference

Before stating our results, we formalize the sense in which the corrupted data
problem is an extended semiparametric problem. Let
$W_{i,\cdot}=(D_{i},X_{i,\cdot},H_{i,\cdot},\pi_{i,\cdot})$ concatenate the
signal and the noise, so that $\mathbb{L}_{2}(\mathcal{W})$ consists of square
integrable functions of the form
$\gamma:(D_{i},X_{i,\cdot},H_{i,\cdot},\pi_{i,\cdot})\rightarrow\mathbb{R}$.
Both the true regression $\gamma_{0}(D_{i},X_{i,\cdot})$ and our error-in-
variable estimator $\hat{\gamma}(D_{i},Z_{i,\cdot})$ belong to this space,
which serves as our hypothesis space for semiparametric analysis. With this
theoretical background, we formalize our distribution shift assumptions.

######  Assumption 5.9 (Marginal distribution shift).

The extended outcome and treatment mechanisms,
$\mathbb{E}[Y_{i}|D_{i},X_{i,\cdot},H_{i,\cdot},\pi_{i,\cdot}]$ and
$\mathbb{E}[D_{i}|X_{i,\cdot},H_{i,\cdot},\pi_{i,\cdot}]$, do not vary across
observations.

Assumption 5.9 implies that $\gamma_{0}(W_{i,\cdot})$ and
$\alpha_{0}(W_{i,\cdot})$ do not vary across observations, though the marginal
distributions $\mathbb{P}_{i}(W_{i})$ may vary. Our corruption model implies
$\gamma_{0}(W_{i,\cdot})=\gamma_{0}(D_{i},X_{i,\cdot})$, and we are agnostic
about whether $\alpha_{0}(W_{i,\cdot})=\alpha_{0}(D_{i},X_{i,\cdot})$ for the
extended hypothesis space.666If
$\mathbb{E}[D_{i}|X_{i,\cdot},H_{i,\cdot},\pi_{i,\cdot}]=\mathbb{E}[D_{i}|X_{i,\cdot}]$,
then $\alpha_{0}(W_{i,\cdot})=\alpha_{0}(D_{i},X_{i,\cdot})$. We place one
final assumption, mildly strengthening common support.

######  Assumption 5.10 (Bounded propensity).

The extended propensity score is bounded away from zero and one, i.e.
$1-\bar{\phi}\leq\mathbb{E}[D_{i}|X_{i,\cdot},H_{i,\cdot},\pi_{i,\cdot}]\leq\bar{\phi}$.

We introduce some additional notation to state the finite sample Gaussian
approximation. Define the oracle influences
$\psi_{i}=\psi(W_{i,\cdot},\theta_{i},\gamma_{0},\alpha_{0})$ where the
influence function is

$\psi(W_{i,\cdot},\theta,\gamma,\alpha)=\gamma(1,X_{i,\cdot},H_{i,\cdot},\pi_{i,\cdot})-\gamma(0,X_{i,\cdot},H_{i,\cdot},\pi_{i,\cdot})+\alpha(W_{i,\cdot})\\{Y_{i}-\gamma(W_{i,\cdot})\\}-\theta.$

$\mathbb{E}[\psi_{i}]=0$ since
$\mathbb{E}[\gamma_{0}(1,X_{i,\cdot})-\gamma_{0}(0,X_{i,\cdot})]=\theta_{i}$
and $\mathbb{E}[\alpha_{0}(W_{i,\cdot})\\{Y_{i}-\gamma_{0}(W_{i,\cdot})\\}]=0$
by law of iterated expectations. We define the higher moments and average
higher moments by

$\sigma_{i}^{2}=\mathbb{E}[\psi_{i}^{2}],\quad\xi_{i}^{3}=\mathbb{E}[|\psi_{i}|^{3}],\quad\chi_{i}^{4}=\mathbb{E}[\psi_{i}^{4}];\quad\sigma^{2}=\frac{1}{n}\sum_{i=1}^{n}\sigma_{i}^{2},\quad\xi^{3}=\frac{1}{n}\sum_{i=1}^{n}\xi_{i}^{3},\quad\chi^{4}=\frac{1}{n}\sum_{i=1}^{n}\chi_{i}^{4}.$

######  Remark 5.5 (General causal parameters).

In the main text, we state results that hold for a broad class of causal
parameters, with parameter specific constants $(\bar{Q},\bar{q})$ in Theorems
5.4 and 5.5. For ATE,
$\bar{Q}=2\left(\frac{1}{\bar{\phi}}+\frac{1}{1-\bar{\phi}}\right)$ and
$\bar{q}=1$ under Assumptions 5.9 and 5.10. In Appendix G, we characterize
$(\bar{Q},\bar{q})$ for several other examples under generalizations of
Assumptions 5.9 and 5.10.

######  Theorem 5.4 (Finite sample Gaussian approximation).

Suppose Assumptions 5.9 and 5.10 hold, $\mathbb{E}[\varepsilon_{i}^{2}\mid
W_{i,\cdot}]\leq\bar{\sigma}^{2},$ and
$\|\alpha_{0}\|_{\infty}\leq\bar{\alpha}.$ Further suppose that for
$(i,j)\in\textsc{test}$,

$\hat{\gamma}(W_{i,\cdot})\raisebox{0.50003pt}{\rotatebox[origin={c}]{90.0}{$\models$}}\hat{\gamma}(W_{j,\cdot})|\textsc{train},\quad\hat{\alpha}(W_{i,\cdot})\raisebox{0.50003pt}{\rotatebox[origin={c}]{90.0}{$\models$}}\hat{\alpha}(W_{j,\cdot})|\textsc{train}.$

Then with probability $1-\epsilon$,

$\sup_{z\in\mathbb{R}}\left|\mathbb{P}\left\\{\frac{n^{1/2}}{\sigma}(\hat{\theta}-\theta_{0})\leq
z\right\\}-\Phi(z)\right|\leq
c^{BE}\left(\frac{\xi}{\sigma}\right)^{3}n^{-\frac{1}{2}}+\frac{\Delta}{(2\pi)^{1/2}}+\epsilon,$

where $\Phi(z)$ is the standard Gaussian cumulative distribution function,
$c^{BE}=0.5600$, and

$\Delta=\frac{3L}{\epsilon\sigma}\left[(\bar{Q}^{1/2}+\bar{\alpha})\\{\mathcal{R}(\hat{\gamma})\\}^{\bar{q}/2}+\bar{\sigma}\\{\mathcal{R}(\hat{\alpha})\\}^{1/2}+\\{n\mathcal{R}(\hat{\gamma})\mathcal{R}(\hat{\alpha})\\}^{1/2}\right].$

######  Theorem 5.5 (Finite sample variance estimation).

Suppose Assumptions 5.9 and 5.10 hold, $\mathbb{E}[\varepsilon_{i}^{2}\mid
W_{i,\cdot}]\leq\bar{\sigma}^{2}$, and
$\|\hat{\alpha}\|_{\infty}\leq\bar{\alpha}^{\prime}$. Then with probability
$1-\epsilon^{\prime}$,

$|\hat{\sigma}^{2}-(\sigma^{2}+\textsc{bias})|\leq\Delta^{\prime}+\Delta^{\prime\prime}+3[(\Delta^{\prime})^{1/2}\\{(\Delta^{\prime\prime})^{1/2}+\sigma+\Delta_{\textsc{out}}^{1/2}\\}+(\Delta^{\prime\prime})^{1/2}\\{\Delta_{\textsc{out}}^{1/2}+(\Delta^{\prime})^{1/4}\Delta_{\textsc{out}}^{1/4}\\}+(\Delta^{\prime})^{1/4}\Delta_{\textsc{out}}^{1/4}\sigma],$

where
$\textsc{bias}=\Delta_{\textsc{out}}+2\Delta_{\textsc{out}}^{1/2}\sigma$,
$\Delta_{\textsc{out}}=\frac{1}{n}\sum_{i=1}^{n}[(\theta_{i}-\theta_{0})^{2}]$,
and

$\Delta^{\prime}=4(\hat{\theta}-\theta_{0})^{2}+\frac{24L}{\epsilon^{\prime}}\left[\\{\bar{Q}+(\bar{\alpha}^{\prime})^{2}\\}\mathcal{R}(\hat{\gamma})^{\bar{q}}+\bar{\sigma}^{2}\mathcal{R}(\hat{\alpha})\right],\quad\Delta^{\prime\prime}=\left(\frac{2}{\epsilon^{\prime}}\right)^{1/2}\chi^{2}n^{-\frac{1}{2}}.$

######  Corollary 5.3 (Confidence interval coverage).

Suppose the conditions of Theorems 5.4 and 5.5 hold. Further assume

  1. 1.

Moment regularity: $\\{(\xi/\sigma)^{3}+\chi^{2}\\}n^{-\frac{1}{2}}\rightarrow
0$;

  2. 2.

Error-in-variable regression rate:
$\left(\bar{Q}^{1/2}+\bar{\alpha}/\sigma+\bar{\alpha}^{\prime}\right)\\{\mathcal{R}(\hat{\gamma})\\}^{\bar{q}/2}\rightarrow
0$;

  3. 3.

Error-in-variable balancing weight rate:
$\bar{\sigma}\\{\mathcal{R}(\hat{\alpha})\\}^{1/2}\rightarrow 0$;

  4. 4.

Product of rates is fast:
$\\{n\mathcal{R}(\hat{\gamma})\mathcal{R}(\hat{\alpha})\\}^{1/2}/\sigma\rightarrow
0$.

$\text{Then}\quad\hat{\theta}\overset{p}{\rightarrow}\theta_{0},\quad\hat{\sigma}^{2}\overset{p}{\rightarrow}\sigma^{2}+\textsc{bias},\quad\mathbb{P}\\{\theta_{0}\in(\hat{\theta}\pm
1.96\hat{\sigma}n^{-1/2})\\}\rightarrow 0.95+c,\quad\textsc{bias},c\geq 0.$

If in addition $\Delta_{\textsc{out}}\rightarrow 0$, i.e. there are not too
many outliers,

$\text{then}\quad\hat{\theta}\overset{p}{\rightarrow}\theta_{0},\quad\hat{\sigma}^{2}\overset{p}{\rightarrow}\sigma^{2},\quad\mathbb{P}\\{\theta_{0}\in(\hat{\theta}\pm
1.96\hat{\sigma}n^{-1/2})\\}\rightarrow 0.95.$

######  Remark 5.6 (General causal parameters and data cleaning).

Corollary 5.3 holds, as stated, for a broad class of semiparametric estimands
such as the average elasticity and nonparametric estimands such as
heterogeneous treatment effects. Moreover, it holds for not only the data
cleaning and estimation procedure that we propose, but for any data cleaning
and estimation procedure satisfying its weak conditions.

The individual rate conditions $\mathcal{R}(\hat{\gamma})\rightarrow 0$ and
$\mathcal{R}(\hat{\alpha})\rightarrow 0$ as well as the product rate condition
$\\{n\mathcal{R}(\hat{\gamma})\mathcal{R}(\hat{\alpha})\\}^{1/2}\rightarrow 0$
suffice for Gaussian approximation with standard deviation $\sigma n^{-1/2}$,
generalizing the main result in [Chernozhukov et al., 2021] to the harder
setting with corrupted and i.n.i.d. data. These rate conditions are in terms
of a more general norm than previous work because of matrix completion in the
data cleaning step. Nonetheless, we recover a familiar product rate condition
from semiparametric theory. The conditions solve the two remaining theoretical
challenges. First, they provide a framework to translate an on-average data
cleaning guarantee into a data cleaning-adjusted confidence interval for the
causal parameter, by using generalized norms. Second, they ensure that the
standard deviation is $\sigma n^{-1/2}$ as long as the product of error-in-
variable rates (and hence the product of data cleaning rates) is of order
$n^{-1/2}$. In summary, they allow for causal inference at rates faster than
matrix completion, which will be essential for the leading application, where
we desire precision for the population while maintaining privacy for
individuals.

A major technical innovation is semiparametric variance estimation in the
i.n.i.d. setting, which is essential to the validity of confidence intervals.
We define $\Delta_{\textsc{out}}$ to quantify the frequency of outliers. Since
$\theta_{i}=\mathbb{E}[\gamma_{0}(1,X_{i,\cdot})-\gamma_{0}(0,X_{i,\cdot})]$,
$\Delta_{\textsc{out}}$ quantifies the shift in the marginal distributions of
true covariates $\mathbb{P}_{i}(X_{i,\cdot})$. At best,
$\Delta_{\textsc{out}}=0$ in the i.i.d. case. At worst,
$\Delta_{\textsc{out}}$ is a constant (when individual treatment effects are
bounded). The condition $\Delta_{\textsc{out}}\rightarrow 0$, i.e. relatively
few outliers, suffices for consistent variance estimation and nominal
confidence intervals. When $\Delta_{\textsc{out}}\not\rightarrow 0$, our
variance estimator is asymptotically biased upwards by
$\textsc{bias}=\Delta_{\textsc{out}}+2\Delta_{\textsc{out}}^{1/2}\sigma$,
implying conservative confidence intervals. At worst, our confidence intervals
are valid but conservative by a theoretically quantifiable amount. Our exact
characterization of bias may have broader consequences for variance upper
bounds and design-based uncertainty, which we pose as a direction for future
work.777We thank Isaiah Andrews for suggesting this connection.

####  5.5 Key assumption holds for nonlinear factor models

Finally, we tie together our various results and revisit our key assumption
that covariates are approximately low rank. We show that nonlinear factor
models (i) encode the intuition of approximate repeated measurements; (ii)
imply that covariates are approximately low rank; and (iii) satisfy the rate
conditions for causal inference. In a nonlinear factor model,
$X_{ij}=g(\lambda_{i},\mu_{j})$ where $(\lambda_{i},\mu_{j})$ are latent
factors corresponding to units and covariates, respectively. We assume that
the function $g$ is smooth in its second argument, formalizing the repeated
measurement intuition.

######  Definition 5.1 (Hölder class).

The Hölder class $\mathcal{H}(q,S,C_{H})$ on $[0,1)^{q}$ is the set of
functions $g:[0,1)^{q}\to\mathbb{R}$ whose partial derivatives satisfy

$\sum_{s:|s|=\lfloor
S\rfloor}\frac{1}{s!}|\nabla_{s}g(\mu)-\nabla_{s}g(\mu^{\prime})|\leq
C_{H}\left\|\mu-\mu^{\prime}\right\|_{\max}^{S-\lfloor
S\rfloor},\quad\forall\mu,\mu^{\prime}\in[0,1)^{q},$

where $\lfloor S\rfloor$ denotes the largest integer strictly smaller than
$S$.

######  Assumption 5.11 (Generalized factor model).

Assume $\boldsymbol{X}$ is generated as $X_{ij}=g(\lambda_{i},\mu_{j})$, where
$\lambda_{i},\mu_{j}\in[0,1)^{q}$ and
$g(\lambda_{i},\cdot)\in\mathcal{H}(q,S,C_{H})$.

A linear factor model is a special case of a generalized factor model where
$g(\lambda_{i},\mu_{j})=\lambda_{i}^{T}\mu_{j}$. Such a model satisfies
Definition 5.1 for all $S\in\mathbb{N}$ and $C_{H}=C$, for some absolute
positive constant $C<\infty$. Assumption 5.11 also allows for smooth nonlinear
factor models, and it implies joint control over $(r,\Delta_{E})$ as desired.
Intuitively, as latent dimension $q$ increases, the rank $r$ increases. As
smoothness $S$ increases, the approximation error $\Delta_{E}$ decreases. Our
final result demonstrates that, as long as the ratio $q/S$ is small enough,
the data cleaning adjusted confidence intervals are valid.

######  Remark 5.7 (General dictionaries).

In the main text, we state results that hold for a broad class of
dictionaries, with the concise notation $q^{\prime}$ in Corollary 5.4. In
Appendix H, we prove that $q^{\prime}\leq d_{\max}q$, where $d_{\max}$ is the
degree of the polynomial dictionary. We articulate restrictions on the class
of dictionaries in Appendix C. For the interacted dictionary, $d_{\max}=2$.

######  Corollary 5.4.

Suppose the conditions of Theorems 5.2, 5.3, 5.4 and 5.5 hold, as well as
Assumption 5.11. For simplicity, consider the semiparametric case where
$\sigma,\bar{\sigma},\bar{\alpha},\bar{\alpha}^{\prime},\bar{Q}$ are bounded
above and $\bar{q}=1$. If in addition

  1. 1.

Moment regularity: $\\{(\xi/\sigma)^{3}+\chi^{2}\\}n^{-\frac{1}{2}}\rightarrow
0$;

  2. 2.

Weak dependence: $(K_{a},\kappa,\bar{K},\rho_{\min}^{-1})$ scale polynomially
in $\ln(np)$;

  3. 3.

Nonlinear sparse approximation:
$m\Delta_{\phi}\leq\|\beta^{*}\|_{1}^{2},<\infty$; and
$m\Delta_{\zeta}\leq\|\eta^{*}\|_{1}^{2}$;

  4. 4.

Enough repeated measurements: $n^{\frac{2}{3}}\lesssim p\lesssim
n^{\frac{3}{2}}$, i.e. $n=p^{\upsilon}$ or $p=n^{\upsilon}$ for
$\upsilon\in[1,\frac{3}{2}]$;

  5. 5.

Small latent dimension to smoothness ratio:
$\frac{q^{\prime}}{S}<\frac{3}{4}-\frac{\upsilon}{2}$.

Then the conclusions of Corollary 5.3 hold.

In summary, we allow either $n>p$ or $p>n$ as long as $(n,p)$ increase at
similar rates. Given $(n,p)$, the ratio of the latent dimension $q$ over
smoothness $S$ in the generalized factor model must be sufficiently low. For
example, if $n=p$ and $q^{\prime}=q$ then we require $q<\frac{S}{4}$: the
latent dimension must be less than a quarter of the smoothness. A sufficiently
low $\frac{q}{S}$ ratio ensures sufficiently fast learning rates
$\mathcal{R}(\hat{\gamma})$ and $\mathcal{R}(\hat{\alpha})$ for causal
inference with standard error $\hat{\sigma}n^{-1/2}$. For the special case of
a linear factor model, the $\frac{q}{S}$ ratio constraint becomes vacuous, and
there is no restriction on the latent dimension $q$. The same is true for a
polynomial factor model where
$g(\lambda_{i},\mu_{j})=\text{polynomial}(\lambda_{i},\mu_{j})$. The doubly
robust framework allows us to slightly relax the conditions stated above and
still obtain valid inference. In particular, either
$\Delta_{\phi}\not\rightarrow 0$ or $\Delta_{\zeta}\not\rightarrow 0$, i.e.
$\gamma_{0}$ or $\alpha_{0}$ may be incorrectly specified. See [Chernozhukov
et al., 2022a, Section 2] for further discussion of mis-specification.

###  6 Case study: Effect of import competition using Census

Equipped with theoretical guarantees, we return to the motivating real world
issue: measurement error, missing values, discretization, and differential
privacy in the US Census. We replicate a seminal paper in labor economics that
uses Census data, while introducing different types and levels of synthetic
corruption. In particular, we implement differential privacy at the level
mandated for the 2020 Census.

####  6.1 Economic research question

We revisit the economic research question studied by [Autor et al., 2013]:
what is the effect of import competition on local labor markets in the US?
Figure 9 illustrates how, between the years 1987 and 2007, Chinese imports
skyrocketed while US manufacturing employment plummeted. Phrased another way,
the research question is whether this correlation is causal. We ask an
additional question: can we recover the same effects after introducing various
types and levels of synthetic corruption?

Figure 9: Is correlation causation? [Autor et al., 2013, Figure 1]

Following the original study, we use Census data at the commuting zone (CZ)
level. A CZ is an aggregate unit interpretable as a local economy. 722 CZs
make up the mainland US, and CZ data are constructed from individual microdata
published by the US government: IPUMS, ACS, BEA, and SSA. The outcome $Y_{i}$
is percent change in US manufacturing employment; the treatment $D_{i}$ is
percent change in imports from China; the instrument $U_{i}$ is percent change
in imports from China to other countries; and the covariates $X_{i}$ are CZ
characteristics. In more detail, we use data from two periods: 1990-2000 and
2000-2007, for a total of 1,444 observations. $(Y_{i},D_{i},U_{i})$ are
changes within a period, e.g. the 2000 level minus the 1990 level, while
$X_{i,\cdot}$ are levels at the beginning of a period, e.g. the 1990 level.
The causal parameter is the partially linear instrumental variable regression
parameter described in Appendix A.

####  6.2 Can we recover the same effects with data corruption?

In Section 3, we have already verified that the covariates $X_{i,\cdot}$ are
approximately low rank. Both the original specification
$X_{i,\cdot}\in\mathbb{R}^{14}$ and the augmented specification
$X_{i,\cdot}\in\mathbb{R}^{30}$ include approximate repeated measurements,
with variables such as percent employment in manufacturing, percent college
educated, and percent employment among women. In particular, the approximate
rank is five.

Figure 10 presents our first semi-synthetic exercises. For reference, we
visualize in red the 2SLS point estimate and confidence interval of [Autor et
al., 2013], using clean data. Immediately next to [Autor et al., 2013]’s
results, we visualize our own point estimate and confidence interval with
clean data. We recover essentially the same point estimate and a somewhat
smaller confidence interval. The true covariates are approximately low rank,
our procedure exploits this fact, and therefore it is is more efficient.
Subsequently, we implement our procedure with increasing levels of measurement
error: 20%, 40%, 60%, 80%, and 100% noise-to-signal ratio. Our point estimates
remain stable, and the standard errors subtly increase in length. We obtain
similar results with missing values, discretization, and differential privacy:
point estimates remain stable and the standard errors adaptively increase in
length for higher noise-to-signal ratios.

(a) Measurement error

(b) Missing values

(c) Discretization

(d) Differential privacy

Figure 10:  Synthetic corruption

####  6.3 Formalizing privacy

Next, we calibrate the semi-synthetic exercises to privacy levels mandated by
the US Census Bureau. To do so, we first clarify how our model of causal
inference with corrupted data accommodates various notions of differential
privacy. With these formal results, we can then calibrate the variance of the
Laplacian noise appropriately. In what follows, we focus on a one-off data
release (formally called the non-interactive setting). There are two relevant
privacy concepts: central differential privacy of summary tables such as the
Census, and per instance differential privacy of microdata such as the current
population survey (CPS).

To begin, we define central differential privacy. To be concrete, we maintain
the following thought experiment: we are the Census Bureau, and our goal is to
publish [Autor et al., 2013]’s CZ-level aggregated data set while protecting
the privacy of individuals within CZs. In particular, we have access to the
individual-level microdata, which we will not publicly share; we will only
publish the CZ-level summaries for aggregate units. Consider a particular
commuting zone $i\in[n]$ with $L_{i}$ individuals, and denote its individual-
level microdata by $\boldsymbol{M}^{(i)}\in\mathbb{R}^{L_{i}\times p}$. We
wish to publish $p$ summary statistics $X_{i,\cdot}$ for this CZ, where
$X_{ij}=\frac{1}{L_{i}}\sum_{\ell=1}^{L_{i}}M^{(i)}_{\ell j}$, however we wish
to maintain plausible deniability that each individual $\ell\in[L_{i}]$
contributed their data. The simulated attack on the 2010 Census found that
Census blocks summary tables did not maintain this plausible deniability.

######  Definition 6.1 (Central differential privacy for summary tables).

A randomized mechanism $\mathcal{M}$ confers central differential privacy with
privacy loss $\epsilon$ if and only if for any two possible individual-level
data sets $\boldsymbol{M}$ and $\boldsymbol{M}^{\prime}$ differing in a single
row, and for all events $E$ in the range of $\mathcal{M}$,

$\frac{\mathbb{P}(\mathcal{M}(\boldsymbol{M})\in
E)}{\mathbb{P}(\mathcal{M}(\boldsymbol{M}^{\prime})\in E)}\leq e^{\epsilon}$

where the randomness is with respect to $\mathcal{M}$.

The canonical mechanism that achieves central differential privacy is to
publish $\mathcal{M}(\boldsymbol{M}^{(i)})=X_{i,\cdot}+H_{i,\cdot}$ instead of
$X_{i,\cdot}$, where $H_{i,\cdot}$ is Laplacian noise with an appropriately
calibrated variance.888More precisely,
$\mathcal{M}_{i}:\mathbb{R}^{L_{i}\times p}\rightarrow\mathbb{R}^{p}$. In
addition to the Laplace mechanism [Dwork et al., 2006], the discrete [Duchi et
al., 2018] and piece-wise uniform [Wang et al., 2019] mechanisms induce
measurement error that is sub-exponential and mean zero, which fits within our
framework. For simplicity, we focus on the Laplace mechanism when relating
privacy to our theoretical results.

######  Proposition 6.1 (Strong protections for aggregate data).

Suppose

  1. 1.

Each entry of microdata is bounded, i.e. $|M^{(i)}_{\ell j}|\leq\bar{A}_{i}$;

  2. 2.

No individual appears in two commuting zones.

Then the mechanism $Z_{ij}=X_{ij}+H_{ij}$ where
$H_{ij}\overset{i.i.d.}{\sim}\text{Laplace}\left(\frac{2\bar{A}_{i}}{\epsilon}\frac{p}{L_{i}}\right)$
confers $\epsilon$ central differential privacy and the measurement error
parameters satisfy
$K_{a},\kappa\leq\max_{i\in[n]}\frac{2^{3/2}\bar{A}_{i}}{\epsilon}\frac{p}{L_{i}}.$

In summary, the calibrated Laplacian variance depends on the privacy loss
$\epsilon$, the most extreme true value $\bar{A}_{i}$, the number of
covariates $p$, and number of individuals $L_{i}$ per aggregate unit. The
condition $\max_{i\in[n]}\frac{p}{L_{i}}\lesssim\ln(np)$ implies that the
measurement error parameters $(K_{a},\kappa)$ diverge slowly with $(n,p)$, so
that our rates of data cleaning and error-in-variable estimation translate
into data cleaning adjusted confidence intervals. This auxiliary condition is
a practical diagnostic: roughly speaking, the number of published covariates
should not greatly exceed the number of individuals per aggregate unit.

######  Remark 6.1 (Limitations for aggregate data).

The theoretical condition $\max_{i\in[n]}\frac{p}{L_{i}}\lesssim\ln(np)$ sheds
new light on limitations. It is perhaps plausible for commuting zones, but
implausible for Census blocks, which have fewer individuals per aggregate
unit. Fortunately, much empirical economic research studies commuting zones,
which we study in our semi-synthetic exercise. An important direction for
future research is to empirically investigate, through simulated attacks, how
vulnerable various data releases may be for different $\frac{p}{L_{i}}$
regimes.

In addition to tabular summaries, the Bureau publishes microdata, for which an
alternative definition of privacy is necessary. We maintain the following
thought experiment: we are the Census Bureau, and our goal is to publish the
microdata that is subsequently aggregated by [Autor et al., 2013] at the CZ
level. Now, the index $i\in[n]$ corresponds to the $i$-th individual with
covariates $X_{i,\cdot}\in\mathbb{R}^{p}$ that we wish to publish while
maintaining plausible deniability that any given covariate profile is
contained in the data release.

######  Definition 6.2 (Per instance differential privacy for microdata).

A randomized mechanism $\mathcal{M}$ confers per instance differential privacy
with privacy loss $\epsilon$ if and only if for any two possible individual-
level vectors $x$ and $x^{\prime}$, and for all events $E$ in the range of
$\mathcal{M}$,

$\frac{\mathbb{P}(\mathcal{M}(x)\in E)}{\mathbb{P}(\mathcal{M}(x^{\prime})\in
E)}\leq e^{\epsilon}$

where the randomness is with respect to $\mathcal{M}$.

A mechanism that achieves per instance differential privacy is to publish
$\mathcal{M}(X_{i,\cdot})=X_{i,\cdot}+H_{i,\cdot}$ instead of $X_{i,\cdot}$,
where $H_{i,\cdot}$ is Laplacian noise with an appropriately calibrated
variance. We focus on the Laplace mechanism when relating privacy to our
theoretical results. In anticipation of a more qualified privacy guarantee, we
consider adding noise to only $T$ out of the $p$ covariates.

######  Proposition 6.2 (Weaker protections for microdata).

Suppose Assumption 5.1 holds. The mechanism $Z_{ij}=X_{ij}+H_{ij}$ with
$j\in[T]$, where
$H_{ij}\overset{i.i.d.}{\sim}\text{Laplace}\left(\frac{2\bar{A}T}{\epsilon}\right)$,
confers $\epsilon$ per instance differential privacy for the initial $T$
covariates. Moreover, the measurement error parameters satisfy
$K_{a},\kappa\leq\frac{2^{3/2}\bar{A}T}{\epsilon}.$

Figure 11: Calibration

In summary, the calibrated Laplacian variance depends on the privacy loss
$\epsilon$ and the subset size $T$. The condition $T\ll(n,p)$ implies that the
measurement error parameters $(K_{a},\kappa)$ diverge slowly with $(n,p)$, so
that our rates of data cleaning and error-in-variable estimation translate
into data cleaning adjusted confidence intervals. This qualification, that
only a subset of variables may be privatized, is an example of event level
rather than user level differential privacy: with microdata, only some aspects
of an individual’s identity are protected, while with summary tables all
aspects are protected.

####  6.4 Privacy calibrated to 2020 US Census levels

As our second semi-synthetic exercise, we implement central differential
privacy for [Autor et al., 2013]’s CZ-level aggregated data set while
protecting the privacy of individuals within CZs. We calibrate the Laplacian
variance according to Proposition 6.1, where $\epsilon=17.14$ according to a
Bureau memo, $p=30$ in the augmented specification, and $(\bar{A}_{i},L_{i})$
are calculated from the microdata for each CZ. As before, we visualize in red
the point estimate and confidence interval of [Autor et al., 2013] using clean
data in Figure 11. Next to that, we visualize the point estimate and
confidence interval we propose. To study the robustness of our results to the
privacy loss parameter, we consider
$(100\epsilon,10\epsilon,\epsilon,0.1\epsilon,0.01\epsilon)$, which
corresponds to adding Laplacian noise both below and above the mandated level.
Across levels, our point estimates and confidence intervals remain remarkable
stable.

###  7 Conclusion

Recent developments in how the US Census Bureau will publish economic data
motivate us to study a class of corruptions that is rich enough to encompass
classical types of corruption, such as measurement error and missingness, as
well as modern types, such as discretization and differential privacy.
Abstractly, our goal is to learn a causal parameter from corrupted data;
concretely, our goal is to characterize scenarios in which it is possible to
achieve both privacy and precision. To do so, we propose new data cleaning-
adjusted confidence intervals that are computationally simple, statistically
rigorous, and empirically robust in settings calibrated to empirical economic
research. We build a framework to use matrix completion as data cleaning for
downstream causal inference, bridging two rich literatures. For future
research, we pose the question of how to extend our results to confounded
noise and sample selection bias. The modular structure of this paper provides
a template to do so.

### References

  * [Abadie and Imbens, 2012] Abadie, A. and Imbens, G. W. (2012).  A martingale representation for matching estimators.  Journal of the American Statistical Association, 107(498):833–843.
  * [Abowd and Schmutte, 2019] Abowd, J. M. and Schmutte, I. M. (2019).  An economic analysis of privacy protection and statistical accuracy as social choices.  American Economic Review, 109(1):171–202.
  * [Agarwal et al., 2020a] Agarwal, A., Shah, D., and Shen, D. (2020a).  On principal component regression in a high-dimensional error-in-variables setting.  arXiv:2010.14449.
  * [Agarwal et al., 2020b] Agarwal, A., Shah, D., and Shen, D. (2020b).  Synthetic interventions.  arXiv:2006.07691.
  * [Agarwal et al., 2021] Agarwal, A., Shah, D., Shen, D., and Song, D. (2021).  On robustness of principal component regression.  Journal of the American Statistical Association, pages 1–34.
  * [Ai and Chen, 2003] Ai, C. and Chen, X. (2003).  Efficient estimation of models with conditional moment restrictions containing unknown functions.  Econometrica, 71(6):1795–1843.
  * [Anatolyev and Mikusheva, 2021] Anatolyev, S. and Mikusheva, A. (2021).  Factor models with many assets: Strong factors, weak factors, and the two-pass procedure.  Journal of Econometrics.
  * [Andrews, 1994] Andrews, D. W. (1994).  Asymptotics for semiparametric econometric models via stochastic equicontinuity.  Econometrica, pages 43–72.
  * [Arkhangelsky et al., 2019] Arkhangelsky, D., Athey, S., Hirshberg, D. A., Imbens, G. W., and Wager, S. (2019).  Synthetic difference in differences.  Technical report, National Bureau of Economic Research.
  * [Athey et al., 2021] Athey, S., Bayati, M., Doudchenko, N., Imbens, G., and Khosravi, K. (2021).  Matrix completion methods for causal panel data models.  Journal of the American Statistical Association, pages 1–15.
  * [Autor et al., 2013] Autor, D. H., Dorn, D., and Hanson, G. H. (2013).  The China syndrome: Local labor market effects of import competition in the United States.  American Economic Review, 103(6):2121–68.
  * [Bai, 2003] Bai, J. (2003).  Inferential theory for factor models of large dimensions.  Econometrica, 71(1):135–171.
  * [Bai, 2009] Bai, J. (2009).  Panel data models with interactive fixed effects.  Econometrica, 77(4):1229–1279.
  * [Bai and Ng, 2002] Bai, J. and Ng, S. (2002).  Determining the number of factors in approximate factor models.  Econometrica, 70(1):191–221.
  * [Bai and Ng, 2006] Bai, J. and Ng, S. (2006).  Confidence intervals for diffusion index forecasts and inference for factor-augmented regressions.  Econometrica, 74(4):1133–1150.
  * [Bai and Ng, 2013] Bai, J. and Ng, S. (2013).  Principal components estimation and identification of static factors.  Journal of Econometrics, 176(1):18–29.
  * [Bai and Ng, 2019] Bai, J. and Ng, S. (2019).  Matrix completion, counterfactuals, and factor analysis of missing data.  arXiv:1910.06677.
  * [Bai and Wang, 2014] Bai, J. and Wang, P. (2014).  Identification theory for high dimensional static and dynamic factor models.  Journal of Econometrics, 178(2):794–804.
  * [Battistin and Chesher, 2014] Battistin, E. and Chesher, A. (2014).  Treatment effect estimation with covariate measurement error.  Journal of Econometrics, 178(2):707–715.
  * [Ben-Michael et al., 2021] Ben-Michael, E., Feller, A., Hirshberg, D. A., and Zubizarreta, J. R. (2021).  The balancing act in causal inference.  arXiv:2110.14831.
  * [Benedetto et al., 2022] Benedetto, G., Linse, K., and Parker, E. (2022).  Improving disclosure avoidance procedures for the Current Population Survey public use file.  United States Census Bureau.
  * [Bia et al., 2020] Bia, M., Huber, M., and Lafférs, L. (2020).  Double machine learning for sample selection models.  arXiv:2012.00745.
  * [Bickel et al., 1993] Bickel, P. J., Klaassen, C. A., Ritov, Y., and Wellner, J. A. (1993).  Efficient and Adaptive Estimation for Semiparametric Models, volume 4.  Johns Hopkins University Press Baltimore.
  * [Bruns-Smith and Feller, 2022] Bruns-Smith, D. A. and Feller, A. (2022).  Outcome assumptions and duality theory for balancing weights.  In International Conference on Artificial Intelligence and Statistics, pages 11037–11055. PMLR.
  * [Candès and Recht, 2009] Candès, E. J. and Recht, B. (2009).  Exact matrix completion via convex optimization.  Foundations of Computational mathematics, 9(6):717–772.
  * [Candès and Tao, 2010] Candès, E. J. and Tao, T. (2010).  The power of convex relaxation: Near-optimal matrix completion.  IEEE Transactions on Information Theory, 56(5):2053–2080.
  * [Chatterjee, 2015] Chatterjee, S. (2015).  Matrix estimation by universal singular value thresholding.  The Annals of Statistics, 43(1):177–214.
  * [Chattopadhyay and Zubizarreta, 2021] Chattopadhyay, A. and Zubizarreta, J. R. (2021).  On the implied weights of linear regression for causal inference.  arXiv:2104.06581.
  * [Chen et al., 2011] Chen, X., Hong, H., and Nekipelov, D. (2011).  Nonlinear models of measurement errors.  Journal of Economic Literature, 49(4):901–37.
  * [Chernozhukov et al., 2018a] Chernozhukov, V., Chetverikov, D., Demirer, M., Duflo, E., Hansen, C., Newey, W. K., and Robins, J. (2018a).  Double/debiased machine learning for treatment and structural parameters.  The Econometrics Journal, 21(1).
  * [Chernozhukov et al., 2016] Chernozhukov, V., Escanciano, J. C., Ichimura, H., Newey, W. K., and Robins, J. M. (2016).  Locally robust semiparametric estimation.  arXiv:1608.00033.
  * [Chernozhukov et al., 2021] Chernozhukov, V., Newey, W. K., and Singh, R. (2021).  A simple and general debiased machine learning theorem with finite sample guarantees.  arXiv:2105.15197.
  * [Chernozhukov et al., 2022a] Chernozhukov, V., Newey, W. K., and Singh, R. (2022a).  Automatic debiased machine learning of causal and structural effects.  Econometrica, 90(3):967–1027.
  * [Chernozhukov et al., 2022b] Chernozhukov, V., Newey, W. K., and Singh, R. (2022b).  Debiased machine learning of global and local parameters using regularized Riesz representers.  The Econometrics Journal.
  * [Chernozhukov et al., 2018b] Chernozhukov, V., Wuthrich, K., and Zhu, Y. (2018b).  A $t$-test for synthetic controls.  arXiv:1812.10820.
  * [Chetty and Hendren, 2018a] Chetty, R. and Hendren, N. (2018a).  The impacts of neighborhoods on intergenerational mobility i: Childhood exposure effects.  The Quarterly Journal of Economics, 133(3):1107–1162.
  * [Chetty and Hendren, 2018b] Chetty, R. and Hendren, N. (2018b).  The impacts of neighborhoods on intergenerational mobility ii: County-level estimates.  The Quarterly Journal of Economics, 133(3):1163–1228.
  * [D’Agostino Jr and Rubin, 2000] D’Agostino Jr, R. B. and Rubin, D. B. (2000).  Estimating and using propensity scores with partially missing data.  Journal of the American Statistical Association, 95(451):749–759.
  * [Das et al., 2003] Das, M., Newey, W. K., and Vella, F. (2003).  Nonparametric estimation of sample selection models.  The Review of Economic Studies, 70(1):33–58.
  * [Datta and Zou, 2017] Datta, A. and Zou, H. (2017).  Cocolasso for high-dimensional error-in-variables regression.  Annals of Statistics, 45(6):2400–2426.
  * [Davis and Kahan, 1970] Davis, C. and Kahan, W. M. (1970).  The rotation of eigenvectors by a perturbation. iii.  SIAM Journal on Numerical Analysis, 7(1):1–46.
  * [Deaner, 2018] Deaner, B. (2018).  Proxy controls and panel data.  arXiv:1810.00283.
  * [Delaigle et al., 2008] Delaigle, A., Hall, P., and Meister, A. (2008).  On deconvolution with repeated measurements.  The Annals of Statistics, 36(2):665–685.
  * [Duchi et al., 2018] Duchi, J. C., Jordan, M. I., and Wainwright, M. J. (2018).  Minimax optimal procedures for locally private estimation.  Journal of the American Statistical Association, 113(521):182–201.
  * [Dwork et al., 2006] Dwork, C., McSherry, F., Nissim, K., and Smith, A. (2006).  Calibrating noise to sensitivity in private data analysis.  Theory of Cryptography, pages 265–284.
  * [Fan et al., 2018] Fan, J., Wang, W., and Zhong, Y. (2018).  An l-$\infty$ eigenvector perturbation bound and its application to robust covariance estimation.  Journal of Machine Learning Research, 18(207):1–42.
  * [Feng, 2020] Feng, Y. (2020).  Causal inference in possibly nonlinear factor models.  arXiv:2008.13651.
  * [Fernández-Val et al., 2020] Fernández-Val, I., Freeman, H., and Weidner, M. (2020).  Low-rank approximations of nonseparable panel models.  arXiv:2010.12439.
  * [Foster and Syrgkanis, 2019] Foster, D. J. and Syrgkanis, V. (2019).  Orthogonal statistical learning.  arXiv:1901.09036.
  * [Gavish and Donoho, 2014] Gavish, M. and Donoho, D. L. (2014).  The optimal hard threshold for singular values is $4/\sqrt{3}$.  IEEE Transactions on Information Theory, 60(8):5040–5053.
  * [Griliches, 1986] Griliches, Z. (1986).  Economic data issues.  Handbook of Econometrics, 3:1465–1514.
  * [Hasminskii and Ibragimov, 1979] Hasminskii, R. Z. and Ibragimov, I. A. (1979).  On the nonparametric estimation of functionals.  In Proceedings of the Second Prague Symposium on Asymptotic Statistics.
  * [Hastie et al., 2015] Hastie, T., Mazumder, R., Lee, J. D., and Zadeh, R. (2015).  Matrix completion and low-rank SVD via fast alternating least squares.  The Journal of Machine Learning Research, 16(1):3367–3402.
  * [Hausman and Wise, 1979] Hausman, J. A. and Wise, D. A. (1979).  Attrition bias in experimental and panel data: the Gary income maintenance experiment.  Econometrica, pages 455–473.
  * [Hawes, 2021] Hawes, M. (2021).  The Census Bureau’s simulated reconstruction-abetted re-identification attack on the 2010 Census.  United States Census Bureau.
  * [Heckman, 1979] Heckman, J. J. (1979).  Sample selection bias as a specification error.  Econometrica, pages 153–161.
  * [Hotz et al., 2022] Hotz, V. J., Bollinger, C. R., Komarova, T., Manski, C. F., Moffitt, R. A., Nekipelov, D., Sojourner, A., and Spencer, B. D. (2022).  Balancing data privacy and usability in the federal statistical system.  Proceedings of the National Academy of Sciences, 119(31):e2104906119.
  * [Hu and Schennach, 2008] Hu, Y. and Schennach, S. M. (2008).  Instrumental variable treatment of nonclassical measurement error models.  Econometrica, 76(1):195–216.
  * [Huber, 2014] Huber, M. (2014).  Treatment evaluation in the presence of sample selection.  Econometric Reviews, 33(8):869–905.
  * [Jarmin, 2019] Jarmin, R. (2019).  Census Bureau adopts cutting edge privacy protections for 2020 Census.  United States Census Bureau.
  * [Jöreskog and Sörbom, 1996] Jöreskog, K. G. and Sörbom, D. (1996).  LISREL 8: User’s reference guide.  Scientific Software International.
  * [Kallus et al., 2018] Kallus, N., Mao, X., and Udell, M. (2018).  Causal inference with noisy and missing covariates via matrix factorization.  arXiv:1806.00811.
  * [Keshavan et al., 2009] Keshavan, R., Montanari, A., and Oh, S. (2009).  Matrix completion from noisy entries.  Advances in Neural Information Processing Systems, 22.
  * [Klaassen, 1987] Klaassen, C. A. (1987).  Consistent estimation of the influence function of locally asymptotically linear estimators.  The Annals of Statistics, pages 1548–1562.
  * [LaLonde, 1986] LaLonde, R. J. (1986).  Evaluating the econometric evaluations of training programs with experimental data.  The American Economic Review, pages 604–620.
  * [Li, 2002] Li, T. (2002).  Robust and consistent estimation of nonlinear errors-in-variables models.  Journal of Econometrics, 110(1):1–26.
  * [Li and Vuong, 1998] Li, T. and Vuong, Q. (1998).  Nonparametric estimation of the measurement error model using multiple indicators.  Journal of Multivariate Analysis, 65(2):139–165.
  * [Loh and Wainwright, 2012] Loh, P.-L. and Wainwright, M. J. (2012).  High-dimensional regression with noisy and missing data: Provable guarantees with nonconvexity.  The Annals of Statistics, 40(3):1637–1664.
  * [Mayer et al., 2020] Mayer, I., Sverdrup, E., Gauss, T., Moyer, J.-D., Wager, S., and Josse, J. (2020).  Doubly robust treatment effect estimation with missing attributes.  Annals of Applied Statistics, 14(3):1409–1431.
  * [Miao et al., 2018] Miao, W., Geng, Z., and Tchetgen Tchetgen, E. J. (2018).  Identifying causal effects with proxy variables of an unmeasured confounder.  Biometrika, 105(4):987–993.
  * [Miao and Tchetgen, 2018] Miao, W. and Tchetgen, E. J. T. (2018).  A confounding bridge approach for double negative control inference on causal effects.  arXiv:1808.04945.
  * [Newey, 1994] Newey, W. K. (1994).  The asymptotic variance of semiparametric estimators.  Econometrica, pages 1349–1382.
  * [Newey, 2001] Newey, W. K. (2001).  Flexible simulated moment estimation of nonlinear errors-in-variables models.  Review of Economics and Statistics, 83(4):616–627.
  * [Onatski, 2009] Onatski, A. (2009).  A formal statistical test for the number of factors in the approximate factor models.  Econometrica, 77(5):1447–1480.
  * [Onatski, 2012] Onatski, A. (2012).  Asymptotics of the principal components estimator of large factor models with weakly influential factors.  Journal of Econometrics, 168(2):244–258.
  * [Poterba et al., 1996] Poterba, J. M., Venti, S. F., and Wise, D. A. (1996).  How retirement saving programs increase saving.  Journal of Economic Perspectives, 10(4):91–112.
  * [Robins et al., 2007] Robins, J., Sued, M., Lei-Gomez, Q., and Rotnitzky, A. (2007).  Comment: Performance of double-robust estimators when “inverse probability” weights are highly variable.  Statistical Science, 22(4):544–559.
  * [Robins and Rotnitzky, 1995] Robins, J. M. and Rotnitzky, A. (1995).  Semiparametric efficiency in multivariate regression models with missing data.  Journal of the American Statistical Association, 90(429):122–129.
  * [Robins et al., 1995] Robins, J. M., Rotnitzky, A., and Zhao, L. P. (1995).  Analysis of semiparametric regression models for repeated outcomes in the presence of missing data.  Journal of the American Statistical Association, 90(429):106–121.
  * [Robinson, 1988] Robinson, P. M. (1988).  Root-n-consistent semiparametric regression.  Econometrica, pages 931–954.
  * [Rosenbaum and Tsybakov, 2013] Rosenbaum, M. and Tsybakov, A. B. (2013).  Improved matrix uncertainty selector.  In From Probability to Statistics and Back: High-Dimensional Models and Processes–A Festschrift in Honor of Jon A. Wellner, pages 276–290. Institute of Mathematical Statistics.
  * [Rosenbaum and Rubin, 1984] Rosenbaum, P. R. and Rubin, D. B. (1984).  Reducing bias in observational studies using subclassification on the propensity score.  Journal of the American Statistical Association, 79(387):516–524.
  * [Rotnitzky et al., 2021] Rotnitzky, A., Smucler, E., and Robins, J. M. (2021).  Characterization of parameters with a mixed bias property.  Biometrika, 108(1):231–238.
  * [Rubin, 1976] Rubin, D. B. (1976).  Inference and missing data.  Biometrika, 63(3):581–592.
  * [Schennach, 2007] Schennach, S. M. (2007).  Instrumental variable estimation of nonlinear errors-in-variables models.  Econometrica, 75(1):201–239.
  * [Schennach, 2016] Schennach, S. M. (2016).  Recent advances in the measurement error literature.  Annual Review of Economics, 8:341–377.
  * [Schennach and Hu, 2013] Schennach, S. M. and Hu, Y. (2013).  Nonparametric identification and semiparametric estimation of classical measurement error models without side information.  Journal of the American Statistical Association, 108(501):177–186.
  * [Shevtsova, 2010] Shevtsova, I. (2010).  An improvement of convergence rate estimates in the Lyapunov theorem.  In Doklady Mathematics, volume 82, pages 862–864. Springer.
  * [Singh, 2020] Singh, R. (2020).  Kernel methods for unobserved confounding: Negative controls, proxies, and instruments.  arXiv:2012.10315.
  * [Singh, 2021] Singh, R. (2021).  Generalized kernel ridge regression for causal inference with missing-at-random sample selection.  arXiv:2111.05277.
  * [Singh et al., 2019] Singh, R., Sahani, M., and Gretton, A. (2019).  Kernel instrumental variable regression.  In Advances in Neural Information Processing Systems, pages 4595–4607.
  * [Singh et al., 2020] Singh, R., Xu, L., and Gretton, A. (2020).  Reproducing kernel methods for nonparametric and semiparametric treatment effects.  arXiv:2010.04855.
  * [Steed et al., 2022] Steed, R., Liu, T., Wu, Z. S., and Acquisti, A. (2022).  Policy impacts of statistical uncertainty and privacy.  Science, 377(6609):928–931.
  * [Stock and Watson, 2002] Stock, J. H. and Watson, M. W. (2002).  Forecasting using principal components from a large number of predictors.  Journal of the American Statistical Association, 97(460):1167–1179.
  * [Tchetgen Tchetgen et al., 2020] Tchetgen Tchetgen, E. J., Ying, A., Cui, Y., Shi, X., and Miao, W. (2020).  An introduction to proximal causal learning.  arXiv:2009.10982.
  * [van der Laan and Rose, 2018] van der Laan, M. J. and Rose, S. (2018).  Targeted Learning in Data Science.  Springer.
  * [van der Laan and Rubin, 2006] van der Laan, M. J. and Rubin, D. (2006).  Targeted maximum likelihood learning.  The International Journal of Biostatistics, 2(1).
  * [Vershynin, 2018] Vershynin, R. (2018).  High-Dimensional Probability: An Introduction with Applications in Data Science, volume 47.  Cambridge University Press.
  * [Wang and Hsiao, 2011] Wang, L. and Hsiao, C. (2011).  Method of moments estimation and identifiability of semiparametric nonlinear errors-in-variables models.  Journal of Econometrics, 165(1):30–44.
  * [Wang et al., 2019] Wang, N., Xiao, X., Yang, Y., Zhao, J., Hui, S. C., Shin, H., Shin, J., and Yu, G. (2019).  Collecting and analyzing multidimensional data with local differential privacy.  In 2019 IEEE 35th International Conference on Data Engineering (ICDE), pages 638–649. IEEE.
  * [Wang and Fan, 2017] Wang, W. and Fan, J. (2017).  Asymptotics of empirical eigenstructure for high dimensional spiked covariance.  Annals of Statistics, 45(3):1342.
  * [Wedin, 1972] Wedin, P.-Å. (1972).  Perturbation bounds in connection with singular value decomposition.  BIT Numerical Mathematics, 12(1):99–111.
  * [Wold, 1982] Wold, H. (1982).  Soft modelling: The basic design and some extensions.  Systems Under Indirect Observation, Part II, pages 36–37.
  * [Wold et al., 1984] Wold, S., Ruhe, A., Wold, H., and Dunn III, W. J. (1984).  The collinearity problem in linear regression. The partial least squares (PLS) approach to generalized inverses.  SIAM Journal on Scientific and Statistical Computing, 5(3):735–743.
  * [Xiong and Pelger, 2019] Xiong, R. and Pelger, M. (2019).  Large dimensional latent factor modeling with missing observations and applications to causal inference.  arXiv:1910.08273.
  * [Zheng and van der Laan, 2011] Zheng, W. and van der Laan, M. J. (2011).  Cross-validated targeted minimum-loss-based estimation.  In Targeted Learning, pages 459–474. Springer Science & Business Media.

## Appendix

###  Appendix A Additional examples

####  A.1 Semiparametric estimands

We consider the goal of estimation and inference of some causal parameter
$\theta_{0}\in\mathbb{R}$ which is a scalar summary of the regression
$\gamma_{0}$, e.g. a treatment effect, policy effect, or elasticity. We
consider a class of causal parameters of the form

$\theta_{0}=\frac{1}{n}\sum_{i=1}^{n}\theta_{i},\quad\theta_{i}=\mathbb{E}[m(W_{i,\cdot},\gamma_{0})]$

in an i.n.i.d. data generating process of the form

$Y_{i}=\gamma_{0}(D_{i},X_{i,\cdot})+\varepsilon_{i},\quad
Z_{i,\cdot}=(X_{i,\cdot}+H_{i,\cdot})\odot\pi_{i,\cdot},\quad
W_{i,\cdot}=(D_{i},X_{i,\cdot},H_{i,\cdot},\pi_{i,\cdot}).$

This model is a restatement of (4). $(D_{i},X_{i,\cdot})$ concatenate the
various arguments of $\gamma_{0}$, which we hereby call regressors. Rather
than observing $X_{i,\cdot}$, we observe $Z_{i,\cdot}$. This model includes
the scenario in which some variables are corrupted and other are not. Which
regressors are corrupted or uncorrupted constrains the construction of
technical regressors; see Appendix C. We concatentate signal and noise as
$W_{i,\cdot}=(D_{i},X_{i,\cdot},H_{i,\cdot},\pi_{i,\cdot})$. A generalization
of Assumption 5.9 imposes invariance of the regression $\gamma_{0}$ and
generalized balancing weight $\alpha_{0}$ across observations, which we
formalize in Appendix G.

######  Example A.1 (Average treatment effect).

Let $(D_{i},X_{i,\cdot})$ concatenate treatment $D_{i}\in\\{0,1\\}$ and
covariates $X_{i,\cdot}\in\mathbb{R}^{p}$. Denote
$\gamma_{0}(D_{i},X_{i,\cdot}):=\mathbb{E}[Y_{i}|D_{i},X_{i,\cdot}]$. Under
the assumption of selection on observables, the average treatment effect is
given by

$\theta_{i}=\mathbb{E}[\gamma_{0}(1,X_{i,\cdot})-\gamma_{0}(0,X_{i,\cdot})].$

With uncorrupted treatment and corrupted covariates,
$W_{i,\cdot}=(D_{i},X_{i,\cdot},H_{i,\cdot},\pi_{i,\cdot})$ where
$(H_{i,\cdot},\pi_{i,\cdot})$ are measurement error and missingness for the
covariates.999More generally, treatment observations may be corrupted as well.
For readability, we exposit the simpler and plausible case that treatment is
uncorrupted.

While the true regression $\gamma_{0}(D_{i},X_{i,\cdot})$ is only a function
of signal $(D_{i},X_{i,\cdot})$, our regression estimator
$\hat{\gamma}(D_{i},Z_{i,\cdot})$ is a function of both signal and noise
$W_{i,\cdot}$. In other words, the hypothesis space for estimation is the
extended space of functions $\mathbb{L}_{2}(\mathcal{W})$, and we must define
an extended functional over $\mathbb{L}_{2}(\mathcal{W})$. In Example A.1, the
extended functional is

$\gamma\mapsto\mathbb{E}[\gamma(1,X_{i,\cdot},H_{i,\cdot},\pi_{i,\cdot})-\gamma(0,X_{i,\cdot},H_{i,\cdot},\pi_{i,\cdot})].$

######  Example A.2 (Local average treatment effect).

Let $(U_{i},X_{i,\cdot})$ concatenate instrument $U_{i}\in\\{0,1\\}$ and
covariates $X_{i,\cdot}\in\mathbb{R}^{p}$. Denote
$\gamma_{0}(U_{i},X_{i,\cdot}):=\mathbb{E}[Y_{i}|U_{i},X_{i,\cdot}]$ and
$\delta_{0}(U_{i},X_{i,\cdot}):=\mathbb{E}[D_{i}|U_{i},X_{i,\cdot}]$. Under
standard instrumental variable assumptions, the local average treatment effect
for the subpopulation of compliers is given by

$\beta_{0}=\frac{\theta_{0}}{\theta^{\prime}_{0}},\quad\theta_{i}=\mathbb{E}[\gamma_{0}(1,X_{i,\cdot})-\gamma_{0}(0,X_{i,\cdot})],\quad\theta_{i}^{\prime}=\mathbb{E}[\delta_{0}(1,X_{i,\cdot})-\delta_{0}(0,X_{i,\cdot})].$

With uncorrupted instrument and corrupted covariates,
$W_{i,\cdot}=(U_{i},X_{i,\cdot},H_{i,\cdot},\pi_{i,\cdot})$ where
$(H_{i,\cdot},\pi_{i,\cdot})$ are measurement error and missingness for the
covariates.

######  Example A.3 (Average policy effect).

Let $X_{i,\cdot}\in\mathbb{R}^{p}$ be the covariates. Consider the
counterfactual transportation of covariates $x_{i,\cdot}\mapsto
t(x_{i,\cdot})$. Denote
$\gamma_{0}(X_{i,\cdot}):=\mathbb{E}[Y_{i}|X_{i,\cdot}]$. The average policy
effect of transporting covariates is given by

$\theta_{i}=\mathbb{E}[\gamma_{0}\\{t(X_{i,\cdot})\\}-\gamma_{0}(X_{i,\cdot})].$

With corrupted covariates,
$W_{i,\cdot}=(X_{i,\cdot},H_{i,\cdot},\pi_{i,\cdot})$ where
$(H_{i,\cdot},\pi_{i,\cdot})$ are measurement error and missingness for the
covariates.

######  Example A.4 (Price elasticity of demand).

Let $Y_{i}$ be price of a particular good. Let $(D_{i},X_{i,\cdot})$
concatenate quantities sold of the particular good $D_{i}$ and other goods
$X_{i,\cdot}\in\mathbb{R}^{p}$. Denote
$\gamma_{0}(D_{i},X_{i,\cdot})=\mathbb{E}[Y_{i}|D_{i},X_{i,\cdot}]$. The
average price elasticity of demand is

$\theta_{i}=\mathbb{E}\left[\nabla_{d}\gamma_{0}(D_{i},X_{i,\cdot})\right].$

With uncorrupted quantity for the particular good and corrupted quantities for
the other goods, $W_{i,\cdot}=(D_{i},X_{i,\cdot},H_{i,\cdot},\pi_{i,\cdot})$
where $(H_{i,\cdot},\pi_{i,\cdot})$ are measurement error and missingness for
the other goods.

####  A.2 Weighted estimands

In empirical economic research with aggregate units, it is common to weight
units by their size. It is also common to consider partially linear models.
For example, the estimand of [Autor et al., 2013] may be viewed as a weighted
partially linear instrumental variable regression. To bridge theory with
practice, we provide these examples next.

A weighted functional $\theta_{0}\in\mathbb{R}$ is a scalar that takes the
form

$\theta_{0}=\frac{1}{n}\sum_{i=1}^{n}\theta_{i},\quad\theta_{i}=\mathbb{E}[\ell_{i}m(W_{i,\cdot},\gamma_{0})]$

where $\ell_{i}$ is the weight for aggregate unit $i$. For simplicity, we take
the weights $\ell_{i}$ to be known, but their uncertainty can be incorporated
as well.

######  Example A.5 (Weighted partially linear regression).

Let $(D_{i},X_{i})$ concatenate treatment $D\in\mathbb{R}$ and covariates
$X_{i,\cdot}\in\mathbb{R}^{p}$. Denote
$\gamma_{0}(D_{i},X_{i,\cdot})=\mathbb{E}[Y_{i}|D_{i},X_{i,\cdot}]$. The
weighted partially regression coefficient is given by

$\theta_{i}=\mathbb{E}\left[\ell_{i}\\{\gamma_{0}(d+1,X_{i,\cdot})-\gamma_{0}(d,X_{i,\cdot})\\}\right].$

With uncorrupted treatment and corrupted covariates,
$W_{i,\cdot}=(D_{i},X_{i,\cdot},H_{i,\cdot},\pi_{i,\cdot})$ where
$(H_{i,\cdot},\pi_{i,\cdot})$ are measurement error and missingness for the
covariates.

######  Example A.6 (Weighted partially linear instrumental variable
regression).

Let $(U_{i},X_{i,\cdot})$ concatenate instrument $U_{i}\in\mathbb{R}$ and
covariates $X_{i,\cdot}\in\mathbb{R}^{p}$. Denote
$\gamma_{0}(U_{i},X_{i,\cdot}):=\mathbb{E}[Y_{i}|U_{i},X_{i,\cdot}]$ and
$\delta_{0}(U_{i},X_{i,\cdot}):=\mathbb{E}[D_{i}|U_{i},X_{i,\cdot}]$. Under
standard instrumental variable assumptions, the weighted partially linear
instrumental variable regression coefficient is given by

$\beta_{0}=\frac{\theta_{0}}{\theta^{\prime}_{0}},\quad\theta_{i}=\mathbb{E}[\ell_{i}\\{\\{\gamma_{0}(u+1,X_{i,\cdot})-\gamma_{0}(u,X_{i,\cdot})\\}],\quad\theta_{i}^{\prime}=\mathbb{E}[\ell_{i}\\{\delta_{0}(u+1,X_{i,\cdot})-\delta_{0}(u,X_{i,\cdot})\\}].$

With uncorrupted instrument and corrupted covariates,
$W_{i,\cdot}=(U_{i},X_{i,\cdot},H_{i,\cdot},\pi_{i,\cdot})$ where
$(H_{i,\cdot},\pi_{i,\cdot})$ are measurement error and missingness for the
covariates.

####  A.3 Nonparametric estimands

A local functional $\theta_{0}^{\lim}\in\mathbb{R}$ is a scalar that takes the
form

$\theta^{\lim}_{0}=\lim_{h\rightarrow
0}\theta_{0}^{h},\quad\theta_{0}^{h}=\frac{1}{n}\sum_{i=1}^{n}\theta^{h}_{i},\quad\theta_{i}^{h}=\mathbb{E}[m_{h}(W_{i,\cdot},\gamma_{0})]=\mathbb{E}[\ell_{h}(W_{ij})m(W_{i,\cdot},\gamma_{0})]$

where $\ell_{h}$ is a Nadaraya Watson weighting with bandwidth $h$ and
$W_{ij}$ is a scalar component of $W_{i,\cdot}$. $\theta^{\lim}_{0}$ is a
nonparametric quantity. However, it can be approximated by the sequence
$\\{\theta_{0}^{h}\\}$. Each $\theta_{0}^{h}$ can be analyzed like a weighted
functional as long as we keep track of how certain quantities depend on $h$.
By this logic, finite sample semiparametric theory for $\theta^{h}_{0}$
translates to finite sample nonparametric theory for $\theta_{0}^{\lim}$ up to
some approximation error. In this sense, our analysis encompasses both
semiparametric and nonparametric estimands. As a leading example, we study
heterogeneous treatment effects.

######  Example A.7 (Heterogeneous treatment effect).

Let $(D_{i},V_{i},X_{i,\cdot})$ concatenate treatment $D_{i}\in\\{0,1\\}$,
covariate of interest $V_{i}\in\mathbb{R}$, and other covariates
$X_{i,\cdot}\in\mathbb{R}^{p}$. Denote
$\gamma_{0}(D_{i},V_{i},X_{i,\cdot}):=\mathbb{E}[Y_{i}|D_{i},V_{i},X_{i,\cdot}]$.
Under the assumption of selection on observables and identicial distribution
of $V_{i}$, the heterogeneous treatment effect for the subpopulation with
subcovariate value $v$ is given by

$\theta_{i}=\mathbb{E}[\gamma_{0}(1,V_{i},X_{i,\cdot})-\gamma_{0}(0,V_{i},X_{i,\cdot})|V_{i}=v]=\lim_{h\rightarrow
0}\mathbb{E}[\ell_{h}(V_{i})\\{\gamma_{0}(1,V_{i},X_{i,\cdot})-\gamma_{0}(0,V_{i},X_{i,\cdot})\\}]$

where

$\ell_{h}(V_{i})=\frac{K\left\\{(V_{i}-v)/h\right\\}}{\omega},\quad\omega=\mathbb{E}[K\left\\{(V_{i}-v)/h\right\\}]$

and $K$ is the standard kernel function. With uncorrupted treatment,
uncorrupted covariate of interest, and corrupted other covariates,
$W_{i,\cdot}=(D_{i},V_{i},X_{i,\cdot},H_{i,\cdot},\pi_{i,\cdot})$ where
$(H_{i,\cdot},\pi_{i,\cdot})$ are measurement error and missingness for the
other covariates.

In Appendix G, we formalize our general class of semiparametric and
nonparametric estimands. We define the class abstractly and verify that each