I'm reading The Central Role of the Propensity Score in Observational Studies for Causal Effects in order to understand why Propensity Scores work. I'm kind of new to this and I'm not understanding an estimator given in corollary 4.3.

Let $r_t$ be the potential outcome response for treatment $t$. $z = t, t = 0, 1$ denotes the treatment and $b(x)$ is a balancing score. This is the notation used in the paper.

They state:

"Suppose treatment assignment is strongly ignorable, so that in particular, $E[r_t| z = t, b(x)] = E[r_t | b(x) ]$ for balancing score $b(x)$. Further suppose that the conditional expectation of $r_t$ given $b(x)$ is linear: $E [ r_t | z = t, b(x) ] = \alpha_t + \beta_t b(x)$ with $t = 0, 1$.

Then the estimator $$(\hat{\alpha}_1 - \hat{\alpha}_0) + (\hat{\beta}_1 - \hat{\beta}_0)b(x)$$ is conditionally unbiased given $b(x_i) (i = 1, ..., n)$ for the treatment effect at $b(x)$. "

My question is, what is this estimator? I am not sure how to relate it to the notation I am familiar with: Let $Y_i$ be the response of the $i$th unit, and $Z_i$ be a treatment indicator. Keep $b(x)$ as a balancing score. Is this estimator the same as the interpretation of treatment effect $\theta$ as the change in the expected response in a regression model of the form: $$E[Y_i| b(x_i)] = \alpha + \theta z_i + \beta b(x_i)$$

where $Z_i = 0, 1$?

More generally, I'm a little confused on the mathematical justification for why a regression adjustment on the propensity score works. I understand that the potential outcomes are independent of the treatment conditional on the propensity score, but why can we use regression coefficients to estimate average treatment effects? If anyone had some references that would be great

  • $\begingroup$ for those who can't access the paper, define the $r$, $t$, $z$, $b(x)$, and $Y$. Is $r$ response or indicator of receipt of treatment? Did you introduce a $Y$ unnecessarily? $\endgroup$ – AdamO Apr 4 at 19:58
  • $\begingroup$ updated the original post. I am introducing $Y$ since that is the notation for the response I am familiar with. $\endgroup$ – Marcel Apr 4 at 20:07
  • $\begingroup$ This just looks like a simple difference in counterfactual outcomes, where the casual effect varies with the probability of take-up. One way this could happen is if people with high benefit are more likely to opt into treatment. Your notation does not seem to distinguish between counterfactual outcomes, as it's just the observed value. $\endgroup$ – Dimitriy V. Masterov Apr 4 at 20:25

Let's switch from $r_t$ to the more standard notation $Y_t$, which is the potential outcome corresponding to setting treatment $z$ to $t$. The whole point of propensity score analysis is to estimate a causal quantity, usually $E[Y_1-Y_0]$, using observed data. Not all of the potential outcomes are observed because only the potential outcome corresponding to the treatment actually received is observed for each individual.

The causal estimand $E[Y_1-Y_0]$ is known as the ATE. The estimand described in that passage is $E[Y_1-Y_0|b(x)=k]$, which is the conditional average treatment effect for those with balancing score $b(x)=k$. The relevant balancing score is the propensity score, so, for example, the effect of interest might be the causal effect of treatment for those with a propensity score ($b(x)$) of .8 (i.e., $k$).

The estimator is the difference between the conditional means of $Y$ at each level of $z$ adjusted for the balancing score $b(x)$ at $b(x)=k$; more precisely, it's $E[Y|z=1,b(x)=k]-E[Y|z=0,b(x)=k]$. This quantity only relies on observed values (i.e., $Y$, $z$, and $b(x)$), but it is unbiased for the causal quantity $E[Y_1-Y_0|b(x_i)=k]$, which otherwise involves counterfactual values.

Here's how you might get that estimator:

  1. Regress $Y$ on $b(x)$ for those with $z=1$. Get the coefficients $\hat{\alpha}_1$ and $\hat{\beta}_1$ from this regression.
  2. Regress $Y$ on $b(x)$ for those with $z=0$. Get the coefficients $\hat{\alpha}_0$ and $\hat{\beta}_0$ from this regression.
  3. Choose a level of the balancing score $b(X)$, let's say, $k$.
  4. Plug in those values into the formula $(\hat{\alpha}_1 - \hat{\alpha}_0) + (\hat{\beta}_1 - \hat{\beta}_0)k$.

The estimated quantity is the conditional ATE at $b(x)=k$.

It's possible to get this in one single step using a regression model, like the one you proposed. The problem with that regression model is it assumes $\beta_1=\beta_0$. If that's true, the coefficient $\theta$ in your model is the average treatment effect. If not, $\theta$ is biased.

The correct model to use would be $E[Y|z,b(x)]=\lambda_0 + \tau z + \lambda_1(b(x)-k) + \lambda_2(b(x)-k)z$. Now $\tau$ represents the conditional ATE at $b(x)=k$.

Following up on some matters in the comments...

Regression adjustment is one way to use the propensity score to estimate a causal effect. It requires that the relationship between the propensity score and the outcome is linear (or correctly modeled) at each level of treatment. If there is no treatment effect modification by the propensity score, a regression of the outcome on the treatment and the propensity score, as you have done, will indeed estimate a valid causal effect. The coefficient on treatment ($\theta$ in your model) corresponds to the ATE.

If there is treatment effect modification, which means the effect of treatment depends on one's propensity score, you need to model the interaction between the treatment and the propensity score on the outcome. If you center the propensity score at some value $k$, the effect estimate is the conditional ATE at the centered value (i.e., $k$) of the propensity score, which is equal to the quantity estimated using $(\hat{\alpha}_1 - \hat{\alpha}_0) + (\hat{\beta}_1 - \hat{\beta}_0)k$. With a linear outcome model, the ATE is the conditional ATE evaluated at the mean of the propensity score in the sample. Within a nonlinear outcome model, estimating the ATE requires using a marginal effects procedure like the margins command in Stata.

You should always allow the treatment effect to vary based on the propensity score or any covariates in your model, which means including an interaction between treatment and that covariate in the outcome model. Centering the covariate at its mean in a linear model allows the coefficient on the main effect of treatment to be equal the ATE. The propensity score literature may not be super clear about this, but it is the best thing to do, and, as you have found, R&R83 even recommend it.

  • $\begingroup$ Thank you for the nice answer. When would we assume that $\beta_1 \neq \beta_0$? I'm not sure how to interpret the coefficient $\beta$, since it's related to the balancing score. Are we saying that the balancing score has a different effect size in those who received treatment when compared to those who did not? What if we let $b(x) = p(x)$, a propensity score? $\endgroup$ – Marcel Apr 7 at 23:28
  • $\begingroup$ You should always make that assumption to be conservative. You would not make that assumption if you thought the effect of treatment did not depend on the level of the propensity score. Because the balancing score is a function of the covariates, this corresponds to when you think no covariates change the size fo the treatment effect. That's a really strogn assumption to make, so it always makes sense to allow the $\beta$s to differ. $\endgroup$ – Noah Apr 8 at 1:57
  • $\begingroup$ $\beta$, and kind of this whole question, are pretty divorced from how propensity score analysis is done. Few people use the propensity score in an outcome regression, and even fewer are interested in the conditional ATE at each level of the propensity score. Most people want the ATE, ATT, or conditional ATE at some levels of the covariates. $\beta_1$ ($\beta_0$) is the relationship between the balancing score and the potential outcome under treatment (control). $\endgroup$ – Noah Apr 8 at 2:02
  • $\begingroup$ I guess I'm confused because I initially thought that a "regression adjustment" (something I am trying to learn how to do, and understand the justification for why it works) was simply estimating the propensity score (probability of receiving treatment) and then using that estimated score as a covariate in a regression model, in addition to the treatment indicator with the response of interest as $Y_i$, then $\theta$ would be the average treatment effect. $\endgroup$ – Marcel Apr 8 at 7:02
  • $\begingroup$ It is, but adding an interaction between the treatment and the propensity score just makes your estimate robust to treatment effect modification by the propensity score. I'll give a more detailed description in my answer. $\endgroup$ – Noah Apr 8 at 18:35

Your Answer

