1. This approach requires enumerating all off-target concepts in order to construct the oracle probe for Z_e. However, this is a common assumption in the causal-inference-in-text literature, so not surprising.
2. A crux of this paper is the development of *good* oracle probes. However, there is very little analysis of how to evaluate the oracle probes (which are then used to evaluate causal probing methods).
3. Currently, this paper only considers binary concepts. This is a common assumption, so not very limiting.
3. Currently, this paper only considers binary concepts. This is a common assumption, so not very limiting.