We elaborate on using importance sampling for causal reasoning, in particular for counterfactual inference. We show how this can be implemented natively in probabilistic programming. By considering the structure of the counterfactual query, one can significantly optimise the inference process. We also consider design choices to enable further optimisations. We introduce MultiVerse, a probabilistic programming prototype engine for approximate causal reasoning. We provide experimental results and compare with Pyro, an existing probabilistic programming framework with some of causal reasoning tools. Machine learning has renewed interest in causal tools to aid reasoning (Pearl, 2018) . Counterfactuals are particularly special causal questions as they involve the full suite of causal tools: posterior 1 inference and interventional reasoning (Pearl, 2000) . Counterfactuals are probabilistic in nature and difficult to infer, but are powerful for explanation (Wachter et al., 2017; Sokol and Flach, 2018; Guidotti et al., 2018; Pedreschi et al., 2019) , fairness Kusner et al. (2017) ; Zhang and Bareinboim (2018) ; Russell et al. (2017) , policy search (e.g. Buesing et al. (2019) ) and are also quantities of interest on their own (e.g. Johansson et al. (2016) ). This has seen counterfactuals applied to medicine (Constantinou et al., 2016; Schulam and Saria, 2017; Richens et al., 2019; Oberst and Sontag, 2019) , advertisement and search (Bottou et al., 2013; Swaminathan and Joachims, 2015; Li et al., 2015; Gilotte et al., 2018) , translation (Lawrence et al., 2017) , and reinforcement learning (Foerster et al., 2018; Forney et al., 2017; Buesing et al., 2019) . Consequently, counterfactual inference generally requires enhanced tools and inference procedures to incorporate both observation and inter-vention. Existing frameworks are not fully equipped to handle them naturally, preventing both easy interventional reasoning, as well as optimizations that emerge when considering the full counterfactual query. Probabilistic programming frameworks handle well OBSERVE(variable, value) statements for observational inference: they incorporate the likelihood of the variables given observations. However, for counterfactual inference, it is generally necessary to represent the noise variables as explicit random variables in the trace because the noise variables should be part of the joint posterior that is received after the abduction step. In MultiVerse, we introduce "Observable" Random Procedures that are similar to regular Random Procedures but also (a) have an explicit noise variable that is the part of the program trace, and (b) have an inverse function that proposes that variable to a specific value to "match" the hyperparameters of the random procedure and the observation. For more details, see Section E. In this paper we discuss how to perform counterfactual queries using importance sampling. Further, we introduce MultiVerse, a probabilistic programming system for causal reasoning that optimises approximate counterfactual inference. For future work, we aim towards an approximate causal inference engine for any counterfactual query expressed in a probabilistic program, taking advantage of the structure of counterfactual queries to optimise the process of the inference and to choose one of many approximate inference methods. As causal queries become more used in machine learning, we believe so will flexible and optimised tools that perform these types of inference.
Probabilistic Programming that Natively Supports Causal, Counterfactual Inference