Title: Uncertainty-aware off policy learning. Abstract: Off-policy learning, referring to the procedure of policy optimization with access only to logged feedback data, has shown importance in various real-world applications, such as search engines, recommender systems, etc.  While the ground-truth logging policy, which generates the logged data, is usually unknown, previous work directly takes its estimated value in off-policy learning, resulting in a biased estimator. This estimator has both high bias and variance on samples with small and inaccurate estimated logging probabilities. 
In this work, we explicitly model the uncertainty in the estimated logging policy and propose a novel  \underline{U}ncertainty-aware \underline{I}nverse  \underline{P}ropensity \underline{S}core estimator (UIPS) for improved off-policy learning. Experiment results on synthetic and three real-world  recommendation datasets demonstrate the advantageous sample efficiency of the proposed UIPS estimator.