in policy gradient methods. In policy optimization based methods, the state coverage, or the various times different states are visited can be estimated from the state distribution induced by the policy. This is often called the discounted (future) state distribution, or the normalized discounted weighting of states. In this work, our objective is to promote exploration in policy gradient methods by using the entropy of the discounted future state distribution $d_{\alpha, \gamma, \pi}$ (which we will denote as $d_{\pi_{\theta}}$) where $\mathbf \alpha$ is the distribution over the initial states and to explicitly highlight that this distribution is dependent on the changes in the policy parameters $\theta$, and we propose a practically feasible algorithm for estimating and regularizing policy gradient methods with the discounted state distribution for exploration and reducing sample complexity. We propose the following state distribution entropy regularized policy gradient objective: ${\tilde{J}(\theta) = \expectation_{\pi_{\theta}} \Bigl[\sum_{t=0}^{\infty} \gamma^{t} r(S_t, A_t) \Bigm | S_0 \Bigr] + \lambda {\mathbb{H}}(d_{\pi_{\theta}})}$, where $d_{\pi_{\theta}}$ is the discounted state distribution induced by the policy $\pi$. We can estimate $\nabla_{\theta} {J}(\theta)$ while using stochastic policies from [@sutton] or deterministic policies from [@dpg]. The regularized policy gradient objective is: $    \nabla_{\theta} \tilde{J}(\theta) = \nabla_{\theta}{J}(\theta) +\lambda \nabla_{\theta} {\mathbb{H}}(d_{\pi_{\theta}})$.

#### Entropy 