First, we examine the significance of the state distribution entropy regularizer in DDPG algorithm [@DDPG]. In DDPG, policy entropy regularization cannot be used due to existence of deterministic policies [@dpg]. In Figure \[fig:ddpg\_control\], we show that by inducing policies to maximize state space coverage, we can enhance exploration that leads to significant improvements on standard benchmark tasks, especially in environments where exploration in the state space plays a key role (e.g HalfCheetah environment)

[0.27]{} ![Significant performance improvements with state entropy regularization with DDPG, especially in tasks like HalfCheetah-v1 and Hopper-v1 where exploration plays a key role. In all the experiments, we use a state entropy regularization coefficient of $\lambda=0.1$ for our approach, and $\lambda=0.0$ for the baseline DDPG. Experiment results are averaged over $10$ random seeds [@DBLP:conf/aaai/0002IBPPM18][]{data-label="fig:ddpg_control"}](DDPG_Final_HalfCheetah-v1.pdf "fig:"){width="\linewidth"}

[0.27]{} ![Significant performance improvements with state entropy regularization with DDPG, especially in tasks like HalfCheetah-v1 and Hopper-v1 where exploration plays a key role. In all the experiments, we use a state entropy regularization coefficient of $\lambda=0.1$ for our approach, and $\lambda=0.0$ for the baseline DDPG. Experiment results are averaged over $10$ random seeds [@DBLP:conf/aaai/0002IBPPM18][]{data-label="fig:ddpg_control"}](DDPG_Final_Hopper-v1.pdf "fig:"){width="\linewidth"}

[0.27]{} ![Significant performance improvements with state entropy regularization with DDPG, especially in tasks like HalfCheetah-v1 and Hopper-v1 