weakness of this paper, as well as the absence of theoretical guarantees about the performance w.r.t. the quality of the approximation of the MDP model. The empirical analysis on Atari games is not satisfying enough, as many results of the baseline are significantly different from the ones reported in literature [1,2]. In general, I am aware Atari experiments are extremely sensitive to the choice of hyperparameters, thus, I’d rather see more reliable benchmarks to complement them. A toy problem to study more rigorously the effect of Doubly Bounded Q-Learning would be desirable. Also, it would be nice to have some evidence of the estimate of the Q-function computed by Doubly Bounded Q-Learning w.r.t. other methods. I encourage authors submitting a revised version addressing these points. Overall, this work is promising, but it needs still much work to be recommendable for publication.
[1] Van Hasselt, Hado, Arthur Guez, and David Silver. "Deep reinforcement learning with double q-learning." Proceedings of the AAAI conference on artificial intelligence. Vol. 30. No. 1. 2016. [2] Wang, Ziyu, et al. "Dueling network architectures for deep reinforcement learning." International conference on machine learning. PMLR, 2016.
30. No.1. 2016. [2] Wang, Ziyu, et al. "Dueling network architectures for deep reinforcement learning." International conference on machine learning. PMLR, 2016.