Weakness: 1.Some experiment is hard to understand. Table1 shows the TD-error and the absolute state-action value which didn’t demonstrate the small approximation error would cause significant estimation error which would cause the sub-optimal fixed points.
2.The effectiveness of lower bound double q-learning is doubtful. In MsPacman of Figure2, the algorithm shows slight performance decrease of Clipped DDQN, in some environment such as WizardOfWor, Zaxxon RoadRunner and BattleZone, these algorithms seems converge into same solutions. Besides, the algorithm would cause the overestimate the true maximum value.
2.The effectiveness of lower bound double q-learning is doubtful. In MsPacman of Figure2, the algorithm shows slight performance decrease of Clipped DDQN, in some environment such as WizardOfWor, Zaxxon RoadRunner and BattleZone, these algorithms seems converge into same solutions. Besides, the algorithm would cause the overestimate the true maximum value.