10331.56
      Road Runner      [**54768.0**]{}        53596.0            49029.74
       Robotank             31.83          [**41.04**]{}           6.68
       Seaquest            20458.6        [**24001.6**]{}        2274.06
    Space Invaders         1890.8         [**2626.55**]{}        1466.01
      Star Gunner          41684.0            47367.0        [**52466.84**]{}
        Tennis             -11.63          [**-7.8**]{}           -20.49
      Time Pilot           6153.0         [**6490.0**]{}         3816.38
       Tutankham       [**208.61**]{}         200.76              132.67
       Up N Down           19528.3        [**19827.3**]{}        8705.64
        Venture         [**78.0**]{}           67.0                0.00
     Video Pinball        343380.29      [**372564.11**]{}       35515.92
     Wizard Of Wor         5451.0         [**5873.0**]{}         3657.65
        Zaxxon             3901.0             3695.0         [**7956.05**]{}
      Times Best             10                 28                  10

  : Comparison of Ensemble Voting, UCB Exploration, both trained with 40 million frames and A3C+ of [@bellemare2016unifying], trained with 200 million frames[]{data-label="tab:compare_with_bellemare"}

      Category      Total   Bootstrapped DQN   Double DQN   Ensemble Voting   UCB-Exploration
  ---------------- ------- ------------------ ------------ ----------------- -----------------
   Human Optimal     23            0               3               5                15
   Score Explicit     8            0               2               1                 5
    Dense Reward      8            0               1               1                 6
   Sparse Reward      5            1               0               2                 2

  : Comparison of each method across different game categories. The Atari games are separated into four categories: human optimal, score explicit, dense reward, and sparse reward. In each row, we present the number of games in this category, the total number of games where each algorithm achieves the optimal performance according to Table \[tab:my\_label\]. The game categories follow the taxonomy in Table 1 of [@ostrovski2017count][]{data-label="tab:compare_each_category"}

InfoGain exploration
====================

