0.9 & Adam optimizer parameter.\
\
$\beta_2$ & 0.99 & Adam optimizer parameter.\
\
$\epsilon$ & $10^{-4}$ & Adam optimizer parameter.\
\
learning rate schedule & $\left\{\begin{matrix} 
10^{-4} & t \leq 10^6\\
Interp(10^{-4}, 5*10^{-5}) & \text{otherwise}\\
5 * 10^{-5} &  t > 5*10^6
\end{matrix}\right.$ & Learning rate for Adam optimizer, as a function of iteration $t$.\
\
exploration schedule & $\left\{\begin{matrix}
Interp(1, 0.1) & t <10^6\\
Interp(0.1, 0.01) & \text{otherwise} \\
0.01 & t > 5*10^6
\end{matrix}\right.$& Probability of random action in $\epsilon$-greedy exploration, as a function of the iteration $t$ .\
\
replay start size & 50000 & Number of uniform random actions taken before learning starts.\

Results tables {#app:table}
==============

                       Bootstrapped DQN      Double DQN       Ensemble Voting    UCB-Exploration
  ------------------- ------------------ ------------------- ----------------- -------------------
         Alien              1445.1             2059.7             2282.8         [**2817.6**]{}
        Amidar              430.58              667.5         [**683.72**]{}          663.8
        Assault            2519.06             2820.61            3213.58        [**3702.76**]{}
        Asterix             3829.0             7639.5         [**8740.0** ]{}        8732.0
       Asteroids            1009.5             1002.3         [**1149.3** ]{}        1007.8
       Atlantis           1314058.0           1982677.0          1786305.0      [**2016145.0**]{}
      Bank Heist            795.1               789.9              869.4          [**906.9**]{}
      Battle Zone          26230.0             24880.0        [**27430.0**]{}        26770.0
      Beam Rider           8006.58             7743.74            7991.9         [**9188.26**]{}
        Bowling             28.62               30.92              32.92          [**38.06**]{}
        Boxing              85.91               94.07              94.47          [**98.08**]{}
       Breakout             400.22         [**467.45**]{}         426.78             411.31
       Centipede           5328.77             5177.51            6153.28        [**6237.18**]{}
    Chopper Command         2153.0             3260.0             3544.0         [**3677.0**]{}
     