title: When using the frame skipping wrapper for OpenAI Gym, what is the purpose of the np.max line?
body: <p>I'm implementing the following wrapper used commonly in OpenAI's Gym for Frame Skipping. It can be found in dqn/atari_wrappers.py</p>
  <p>I'm very confused about the following line:</p>
  <pre><code>max_frame = np.max(np.stack(self._obs_buffer), axis=0)
  </code></pre>
  <p>I have added comments throughout the code for the parts I understand and to aid anyone who may be able to help.</p>
  <p><code>np.stack(self._obs_buffer)</code> stacks the two states in <code>_obs_buffer</code>.</p>
  <p><code>np.max</code> returns the maximum along axis 0.</p>
  <p>But what I don't understand is why we're doing this or what it's really doing.</p>
  <pre class="lang-py prettyprint-override"><code>class MaxAndSkipEnv(gym.Wrapper):
      &quot;&quot;&quot;Return only every 4th frame&quot;&quot;&quot;
      def __init__(self, env=None, skip=4):
          super(MaxAndSkipEnv, self).__init__(env)
          # Initialise a double ended queue that can store a maximum of two states
          self._obs_buffer = deque(maxlen=2)
          # _skip = 4
          self._skip       = skip
  
      def _step(self, action):
          total_reward = 0.0
          done = None
          for _ in range(self._skip):
              # Take a step 
              obs, reward, done, info = self.env.step(action)
              # Append the new state to the double ended queue buffer 
              self._obs_buffer.append(obs)
              # Update the total reward by summing the (reward obtained from the step taken) + (the current 
              # total reward)
              total_reward += reward
              # If the game ends, break the for loop 
              if done:
                  break
  
          max_frame = np.max(np.stack(self._obs_buffer), axis=0)
  
          return max_frame, total_reward, done, info
  
  </code></pre>