title: Record OpenAI gym Video with Monitor
body: <p>I want to record a video of my rollouts of OpenAIs gym. I use the Monitor class, but other solutions are also appreciated. This is a minimal example I created, that runs without exceptions or warnings:</p>
  <pre><code>import gym
  from gym.wrappers import Monitor
  env = Monitor(gym.make('CartPole-v0'), './video', force=True)
  state = env.reset()
  done = False
  while not done:
      action = env.action_space.sample()
      state_next, reward, done, info = env.step(action)
  env.close()
  </code></pre>
  <p>This saves a video and some metadata to the <code>'./video'</code> folder. However, the mp4-file that is saved there has always just a size of 262 byte and the &quot;file contains no playable steams&quot;. So it seems it is a 'video' but made up of 0 frames? I don't understand what the problem is and how to debug from here.</p>