title: OpenAI GYM's env.step(): what are the values?
body: <p>I am getting to know OpenAI's GYM (0.25.1) using Python3.10 with gym's environment set to <code>'FrozenLake-v1</code> (code below).</p>
  <p>According to the <a href="https://www.gymlibrary.ml/" rel="nofollow noreferrer">documentation</a>, calling <code>env.step()</code> should return a tuple containing 4 values (observation, reward, done, info). However, when running my code accordingly, I get a ValueError:</p>
  <p>Problematic code:</p>
  <pre><code>observation, reward, done, info = env.step(new_action)
  </code></pre>
  <p>Error:</p>
  <pre><code>      3 new_action = env.action_space.sample()
  ----&gt; 5 observation, reward, done, info = env.step(new_action)
        7 # here's a look at what we get back
        8 print(f&quot;observation: {observation}, reward: {reward}, done: {done}, info: {info}&quot;)
  
  ValueError: too many values to unpack (expected 4)
  </code></pre>
  <p>Adding one more variable fixes the error:</p>
  <pre><code>a, b, c, d, e = env.step(new_action)
  print(a, b, c, d, e)
  </code></pre>
  <p>Output:</p>
  <pre><code>5 0 True True {'prob': 1.0}
  </code></pre>
  <p>My interpretation:</p>
  <ul>
  <li><code>5</code> should be observation</li>
  <li><code>0</code> is reward</li>
  <li><code>prob: 1.0</code> is info</li>
  <li>One of the <code>True</code>'s is done</li>
  </ul>
  <p>So what's the leftover boolean standing for?</p>
  <p>Thank you for your help!</p>
  <hr />
  <p>Complete code:</p>
  <pre><code>import gym
  
  env = gym.make('FrozenLake-v1', new_step_api=True, render_mode='ansi') # build environment
  
  current_obs = env.reset() # start new episode
  
  for e in env.render():
      print(e)
      
  new_action = env.action_space.sample() # random action
  
  observation, reward, done, info = env.step(new_action) # perform action, ValueError!
  
  for e in env.render():
      print(e)
  </code></pre>