original_docstring: Calculate expected value for a given action
  
      :param state: state which value is updated
      :type state: tuple
      :param action: action taken in this state
      :type action: int
      :param discount: discount
      :type discount: float
      :return: new value
      :rtype: float
      
docstring: Calculate expected value for a given action
docstring_tokens: Calculate expected value for a given action
code: def expected_value(values, state, action, discount=0.9):
      """Calculate expected value for a given action
  
      :param state: state which value is updated
      :type state: tuple
      :param action: action taken in this state
      :type action: int
      :param discount: discount
      :type discount: float
      :return: new value
      :rtype: float
      """
      x, y = state
  
      # move cars overnight, actions are validated during the policy improvement
      x, y = min(x - action, 20), min(y + action, 20)
  
      if action < 0:
          action += 1
      # pay $2 for each moved car
      new_value = -2 * abs(action)
  
      if x > 10:
          new_value -= 4
      if y > 10:
          new_value -= 4
  
      # Bellman equation
      new_value += np.sum(rewards[x, y]) + np.sum(transition_probs[x, y] * values) * discount
  
      return new_value
code_tokens: def expected_value ( values , state , action , discount = 0.9 ) : x , y = state x , y = min ( x - action , 20 ) , min ( y + action , 20 ) if action < 0 : action += 1 new_value = - 2 * abs ( action ) if x > 10 : new_value -= 4 if y > 10 : new_value -= 4 new_value += np . sum ( rewards [ x , y ] ) + np . sum ( transition_probs [ x , y ] * values ) * discount return new_value
short_docstring: Calculate expected value for a given action
short_docstring_tokens: Calculate expected value for a given action
comment: """Calculate expected value for a given action
  
      :param state: state which value is updated
      :type state: tuple
      :param action: action taken in this state
      :type action: int
      :param discount: discount
      :type discount: float
      :return: new value
      :rtype: float
      """$SEP$# move cars overnight, actions are validated during the policy improvement$SEP$# pay $2 for each moved car$SEP$# Bellman equation