This is the standard experience function for the Roth-Erev
        algorithm. Here propensities for all actions are updated and similarity
        does not come into play. That is, all action choices are assumed to be
        equally similar. If the actionIndex points to the action the reward is
        associated with (usually the last action taken) then simply adjust the
        weight by the experimentation. Otherwise, adjust the weight by a
        smaller portion of the reward.

        If j is the index of the last action chosen, r_j is the reward received
        for performing j, i is the current action being updated, n is the size
        of the action domain and e is the experimentation parameter, then this
        experience function  can be expressed as::
                         _
                        |  r_j * (1-e)         if i = j
            E(i, r_j) = |
                        |_ r_j * (e /(n-1))    if i != j

def _experience(self, lastState, action, previousAction, reward):
        """ This is the standard experience function for the Roth-Erev
        algorithm. Here propensities for all actions are updated and similarity
        does not come into play. That is, all action choices are assumed to be
        equally similar. If the actionIndex points to the action the reward is
        associated with (usually the last action taken) then simply adjust the
        weight by the experimentation. Otherwise, adjust the weight by a
        smaller portion of the reward.

        If j is the index of the last action chosen, r_j is the reward received
        for performing j, i is the current action being updated, n is the size
        of the action domain and e is the experimentation parameter, then this
        experience function  can be expressed as::
                         _
                        |  r_j * (1-e)         if i = j
            E(i, r_j) = |
                        |_ r_j * (e /(n-1))    if i != j
        """
        e = self.experimentation

        if action == previousAction:
            experience = reward * (1 - e)
        else:
            experience = reward * (e / (self.module.numActions - 1))

        return experience