This is an altered version of the experience function for used in
        the standard Roth-Erev algorithm.  Like in RELearner, propensities for
        all actions are updated and similarity does not come into play. If the
        actionIndex points to the action the reward is associated with (usually
        the last action taken) then simply adjust the weight by the
        experimentation. Otherwise increase the weight of the action by a small
        portion of its current propensity.

        If j is the index of the last action chosen, r_j is the reward received
        for performing j, i is the current action being updated, q_i is the
        propensity for i, n is the size of the action domain and e is the
        experimentation parameter, then this experience function can be
        expressed as::

                        |  r_j * (1-e)         if i = j
            E(i, r_j) = |
                        |_ q_i * (e /(n-1))    if i != j

def _experience(self, previousState, action, previousAction, reward):
        """ This is an altered version of the experience function for used in
        the standard Roth-Erev algorithm.  Like in RELearner, propensities for
        all actions are updated and similarity does not come into play. If the
        actionIndex points to the action the reward is associated with (usually
        the last action taken) then simply adjust the weight by the
        experimentation. Otherwise increase the weight of the action by a small
        portion of its current propensity.

        If j is the index of the last action chosen, r_j is the reward received
        for performing j, i is the current action being updated, q_i is the
        propensity for i, n is the size of the action domain and e is the
        experimentation parameter, then this experience function can be
        expressed as::

                        |  r_j * (1-e)         if i = j
            E(i, r_j) = |
                        |_ q_i * (e /(n-1))    if i != j
        """
        e = self.experimentation

        if action == previousAction:
            experience = reward * (1 - e)
        else:
            propensity = self.module.getValue(previousState, action)
            experience = propensity * (e / (self.module.numActions - 1))

        return experience