title: Implementation of RPROP fails when current partial derivative of error gradient equal to 0
body: <p>I am currently implementing the RPROP algorithm and am finding it fails to work when solving the XOR problem. This seems to me to be because when you give the Neural Network the input values, 2 of the input neurons have an output of 0. Therefore when you calculate the partial derivative to the error rate of that neuron, you get a 0 value. This means when the RPROP algorithm runs, it works out the sign of the current partial derivative against the previous partial derivative and this says the sign is 0 correctly. Thus, the delta isn't calculated but the previous one is used with a sign change of the current partial derivative. This causes the update value to be 0 and therefore the algorithm fails. </p>
  
  <p>I wondered if anyone could tell me what exactly I am doing wrong, as i have read over the algorithm and seen the encog implementation and am still struggling to solve this problem. </p>
  
  <p>The error gradient is calculated for hidden layers as: </p>
  
  <pre><code>public void SetNeuronErrorGradient(Neuron neuron)
      {
          neuron.ErrorGradient = neuron.OutputSynapses.Sum(a =&gt; a.OutputNeuron.ErrorGradient * a.Weight) *
                             neuron.ActivationFunction.Derivative(neuron.LatestFedValueFromInputSynapses);
      }
  </code></pre>
  
  <p>The error gradient of the output layer is calculated as:</p>
  
  <pre><code>public void SetNeuronErrorGradient(Neuron neuron, double target)
  {
      neuron.ErrorGradient = CalculateErrorForOutputAgainstTarget(neuron, target) *
                         neuron.ActivationFunction.Derivative(neuron.LatestFedValueFromInputSynapses);
  }
  </code></pre>
  
  <p>The algorithm for RPROP is then implemented as follows, where a Synapse consists on an input neuron, output neuron, and weight and weight delta.: </p>
  
  <pre><code>public sealed partial class ResilientBackPropagationSynapseWeightCalculator : IUpdateSynapseWeights
  {
      private static ILogger Log =&gt; LoggerProvider.For&lt;ResilientBackPropagationSynapseWeightCalculator&gt;();
  
      private readonly Dictionary&lt;Synapse, double&gt; synapseToPreviousPartialDerivative
          = new Dictionary&lt;Synapse, double&gt;(SynapseEqualityComparer.Instance());
  
      private readonly Dictionary&lt;Synapse, double&gt; synapseToPreviousDeltaWithoutDirection
          = new Dictionary&lt;Synapse, double&gt;(SynapseEqualityComparer.Instance());
  
      private ResilientBackPropagationSynapseWeightCalculator()
      {}
  
      public void CalculateAndUpdateInputSynapseWeights(Neuron neuron, ParallelOptions parallelOptions)
      {
          neuron.InputSynapses.ForEach(
              synapse =&gt;
              {
                  var previousPartialDerivative = GetPreviousPartialDerivativeOfSynapse(synapse);
                  var currentPartialDerivative = synapse.OutputNeuron.ErrorGradient * synapse.InputNeuron.Output;
  
                  var weightDelta = CalculateSynapseWeightDelta(
                      synapse,
                      currentPartialDerivative,
                      previousPartialDerivative);
  
                  synapse.WeightDelta = weightDelta;
                  synapse.Weight = synapse.Weight + weightDelta;
              });
      }
  
      private double GetPreviousPartialDerivativeOfSynapse(Synapse synapse)
      {
          if (synapseToPreviousPartialDerivative.TryGetValue(synapse, out var previousPartialDerivative))
          {
              return previousPartialDerivative;
          }
  
          synapseToPreviousPartialDerivative[synapse] = 0;
          return 0;
      }
  
      private double CalculateSynapseWeightDelta(
          Synapse synapse,
          double currentPartialDerivative,
          double previousPartialDerivative)
      {
          var errorGradientSign =
              ResilientBackPropagationHelper.Sign(currentPartialDerivative * previousPartialDerivative);
  
          double weightDelta;
  
          if (errorGradientSign &gt; 0)
          {
              weightDelta = ResilientBackPropagationHelper
                  .CalculateDeltaToContinueTowardsErrorGraidentMinimum(
                      synapse,
                      currentPartialDerivative,
                      synapseToPreviousDeltaWithoutDirection);
  
              synapseToPreviousPartialDerivative[synapse] = currentPartialDerivative;
          }
          else if (errorGradientSign &lt; 0)
          {
              weightDelta = ResilientBackPropagationHelper
                  .CalculateDeltaToRevertPreviousWeightAdjustment(
                      synapse,
                      currentPartialDerivative,
                      synapseToPreviousDeltaWithoutDirection);
  
              synapseToPreviousPartialDerivative[synapse] = 0; //0 so no adjustment next iteration.
          }
          else
          {
              weightDelta = ResilientBackPropagationHelper
                  .CalculateDeltaDirection(
                      synapse,
                      currentPartialDerivative,
                      synapseToPreviousDeltaWithoutDirection);
  
              synapseToPreviousPartialDerivative[synapse] = currentPartialDerivative; 
          }
  
          return weightDelta;
      }
  
      public static ResilientBackPropagationSynapseWeightCalculator Create()
          =&gt; new ResilientBackPropagationSynapseWeightCalculator();
  }
  </code></pre>
  
  <p>With the helper function class implemented as: </p>
  
  <pre><code>internal static class ResilientBackPropagationHelper
  {
      private const double NegativeWeightUpdateAmount = 0.5;
      private const double PositiveWeightUpdateAmount = 1.2;
      private const double MaximumWeightUpdate = 50.0;
      private const double MinimumWeightUpdate = 1.0E-6;
      private const double InitialUpdateValue = 0.1;
  
      private const double ZeroTolerance = 0.00000000000000001d;
  
      public static int Sign(double value)
      {
          if (Math.Abs(value) &lt; ZeroTolerance)
          {
              return 0;
          }
          if (value &gt; 0)
          {
              return 1;
          }
          return -1;
      }
  
      public static double CalculateDeltaToContinueTowardsErrorGraidentMinimum(
          Synapse synapse,
          double currentPartialDerivative,
          Dictionary&lt;Synapse, double&gt; synapseToPreviousDeltaWithoutDirection)
      {
          if (synapseToPreviousDeltaWithoutDirection.TryGetValue(synapse, out var previousUpdateValue))
          {
              var delta = Math.Min(previousUpdateValue * PositiveWeightUpdateAmount, MaximumWeightUpdate);
              synapseToPreviousDeltaWithoutDirection[synapse] = delta;
              return Sign(currentPartialDerivative) * delta;
          }
  
          throw new InvalidOperationException($"You cannot increase a prevous delta is none is present.");
      }
  
      public static double CalculateDeltaToRevertPreviousWeightAdjustment(
          Synapse synapse,
          double currentPartialDerivative,
          Dictionary&lt;Synapse, double&gt; synapseToPreviousDeltaWithoutDirection)
      {
          if (synapseToPreviousDeltaWithoutDirection.TryGetValue(synapse, out var previousUpdateValue))
          {
              var delta = Math.Max(previousUpdateValue * NegativeWeightUpdateAmount, MinimumWeightUpdate);
              synapseToPreviousDeltaWithoutDirection[synapse] = delta;
              return -synapse.WeightDelta;
          }
  
          throw new InvalidOperationException($"You cannot revert a previous change if none is known.");
      }
  
      public static double CalculateDeltaDirection(
          Synapse synapse,
          double currentPartialDerivative,
          Dictionary&lt;Synapse, double&gt; synapseToPreviousDeltaWithoutDirection)
      {
          if (synapseToPreviousDeltaWithoutDirection.TryGetValue(synapse, out var previousUpdateValue))
          {
              return Sign(currentPartialDerivative) * previousUpdateValue;
          }
  
          synapseToPreviousDeltaWithoutDirection.Add(synapse, InitialUpdateValue);
  
          return CalculateDeltaDirection(
              synapse,
              currentPartialDerivative,
              synapseToPreviousDeltaWithoutDirection);
      }
  }
  </code></pre>
  
  <p>Any help as to where my logic is failing would be really appreciated as I have tried numerous ways of getting this to work now against the algorithm. </p>
  
  <p>Many Thanks
  Joe</p>