title: Questions about the Backpropogation Algorithm
body: <p>I have a few questions concerning backpropogation. I'm trying to learn the fundamentals behind neural network theory and wanted to start small, building a simple XOR classifier. I've read a lot of articles and skimmed multiple textbooks - but I can't seem to teach this thing the pattern for XOR. </p>
  
  <p>Firstly, I am unclear about the learning model for backpropogation. Here is some pseudo-code to represent how I am trying to train the network. [Lets assume my network is setup properly (ie: multiple inputs connect to a hidden layer connect to an output layer and all wired up properly)].</p>
  
  <pre><code>SET guess = getNetworkOutput() // Note this is using a sigmoid activation function.
  SET error = desiredOutput - guess
  SET delta = learningConstant * error * sigmoidDerivative(guess)
  For Each Node in inputNodes
     For Each Weight in inputNodes[n]
        inputNodes[n].weight[j] += delta;
  
  // At this point, I am assuming the first layer has been trained.
  // Then I recurse a similar function over the hidden layer and output layer.
  // The prime difference being that it further divi's up the adjustment delta.
  </code></pre>
  
  <p>I realize this is probably not enough to go off of, and I will gladly expound on any part of my implementation. Using the above algorithm, my neural network <em>does</em> get trained, kind of. But not properly. The output is always</p>
  
  <pre><code>XOR 1 1 [smallest number]
  XOR 0 0 [largest number]
  XOR 1 0 [medium number]
  XOR 0 1 [medium number]
  </code></pre>
  
  <p>I can never train the [1,1] [0,0] to be the same value.</p>
  
  <p>If you have any suggestions, additional resources, articles, blogs, etc for me to look at I am very interested in learning more about this topic. Thank you for your assistance, I appreciate it greatly!</p>