title: R package DARCH deep belief neural network cannot learn 'exclusive or' it seems
body: <p>Thank you in advance for any help. I am trying to implement a deep learning neural network to predict a number of variables (a sort of multivariate non-linear regression). As a first step I am looking at the Darch package in R and working through the code snippets in </p>
  
  <p><a href="http://cran.r-project.org/web/packages/darch/darch.pdf" rel="nofollow">http://cran.r-project.org/web/packages/darch/darch.pdf</a></p>
  
  <p>When I run the following code from p 10, which appears to be training on 'exclusive or', then the resultant neural network appears to be unable to learn the function. It either learns the (1,0) pattern or the (0,1) pattern as true, but not both, and sometimes additionally the (1,1) pattern, which should be false. My understanding was that these kind of networks should be able to learning almost any function, including for starters 'exclusive or': was this not resolved by the original backpropagation work, which this network utilizes in the fine tuning. I think I could be missing something, so any advice or help is very much appreciated? (I have even increased the epochs upto 10,000, but to no avail.)</p>
  
  <pre><code># Generating the datasets
  inputs &lt;- matrix(c(0,0,0,1,1,0,1,1),ncol=2,byrow=TRUE)
  outputs &lt;- matrix(c(0,1,1,0),nrow=4)
  # Generating the darch
  darch &lt;- newDArch(c(2,4,1),batchSize=2)
  # Pre-Train the darch
  darch &lt;- preTrainDArch(darch,inputs,maxEpoch=100)
  # Prepare the layers for backpropagation training for
  # backpropagation training the layer functions must be
  # set to the unit functions which calculates the also
  # derivatives of the function result.
  layers &lt;- getLayers(darch)
  for(i in length(layers):1){
      layers[[i]][[2]] &lt;- sigmoidUnitDerivative
  
  }
  setLayers(darch) &lt;- layers
  rm(layers)
  # Setting and running the Fine-Tune function
  setFineTuneFunction(darch) &lt;- backpropagation
  darch &lt;- fineTuneDArch(darch,inputs,outputs,maxEpoch=100)
  # Running the darch
  darch &lt;- darch &lt;- getExecuteFunction(darch)(darch,inputs)
  outputs &lt;- getExecOutputs(darch)
  cat(outputs[[length(outputs)]])
  ## End(Not run)
  
  
  #### Example results
  
  
  &gt; cat(outputs[[length(outputs)]])
  0.02520016 0.8923063 0.1264799 0.9803244
  
  ## Different run
  
  &gt; cat(outputs[[length(outputs)]])
  0.02702418 0.1061477 0.9833059 0.9813462
  </code></pre>