title: Neural network not converging
body: <p>I'm new to Neural Networks, and programming generally. I've written a neural network in java, and i'm looking at football data. I have two inputs:</p>
  
  <p>1) Home team win % over n games
  2) Away team win % over n games</p>
  
  <p>Using 'standard statistical models' one can predict the number of goals that will occur in a match using these two numbers alone, with a reasonable degree of accuracy. However, when i attempt to train my NN to predict the number of goals, it simply doesn't converge :(</p>
  
  <p>I'm using a genetic algorithm to train the network, here is the fittest individual from the first few generations with a population size of 100,000:</p>
  
  <p>1) 0.1407408056662556
  2) 0.13406266176967252
  3) 0.13406267600215235
  4) 0.1338753567259805
  5) 0.13280257001618265
  6) 0.13275165964860766
  7) 0.1319768652096691
  8) 0.13161029326238236</p>
  
  <p>Now i know it looks like it is converging, but it is converging at a painfully slow rate, and i have run this multiple times over many generations and it will not go below 0.13.</p>
  
  <p>I am using a feedforward neural network, with one hidden layer of 10 neurons, and one output neuron. I am using a hyperbolic tangent sigmoid function in the hidden layers, and a sigmoid function for the output layer. I have divided the number of goals by 10, to give an output between 0 and 1.</p>
  
  <p>Before i began running this, i assumed that the NN would outperform simple statistical models, but it doesn't come close. My question is:</p>
  
  <p>From the results you can see, does it look like there is an error in the code somewhere ?
  Do i need to make a change to the architecture of the network ?
  Do i need to change the network inputs/training data in some way ?</p>
  
  <p>I've been trying to identify the issue for a while now, and it's driving me mental. Any adive is greatly appreciated.</p>
  
  <p>Many Thanks.</p>