<p>Make sure your test set is large enough compared to the training set (e.g. 10% of the overall data) and check it regarding diversity. If your test set only covers very specific cases, this could be a reason. Also make sure you always use the same test set. Alternatively you should google the term <em>cross-validation</em>.</p>

<p>Furthermore, observing good training set accuracy while observing bad test set accuracy is a sign for <em>overfitting</em>. Try to apply regularization like a simple L2 weight decay (simply multiply your weight matrices with e.g. 0.999 after each weight update). Depending on your data, <a href="http://arxiv.org/pdf/1207.0580.pdf" rel="nofollow">Dropout</a> or L1 regularization could also help (especially if you have a lot of redundancies in your input data). Also try to choose a smaller network topology (fewer layers and/or fewer neurons per layer).</p>

<p>To speed up training, you could also try alternative learning algorithms like <a href="https://class.coursera.org/neuralnets-2012-001/lecture/preview" rel="nofollow">RPROP+, RPROP- or RMSProp</a> instead of plain backpropagation.</p>
ANN different results for same train-test sets
<p>I'm implementing a neural network for a supervised classification task in MATLAB.</p>

<p>I have a training set and a test set to evaluate the results.
The problem is that every time I train the network for the same training set I get very different results (sometimes I get a 95% classification accuracy and sometimes like 60%) for the same test set. </p>

<p>Now I know this is because I get different initial weights and I know that I can use 'seed' to set the same initial weights but the question is what does this say about my data and what is the right way to look at this? How do I define the accuracy I'm getting using my designed ANN? Is there a protocol for this (like running the ANN 50 times and get an average accuracy or something)?</p>

<p>Thanks</p>