<p>For handwritten character recognition you need</p>

<ol>
<li>many training examples (maybe you should create distortions of your training set)</li>
<li>softmax activation function in the output layer</li>
<li>cross entropy error function</li>
<li>training with <strong>stochastic</strong> gradient descent</li>
<li>a bias in each layer</li>
</ol>

<p>A good test problem is the handwritten digit data set <a href="http://yann.lecun.com/exdb/mnist/" rel="noreferrer">MNIST</a>. Here are papers that successfully applied neural networks on this data set:</p>

<p>Y. LeCun, L. Bottou, Y. Bengio and P. Haffner: Gradient-Based Learning Applied to Document Recognition, <a href="http://yann.lecun.com/exdb/publis/pdf/lecun-98.pdf" rel="noreferrer">http://yann.lecun.com/exdb/publis/pdf/lecun-98.pdf</a></p>

<p>Dan Claudiu Ciresan, Ueli Meier, Luca Maria Gambardella, Juergen Schmidhuber: Deep Big Simple Neural Nets Excel on Handwritten Digit Recognition, <a href="http://arxiv.org/abs/1003.0358" rel="noreferrer">http://arxiv.org/abs/1003.0358</a></p>

<p>I trained an MLP with 784-200-50-10 architecture and got >96% accuracy on the test set.</p>
Training feedforward neural network for OCR
<p>Currently I'm learning about neural networks and I'm trying to create an application that can be trained to recognize handwritten characters. 
For this problem I use a feed-forward neural network and it seems to work when I train it to recognize 1, 2 or 3 different characters. But when I try to make the network learn more than 3 characters it will stagnate at a error percentage around the 40 - 60%. </p>

<p>I tried with multiple layers and less/more neurons but I can't seem to get it right, now I'm wondering if a feedforward neural network is capable of recognizing that much information. </p>

<p>Some statistics:</p>

<p><strong>Network type:</strong> Feed-forward neural network</p>

<p><strong>Input neurons:</strong> 100 (a 10 * 10) grid is used to draw the characters</p>

<p><strong>Output neurons:</strong> The amount of characters to regocnize</p>

<p><em>Does anyone know what's the possible flaw in my architecture is? Are there too much input neurons? Is the feedforward neural network not capable of character regocnition?</em></p>