Take the 2-minute tour ×

I am very curious about making a handwriting recognition application in a web browser. Users draw a letter, ajax sends the data to the server, neural network finds the closest match, and returns results. So if you draw an a, the first result should be an a, then o, then e, something like that.

I don't know much about neural networks. What kinda data would I need to pass to the NN. Could it be an array of the x/y coordinates where the user has drawn on a pad. Or what type of data is the neural network expecting or would produce the best results for handwriting?

share|improve this question

3 Answers 3

up vote 1 down vote accepted

Commonly, simple NNs for image/handwriting recognition take a 2-d boolean matrix as input; i.e., a black-and-white bitmap. Make sure you have a training set of these available; or let the user train the algorithm using online backprop learning.

@FrustratedWithFormsDesigner's suggestion of also sending the order could make the NN a lot "smarter", but if you're just learning, try the bitmap version first and see how well it works. Also, play with the bitmap granularity. Maybe try digit recognition first, there are standard datasets for that problem on the web.

share|improve this answer
can you guys recommend some resources that you have used, if you have, for getting started in this type of project :) –  Dark Passenger Jan 6 '11 at 16:47

Not only would you need to send the X/Y coordinates, but also the ORDER they were drawn in. So a path might be better than just a set of points. A neural net should be able to handle it, and there are many ways it could. One way might be to divide up the path into n segments for n neurons and have each neuron recognize a piece of the letter.

share|improve this answer
what do you mean n segments for n neurons –  Dark Passenger Jan 6 '11 at 16:46
I admit I have never actually attempted this with neural networks, but if you give me some time I might be able to dig out some old links from the dusty parts of my bookmarks file. In the meantime, start googling things like "intro to neural networks" or "neural network tutorial" to get started. Do you have an AI book that has any chapters on neural nets? –  FrustratedWithFormsDesigner Jan 6 '11 at 16:47
@Sam: A neural network will have some number of neurons at the input layer, so the easiest thing to do would be to break the path into enough segments so that each neuron will try to recognize a single segment. If the path is very short, it's possible that you may have < n segments. I reiterate, I have never attempted this, I just came up with it now. ;) –  FrustratedWithFormsDesigner Jan 6 '11 at 16:48
The larger the data-set it have to recognize, the bigger the neural network have to be, and the longer it takes to train it. For something as complex as CJK-ideograms, I recommend breaking it up into sub-tasks; Such as recognizing each radical, and then another network/layer to recognize the full characters. –  Markus Jarderot Jan 10 '11 at 19:53

The basic process is to accumulate a number of examples of each letter to be identified, pre-process the raw data, train a collection of candidate models and choose a final model based on test performance on a separate, holdout set of data.

The nature of the pre-processing will depend on the data you collect. If it is "connect the dots" pen movement data, then it may be simplest to divide the image into regions, and summarize by the number of dots per region. If, instead, you are recording a raster image, other pre-processing would be useful, such as simple statistics and vertical and horizontal projection profiles (row and column averages).

"Dr. Dobb's Journal" ran a handprinting recognition contest some years ago (using electronic ink data). You can read about it here:


...and here:


share|improve this answer

Your Answer


