<p><strong>Classifying Digits</strong></p>

<p>You clarified in comments that you've already isolated the number part of the image pre-detection, so I'll start under that assumption.</p>

<p>Perhaps you can approximate the perspective effects and "blurriness" of the number by treating it as a hand-written number. In this case, there is a famous data-set of handwritten numerals for classification training called mnist. </p>

<p>Yann LeCun has enumerated the state of the art on this dataset here <a href="http://yann.lecun.com/exdb/mnist/" rel="nofollow noreferrer">mnist hand-written dataset</a>. </p>

<p>At the far end of the spectrum, convolutional neural networks yield <a href="http://arxiv.org/abs/1202.2745" rel="nofollow noreferrer">outrageously low error rates</a> (fractions of 1% error). For a simpler solution, k-nearest neighbours using deskewing, noise removal, blurring, and 2 pixel shift, yielded about 1% error, and is significantly faster to implement. <a href="http://docs.opencv.org/2.4/modules/ml/doc/k_nearest_neighbors.html" rel="nofollow noreferrer">Python opencv has an implementation</a>. Neural networks and support vector machines with deskewing also have some pretty impressive performance rates.</p>

<p>Note that convolutional networks don't have you pick your own features, so the important color-differential information here might just be used for narrowing the region-of-interest. Other approaches, where you define your feature space, might incorporate the known color difference more precisely.</p>

<p>Python supports a lot of machine learning techniques in the terrific package sklearn - <a href="http://scikit-learn.org/stable/auto_examples/classification/plot_digits_classification.html" rel="nofollow noreferrer">here are examples of sklearn applied to mnist</a>. <em>If you're looking for an tutorialized explanation of machine learning in python, <a href="http://scikit-learn.org/stable/supervised_learning.html" rel="nofollow noreferrer">sklearn's own tutorial is very verbose</a></em></p>

<p>From the sklearn link:
<a href="https://i.stack.imgur.com/PHDCw.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/PHDCw.png" alt="Classifying mnist"></a></p>

<p>Those are the kinds of items you're trying to classify if you learn using this approach. To emphasize how easy it is to start training some of these machine learning-based classifiers, here is an abridged section from the example code in the linked sklearn package:</p>

<pre><code>digits = datasets.load_digits() # built-in to sklearn!
data = digits.images.reshape((len(digits.images), -1))

# Create a classifier: a support vector classifier
classifier = svm.SVC(gamma=0.001)

# We learn the digits on the first half of the digits
classifier.fit(data[:n_samples / 2], digits.target[:n_samples / 2])
</code></pre>

<p>If you're wedded to openCv (possibly because you want to port to a real-time system in the future), opencv3/python <a href="http://docs.opencv.org/3.0-beta/doc/py_tutorials/py_ml/py_knn/py_knn_opencv/py_knn_opencv.html#knn-opencv" rel="nofollow noreferrer">has a tutorial on this exact topic too</a>! Their demo uses k-nearest-neighbor (listed in the LeCun page), but they also <a href="http://docs.opencv.org/3.0-beta/doc/py_tutorials/py_ml/py_svm/py_svm_opencv/py_svm_opencv.html#svm-opencv" rel="nofollow noreferrer">have svms</a> and the many of the other tools in sklearn. Their ocr page using SVMs uses deskewing, which might be useful with the perspective effect in your problem:</p>

<p><a href="https://i.stack.imgur.com/NT190.jpg" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/NT190.jpg" alt="Deskewed digit"></a></p>

<hr>

<p><strong>UPDATE:</strong> I used the out-of-the box skimage approach outlined above on your image, heavily cropped, and it <strong>correctly classified it</strong>. A <strong>lot</strong> more testing would be required to see if this is rhobust in practice</p>

<p><a href="https://i.stack.imgur.com/Gpgsk.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/Gpgsk.png" alt="enter image description here"></a></p>

<p>^^ That tiny image is the 8x8 crop of the image you embedded in your question. mnist is 8x8 images. That's why it trains in less than a second with default arguments in skimage.</p>

<p>I converted it the correct format by scaling it up to the mnist range using</p>

<pre><code>number = scipy.misc.imread("cropped_image.png")
datum  =  (number[:,:,0]*15).astype(int).reshape((64,))
classifier.predict(datum) # returns 8
</code></pre>

<p>I didn't change anything else from the example; here, I'm only using the first channel for classification, and no smart feature computation. 15 looked about right to me; you'll need to tune it to get within the target range or (ideally) provide your own training and testing set</p>

<hr>

<p><strong>Object Detection</strong></p>

<p>If you haven't isolated the number in the image you'll need an object detector. The literature space on this problem is gigantic and I won't start down that rabbit hole (google Viola and Jones, maybe?) <a href="http://www.pyimagesearch.com/2015/03/23/sliding-windows-for-object-detection-with-python-and-opencv/" rel="nofollow noreferrer">This blog</a> covers the fundamentals of a "sliding window" detector in python. Adrian Rosebrock looks like he's even a contributor on SO, and that page has some good examples of opencv and python-based object detectors fairly tutorialized (you actually linked to that blog in your question, I didn't realize). </p>

<p>In short, classify windows across the image and pick the window of highest confidence. Narrowing down the search space with a region of interest will of course yield huge improvements in all areas of performance</p>
How to classify blurry numbers with openCV
<p>I would like to capture the number from this kind of picture. </p>

<p><a href="https://i.stack.imgur.com/S8esF.png" rel="noreferrer"><img src="https://i.stack.imgur.com/S8esF.png" alt="enter image description here"></a></p>

<p>I tried multi-scale matching from the following link. </p>

<p><a href="http://www.pyimagesearch.com/2015/01/26/multi-scale-template-matching-using-python-opencv/" rel="noreferrer">http://www.pyimagesearch.com/2015/01/26/multi-scale-template-matching-using-python-opencv/</a></p>

<p>All I want to know is the red number. But the problem is, the red number is blurry for openCV recognize/match template. Would there be other possible way to detect this red number on the black background?</p>