<p>Your task is really challenging. I have several ideas, may be it will help you on the way. First, if you get the images right, you can use <a href="https://www.jaided.ai/easyocr/" rel="nofollow noreferrer">EasyOCR</a>. It uses a sophisticated algorithm for detecting letters in the image called <a href="https://arxiv.org/abs/1904.01941" rel="nofollow noreferrer">CRAFT</a> and then recognizes them using CRNN. It provides very fine grained control over symbol detection and recognition parts. For example, after some manual manipulations on the images (greyscaling, contrast enhancing and sharpening) I got</p>
<p><a href="https://i.stack.imgur.com/6rXHr.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/6rXHr.png" alt="enter image description here" /></a> <a href="https://i.stack.imgur.com/Xysuj.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/Xysuj.png" alt="enter image description here" /></a>
and using the following code</p>
<pre><code>import easyocr
reader = easyocr.Reader(['en']) # need to run only once to load model into memory
reader.readtext(path_to_file, allowlist='0123456789')
</code></pre>
<p>the results are <code>31197432</code> and <code>31197396</code>.</p>
<p>Now, for the contrast restoration part, <code>opencv</code> has a tool called <a href="https://en.wikipedia.org/wiki/Adaptive_histogram_equalization#Contrast_Limited_AHE" rel="nofollow noreferrer">CLAHE</a>. If you run following code</p>
<pre><code>img = cv2.imread(fileName)
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
blurred = cv2.GaussianBlur(gray, (25, 25), 0)
grayscaleImage = gray * ((gray / blurred) &gt; 0.01)  
clahe = cv2.createCLAHE(clipLimit=6.0, tileGridSize=(16,6))
contrasted = clahe.apply(grayscaleImage)
</code></pre>
<p>on the original images, you will get
<a href="https://i.stack.imgur.com/4Z5bv.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/4Z5bv.png" alt="enter image description here" /></a>
<a href="https://i.stack.imgur.com/L21Fb.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/L21Fb.png" alt="enter image description here" /></a>
which are visually very similarly to those above. I believe that after some cleaning you can get it to be recognizable without too much fiddling with hyperparameters.</p>
<p>And finally, if you want to train your own deep learning OCR, I suggest you use <a href="https://keras-ocr.readthedocs.io/en/latest/index.html#" rel="nofollow noreferrer">keras-ocr</a> . It uses the same algorithms as EasyOCR, but provides an end-to-end training pipeline to build new OCR model. It has all the necessary steps covered: data sets downloading, data generation, augmentation, training and inferencing.</p>
<p>Take into account that deep learning solutions are very computationally heavy. Good luck!</p>
Deep Learning solution for digit recognition on natural scene
<p>I am working on a problem, where I want to automatically read the number on images as follows:</p>
<p><a href="https://i.stack.imgur.com/bOjqi.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/bOjqi.png" alt="enter image description here" /></a></p>
<p><a href="https://i.stack.imgur.com/aeNUi.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/aeNUi.png" alt="enter image description here" /></a></p>
<p>As can be seen, the images are quite challenging! Not only are these not connected lines in all cases, but also the contrast differs a lot. My first attempt was using pytesseract after some preprocessing. I also created a StackOverflow post <a href="https://stackoverflow.com/questions/65666507/local-contrast-enhancement-for-digit-recognition-with-cv2-pytesseract/65675638?noredirect=1#comment116147683_65675638">here</a>.</p>
<p>While this approach works fine on an individual image, it is not universal, as it requires too much manual information for the preprocessing. The best solution I have so far, is to iterate over some hyperparameters such as threshold value, filter size of erosion/dilation, etc. However, this is computationally expensive!</p>
<p>Therefore I came to believe, that the solution I am looking for must be deep-learning based. I have two ideas here:</p>
<ul>
<li>Using a pre-trained network on a similar task</li>
<li>Splitting the input images into separate digits and train / finetune a network myself in an MNIST fashion</li>
</ul>
<p>Regarding the first approach, I have not found something good yet. Does anyone have an idea for that?</p>
<p>Regarding the second approach, I would need a method first to automatically generate images of the separate digits. I guess this should also be deep-learning-based. Afterward, I could maybe achieve some good results with some data augmentation.</p>
<p>Does anyone have ideas? :)</p>