<p>Regarding to your first approach,</p>
<p>There are two synthetically prepared datasets available:</p>
<ol>
<li><a href="https://www.robots.ox.ac.uk/%7Evgg/data/text/" rel="nofollow noreferrer">Text Recognition Data</a> consists from 9M images.</li>
<li><a href="https://www.robots.ox.ac.uk/%7Evgg/data/scenetext/" rel="nofollow noreferrer">SynthText in the Wild</a> consists from 8M images.</li>
</ol>
<p>I have used above datasets for text recognition on slab images. Images were quite challenging however now I achieved more than 90% accuracy for that. I have implemented following models to solve this task. These are:</p>
<ol>
<li><a href="https://github.com/clovaai/CRAFT-pytorch" rel="nofollow noreferrer">CRAFT</a> for text localization.</li>
<li><a href="https://github.com/clovaai/deep-text-recognition-benchmark" rel="nofollow noreferrer">Deep Text Recognition</a> for text recognition.</li>
</ol>
<p>If you are working with <a href="https://i.stack.imgur.com/crUUR.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/crUUR.png" alt="enter image description here" /></a> kinds of images only, I highly encourage you to try <strong>Deep Text Recognition</strong>. It is 4 stage framework.
<a href="https://i.stack.imgur.com/PWXo8.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/PWXo8.png" alt="enter image description here" /></a></p>
<ol>
<li><p>For Transformation, you can choose <strong>TPS</strong> or <strong>None</strong>. With <strong>TPS</strong>, it has showed higher performance. They implemented <a href="https://arxiv.org/abs/1506.02025" rel="nofollow noreferrer">Spatial Transformer Networks</a>.</p>
</li>
<li><p>On Feature Extraction stage, you will have options: <strong>ResNet</strong> or <strong>VGG</strong></p>
</li>
<li><p>For Sequential Stage, <strong>BiLSTM</strong></p>
</li>
<li><p><strong>Attn</strong> or <strong>CTC</strong> for prediction stage.</p>
</li>
</ol>
<p>They achieved best accuracy on <strong>TPS-ResNet-BiLSTM-Attn</strong> version. You can easily fine tune this network and I hope it can solve your task. The model trained with above mentioned datasets.</p>
Deep Learning solution for digit recognition on natural scene
<p>I am working on a problem, where I want to automatically read the number on images as follows:</p>
<p><a href="https://i.stack.imgur.com/bOjqi.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/bOjqi.png" alt="enter image description here" /></a></p>
<p><a href="https://i.stack.imgur.com/aeNUi.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/aeNUi.png" alt="enter image description here" /></a></p>
<p>As can be seen, the images are quite challenging! Not only are these not connected lines in all cases, but also the contrast differs a lot. My first attempt was using pytesseract after some preprocessing. I also created a StackOverflow post <a href="https://stackoverflow.com/questions/65666507/local-contrast-enhancement-for-digit-recognition-with-cv2-pytesseract/65675638?noredirect=1#comment116147683_65675638">here</a>.</p>
<p>While this approach works fine on an individual image, it is not universal, as it requires too much manual information for the preprocessing. The best solution I have so far, is to iterate over some hyperparameters such as threshold value, filter size of erosion/dilation, etc. However, this is computationally expensive!</p>
<p>Therefore I came to believe, that the solution I am looking for must be deep-learning based. I have two ideas here:</p>
<ul>
<li>Using a pre-trained network on a similar task</li>
<li>Splitting the input images into separate digits and train / finetune a network myself in an MNIST fashion</li>
</ul>
<p>Regarding the first approach, I have not found something good yet. Does anyone have an idea for that?</p>
<p>Regarding the second approach, I would need a method first to automatically generate images of the separate digits. I guess this should also be deep-learning-based. Afterward, I could maybe achieve some good results with some data augmentation.</p>
<p>Does anyone have ideas? :)</p>