<p>I've also encountered this issue recently and raise a similar <a href="https://stackoverflow.com/questions/64808986/scene-text-image-super-resolution-for-ocr">question</a> with more details and with a recent approach. It seems to be an unsolved problem until now. There are some recent research works that try to address such problems with deep learning. Unfortunately, none of the works reach our expectations. However, I'm sharing the info in case it may come helpful to anyone.</p>
<h2>1. Scene Text Image Super-Resolution in the Wild</h2>
<p>In our case, it may be our last choice; comparatively, perform well enough. It's a recent research work (<a href="https://arxiv.org/abs/2005.03341" rel="noreferrer">TSRN</a>) mainly focuses on such cases. The main intuitive of it is to introduce super-resolution (SR) techniques as pre-processing. This <a href="https://github.com/JasonBoy1/TextZoom" rel="noreferrer">implementation</a> looks by far the most promising. Here is the illustration of their achievement, improve blur to clean image.</p>
<p><img src="https://i.stack.imgur.com/N0vc5.jpg" alt="" /></p>
<h2>2. Neural Enhance</h2>
<p>From their <a href="https://github.com/alexjc/neural-enhance" rel="noreferrer">repo</a> demonstration, It's appearing that It may have some potential to improve blur text either. However, the author probably doesn't maintain the repo for about 4 years.</p>
<p><img src="https://i.stack.imgur.com/PDPIL.jpg" alt="" /></p>
<h2>3. Blind Motion Deblurring with GAN</h2>
<p>The attractive part is the <strong>Blind Motion Deblurring</strong> mechanism in it, named <a href="https://github.com/KupynOrest/DeblurGAN" rel="noreferrer">DeblurGAN</a>. It looks very promising.</p>
<p><a href="https://i.stack.imgur.com/SBsow.gif" rel="noreferrer"><img src="https://i.stack.imgur.com/SBsow.gif" alt="enter image description here" /></a></p>
<h2>4. Real-World Super-Resolution via Kernel Estimation and Noise Injection</h2>
<p>An interesting fact about <a href="https://openaccess.thecvf.com/content_CVPRW_2020/papers/w31/Ji_Real-World_Super-Resolution_via_Kernel_Estimation_and_Noise_Injection_CVPRW_2020_paper.pdf" rel="noreferrer">their work</a> is that unlike other literary works they first design a novel <strong>degradation framework</strong> for realworld images by <strong>estimating various blur kernels</strong> as well as real <strong>noise distributions</strong>. Based on that they acquire <strong>LR</strong> images sharing a common domain with real-world images. Then, they propose a realworld super-resolution model aiming at better <strong>perception</strong>. From their article:</p>
<p><a href="https://i.stack.imgur.com/nQjLJ.png" rel="noreferrer"><img src="https://i.stack.imgur.com/nQjLJ.png" alt="enter image description here" /></a> <a href="https://i.stack.imgur.com/hWEqX.png" rel="noreferrer"><img src="https://i.stack.imgur.com/hWEqX.png" alt="enter image description here" /></a></p>
<p>However, in my observation, I couldn't get the expected results. I've raised an <a href="https://github.com/jixiaozhong/RealSR/issues/29" rel="noreferrer">issue on github</a> and until now didn't get any response.</p>
<hr />
<h2>Convolutional Neural Networks for Direct Text Deblurring</h2>
<p>The <a href="http://www.fit.vutbr.cz/%7Eihradis/CNN-Deblur/" rel="noreferrer">paper</a> that was shared by @Ali looks very interesting and the outcomes are extremely good. It's nice that they have shared the pre-trained weight of their trained model and also shared python scripts for easier use. However, they've experimented with the <strong>Caffe</strong> library. I would prefer to convert into <strong>PyTorch</strong> to better control. Below are the provided python scripts with <strong>Caffe</strong> imports. Please note, I couldn't port it completely until now because of a lack of Caffe knowledge, please correct me if you are aware of it.</p>
<pre class="lang-py prettyprint-override"><code>from __future__ import print_function
import numpy as np
import os, sys, argparse, glob, time, cv2, Queue, caffe

# Some Helper Functins 
def getCutout(image, x1, y1, x2, y2, border):
    assert(x1 &gt;= 0 and y1 &gt;= 0)
    assert(x2 &gt; x1 and y2 &gt;y1)
    assert(border &gt;= 0)
    return cv2.getRectSubPix(image, (y2-y1 + 2*border, x2-x1 + 2*border), (((y2-1)+y1) / 2.0, ((x2-1)+x1) / 2.0))

def fillRndData(data, net):
    inputLayer = 'data'
    randomChannels = net.blobs[inputLayer].data.shape[1]
    rndData = np.random.randn(data.shape[0], randomChannels, data.shape[2], data.shape[3]).astype(np.float32) * 0.2
    rndData[:,0:1,:,:] = data
    net.blobs[inputLayer].data[...] = rndData[:,0:1,:,:]

def mkdirp(directory):
    if not os.path.isdir(directory):
        os.makedirs(directory)

</code></pre>
<p>The main function start here</p>
<pre class="lang-py prettyprint-override"><code>def main(argv):
    pycaffe_dir = os.path.dirname(__file__)

    parser = argparse.ArgumentParser()
    # Optional arguments.
    parser.add_argument(
        &quot;--model_def&quot;,
        help=&quot;Model definition file.&quot;,
        required=True
    )
    parser.add_argument(
        &quot;--pretrained_model&quot;,
        help=&quot;Trained model weights file.&quot;,
        required=True
    )
    parser.add_argument(
        &quot;--out_scale&quot;,
        help=&quot;Scale of the output image.&quot;,
        default=1.0,
        type=float
    )
    parser.add_argument(
        &quot;--output_path&quot;,
        help=&quot;Output path.&quot;,
        default=''
    )
    parser.add_argument(
        &quot;--tile_resolution&quot;,
        help=&quot;Resolution of processing tile.&quot;,
        required=True,
        type=int
    )
    parser.add_argument(
        &quot;--suffix&quot;,
        help=&quot;Suffix of the output file.&quot;,
        default=&quot;-deblur&quot;,
    )
    parser.add_argument(
        &quot;--gpu&quot;,
        action='store_true',
        help=&quot;Switch for gpu computation.&quot;
    )
    parser.add_argument(
        &quot;--grey_mean&quot;,
        action='store_true',
        help=&quot;Use grey mean RGB=127. Default is the VGG mean.&quot;
    )
    parser.add_argument(
        &quot;--use_mean&quot;,
        action='store_true',
        help=&quot;Use mean.&quot;
    )
    parser.add_argument(
        &quot;--adversarial&quot;,
        action='store_true',
        help=&quot;Use mean.&quot;
    )
    args = parser.parse_args()

    mkdirp(args.output_path)

    if hasattr(caffe, 'set_mode_gpu'):
        if args.gpu:
            print('GPU mode', file=sys.stderr)
            caffe.set_mode_gpu()
        net = caffe.Net(args.model_def, args.pretrained_model, caffe.TEST)
    else:
        if args.gpu:
            print('GPU mode', file=sys.stderr)
        net = caffe.Net(args.model_def, args.pretrained_model, gpu=args.gpu)


    inputs = [line.strip() for line in sys.stdin]

    print(&quot;Classifying %d inputs.&quot; % len(inputs), file=sys.stderr)


    inputBlob = net.blobs.keys()[0] # [innat]: input shape 
    outputBlob = net.blobs.keys()[-1]

    print( inputBlob, outputBlob)
    channelCount = net.blobs[inputBlob].data.shape[1]
    net.blobs[inputBlob].reshape(1, channelCount, args.tile_resolution, args.tile_resolution)
    net.reshape()

    if channelCount == 1 or channelCount &gt; 3:
        color = 0
    else:
        color = 1

    outResolution = net.blobs[outputBlob].data.shape[2]
    inResolution = int(outResolution / args.out_scale)
    boundary = (net.blobs[inputBlob].data.shape[2] - inResolution) / 2

    for fileName in inputs:
        img = cv2.imread(fileName, flags=color).astype(np.float32)
        original = np.copy(img)
        img = img.reshape(img.shape[0], img.shape[1], -1)
        if args.use_mean:
            if args.grey_mean or channelCount == 1:
                img -= 127
            else:
                img[:,:,0] -= 103.939
                img[:,:,1] -= 116.779
                img[:,:,2] -= 123.68
        img *= 0.004

        outShape = [int(img.shape[0] * args.out_scale) ,
                    int(img.shape[1] * args.out_scale) ,
                    net.blobs[outputBlob].channels]
        imgOut = np.zeros(outShape)

        imageStartTime = time.time()
        for x, xOut in zip(range(0, img.shape[0], inResolution), range(0, imgOut.shape[0], outResolution)):
            for y, yOut in zip(range(0, img.shape[1], inResolution), range(0, imgOut.shape[1], outResolution)):

                start = time.time()

                region = getCutout(img, x, y, x+inResolution, y+inResolution, boundary)
                region = region.reshape(region.shape[0], region.shape[1], -1)
                data = region.transpose([2, 0, 1]).reshape(1, -1, region.shape[0], region.shape[1])

                if args.adversarial:
                    fillRndData(data, net)
                    out = net.forward()
                else:
                    out = net.forward_all(data=data)

                out = out[outputBlob].reshape(out[outputBlob].shape[1], out[outputBlob].shape[2], out[outputBlob].shape[3]).transpose(1, 2, 0)

                if imgOut.shape[2] == 3 or imgOut.shape[2] == 1:
                    out /= 0.004
                    if args.use_mean:
                        if args.grey_mean:
                            out += 127
                        else:
                            out[:,:,0] += 103.939
                            out[:,:,1] += 116.779
                            out[:,:,2] += 123.68

                if out.shape[0] != outResolution:
                    print(&quot;Warning: size of net output is %d px and it is expected to be %d px&quot; % (out.shape[0], outResolution))
                if out.shape[0] &lt; outResolution:
                    print(&quot;Error: size of net output is %d px and it is expected to be %d px&quot; % (out.shape[0], outResolution))
                    exit()

                xRange = min((outResolution, imgOut.shape[0] - xOut))
                yRange = min((outResolution, imgOut.shape[1] - yOut))

                imgOut[xOut:xOut+xRange, yOut:yOut+yRange, :] = out[0:xRange, 0:yRange, :]
                imgOut[xOut:xOut+xRange, yOut:yOut+yRange, :] = out[0:xRange, 0:yRange, :]

                print(&quot;.&quot;, end=&quot;&quot;, file=sys.stderr)
                sys.stdout.flush()


        print(imgOut.min(), imgOut.max())
        print(&quot;IMAGE DONE %s&quot; % (time.time() - imageStartTime))
        basename = os.path.basename(fileName)
        name = os.path.join(args.output_path, basename + args.suffix)
        print(name, imgOut.shape)
        cv2.imwrite( name, imgOut)

if __name__ == '__main__':
    main(sys.argv)
</code></pre>
<p>To run the program:</p>
<blockquote>
<p>cat fileListToProcess.txt | python processWholeImage.py --model_def
./BMVC_nets/S14_19_200.deploy --pretrained_model
./BMVC_nets/S14_19_FQ_178000.model --output_path ./out/
--tile_resolution 300 --suffix _out.png --gpu --use_mean</p>
</blockquote>
<p>The weight files and also the above scripts can be download from <a href="http://www.fit.vutbr.cz/%7Eihradis/CNN-Deblur/" rel="noreferrer">here (BMVC_net)</a>. However, you may want to convert <a href="https://github.com/vadimkantorov/caffemodel2pytorch" rel="noreferrer">caffe2pytorch</a>. In order to do that, here is the basic starting point:</p>
<ul>
<li>install <a href="http://google.github.io/proto-lens/installing-protoc.html" rel="noreferrer">proto-lens</a></li>
<li>clone <a href="https://github.com/vadimkantorov/caffemodel2pytorch" rel="noreferrer">caffemodel2pytorch</a></li>
</ul>
<p>Next,</p>
<pre class="lang-py prettyprint-override"><code># BMVC_net, you need to download it from authors website, link above
model = caffemodel2pytorch.Net(
    prototxt = './BMVC_net/S14_19_200.deploy', 
    weights = './BMVC_net/S14_19_FQ_178000.model',
    caffe_proto = 'https://raw.githubusercontent.com/BVLC/caffe/master/src/caffe/proto/caffe.proto'
)

model.cuda()
model.eval()
torch.set_grad_enabled(False)
</code></pre>
<p>Run-on a demo tensor,</p>
<pre class="lang-py prettyprint-override"><code># make sure to have right procedure of image normalization and channel reordering
image = torch.Tensor(8, 3, 98, 98).cuda()

# outputs dict of PyTorch Variables
# in this example the dict contains the only key &quot;prob&quot;
#output_dict = model(data = image)

# you can remove unneeded layers:
#del model.prob
#del model.fc8

# a single input variable is interpreted as an input blob named &quot;data&quot;
# in this example the dict contains the only key &quot;fc7&quot;
output_dict = model(image)
# print(output_dict)
print(output_dict.keys())
</code></pre>
<p>Please note, there are some basic things to consider; the networks expect text at DPI 120-150, reasonable orientation, and reasonable black and white levels. The networks expect to mean [103.9, 116.8, 123.7] to be subtracted from inputs. The inputs should be further multiplied by 0.004.</p>
Deblur image with text to be recognized by OCR
<p>I have an image that is blurred:<br />
<img src="https://i.stack.imgur.com/gAQlO.png" alt="1" />
This is a part of the business card and it is one of the frames taken by the camera and without proper focus.</p>
<p>The clear image looks like this
<img src="https://i.stack.imgur.com/Kp124.png" alt="2" />
I'm looking for a method that could give me an image of better quality, so that image could be recognized by OCR, but also should be quite fast. The image is not blurred too much (I think so) but isn't good for OCR. I tried:</p>
<ul>
<li>different kinds of HPF,</li>
<li>Laplacian,</li>
<li>Canny detector,</li>
<li>combinations of morphological operations (opening, closing).</li>
</ul>
<p>I also tried:</p>
<ul>
<li>deconvolution with Wiener filter,</li>
<li>deconvolution and the Lucy-Richardson method.</li>
</ul>
<p>But it was not easy to find the right PSF (Point Spread Function). These methods are considered effective, but not so fast enough. I also tried FFT and then IFFT with a Gaussian mask, but the results were not satisfactory. I'm looking for some kind of general method of deblurring images with text, not only this image. Could someone help me with this problem? I'll be grateful for any advice. I'm working with OpenCV 3 (C++ and sometimes Python).</p>