# Exploring OCR Capabilities of GPT-4V(ision) :
A Quantitative and In-depth Evaluation

Yongxin Shi, Dezhi Peng, Wenhui Liao, Zening Lin, Xinhong Chen, Chongyu Liu,
Yuyi Zhang, Lianwen Jin
South China University of Technology
<EMAIL_ADDRESS><EMAIL_ADDRESS>
Corresponding Author.

###### Abstract

This paper presents a comprehensive evaluation of the Optical Character
Recognition (OCR) capabilities of the recently released GPT-4V(ision), a Large
Multimodal Model (LMM). We assess the model’s performance across a range of
OCR tasks, including scene text recognition, handwritten text recognition,
handwritten mathematical expression recognition, table structure recognition,
and information extraction from visually-rich document. The evaluation reveals
that GPT-4V performs well in recognizing and understanding Latin contents, but
struggles with multilingual scenarios and complex tasks. Specifically, it
showed limitations when dealing with non-Latin languages and complex tasks
such as handwriting mathematical expression recognition, table structure
recognition, and end-to-end semantic entity recognition and pair extraction
from document image. Based on these observations, we affirm the necessity and
continued research value of specialized OCR models. In general, despite its
versatility in handling diverse OCR tasks, GPT-4V does not outperform existing
state-of-the-art OCR models. How to fully utilize pre-trained general-purpose
LMMs such as GPT-4V for OCR downstream tasks remains an open problem. The
study offers a critical reference for future research in OCR with LMMs.
Evaluation pipeline and results are available at https://github.com/SCUT-
DLVCLab/GPT-4V_OCR.

##  1 Introduction

The emergence of ChatGPT [1] marks a significant milestone in the field of
Artificial Intelligence (AI). Concurrently, it has ignited a surge in Large
Language Models (LLMs) research across both academia and industry, with models
such as GLM-130B [2], Alpaca [3], Vicuna [4], LLaMA [5], ERNIE Bot [6], Qwen
[7], Baichuan2 [8]. The success of LLMs has also spurred the development of
Large Multimodal Models (LMMs). Many initiatives are now striving to expand
the multimodal capabilities of LLMs, including BLIP-2 [9], OpenFlamingo [10],
LLaVA [11], MiniGPT4 [12], and mPLUG-Owl [13].

Particularly, the recent release of GPT-4V(ision) [14] presents a significant
breakthrough in the domain of LMMs. Researchers across diverse fields are
eager to comprehend the capabilities of GPT-4V, with those in the Optical
Character Recognition (OCR) domain displaying particular curiosity in its
potential to address OCR tasks. While the official report qualitatively
demonstrates GPT-4V’s abilities in several OCR-related tasks (including text
recognition, expression recognition, and document understanding), quantitative
assessment and in-depth analysis are urgently needed, which will provide
valuable insights and essential references for future research.

To this end, we conduct a quantitative evaluation of GPT-4V on mainstream OCR
tasks, including Scene Text Recognition (STR) [15, 16, 17, 18, 19, 20, 21, 22,
23, 24, 25, 26, 27, 28, 29, 30], Handwritten Text Recognition (HTR) [31, 32,
33, 34, 35, 36, 37, 38, 39], Handwritten Mathematical Expression Recognition
(HMER) [40, 41, 42, 43, 44, 45, 46, 47], Table Structure Recognition (TSR)
[48, 49, 50, 51, 52, 53, 54, 55], and Information Extraction from Visually-
rich Document (VIE) [56, 57, 58, 59, 60, 61, 62, 63, 64, 65]. For the above
tasks, we employ some commonly used benchmarks in the OCR domain for
evaluation: (1) STR: CUTE80 [66], SCUT-CTW1500 [67], Total-Text [68], WordArt
[69], ReCTS [70] and MLT19 [71], (2) HTR: IAM [72] and CASIA-HWDB [73], (3)
HMER: CROHME2014 [74] and HME100K [42], (4) TSR: SciTSR [75] and WTW [76], (5)
VIE: FUNSD [77] and XFUND [78] Chinese subset (XFUND-zh).

The evaluation results suggest that GPT-4V does not match the performance of
specialized OCR models. Specifically, GPT-4V demonstrates superior performance
in Latin content but encounters limitations when dealing with other languages.
Furthermore, GPT-4V struggles in complex scenarios for tasks such as HMER,
TSR, and VIE.

Based on the experimental results, we try to address an important question: do
specialized models still hold research value in the OCR field? Given the three
critical drawbacks of GPT-4V, namely, limited performance in multilingual and
complex scenarios, high inference costs, and challenges in updating, we argue
that existing LMMs struggle to simultaneously handle various OCR tasks [79].
Therefore, we affirm the continued research value of specialized models in the
OCR field. However, it is still crucial to leverage the potential of LMMs like
GPT-4V for future OCR research. There may be three potential directions worth
investigating, including semantic understanding enhancement, downstream task
finetuning, and auto/semi-auto data construction.

##  2 Experiments

We evaluate GPT-4V on the following OCR tasks: scene text recognition,
handwritten text recognition, handwritten mathematical expression recognition,
table structure recognition, and information extraction from visually-rich
document. The evaluation process was conducted within the web-based dialogue
interface with GPT-4V, of which we directly uploaded the image and prompt, and
then extracted relevant answers from the generated responses. The prompts for
each task were meticulously designed. Additionally, to prevent interference
from contextual information, we used a separate dialogue window for each
image. Due to the conversation limits (50 conversations per 3 hours) of
GPT-4V, we conducted sampling on datasets with a large number of samples.

###  2.1 Scene text recognition

#### Dataset

We focus on both word-level text recognition and end-to-end text spotting. For
word-level text recognition, we employ CUTE80 [66], SCUT-CTW1500 [67], Total-
Text [68], WordArt [69] in English and ReCTS [70] in Chinese. We randomly
select 50 images from each dataset above for evaluation. The datasets are
downloaded from [1][1][1]https://github.com/Yuliang-Liu/MultimodalOCR.

  * •

CUTE80 comprises 80 images specifically curated for the purpose of evaluating
curved text.

  * •

SCUT-CTW1500 is a comprehensive curved text dataset encompassing a total of
1500 images.

  * •

Total-Text has 1,555 scene images which collected with curved text in mind.

  * •

WordArt consists of 6316 artistic text images, which primarily features
challenging artistic text.

  * •

ReCTS is a large-scale dataset of 25,000 images, which mainly focuses on
reading Chinese text on signboard.

In the end-to-end text spotting task, we use MLT19 [71] to evaluate the
multilingual capabilities of GPT-4V. For each language, we randomly select 20
images from the training set. Additionally, to investigate the impact of image
resolution on recognition results, we select 20 English images from the
aforementioned subset and resize their long sides to 128, 256, 512, 1024, and
2048 pixels, respectively.

  * •

MLT19 is a dataset for Multi-Lingual scene Text (MLT) detection and
recognition, which consists of 20,000 images containing text from 10
languages.

#### Prompt

For word-level English text recognition, we use the following prompt: "What is
the scene text in the image?", while for ReCTS in Chinese, we translate the
prompt into Chinese, resulting in: "图片中的场景文字是什么？" The prompt in end-to-end
text spotting is: "What are all the scene text in the image? Do not
translate."

#### Metric

For the evaluation of word-level recognition, we employ word accuracy ignoring
case and symbols (WAICS) [80] as metric. In the task of end-to-end text
spotting, the predictions of GPT-4V and ground truths (GT) are split with
spaces and then evaluated using precision and recall. Precision represents the
ratio of correctly identified words to those generated by GPT-4V, while recall
is the ratio of correctly identified words to the total number of GT words. We
also compute the F1 score as follow.

$F1=\frac{2\cdot\text{precision}\cdot\text{recall}}{\text{precision}+\text{recall}}$
(1)

#### Results and analysis

The results are shown in Table 1, Table 2 and Table 3, respectively. We
visualize some examples in Figure 1. Based on the results, we draw the
following insights:

(1) There is a substantial accuracy disparity between the recognition of
English and Chinese text.  As shown in Table 1, the performance of English
text recognition is commendable. Conversely, the accuracy of Chinese text
recognition is zero (ReCTS). We speculate that this may be due to the lack of
Chinese scene text images as training data in GPT-4V.

(2) GPT-4V exhibits a strong ability to recognize Latin characters, surpassing
its performance in other languages. As shown in Table 2, it can be observed
that GPT-4V performs significantly better in English, French, German, and
Italian, compared to non-Latin alphabet languages. This suggests noticeable
limitations in GPT-4V’s multilingual OCR capabilities.

(3) GPT-4V supports input images with different resolutions. As shown in Table
3, there is a positive correlation between the input image resolution and the
recognition performance. This suggests that, unlike previous LMMs that resize
images to a fixed size, GPT-4V supports input images with variable
resolutions. Meanwhile, we hypothesize that the image encoder of GPT-4V
employs a fixed patch size, therefore increasing the resolution of the input
image leads to a longer sequence, which help the model to capture more
information.

Table 1: Results of word-level scene text recognition. The SOTA of CUTE80 and WordArt are achieved by [80] and [81], respectively. [82] reported the SOTA on SCUT-CTW1500 and Total-Text. The SOTA of ReCTS can be found at 333https://rrc.cvc.uab.es/?ch=12&com=evaluation&task=2. Method | CUTE80 | SCUT-CTW1500 | Total-Text | WordArt | ReCTS
---|---|---|---|---|---
GPT-4V | 88.0% | 62.0% | 66.0% | 62.0% | 0
Supervised-SOTA | 98.6% | 87.0% | 90.1% | 68.2% | 97.4%
Table 2: Results of MLT19. The SOTA of end-to-end text spotting in MLT19 can be found at 444https://rrc.cvc.uab.es/?ch=15&com=evaluation&task=4. Method | Language | Precision ↑ | Recall ↑ | F1 ↑ |
---|---|---|---|---|---
GPT-4V | Arabic | 16.44% | 16.67% | 16.55% |
English | 86.57% | 78.77% | 82.49% |
French | 83.0% | 83.84% | 83.42% |
Chinese | 1.2% | 1.56% | 1.36% |
German | 73.65% | 86.29% | 79.47% |
Korean | 10.83% | 12.39% | 11.56% |
Japanese | 11.9% | 11.9% | 11.9% |
Italian | 62.7% | 67.52% | 65.02% |
Bangla | 2.53% | 2.63% | 2.58% |
Hindi | 7.29% | 8.33% | 7.78% |
| All language | 43.04% | 45.42% | 44.2% |
Supervised-SOTA | All language | 74.16% | 52.91% | 61.76% |
Table 3: Impact of image resolution for recognition performance on MLT19 English subset.  Image size | Precision ↑ | Recall ↑ | F1 ↑
---|---|---|---
128 | 45.52% | 57.28% | 50.73%
256 | 73.88% | 86.21% | 79.57%
512 | 85.82% | 83.21% | 84.49%
1024 | 90.30% | 84.72% | 87.42%
2048 | 92.54% | 86.01% | 89.16%
Figure 1: Illustration of word-level scene text recognition. In the answers of
GPT-4V, we highlight the characters that match the GT in green and characters
that do not match in red. GPT-4V can recognize curved, slanted, and artistic
English text, while common-style Chinese text can not be recognized.

###  2.2 Handwritten text recognition

#### Dataset

To evaluate GPT-4V’s capability in handwritten text recognition, we employ two
commonly used handwritten datasets: IAM [72] (in English) and CASIA-HWDB [73]
(in Chinese). We randomly sample 50 pages and 50 text lines from each of the
test sets of IAM and CASIA-HWDB for evaluation.

  * •

IAM comprises 1,539 pages and 13,353 lines of handwritten English text.

  * •

CASIA-HWDB is an offline handwritten Chinese dataset, which contains about
5,090 pages and 1.35 million character samples of 7,356 classes (7,185 Chinese
characters and 171 symbols).

#### Prompt

For IAM, we use the prompt: "Recognize the text in the image." as input. And
for CASIA-HWDB, we use the Chinese prompt "请直接告诉我，图片中的文字都是什么？" , which means
"Please tell me directly, what are all the text in the image?"

#### Metric

Two metrics are used for evaluation in the handwritten English text: Word
Error Rate (WER) and Character Error Rate (CER) [83]. To evaluate the
performance in handwritten Chinese text, we use AR and CR metrics [36].

#### Results and analysis

As shown in Table 4 and 5.

(1) There’s also a significant performance gap between English and Chinese
handwritten text. This phenomenon is consistent with the findings in Section
2.1, which collectively suggests that GPT-4V performs well in English text
recognition while facing notable challenges in Chinese.

(2) GPT-4V exhibits significant hallucinations in Chinese text recognition. As
shown in Figure 3 (c) and (d), the responses generated by GPT-4V demonstrate a
high degree of fluency in both grammar and semantics. However, they
substantially deviate from the textual content of the ground truth (GT),
appearing to produce nonsensical information in a seemingly earnest manner.

Table 4: Results of IAM. The SOTA of page-level IAM in WER and CER metric are achieved by [84] and [85], respectively. And the line-level SOTA is achieved by [86].  Method | Page-level | Line-level
---|---|---
WER ↓ | CER ↓ | WER ↓ | CER ↓
GPT-4V | 9.84% | 3.32% | 33.42% | 13.75%
Supervised-SOTA | 8.29% | 2.89% | 21.47% | 6.52%
Table 5: Results of CASIA-HWDB. The SOTA of page-level CASIA-HWDB in AR and CR metric are achieved by [87] and [88], respectively. And the line-level SOTA is achieved by [36]. Method | Page-level | Line-level
---|---|---
AR ↑ | CR ↑ | AR ↑ | CR ↑
GPT-4V | 0.97% | 36.54% | -3.45% | 11.85%
Supervised-SOTA | 96.83% | 96.99% | 97.70% | 97.91%
Figure 2: Illustration of handwritten text recognition. (a), (b), (c), and (d)
are samples of page-level IAM, line-level IAM, page-level CASIA-HWDB, and
line-level CASIA-HWDB, respectively. In the responses of GPT-4V, we highlight
characters that match the GT in green and characters that do not match in red.
For English text, GPT-4V demonstrates excellent performance. In contrast, for
Chinese text, GPT-4V has generated a passage of text that is semantically
coherent, but it is not associated with the ground truth text (GT).

###  2.3 Handwritten mathematical expression recognition

#### Dataset

For this task, we employ two representative dataset includes CROHME2014 [74]
and HME100K [42]. We randomly select 50 images from the test sets of each of
these two datasets for evaluation.

  * •

CROHME2014 is a classical online dataset for handwritten mathematical
expression recognition, which comprises 9,820 samples of mathematical
expressions.

  * •

HME100K is a large-scale handwritten mathematical expression recognition
dataset, which contains 100k images from ten thousand writers and is mainly
captured by cameras.

#### Prompt

In this task, we use "This is an image of a handwritten mathematical
expression. Please recognize the expression above as LaTeX." as prompt.

#### Metric

The metrics we employed include the correct rates at the expression level, and
with at most one to three errors [74].

#### Results and analysis

The results are shown in Table 6. Based on the analysis of the failed case, we
draw the following findings.

(1) GPT-4V appears to be limited when dealing with camera-captured and poor
handwriting scenarios. As shown in Table 6, the performance on HEM100K (which
features camera-captured images and poor handwriting) significantly drops
compared to CROHME2014. As shown in Figure 3, (a) and (c) are examples from
CROHME2014, (b) and (d) are from HEM100K, GPT-4V performs well on the former,
but poorly on the latter.

(2) GPT-4V exhibits certain challenges in fine-grained character recognition.
Among the failed cases, we observed instances where GPT-4V occasionally missed
small-scale characters. Two examples are shown in Figure 3 (e) and (f). For
these two examples, GPT-4V has omitted a superscript and a subscript,
respectively. This finding aligns with the evaluation results of Liu et al.
[79] on other multimodal models, suggesting that GPT-4V may also suffer from
certain fine-grained perceptual issues.

Figure 3: Illustration of handwritten mathematical expression recognition. In each example, the left side displays the input image, while the right side shows the image rendered from the LaTeX sequence output by GPT-4V. In the answer of GPT-4V, we highlight elements that match the GT in green and elements that do not match in red. The symbol _ in red represents the missing elements in the output. Table 6: Results of handwritten mathematical expression recognition. The SOTA of CROHME2014 and HME100K are both achieved by [46]. Method | CROHME2014 | HME100K
---|---|---
Exp rate ↑ | <=1 ↑ | <=2 ↑ | <=3 ↑ | Exp rate ↑ | <=1 ↑ | <=2 ↑ | <=3 ↑
GPT-4V | 34.0% | 44.0% | 50.0% | 54.0% | 16.0% | 18.0% | 22.0% | 28.0%
Supervised-SOTA [42] | 65.89% | 77.97% | 84.16% | - | 68.09% | 83.22% | 89.91% | -

###  2.4 Table structure recognition

#### Dataset

The datasets we used for this task includes SciTSR [75] and WTW [76]. We
randomly select 50 tables from each of the test sets of SciTSR and WTW for
evaluation. Following [53], we crop table regions from original images for
evaluation.

  * •

SciTSR is a dedicated dataset created to address the task of table structure
recognition in scientific papers. The dataset consists of 12,000 training
samples and 3,000 test samples.

  * •

WTW’s images are collected in the wild. The dataset is split into
training/testing sets with 10,970 and 3,611 samples respectively.

#### Prompt

For both SciTSR and WTW, we use the prompt "Please read the table in this
image and return a html-style reconstructed table in text, do not omit
anything." as input.

#### Metric

To evaluate the performance of GPT-4V in table structure recognition, we use
TEDS-S metrics [48], which is a variation of Tree-Edit-Distance-Based
Similarity (TEDS) [48] that disregards the textual content of the cells and
only evaluates the accuracy of the table structure prediction.

#### Results and analysis

The results are shown in Table 7. We gain two important findings based on the
results:

(1) GPT-4V struggles with complex tables. GPT-4V demonstrates outstanding
performance when handling tables with structured layouts and consistent text
distributions, such as Figure 4 (a). However, when dealing with other types of
tables, including those with numerous empty cells, uneven text distribution,
skewing, rotation, or densely packed arrangements, its performance noticeably
declines.

(2) Content omission issues are observed in GPT-4V when processing lengthy
tables. Despite emphasizing the requirement of "do not omit anything" in the
prompt, we still observed some instances of content omission in the responses,
particularly in the case of a large table. A typical example is shown in
Figure 4 (e), the table image Figure 4 (c) contains many rows, but GPT-4V only
reconstructs three of them.

Table 7: The TEDS-S of SciTSR and WTW. The SOTA of SciTSR and WTW are both achieved by [52]. Method | SciTSR | WTW
---|---|---
GPT-4V | 87.47% | 25.60%
Supervised-SOTA | 99.19% | 91.91%
Figure 4: Illustration of table structure recognition. (a) and (c) are two
input images, (b) and (d) are the corresponding visualized images of GPT-4V’s
html-style output sequence. (e) is the output sequence of (c), where the
elements that GPT-4V indicate the omitted content are highlighted in red.

###  2.5 Information Extraction from Visually-rich Document

#### Dataset

We evaluate GPT-4V on FUNSD [77] and XFUND [78] Chinese subset (XFUND-zh).

  * •

FUNSD dataset is a commonly used form understanding benchmark, which contains
199 scanned form-like documents with noisy images.

  * •

XFUND dataset is a multilingual extension of FUNSD that covers seven languages
(Chinese, Japanese, French, Italian, German, Spanish, and Portuguese).

We evaluate GPT-4V on the Semantic Entity Recognition (SER) and the end-to-end
Pair Extraction tasks. The SER task requires the model to identify the
category of each text segments, which are predefined as header, question,
answer, and other in FUNSD and XFUND. The end-to-end pair extraction task asks
the model to extract all the key-value pairs in the given document image. We
use the full test set (both FUNSD and XFUND-zh contain 50 samples) for
performance evaluation.

#### Prompt

For FUNSD, we use the following prompt for SER:

Please read the text in this image and return the information in the following
JSON format (note xxx is placeholder, if the information is not available in
the image, put "N/A" instead). "header": [xxx, …], "key": [xxx, …], "value":
[xxx, …]

It’s important to highlight that, we redefined the official entity type of
"question" and "answer" as "key" and "value" to maintain consistency with the
Pair Extraction task.

For end-to-end Pair Extraction, we use the following prompt:

You are a document understanding AI, who reads the contents in the given
document image and tells the information that the user needs. Respond with the
original content in the document image, do not reformat. No extra explanation
is needed. Extract all the key-value pairs from the document image.

#### Metric

For the SER task, we employ the entity-level F1-score [60] for performance
evaluation. Additionally, Normalized Edit Distance (NED) is also calculated as
is done in other end-to-end VIE methods [65]. However, due to limitations in
GPT-4V’s ability to generate precise bounding boxes for entities, we aligned
predictions with ground-truth using the principle of minimum edit distance.

#### Results and analysis

The SER and Pair Extraction results are shown in 8 and 9, respectively. We
found that:

(1) GPT-4V might have constraints in comprehending the spatial arrangement of
documents. As shown in Figure 5, some text content located near the top of the
page, which lacks both visual and semantic alignment with the header category,
is erroneously identified as a header. Additional visualizations are presented
in 6. It is evident that GPT-4V excels in analyzing documents with
straightforward layouts but struggles to comprehend those featuring intricate
layouts.

(2) GPT4V tends to generate new keys for non-kv-pair contents. For instance,
as shown in Figure 7, contents "09 / 17 / 97 10:55" at the header part are
recognized as "Date: 09/18/97", "Time: 10:55", "Fax Number: 503 841 1898",
"Company: LORILLARD PTLD", "Page Number: 001".

Table 8: SER Results of FUNSD and XFUND-zh. The SOTA of FUNSD is provided in
[65].

Method | FUNSD | XFUND-zh
---|---|---
Precision ↑ | Recall ↑ | F1 ↑ | 1-NED ↑ | Precision ↑ | Recall ↑ | F1 ↑ | 1-NED ↑
GPT-4V | 41.85% | 29.36% | 34.51% | 0.2697 | 25.87% | 15.15% | 19.11% | 0.1544
Supervised-SOTA [65] | - | - | - | 0.5500 | - | - | - | -

Figure 5: Illustration of error cases of the SER task. The text content enclosed within the red box is incorrectly identified as header entities. Figure 6: Illustration of Entity Prediction on Full Document Images in the FUNSD Dataset. Due to GPT-4V’s limited capability in recognizing Chinese characters, we have excluded examples from the XFUND-zh dataset in this context. Zoom in for the best review. Table 9: Pair Extraction Results of FUNSD and XFUND-zh. Dataset | Precision ↑ | Recall ↑ | F1 ↑ | 1-NED ↑
---|---|---|---|---
FUNSD | 20.69% | 10.25% | 13.71% | 0.1979
XFUND-zh | 0.07% | 0.02% | 0.03% | 0.0420
Figure 7: Illustration of error cases of the Pair Extraction task. The text
content enclosed within the red box is incorrectly identified as entity pairs.

##  3 Discussions

#### Do specialized models still hold research value in the OCR field?

There are three main drawbacks of GPT-4V. (1) Based on the experimental
results in Section 2, GPT-4V’s ability in OCR is limited to Latin contents and
struggles to cope with multilingual and complex scenarios. (2) The inference
cost and delay are significantly high, thereby posing usability challenges in
some practical scenarios. (3) The long cycle and complex process of updating
make it difficult to promptly address minor issues. Considering the
aforementioned shortcomings and limited OCR capabilities of some other LMMs
[79], we believe that existing LMMs struggle to simultaneously excel in
various OCR tasks. Therefore, we contend that specialized models in the field
of OCR continue to hold significant value for research.

#### How can we fully leverage the potential of LMMs like GPT-4V in the OCR
domain?

These are some possible strategies. (1) Semantic understanding enhancement: A
significant characteristic of LMMs lies in their outstanding semantic
capabilities after extensive training on large-scale data. Since semantic
understanding is a crucial factor in document comprehension and some related
tasks, harnessing the semantic potential of LMMs can greatly enhance the
performance in these tasks. (2) Downstream task finetuning: Another approach
that fully leverages the prior knowledge of LMMs is fine-tuning, especially in
scenarios with limited data. Fine-tuning allows the model to adapt to specific
tasks or domains, thus improving the performance [89]. (3) Auto/semi-auto data
construction: Using LMMs for automatic/semi-automatic data annotation and
generation will substantially reduce the cost of manual labeling, which is an
effective strategy for tackling the difficulties of data acquisition [90].

##  4 Limitations

There are three main limitations of our work. First, the test sample of our
evaluation is small-scale (mostly 50 samples per dataset) due to the
conversation limits (50 conversations per 3 hours) of GPT-4V. This could
potentially limit the generalizability of the results. Second, our assessment
primarily focuses on mainstream OCR tasks and does not include other OCR-
related tasks. Hence, the findings might not cover the full spectrum of OCR
capabilities of GPT-4V. Third, only the zero-shot capacity of GPT-4V in OCR
was evaluated, without exploring few-shot scenarios. As a result, the
potential benefits of further training or fine-tuning the LLM model for
specific tasks are not addressed. Few-shot scenarios with technology such as
in-context learning [91] are worth of exploring in the future.

##  5 Conclusion

In this paper, we present a comprehensive evaluation of the OCR capabilities
of GPT-4V through a variety of experiments. For the first time, we offer not
only qualitative demonstrations but also quantitative performance analysis of
GPT-4V across a wide spectrum of tasks. These tasks encompass scene text
recognition, handwritten text recognition, handwritten mathematical expression
recognition, table structure recognition, and information extraction from
visually rich documents.

Our findings, grounded in meticulous experimental results, provide an in-depth
analysis of the strengths and limitations of GPT-4V. Although the model shows
a strong ability to accurately recognize Latin content and supports input
images of variable resolutions, it displays notable struggles with
multilingual and complex scenarios. Additionally, the high inference costs and
the challenges associated with continuous updating pose significant barriers
to the real-world deployment of GPT-4V. Therefore, we contend that specialized
models in the field of OCR continue to hold significant value for research.
Despite these limitations, GPT-4V and other existing general LMMs could still
significantly contribute to the development of the OCR field in several ways.
These would include enhancing semantic understanding, fine-tuning for
downstream tasks, and facilitating auto/semi-auto data construction.

In summary, this paper presents a first-of-its-kind, in-depth quantitative
evaluation of GPT-4V’s performance in OCR tasks. We will continuously update
the evaluation results in the future, and we hope the findings in this paper
will provide valuable insights and strategies for researchers and
practitioners working on OCR tasks using large multi-modal models.

## References

  * [1] OpenAI.  ChatGPT.  https://openai.com/blog/chatgpt/, 2023.
  * [2] Aohan Zeng, Xiao Liu, Zhengxiao Du, Zihan Wang, Hanyu Lai, Ming Ding, Zhuoyi Yang, Yifan Xu, Wendi Zheng, Xiao Xia, et al.  GLM-130B: An open bilingual pre-trained model.  arXiv preprint arXiv:2210.02414, 2022.
  * [3] Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto.  Stanford Alpaca: An instruction-following LLaMA model.  https://github.com/tatsu-lab/stanford_alpaca, 2023.
  * [4] Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E Gonzalez, et al.  Vicuna: An open-source chatbot impressing GPT-4 with 90%* ChatGPT quality.  See https://vicuna. lmsys. org (accessed 14 April 2023), 2023.
  * [5] Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al.  LLaMA: Open and efficient foundation language models.  arXiv preprint arXiv:2302.13971, 2023.
  * [6] Baidu.  ERNIE Bot.  https://yiyan.baidu.com/, 2023.
  * [7] Ali.  Qwen.  https://tongyi.aliyun.com/, 2023.
  * [8] Baichuan.  Baichuan 2: Open large-scale language models.  arXiv preprint arXiv:2309.10305, 2023.
  * [9] Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi.  BLIP-2: Bootstrapping language-image pre-training with frozen image encoders and large language models, 2023.
  * [10] Anas Awadalla, Irena Gao, Joshua Gardner, Jack Hessel, Yusuf Hanafy, Wanrong Zhu, Kalyani Marathe, Yonatan Bitton, Samir Gadre, Jenia Jitsev, Simon Kornblith, Pang Wei Koh, Gabriel Ilharco, Mitchell Wortsman, and Ludwig Schmidt.  Openflamingo, March 2023.
  * [11] Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee.  Visual instruction tuning.  arXiv preprint arXiv:2304.08485, 2023.
  * [12] Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny.  MiniGPT-4: Enhancing vision-language understanding with advanced large language models, 2023.
  * [13] Qinghao Ye, Haiyang Xu, Guohai Xu, Jiabo Ye, Ming Yan, Yiyang Zhou, Junyang Wang, Anwen Hu, Pengcheng Shi, Yaya Shi, Chenliang Li, Yuanhong Xu, Hehong Chen, Junfeng Tian, Qian Qi, Ji Zhang, and Fei Huang.  mPLUG-Owl: Modularization empowers large language models with multimodality, 2023.
  * [14] Zhengyuan Yang, Linjie Li, Kevin Lin, Jianfeng Wang, Chung-Ching Lin, Zicheng Liu, and Lijuan Wang.  The dawn of LMMs: Preliminary explorations with GPT-4V (ision).  arXiv preprint arXiv:2309.17421, 2023.
  * [15] Baoguang Shi, Xiang Bai, and Cong Yao.  An end-to-end trainable neural network for image-based sequence recognition and its application to scene text recognition.  IEEE Trans. Pattern Anal. Mach. Intell., 39(11):2298–2304, 2016.
  * [16] Baoguang Shi, Mingkun Yang, Xinggang Wang, Pengyuan Lyu, Cong Yao, and Xiang Bai.  Aster: An attentional scene text recognizer with flexible rectification.  IEEE Trans. Pattern Anal. Mach. Intell., 41(9):2035–2048, 2018.
  * [17] Canjie Luo, Lianwen Jin, and Zenghui Sun.  Moran: A multi-object rectified attention network for scene text recognition.  Pattern Recognit., 90:109–118, 2019.
  * [18] Tianwei Wang, Yuanzhi Zhu, Lianwen Jin, Canjie Luo, Xiaoxue Chen, Yaqiang Wu, Qianying Wang, and Mingxiang Cai.  Decoupled attention network for text recognition.  In Proc. AAAI, volume 34, pages 12216–12224, 2020.
  * [19] Minghui Liao, Guan Pang, Jing Huang, Tal Hassner, and Xiang Bai.  Mask TextSpotter v3: Segmentation proposal network for robust scene text spotting.  In Proc. ECCV, pages 706–722. Springer, 2020.
  * [20] Shancheng Fang, Hongtao Xie, Yuxin Wang, Zhendong Mao, and Yongdong Zhang.  Read like humans: Autonomous, bidirectional and iterative language modeling for scene text recognition.  In Proc. CVPR, pages 7098–7107, 2021.
  * [21] Yuliang Liu, Chunhua Shen, Lianwen Jin, Tong He, Peng Chen, Chongyu Liu, and Hao Chen.  ABCNet v2: Adaptive bezier-curve network for real-time end-to-end text spotting.  IEEE Trans. Pattern Anal. Mach. Intell., 44(11):8048–8064, 2021.
  * [22] Dajian Zhong, Shujing Lyu, Palaiahnakote Shivakumara, Bing Yin, Jiajia Wu, Umapada Pal, and Yue Lu.  SGBANet: semantic gan and balanced attention network for arbitrarily oriented scene text recognition.  In Proc. ECCV, pages 464–480. Springer, 2022.
  * [23] Mingxin Huang, Yuliang Liu, Zhenghao Peng, Chongyu Liu, Dahua Lin, Shenggao Zhu, Nicholas Yuan, Kai Ding, and Lianwen Jin.  SwinTextSpotter: Scene text spotting via better synergy between text detection and text recognition.  In Proc. CVPR, pages 4593–4603, 2022.
  * [24] Maoyuan Ye, Jing Zhang, Shanshan Zhao, Juhua Liu, Tongliang Liu, Bo Du, and Dacheng Tao.  DeepSolo: Let transformer decoder with explicit points solo for text spotting.  In Proc. CVPR, pages 19348–19357, 2023.
  * [25] Mingxin Huang, Jiaxin Zhang, Dezhi Peng, Hao Lu, Can Huang, Yuliang Liu, Xiang Bai, and Lianwen Jin.  ESTextSpotter: Towards better scene text spotting with explicit synergy in transformer.  In Proc. ICCV, pages 19495–19505, 2023.
  * [26] Pengyuan Lyu, Minghui Liao, Cong Yao, Wenhao Wu, and Xiang Bai.  Mask TextSpotter: An end-to-end trainable neural network for spotting text with arbitrary shapes.  In Proc. ECCV, pages 67–83, 2018.
  * [27] Minghui Liao, Pengyuan Lyu, Minghang He, Cong Yao, Wenhao Wu, and Xiang Bai.  Mask TextSpotter: An end-to-end trainable neural network for spotting text with arbitrary shapes.  IEEE Trans. Pattern Anal. Mach. Intell., 43(2):532–548, 2019.
  * [28] Dezhi Peng, Xinyu Wang, Yuliang Liu, Jiaxin Zhang, Mingxin Huang, Songxuan Lai, Jing Li, Shenggao Zhu, Dahua Lin, Chunhua Shen, et al.  SPTS: single-point text spotting.  In Proc. ACM MM, pages 4272–4281, 2022.
  * [29] Yuliang Liu, Jiaxin Zhang, Dezhi Peng, Mingxin Huang, Xinyu Wang, Jingqun Tang, Can Huang, Dahua Lin, Chunhua Shen, Xiang Bai, and Lianwen Jin.  SPTS v2: Single-point scene text spotting.  IEEE Trans. Pattern Anal. Mach. Intell., pages 1–15, 2023.
  * [30] Shi-Xue Zhang, Chun Yang, Xiaobin Zhu, and Xu-Cheng Yin.  Arbitrary shape text detection via boundary transformer.  IEEE Trans. Multimedia, 2023.
  * [31] Qiu-Feng Wang, Fei Yin, and Cheng-Lin Liu.  Handwritten Chinese text recognition by integrating multiple contexts.  IEEE Trans. Pattern Anal. Mach. Intell., 34(8):1469–1481, 2011.
  * [32] Théodore Bluche.  Joint line segmentation and transcription for end-to-end handwritten paragraph recognition.  Proc. NIPS, 29, 2016.
  * [33] Zecheng Xie, Zenghui Sun, Lianwen Jin, Hao Ni, and Terry Lyons.  Learning spatial-semantic context with fully convolutional recurrent network for online handwritten Chinese text recognition.  IEEE Trans. Pattern Anal. Mach. Intell., 40(8):1903–1917, 2017.
  * [34] Dezhi Peng, Lianwen Jin, Yaqiang Wu, Zhepeng Wang, and Mingxiang Cai.  A fast and accurate fully convolutional network for end-to-end handwritten Chinese text segmentation and recognition.  In Proc. ICDAR, pages 25–30. IEEE, 2019.
  * [35] Mohamed Yousef and Tom E Bishop.  OrigamiNet: weakly-supervised, segmentation-free, one-step, full page text recognition by learning to unfold.  In Proc. CVPR, pages 14710–14719, 2020.
  * [36] Dezhi Peng, Lianwen Jin, Weihong Ma, Canyu Xie, Hesuo Zhang, Shenggao Zhu, and Jing Li.  Recognition of handwritten Chinese text by segmentation: a segment-annotation-free approach.  IEEE Trans. Multimedia, 2022.
  * [37] Dezhi Peng, Lianwen Jin, Yuliang Liu, Canjie Luo, and Songxuan Lai.  PageNet: Towards end-to-end weakly supervised page-level handwritten Chinese text recognition.  Int. J. Comput. Vis., 130(11):2623–2645, 2022.
  * [38] Jiarong Huang, Dezhi Peng, Hongliang Li, Hao Ni, and Lianwen Jin.  SegCTC: Offline handwritten Chinese text recognition via better fusion between explicit and implicit segmentation.  In Proc. ICDAR, pages 332–349. Springer, 2023.
  * [39] Denis Coquenet, Clément Chatelain, and Thierry Paquet.  DAN: a segmentation-free document attention network for handwritten document recognition.  IEEE Trans. Pattern Anal. Mach. Intell., 2023.
  * [40] Jianshu Zhang, Jun Du, Shiliang Zhang, Dan Liu, Yulong Hu, Jinshui Hu, Si Wei, and Lirong Dai.  Watch, attend and parse: An end-to-end neural network based approach to handwritten mathematical expression recognition.  Pattern Recognit., 71:196–206, 2017.
  * [41] Jianshu Zhang, Jun Du, and Lirong Dai.  Multi-scale attention with dense encoder for handwritten mathematical expression recognition.  In Proc. ICPR, pages 2245–2250. IEEE, 2018.
  * [42] Ye Yuan, Xiao Liu, Wondimu Dikubab, Hui Liu, Zhilong Ji, Zhongqin Wu, and Xiang Bai.  Syntax-aware network for handwritten mathematical expression recognition.  In Proc. CVPR, pages 4553–4562, 2022.
  * [43] Shuhan Zhong, Sizhe Song, Guanyao Li, and S-H Gary Chan.  A tree-based structure-aware transformer decoder for image-to-markup generation.  In Proc. ACM MM, pages 5751–5760, 2022.
  * [44] Xiaohang Bian, Bo Qin, Xiaozhe Xin, Jianwu Li, Xuefeng Su, and Yanfeng Wang.  Handwritten mathematical expression recognition via attention aggregation based bi-directional mutual learning.  In Proc. AAAI, volume 36, pages 113–121, 2022.
  * [45] Wenqi Zhao and Liangcai Gao.  Comer: Modeling coverage for transformer-based handwritten mathematical expression recognition.  In Proc. ECCV, pages 392–408. Springer, 2022.
  * [46] Bohan Li, Ye Yuan, Dingkang Liang, Xiao Liu, Zhilong Ji, Jinfeng Bai, Wenyu Liu, and Xiang Bai.  When counting meets HMER: Counting-aware network for handwritten mathematical expression recognition.  In Proc. ECCV, pages 197–214. Springer, 2022.
  * [47] Zhe Li, Xinyu Wang, Yuliang Liu, Lianwen Jin, Yichao Huang, and Kai Ding.  Improving handwritten mathematical expression recognition via similar symbol distinguishing.  IEEE Trans. Multimedia, 2023.
  * [48] Xu Zhong, Elaheh ShafieiBavani, and Antonio Jimeno Yepes.  Image-based table recognition: data, model, and evaluation.  In Proc. ECCV, pages 564–580. Springer, 2020.
  * [49] Jiaquan Ye, Xianbiao Qi, Yelin He, Yihao Chen, Dengyi Gu, Peng Gao, and Rong Xiao.  Pingan-vcgroup’s solution for ICDAR 2021 competition on scientific literature parsing task b: table recognition to html.  arXiv preprint arXiv:2105.01848, 2021.
  * [50] Hao Liu, Xin Li, Bing Liu, Deqiang Jiang, Yinsong Liu, Bo Ren, and Rongrong Ji.  Show, read and reason: Table structure recognition with flexible context aggregator.  In Proc. ACM MM, pages 1084–1092, 2021.
  * [51] Hao Liu, Xin Li, Bing Liu, Deqiang Jiang, Yinsong Liu, and Bo Ren.  Neural collaborative graph machines for table structure recognition.  In Proc. CVPR, pages 4533–4542, 2022.
  * [52] Bangdong Chen, Dezhi Peng, Jiaxin Zhang, Yujin Ren, and Lianwen Jin.  Complex table structure recognition in the wild using transformer and identity matrix-based augmentation.  In Proc. ICFHR, pages 545–561. Springer, 2022.
  * [53] Weihong Lin, Zheng Sun, Chixiang Ma, Mingze Li, Jiawei Wang, Lei Sun, and Qiang Huo.  TSRFormer: Table structure recognition with transformers.  In Proc. ACM MM, pages 6473–6482, 2022.
  * [54] Yongshuai Huang, Ning Lu, Dapeng Chen, Yibo Li, Zecheng Xie, Shenggao Zhu, Liangcai Gao, and Wei Peng.  Improving table structure recognition with visual-alignment sequential coordinate modeling.  In Proc. CVPR, pages 11134–11143, 2023.
  * [55] Huawen Shen, Xiang Gao, Jin Wei, Liang Qiao, Yu Zhou, Qiang Li, and Zhanzhan Cheng.  Divide rows and conquer cells: Towards structure recognition for large tables.  IJCAI, 2023.
  * [56] Wenwen Yu, Ning Lu, Xianbiao Qi, Ping Gong, and Rong Xiao.  PICK: Processing key information extraction from documents using improved graph learning-convolutional networks.  In Proc. ICPR, pages 4363–4370, 2021.
  * [57] Jiapeng Wang, Lianwen Jin, and Kai Ding.  LiLT: A simple yet effective language-independent layout transformer for structured document understanding.  In Proc. ACL, pages 7747–7757, 2022.
  * [58] Jiapeng Wang, Chongyu Liu, Lianwen Jin, Guozhi Tang, Jiaxin Zhang, Shuaitao Zhang, Qianying Wang, Yaqiang Wu, and Mingxiang Cai.  Towards robust visual information extraction in real world: New dataset and novel solution.  35:2738–2745, 2021.
  * [59] Jiapeng Wang, Tianwei Wang, Guozhi Tang, Lianwen Jin, Weihong Ma, Kai Ding, and Yichao Huang.  Tag, copy or predict: A unified weakly-supervised learning framework for visual information extraction using sequences.  In Proc. IJCAI, pages 1082–1090, 2021.
  * [60] Yiheng Xu, Minghao Li, Lei Cui, Shaohan Huang, Furu Wei, and Ming Zhou.  LayoutLM: Pre-training of text and layout for document image understanding.  In Proc. SIGKDD, page 1192–1200, 2020.
  * [61] Yupan Huang, Tengchao Lv, Lei Cui, Yutong Lu, and Furu Wei.  LayoutLMv3: Pre-training for document ai with unified text and image masking.  In Proc. ACM MM, page 4083–4091, 2022.
  * [62] Teakgyu Hong, Donghyun Kim, Mingi Ji, Wonseok Hwang, Daehyun Nam, and Sungrae Park.  BROS: A pre-trained language model focusing on text and layout for better key information extraction from documents.  In Proc. AAAI, pages 10767–10775, 2022.
  * [63] Peizhao Li, Jiuxiang Gu, Jason Kuen, Vlad I. Morariu, Handong Zhao, Rajiv Jain, Varun Manjunatha, and Hongfu Liu.  SelfDoc: Self-supervised document representation learning.  In Proc. CVPR, pages 5652–5660, June 2021.
  * [64] Srikar Appalaraju, Bhavan Jasani, Bhargava Urala Kota, Yusheng Xie, and R. Manmatha.  DocFormer: End-to-end transformer for document understanding.  In Proc. ICCV, pages 993–1003, October 2021.
  * [65] Yuechen Yu, Yulin Li, Chengquan Zhang, Xiaoqiang Zhang, Zengyuan Guo, Xiameng Qin, Kun Yao, Junyu Han, Errui Ding, and Jingdong Wang.  StrucTexTv2: Masked visual-textual prediction for document image pre-training.  In Proc. ICLR, 2023.
  * [66] Anhar Risnumawan, Palaiahnakote Shivakumara, Chee Seng Chan, and Chew Lim Tan.  A robust arbitrary text detection system for natural scene images.  Expert Syst. Appl., 41:8027–8048, 2014.
  * [67] Yuliang Liu, Lianwen Jin, Shuaitao Zhang, Canjie Luo, and Sheng Zhang.  Curved scene text detection via transverse and longitudinal sequence connection.  Pattern Recognit., 90:337–345, 2019.
  * [68] Chee-Kheng Chng and Chee Seng Chan.  Total-Text: A comprehensive dataset for scene text detection and recognition.  In Proc. ICDAR, volume 01, pages 935–942, 2017.
  * [69] Xudong Xie, Ling Fu, Zhifei Zhang, Zhaowen Wang, and Xiang Bai.  Toward understanding WordArt: Corner-guided transformer for scene text recognition.  In Proc. ECCV, pages 303–321. Springer, 2022.
  * [70] Rui Zhang, Yongsheng Zhou, Qianyi Jiang, Qi Song, Nan Li, Kai Zhou, Lei Wang, Dong Wang, Minghui Liao, Mingkun Yang, et al.  ICDAR 2019 robust reading challenge on reading Chinese text on signboard.  In Proc. ICDAR, pages 1577–1581. IEEE, 2019.
  * [71] Nibal Nayef, Yash Patel, Michal Busta, Pinaki Nath Chowdhury, Dimosthenis Karatzas, Wafa Khlif, Jiri Matas, Umapada Pal, Jean-Christophe Burie, Cheng-lin Liu, et al.  ICDAR2019 robust reading challenge on multi-lingual scene text detection and recognition—RRC-MLT-2019.  In Proc. ICDAR, pages 1582–1587. IEEE, 2019.
  * [72] U-V Marti and Horst Bunke.  The IAM-database: an english sentence database for offline handwriting recognition.  Int. J. Doc. Anal. Recognit., 5:39–46, 2002.
  * [73] Fei Yin, Qiu-Feng Wang, Xu-Yao Zhang, and Cheng-Lin Liu.  ICDAR 2013 Chinese handwriting recognition competition.  In Proc. ICDAR, pages 1464–1470. IEEE, 2013.
  * [74] Harold Mouchere, Christian Viard-Gaudin, Richard Zanibbi, and Utpal Garain.  ICFHR 2014 competition on recognition of on-line handwritten mathematical expressions (CROHME 2014).  In Proc. ICFHR, pages 791–796. IEEE, 2014.
  * [75] Zewen Chi, Heyan Huang, Heng-Da Xu, Houjin Yu, Wanxuan Yin, and Xian-Ling Mao.  Complicated table structure recognition.  arXiv preprint arXiv:1908.04729, 2019.
  * [76] Rujiao Long, Wen Wang, Nan Xue, Feiyu Gao, Zhibo Yang, Yongpan Wang, and Gui-Song Xia.  Parsing table structures in the wild.  In Proc. ICCV, pages 944–952, 2021.
  * [77] Guillaume Jaume, Hazim Kemal Ekenel, and Jean-Philippe Thiran.  FUNSD: A dataset for form understanding in noisy scanned documents.  In Proc. ICDAR Workshops, volume 2, pages 1–6, 2019.
  * [78] Yiheng Xu, Tengchao Lv, Lei Cui, Guoxin Wang, Yijuan Lu, Dinei Florencio, Cha Zhang, and Furu Wei.  XFUND: A benchmark dataset for multilingual visually rich form understanding.  In Proc. ACL Findings, pages 3214–3224, May 2022.
  * [79] Yuliang Liu, Zhang Li, Hongliang Li, Wenwen Yu, Mingxin Huang, Dezhi Peng, Mingyu Liu, Mingrui Chen, Chunyuan Li, Lianwen Jin, et al.  On the hidden mystery of OCR in large multimodal models.  arXiv preprint arXiv:2305.07895, 2023.
  * [80] Qing Jiang, Jiapeng Wang, Dezhi Peng, Chongyu Liu, and Lianwen Jin.  Revisiting scene text recognition: A data perspective.  In Proc. ICCV, pages 20543–20554, 2023.
  * [81] Wenwen Yu, Mingyu Liu, Biao Yang, Enming Zhang, Deqiang Jiang, Xing Sun, Yuliang Liu, and Xiang Bai.  Looking and listening: Audio guided text recognition.  arXiv preprint arXiv:2306.03482, 2023.
  * [82] Mingkun Yang, Minghui Liao, Pu Lu, Jing Wang, Shenggao Zhu, Hualin Luo, Qi Tian, and Xiang Bai.  Reading and writing: Discriminative and generative modeling for self-supervised text recognition.  In Proc. ACM MM, pages 4214–4223, 2022.
  * [83] Jorge Sueiras, Victoria Ruiz, Angel Sanchez, and Jose F Velez.  Offline continuous handwriting recognition using sequence to sequence neural networks.  Neurocomputing, 289:119–128, 2018.
  * [84] Lalita Kumari, Sukhdeep Singh, Vaibhav Varish Singh Rathore, and Anuj Sharma.  A comprehensive handwritten paragraph text recognition system: Lexiconnet.  In Proc. ICDAR, pages 226–241. Springer, 2023.
  * [85] Minghao Li, Tengchao Lv, Jingye Chen, Lei Cui, Yijuan Lu, Dinei Florencio, Cha Zhang, Zhoujun Li, and Furu Wei.  TrOCR: Transformer-based optical character recognition with pre-trained models.  In Proc. AAAI, volume 37, pages 13094–13102, 2023.
  * [86] BN Shashank, S Nagesh Bhattu, and K Sri Phani Krishna.  Improvising the CNN feature maps through integration of channel attention for handwritten text recognition.  In Proc. ICCVAIP, pages 490–502. Springer, 2022.
  * [87] Zi-Rui Wang, Jun Du, and Jia-Ming Wang.  Writer-aware CNN for parsimonious HMM-based offline handwritten Chinese text recognition.  Pattern Recognit., 100:107102, 2020.
  * [88] Canyu Xie, Songxuan Lai, Qianying Liao, and Lianwen Jin.  High performance offline handwritten Chinese text recognition with a new data preprocessing and augmentation pipeline.  In Proc. DAS, pages 45–59. Springer, 2020.
  * [89] Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale Fung, and Steven Hoi.  InstructBLIP: Towards general-purpose vision-language models with instruction tuning, 2023.
  * [90] Lai Wei, Zihao Jiang, Weiran Huang, and Lichao Sun.  InstructionGPT-4: A 200-instruction paradigm for fine-tuning MiniGPT-4.  arXiv preprint arXiv:2308.12067, 2023.
  * [91] Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al.  Language models are few-shot learners.  Proc. NIPS, 33:1877–1901, 2020.