echo840 commited on
Commit
ce7bc2e
·
verified ·
1 Parent(s): e2f1716

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1089 -558
README.md CHANGED
@@ -16,6 +16,7 @@ MonkeyOCR: Document Parsing with a Structure-Recognition-Relation Triplet Paradi
16
  [![HuggingFace](https://img.shields.io/badge/HuggingFace%20Weights-black.svg?logo=HuggingFace)](https://huggingface.co/echo840/MonkeyOCR)
17
  [![GitHub issues](https://img.shields.io/github/issues/Yuliang-Liu/MonkeyOCR?color=critical&label=Issues)](https://github.com/Yuliang-Liu/MonkeyOCR/issues?q=is%3Aopen+is%3Aissue)
18
  [![GitHub closed issues](https://img.shields.io/github/issues-closed/Yuliang-Liu/MonkeyOCR?color=success&label=Issues)](https://github.com/Yuliang-Liu/MonkeyOCR/issues?q=is%3Aissue+is%3Aclosed)
 
19
  [![GitHub views](https://komarev.com/ghpvc/?username=Yuliang-Liu&repo=MonkeyOCR&color=brightgreen&label=Views)](https://github.com/Yuliang-Liu/MonkeyOCR)
20
  </div>
21
 
@@ -23,9 +24,10 @@ MonkeyOCR: Document Parsing with a Structure-Recognition-Relation Triplet Paradi
23
  > **MonkeyOCR: Document Parsing with a Structure-Recognition-Relation Triplet Paradigm**<br>
24
  > Zhang Li, Yuliang Liu, Qiang Liu, Zhiyin Ma, Ziyang Zhang, Shuo Zhang, Zidun Guo, Jiarui Zhang, Xinyu Wang, Xiang Bai <br>
25
  [![arXiv](https://img.shields.io/badge/Arxiv-b31b1b.svg?logo=arXiv)](https://arxiv.org/abs/2506.05218)
26
- [![Source_code](https://img.shields.io/badge/Code-Available-white)](https://github.com/Yuliang-Liu/MonkeyOCR)
27
  [![Model Weight](https://img.shields.io/badge/HuggingFace-gray)](https://huggingface.co/echo840/MonkeyOCR)
28
  [![Model Weight](https://img.shields.io/badge/ModelScope-green)](https://modelscope.cn/models/l1731396519/MonkeyOCR)
 
29
  [![Demo](https://img.shields.io/badge/Demo-blue)](http://vlrlabmonkey.xyz:7685/)
30
 
31
 
@@ -33,66 +35,273 @@ MonkeyOCR: Document Parsing with a Structure-Recognition-Relation Triplet Paradi
33
  ## Introduction
34
  MonkeyOCR adopts a Structure-Recognition-Relation (SRR) triplet paradigm, which simplifies the multi-tool pipeline of modular approaches while avoiding the inefficiency of using large multimodal models for full-page document processing.
35
 
36
- 1. Compared with the pipeline-based method MinerU, our approach achieves an average improvement of 5.1% across nine types of Chinese and English documents, including a 15.0% gain on formulas and an 8.6% gain on tables.
37
- 2. Compared to end-to-end models, our 3B-parameter model achieves the best average performance on English documents, outperforming models such as Gemini 2.5 Pro and Qwen2.5 VL-72B.
38
- 3. For multi-page document parsing, our method reaches a processing speed of 0.84 pages per second, surpassing MinerU (0.65) and Qwen2.5 VL-7B (0.12).
39
-
40
- <img src="https://v1.ax1x.com/2025/06/05/7jQ3cm.png" alt="7jQ3cm.png" border="0" />
41
-
42
- MonkeyOCR currently does not support photographed documents, but we will continue to improve it in future updates. Stay tuned!
43
- Currently, our model is deployed on a single GPU, so if too many users upload files at the same time, issues like “This application is currently busy” may occur. We're actively working on supporting Ollama and other deployment solutions to ensure a smoother experience for more users. Additionally, please note that the processing time shown on the demo page does not reflect computation time alone—it also includes result uploading and other overhead. During periods of high traffic, this time may be longer. The inference speeds of MonkeyOCR, MinerU, and Qwen2.5 VL-7B were measured on an H800 GPU.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
44
 
45
- 🚀🚀🚀 Chinese Video Tutorial (Thanks to [leo009](https://huggingface.co/leo009) for sharing!)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
46
 
47
- https://youtu.be/T9oaqp-IaZ0
48
 
49
- https://www.bilibili.com/video/BV1sxKhztEx1/
 
50
 
51
 
52
  ## News
53
- * ```2025.06.05 ``` 🚀 We release MonkeyOCR, which supports the parsing of various types of Chinese and English documents.
 
 
54
 
55
 
56
- ## Quick Start
57
-
58
  ### 1. Install MonkeyOCR
59
- ```bash
60
- conda create -n MonkeyOCR python=3.10
61
- conda activate MonkeyOCR
62
-
63
- git clone https://github.com/Yuliang-Liu/MonkeyOCR.git
64
- cd MonkeyOCR
65
-
66
- # Install pytorch, see https://pytorch.org/get-started/previous-versions/ for your cuda version
67
- pip install torch==2.5.1 torchvision==0.20.1 torchaudio==2.5.1 --index-url https://download.pytorch.org/whl/cu124
68
- pip install -e .
69
- ```
70
  ### 2. Download Model Weights
71
  Download our model from Huggingface.
72
  ```python
73
  pip install huggingface_hub
74
 
75
- python tools/download_model.py
76
  ```
77
  You can also download our model from ModelScope.
78
 
79
  ```python
80
  pip install modelscope
81
 
82
- python tools/download_model.py -t modelscope
83
  ```
84
  ### 3. Inference
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
85
  ```bash
86
- # Make sure in MonkeyOCR directory
87
- python parse.py path/to/your.pdf
88
- # or with image as input
89
- pyhton parse.py path/to/your/image
90
- # Specify output path and model configs path
91
- python parse.py path/to/your.pdf -o ./output -c config.yaml
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
92
  ```
93
 
94
- #### Output Results
95
- MonkeyOCR generates three types of output files:
 
 
 
 
96
 
97
  1. **Processed Markdown File** (`your.md`): The final parsed document content in markdown format, containing text, formulas, tables, and other structured elements.
98
  2. **Layout Results** (`your_layout.pdf`): The layout results drawed on origin PDF.
@@ -103,562 +312,882 @@ MonkeyOCR generates three types of output files:
103
 
104
  These files provide both the final formatted output and detailed intermediate results for further analysis or processing.
105
 
 
 
106
  ### 4. Gradio Demo
107
  ```bash
108
- # Prepare your env for gradio
109
- pip install gradio==5.23.3
110
- pip install pdf2image==1.17.0
111
  ```
 
 
 
 
112
  ```bash
113
- # Start demo
114
- python demo/demo_gradio.py
115
  ```
116
- ### Fix **shared memory error** on **RTX 3090 / 4090 / ...** GPUs (Optional)
 
 
117
 
118
- Our 3B model runs efficiently on NVIDIA RTX 3090. However, when using **LMDeploy** as the inference backend, you may encounter compatibility issues on **RTX 3090 / 4090** GPUs — particularly the following error:
119
 
120
- ```
121
- triton.runtime.errors.OutOfResources: out of resource: shared memory
122
- ```
123
 
124
- To work around this issue, you can apply the patch below:
 
 
125
 
126
- ```bash
127
- python tools/lmdeploy_patcher.py patch
128
- ```
129
 
130
- > ⚠️ **Note:** This command will modify LMDeploy's source code in your environment.
131
- > To revert the changes, simply run:
 
132
 
133
- ```bash
134
- python tools/lmdeploy_patcher.py restore
135
- ```
136
 
137
- **Special thanks to [@pineking](https://github.com/pineking) for the solution!**
 
 
138
 
139
- ### Switch inference backend (Optional)
 
 
 
 
 
 
 
 
140
 
141
- You can switch inference backend to `transformers` following the steps below:
142
 
143
- 1. Install required dependency (if not already installed):
144
  ```bash
145
- # install flash attention 2, you can download the corresponding version from https://github.com/Dao-AILab/flash-attention/releases/
146
- pip install flash-attn==2.7.4.post1 --no-build-isolation
147
  ```
148
- 2. Open the `model_configs.yaml` file
149
- 3. Set `chat_config.backend` to `transformers`
150
- 4. Adjust the `batch_size` according to your GPU's memory capacity to ensure stable performance
151
 
152
- Example configuration:
153
 
154
- ```yaml
155
- chat_config:
156
- backend: transformers
157
- batch_size: 10 # Adjust based on your available GPU memory
158
- ```
159
 
 
 
 
 
 
160
 
161
- ## Benchmark Results
162
 
 
163
 
164
- Here are the evaluation results of our model on OmniDocBench. MonkeyOCR-3B uses DocLayoutYOLO as the structure detection model, while MonkeyOCR-3B* uses our trained structure detection model with improved Chinese performance.
165
 
 
166
 
167
- ### 1. The end-to-end evaluation results of different tasks.
168
 
169
- <table style="width:100%; border-collapse:collapse; text-align:center;" border="0">
170
- <thead>
171
- <tr>
172
- <th rowspan="2">Model Type</th>
173
- <th rowspan="2">Methods</th>
174
- <th colspan="2">Overall Edit↓</th>
175
- <th colspan="2">Text Edit↓</th>
176
- <th colspan="2">Formula Edit↓</th>
177
- <th colspan="2">Formula CDM↑</th>
178
- <th colspan="2">Table TEDS↑</th>
179
- <th colspan="2">Table Edit↓</th>
180
- <th colspan="2">Read Order Edit↓</th>
181
- </tr>
182
- <tr>
183
- <th>EN</th>
184
- <th>ZH</th>
185
- <th>EN</th>
186
- <th>ZH</th>
187
- <th>EN</th>
188
- <th>ZH</th>
189
- <th>EN</th>
190
- <th>ZH</th>
191
- <th>EN</th>
192
- <th>ZH</th>
193
- <th>EN</th>
194
- <th>ZH</th>
195
- <th>EN</th>
196
- <th>ZH</th>
197
- </tr>
198
- </thead>
199
- <tbody>
200
- <tr>
201
- <td rowspan="7">Pipeline Tools</td>
202
- <td>MinerU</td>
203
- <td>0.150</td>
204
- <td>0.357</td>
205
- <td>0.061</td>
206
- <td>0.215</td>
207
- <td>0.278</td>
208
- <td>0.577</td>
209
- <td>57.3</td>
210
- <td>42.9</td>
211
- <td>78.6</td>
212
- <td>62.1</td>
213
- <td>0.180</td>
214
- <td>0.344</td>
215
- <td><strong>0.079</strong></td>
216
- <td>0.292</td>
217
- </tr>
218
- <tr>
219
- <td>Marker</td>
220
- <td>0.336</td>
221
- <td>0.556</td>
222
- <td>0.080</td>
223
- <td>0.315</td>
224
- <td>0.530</td>
225
- <td>0.883</td>
226
- <td>17.6</td>
227
- <td>11.7</td>
228
- <td>67.6</td>
229
- <td>49.2</td>
230
- <td>0.619</td>
231
- <td>0.685</td>
232
- <td>0.114</td>
233
- <td>0.340</td>
234
- </tr>
235
- <tr>
236
- <td>Mathpix</td>
237
- <td>0.191</td>
238
- <td>0.365</td>
239
- <td>0.105</td>
240
- <td>0.384</td>
241
- <td>0.306</td>
242
- <td><strong>0.454</strong></td>
243
- <td>62.7</td>
244
- <td><strong>62.1</strong></td>
245
- <td>77.0</td>
246
- <td>67.1</td>
247
- <td>0.243</td>
248
- <td>0.320</td>
249
- <td>0.108</td>
250
- <td>0.304</td>
251
- </tr>
252
- <tr>
253
- <td>Docling</td>
254
- <td>0.589</td>
255
- <td>0.909</td>
256
- <td>0.416</td>
257
- <td>0.987</td>
258
- <td>0.999</td>
259
- <td>1</td>
260
- <td>-</td>
261
- <td>-</td>
262
- <td>61.3</td>
263
- <td>25.0</td>
264
- <td>0.627</td>
265
- <td>0.810</td>
266
- <td>0.313</td>
267
- <td>0.837</td>
268
- </tr>
269
- <tr>
270
- <td>Pix2Text</td>
271
- <td>0.320</td>
272
- <td>0.528</td>
273
- <td>0.138</td>
274
- <td>0.356</td>
275
- <td>0.276</td>
276
- <td>0.611</td>
277
- <td>78.4</td>
278
- <td>39.6</td>
279
- <td>73.6</td>
280
- <td>66.2</td>
281
- <td>0.584</td>
282
- <td>0.645</td>
283
- <td>0.281</td>
284
- <td>0.499</td>
285
- </tr>
286
- <tr>
287
- <td>Unstructured</td>
288
- <td>0.586</td>
289
- <td>0.716</td>
290
- <td>0.198</td>
291
- <td>0.481</td>
292
- <td>0.999</td>
293
- <td>1</td>
294
- <td>-</td>
295
- <td>-</td>
296
- <td>0</td>
297
- <td>0.06</td>
298
- <td>1</td>
299
- <td>0.998</td>
300
- <td>0.145</td>
301
- <td>0.387</td>
302
- </tr>
303
- <tr>
304
- <td>OpenParse</td>
305
- <td>0.646</td>
306
- <td>0.814</td>
307
- <td>0.681</td>
308
- <td>0.974</td>
309
- <td>0.996</td>
310
- <td>1</td>
311
- <td>0.11</td>
312
- <td>0</td>
313
- <td>64.8</td>
314
- <td>27.5</td>
315
- <td>0.284</td>
316
- <td>0.639</td>
317
- <td>0.595</td>
318
- <td>0.641</td>
319
- </tr>
320
- <tr>
321
- <td rowspan="5">Expert VLMs</td>
322
- <td>GOT-OCR</td>
323
- <td>0.287</td>
324
- <td>0.411</td>
325
- <td>0.189</td>
326
- <td>0.315</td>
327
- <td>0.360</td>
328
- <td>0.528</td>
329
- <td>74.3</td>
330
- <td>45.3</td>
331
- <td>53.2</td>
332
- <td>47.2</td>
333
- <td>0.459</td>
334
- <td>0.520</td>
335
- <td>0.141</td>
336
- <td>0.280</td>
337
- </tr>
338
- <tr>
339
- <td>Nougat</td>
340
- <td>0.452</td>
341
- <td>0.973</td>
342
- <td>0.365</td>
343
- <td>0.998</td>
344
- <td>0.488</td>
345
- <td>0.941</td>
346
- <td>15.1</td>
347
- <td>16.8</td>
348
- <td>39.9</td>
349
- <td>0</td>
350
- <td>0.572</td>
351
- <td>1.000</td>
352
- <td>0.382</td>
353
- <td>0.954</td>
354
- </tr>
355
- <tr>
356
- <td>Mistral OCR</td>
357
- <td>0.268</td>
358
- <td>0.439</td>
359
- <td>0.072</td>
360
- <td>0.325</td>
361
- <td>0.318</td>
362
- <td>0.495</td>
363
- <td>64.6</td>
364
- <td>45.9</td>
365
- <td>75.8</td>
366
- <td>63.6</td>
367
- <td>0.600</td>
368
- <td>0.650</td>
369
- <td>0.083</td>
370
- <td>0.284</td>
371
- </tr>
372
- <tr>
373
- <td>OLMOCR-sglang</td>
374
- <td>0.326</td>
375
- <td>0.469</td>
376
- <td>0.097</td>
377
- <td>0.293</td>
378
- <td>0.455</td>
379
- <td>0.655</td>
380
- <td>74.3</td>
381
- <td>43.2</td>
382
- <td>68.1</td>
383
- <td>61.3</td>
384
- <td>0.608</td>
385
- <td>0.652</td>
386
- <td>0.145</td>
387
- <td>0.277</td>
388
- </tr>
389
- <tr>
390
- <td>SmolDocling-256M</td>
391
- <td>0.493</td>
392
- <td>0.816</td>
393
- <td>0.262</td>
394
- <td>0.838</td>
395
- <td>0.753</td>
396
- <td>0.997</td>
397
- <td>32.1</td>
398
- <td>0.55</td>
399
- <td>44.9</td>
400
- <td>16.5</td>
401
- <td>0.729</td>
402
- <td>0.907</td>
403
- <td>0.227</td>
404
- <td>0.522</td>
405
- </tr>
406
- <tr>
407
- <td rowspan="3">General VLMs</td>
408
- <td>GPT4o</td>
409
- <td>0.233</td>
410
- <td>0.399</td>
411
- <td>0.144</td>
412
- <td>0.409</td>
413
- <td>0.425</td>
414
- <td>0.606</td>
415
- <td>72.8</td>
416
- <td>42.8</td>
417
- <td>72.0</td>
418
- <td>62.9</td>
419
- <td>0.234</td>
420
- <td>0.329</td>
421
- <td>0.128</td>
422
- <td>0.251</td>
423
- </tr>
424
- <tr>
425
- <td>Qwen2.5-VL-7B</td>
426
- <td>0.312</td>
427
- <td>0.406</td>
428
- <td>0.157</td>
429
- <td>0.228</td>
430
- <td>0.351</td>
431
- <td>0.574</td>
432
- <td><strong>79.0</strong></td>
433
- <td>50.2</td>
434
- <td>76.4</td>
435
- <td>72.2</td>
436
- <td>0.588</td>
437
- <td>0.619</td>
438
- <td>0.149</td>
439
- <td>0.203</td>
440
- </tr>
441
- <tr>
442
- <td>InternVL3-8B</td>
443
- <td>0.314</td>
444
- <td>0.383</td>
445
- <td>0.134</td>
446
- <td>0.218</td>
447
- <td>0.417</td>
448
- <td>0.563</td>
449
- <td>78.3</td>
450
- <td>49.3</td>
451
- <td>66.1</td>
452
- <td>73.1</td>
453
- <td>0.586</td>
454
- <td>0.564</td>
455
- <td>0.118</td>
456
- <td>0.186</td>
457
- </tr>
458
- <tr>
459
- <td rowspan="2">Mix</td>
460
- <td>MonkeyOCR-3B <a href="https://huggingface.co/echo840/MonkeyOCR/blob/main/Structure/doclayout_yolo_docstructbench_imgsz1280_2501.pt">[Weight]</a></td>
461
- <td><strong>0.140</strong></td>
462
- <td>0.297</td>
463
- <td><strong>0.058</strong></td>
464
- <td>0.185</td>
465
- <td><strong>0.238</strong></td>
466
- <td>0.506</td>
467
- <td>78.7</td>
468
- <td>51.4</td>
469
- <td><strong>80.2</strong></td>
470
- <td><strong>77.7</strong></td>
471
- <td><strong>0.170</strong></td>
472
- <td><strong>0.253</strong></td>
473
- <td>0.093</td>
474
- <td>0.244</td>
475
- </tr>
476
- <tr>
477
- <td>MonkeyOCR-3B* <a href="https://huggingface.co/echo840/MonkeyOCR/blob/main/Structure/layout_zh.pt">[Weight]</a></td>
478
- <td>0.154</td>
479
- <td><strong>0.277</strong></td>
480
- <td>0.073</td>
481
- <td><strong>0.134</strong></td>
482
- <td>0.255</td>
483
- <td>0.529</td>
484
- <td>78.5</td>
485
- <td>50.8</td>
486
- <td>78.2</td>
487
- <td>76.2</td>
488
- <td>0.182</td>
489
- <td>0.262</td>
490
- <td>0.105</td>
491
- <td><strong>0.183</strong></td>
492
- </tr>
493
- </tbody>
494
- </table>
495
 
 
496
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
497
 
498
 
499
  ### 2. The end-to-end text recognition performance across 9 PDF page types.
500
- <table style="width: 100%; border-collapse: collapse; text-align: center;">
501
- <thead>
502
- <tr style="border-bottom: 2px solid #000;">
503
- <th><b>Model Type</b></th>
504
- <th><b>Models</b></th>
505
- <th><b>Book</b></th>
506
- <th><b>Slides</b></th>
507
- <th><b>Financial Report</b></th>
508
- <th><b>Textbook</b></th>
509
- <th><b>Exam Paper</b></th>
510
- <th><b>Magazine</b></th>
511
- <th><b>Academic Papers</b></th>
512
- <th><b>Notes</b></th>
513
- <th><b>Newspaper</b></th>
514
- <th><b>Overall</b></th>
515
- </tr>
516
- </thead>
517
- <tbody>
518
- <tr>
519
- <td rowspan="3"><b>Pipeline Tools</b></td>
520
- <td>MinerU</td>
521
- <td><u>0.055</u></td>
522
- <td>0.124</td>
523
- <td><u>0.033</u></td>
524
- <td><u>0.102</u></td>
525
- <td><u>0.159</u></td>
526
- <td><b>0.072</b></td>
527
- <td><u>0.025</u></td>
528
- <td>0.984</td>
529
- <td>0.171</td>
530
- <td>0.206</td>
531
- </tr>
532
- <tr>
533
- <td>Marker</td>
534
- <td>0.074</td>
535
- <td>0.340</td>
536
- <td>0.089</td>
537
- <td>0.319</td>
538
- <td>0.452</td>
539
- <td>0.153</td>
540
- <td>0.059</td>
541
- <td>0.651</td>
542
- <td>0.192</td>
543
- <td>0.274</td>
544
- </tr>
545
- <tr>
546
- <td>Mathpix</td>
547
- <td>0.131</td>
548
- <td>0.220</td>
549
- <td>0.202</td>
550
- <td>0.216</td>
551
- <td>0.278</td>
552
- <td>0.147</td>
553
- <td>0.091</td>
554
- <td>0.634</td>
555
- <td>0.690</td>
556
- <td>0.300</td>
557
- </tr>
558
- <tr>
559
- <td rowspan="2"><b>Expert VLMs</b></td>
560
- <td>GOT-OCR</td>
561
- <td>0.111</td>
562
- <td>0.222</td>
563
- <td>0.067</td>
564
- <td>0.132</td>
565
- <td>0.204</td>
566
- <td>0.198</td>
567
- <td>0.179</td>
568
- <td>0.388</td>
569
- <td>0.771</td>
570
- <td>0.267</td>
571
- </tr>
572
- <tr>
573
- <td>Nougat</td>
574
- <td>0.734</td>
575
- <td>0.958</td>
576
- <td>1.000</td>
577
- <td>0.820</td>
578
- <td>0.930</td>
579
- <td>0.830</td>
580
- <td>0.214</td>
581
- <td>0.991</td>
582
- <td>0.871</td>
583
- <td>0.806</td>
584
- </tr>
585
- <tr>
586
- <td rowspan="3"><b>General VLMs</b></td>
587
- <td>GPT4o</td>
588
- <td>0.157</td>
589
- <td>0.163</td>
590
- <td>0.348</td>
591
- <td>0.187</td>
592
- <td>0.281</td>
593
- <td>0.173</td>
594
- <td>0.146</td>
595
- <td>0.607</td>
596
- <td>0.751</td>
597
- <td>0.316</td>
598
- </tr>
599
- <tr>
600
- <td>Qwen2.5-VL-7B</td>
601
- <td>0.148</td>
602
- <td><b>0.053</b></td>
603
- <td>0.111</td>
604
- <td>0.137</td>
605
- <td>0.189</td>
606
- <td>0.117</td>
607
- <td>0.134</td>
608
- <td>0.204</td>
609
- <td>0.706</td>
610
- <td>0.205</td>
611
- </tr>
612
- <tr>
613
- <td>InternVL3-8B</td>
614
- <td>0.163</td>
615
- <td><u>0.056</u></td>
616
- <td>0.107</td>
617
- <td>0.109</td>
618
- <td><b>0.129</b></td>
619
- <td>0.100</td>
620
- <td>0.159</td>
621
- <td><b>0.150</b></td>
622
- <td>0.681</td>
623
- <td>0.188</td>
624
- </tr>
625
- <tr>
626
- <td rowspan="2"><b>Mix</b></td>
627
- <td>MonkeyOCR-3B <a href="https://huggingface.co/echo840/MonkeyOCR/blob/main/Structure/doclayout_yolo_docstructbench_imgsz1280_2501.pt">[Weight]</a></td>
628
- <td><b>0.046</b></td>
629
- <td>0.120</td>
630
- <td><b>0.024</b></td>
631
- <td><b>0.100</b></td>
632
- <td><b>0.129</b></td>
633
- <td><u>0.086</u></td>
634
- <td><b>0.024</b></td>
635
- <td>0.643</td>
636
- <td><b>0.131</b></td>
637
- <td><u>0.155</u></td>
638
- </tr>
639
- <tr>
640
- <td>MonkeyOCR-3B* <a href="https://huggingface.co/echo840/MonkeyOCR/blob/main/Structure/layout_zh.pt">[Weight]</a></td>
641
- <td>0.054</td>
642
- <td>0.203</td>
643
- <td>0.038</td>
644
- <td>0.112</td>
645
- <td>0.138</td>
646
- <td>0.111</td>
647
- <td>0.032</td>
648
- <td><u>0.194</u></td>
649
- <td><u>0.136</u></td>
650
- <td><b>0.120</b></td>
651
- </tr>
652
- </tbody>
653
- </table>
654
 
655
- ### 3. Comparing MonkeyOCR with closed-source and extra large open-source VLMs.
656
- <img src="https://v1.ax1x.com/2025/06/05/7jQlj4.png" alt="7jQlj4.png" border="0" />
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
657
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
658
 
659
  ## Visualization Demo
660
 
661
- Get a Quick Hands-On Experience with Our Demo: http://vlrlabmonkey.xyz:7685
662
 
663
  > Our demo is simple and easy to use:
664
  >
@@ -701,8 +1230,10 @@ If you wish to refer to the baseline results published here, please use the foll
701
 
702
 
703
  ## Acknowledgments
704
- We would like to thank [MinerU](https://github.com/opendatalab/MinerU), [DocLayout-YOLO](https://github.com/opendatalab/DocLayout-YOLO), [PyMuPDF](https://github.com/pymupdf/PyMuPDF), [layoutreader](https://github.com/ppaanngggg/layoutreader), [Qwen2.5-VL](https://github.com/QwenLM/Qwen2.5-VL), [LMDeploy](https://github.com/InternLM/lmdeploy), and [InternVL3](https://github.com/OpenGVLab/InternVL) for providing base code and models, as well as their contributions to this field. We also thank [M6Doc](https://github.com/HCIILAB/M6Doc), [DocLayNet](https://github.com/DS4SD/DocLayNet), [CDLA](https://github.com/buptlihang/CDLA), [D4LA](https://github.com/AlibabaResearch/AdvancedLiterateMachinery), [DocGenome](https://github.com/Alpha-Innovator/DocGenome), [PubTabNet](https://github.com/ibm-aur-nlp/PubTabNet), and [UniMER-1M](https://github.com/opendatalab/UniMERNet) for providing valuable datasets.
705
 
 
 
706
 
707
  ## Copyright
708
- Please don’t hesitate to share your valuable feedback — it’s a key motivation that drives us to continuously improve our framework. The current technical report only presents the results of the 3B model. Our model is intended for non-commercial use. If you are interested in larger one, please contact us at [email protected] or [email protected].
 
16
  [![HuggingFace](https://img.shields.io/badge/HuggingFace%20Weights-black.svg?logo=HuggingFace)](https://huggingface.co/echo840/MonkeyOCR)
17
  [![GitHub issues](https://img.shields.io/github/issues/Yuliang-Liu/MonkeyOCR?color=critical&label=Issues)](https://github.com/Yuliang-Liu/MonkeyOCR/issues?q=is%3Aopen+is%3Aissue)
18
  [![GitHub closed issues](https://img.shields.io/github/issues-closed/Yuliang-Liu/MonkeyOCR?color=success&label=Issues)](https://github.com/Yuliang-Liu/MonkeyOCR/issues?q=is%3Aissue+is%3Aclosed)
19
+ [![License](https://img.shields.io/badge/License-Apache%202.0-yellow)](https://github.com/Yuliang-Liu/MonkeyOCR/blob/main/LICENSE.txt)
20
  [![GitHub views](https://komarev.com/ghpvc/?username=Yuliang-Liu&repo=MonkeyOCR&color=brightgreen&label=Views)](https://github.com/Yuliang-Liu/MonkeyOCR)
21
  </div>
22
 
 
24
  > **MonkeyOCR: Document Parsing with a Structure-Recognition-Relation Triplet Paradigm**<br>
25
  > Zhang Li, Yuliang Liu, Qiang Liu, Zhiyin Ma, Ziyang Zhang, Shuo Zhang, Zidun Guo, Jiarui Zhang, Xinyu Wang, Xiang Bai <br>
26
  [![arXiv](https://img.shields.io/badge/Arxiv-b31b1b.svg?logo=arXiv)](https://arxiv.org/abs/2506.05218)
27
+ [![Source_code](https://img.shields.io/badge/Code-Available-white)](README.md)
28
  [![Model Weight](https://img.shields.io/badge/HuggingFace-gray)](https://huggingface.co/echo840/MonkeyOCR)
29
  [![Model Weight](https://img.shields.io/badge/ModelScope-green)](https://modelscope.cn/models/l1731396519/MonkeyOCR)
30
+ [![Public Courses](https://img.shields.io/badge/Openbayes-yellow)](https://openbayes.com/console/public/tutorials/91ESrGvEvBq)
31
  [![Demo](https://img.shields.io/badge/Demo-blue)](http://vlrlabmonkey.xyz:7685/)
32
 
33
 
 
35
  ## Introduction
36
  MonkeyOCR adopts a Structure-Recognition-Relation (SRR) triplet paradigm, which simplifies the multi-tool pipeline of modular approaches while avoiding the inefficiency of using large multimodal models for full-page document processing.
37
 
38
+ 1. MonkeyOCR-pro-1.2B surpasses MonkeyOCR-3B by 7.4% on Chinese documents.
39
+ 2. MonkeyOCR-pro-1.2B delivers approximately a 36% speed improvement over MonkeyOCR-pro-3B, with approximately 1.6% drop in performance.
40
+ 3. On olmOCR-Bench, MonkeyOCR-pro-1.2B outperforms Nanonets-OCR-3B by 7.3%.
41
+ 4. On OmniDocBench, MonkeyOCR-pro-3B achieves the best overall performance on both English and Chinese documents, outperforming even closed-source and extra-large open-source VLMs such as Gemini 2.0-Flash, Gemini 2.5-Pro, Qwen2.5-VL-72B, GPT-4o, and InternVL3-78B.
42
+
43
+ See detailed results below.
44
+
45
+ ### Comparing MonkeyOCR with closed-source and extra large open-source VLMs.
46
+ <a href="https://zimgs.com/i/EKhkhY"><img src="https://v1.ax1x.com/2025/07/15/EKhkhY.png" alt="EKhkhY.png" border="0" /></a>
47
+
48
+ ## Inference Speed (Pages/s) on Different GPUs and [PDF](https://drive.google.com/drive/folders/1geumlJmVY7UUKdr8324sYZ0FHSAElh7m?usp=sharing) Page Counts
49
+
50
+ <table>
51
+ <thead>
52
+ <tr align='center'>
53
+ <th>Model</th>
54
+ <th>GPU</th>
55
+ <th>50 Pages</th>
56
+ <th>100 Pages</th>
57
+ <th>300 Pages</th>
58
+ <th>500 Pages</th>
59
+ <th>1000 Pages</th>
60
+ </tr>
61
+ </thead>
62
+ <tbody>
63
+ <tr align='center'>
64
+ <td rowspan='4'>MonkeyOCR-pro-3B</td>
65
+ <td>3090</td>
66
+ <td>0.492</td>
67
+ <td>0.484</td>
68
+ <td>0.497</td>
69
+ <td>0.492</td>
70
+ <td>0.496</td>
71
+ </tr>
72
+ <tr align='center'>
73
+ <td>A6000</td>
74
+ <td>0.585</td>
75
+ <td>0.587</td>
76
+ <td>0.609</td>
77
+ <td>0.598</td>
78
+ <td>0.608</td>
79
+ </tr>
80
+ <tr align='center'>
81
+ <td>H800</td>
82
+ <td>0.923</td>
83
+ <td>0.768</td>
84
+ <td>0.897</td>
85
+ <td>0.930</td>
86
+ <td>0.891</td>
87
+ </tr>
88
+ <tr align='center'>
89
+ <td>4090</td>
90
+ <td>0.972</td>
91
+ <td>0.969</td>
92
+ <td>1.006</td>
93
+ <td>0.986</td>
94
+ <td>1.006</td>
95
+ </tr>
96
+ <tr align='center'>
97
+ <td rowspan='4'>MonkeyOCR-pro-1.2B</td>
98
+ <td>3090</td>
99
+ <td>0.615</td>
100
+ <td>0.660</td>
101
+ <td>0.677</td>
102
+ <td>0.687</td>
103
+ <td>0.683</td>
104
+ </tr>
105
+ <tr align='center'>
106
+ <td>A6000</td>
107
+ <td>0.709</td>
108
+ <td>0.786</td>
109
+ <td>0.825</td>
110
+ <td>0.829</td>
111
+ <td>0.825</td>
112
+ </tr>
113
+ <tr align='center'>
114
+ <td>H800</td>
115
+ <td>0.965</td>
116
+ <td>1.082</td>
117
+ <td>1.101</td>
118
+ <td>1.145</td>
119
+ <td>1.015</td>
120
+ </tr>
121
+ <tr align='center'>
122
+ <td>4090</td>
123
+ <td>1.194</td>
124
+ <td>1.314</td>
125
+ <td>1.436</td>
126
+ <td>1.442</td>
127
+ <td>1.434</td>
128
+ </tr>
129
+ </tbody>
130
+ </table>
131
 
132
+ ## VLM OCR Speed (Pages/s) on Different GPUs and [PDF](https://drive.google.com/drive/folders/1geumlJmVY7UUKdr8324sYZ0FHSAElh7m?usp=sharing) Page Counts
133
+
134
+ <table>
135
+ <thead>
136
+ <tr align='center'>
137
+ <th>Model</th>
138
+ <th>GPU</th>
139
+ <th>50 Pages</th>
140
+ <th>100 Pages</th>
141
+ <th>300 Pages</th>
142
+ <th>500 Pages</th>
143
+ <th>1000 Pages</th>
144
+ </tr>
145
+ </thead>
146
+ <tbody>
147
+ <tr align='center'>
148
+ <td rowspan='4'>MonkeyOCR-pro-3B</td>
149
+ <td>3090</td>
150
+ <td>0.705</td>
151
+ <td>0.680</td>
152
+ <td>0.711</td>
153
+ <td>0.700</td>
154
+ <td>0.724</td>
155
+ </tr>
156
+ <tr align='center'>
157
+ <td>A6000</td>
158
+ <td>0.885</td>
159
+ <td>0.860</td>
160
+ <td>0.915</td>
161
+ <td>0.892</td>
162
+ <td>0.934</td>
163
+ </tr>
164
+ <tr align='center'>
165
+ <td>H800</td>
166
+ <td>1.371</td>
167
+ <td>1.135</td>
168
+ <td>1.339</td>
169
+ <td>1.433</td>
170
+ <td>1.509</td>
171
+ </tr>
172
+ <tr align='center'>
173
+ <td>4090</td>
174
+ <td>1.321</td>
175
+ <td>1.300</td>
176
+ <td>1.384</td>
177
+ <td>1.343</td>
178
+ <td>1.410</td>
179
+ </tr>
180
+ <tr align='center'>
181
+ <td rowspan='4'>MonkeyOCR-pro-1.2B</td>
182
+ <td>3090</td>
183
+ <td>0.919</td>
184
+ <td>1.086</td>
185
+ <td>1.166</td>
186
+ <td>1.182</td>
187
+ <td>1.199</td>
188
+ </tr>
189
+ <tr align='center'>
190
+ <td>A6000</td>
191
+ <td>1.177</td>
192
+ <td>1.361</td>
193
+ <td>1.506</td>
194
+ <td>1.525</td>
195
+ <td>1.569</td>
196
+ </tr>
197
+ <tr align='center'>
198
+ <td>H800</td>
199
+ <td>1.466</td>
200
+ <td>1.719</td>
201
+ <td>1.763</td>
202
+ <td>1.875</td>
203
+ <td>1.650</td>
204
+ </tr>
205
+ <tr align='center'>
206
+ <td>4090</td>
207
+ <td>1.759</td>
208
+ <td>1.987</td>
209
+ <td>2.260</td>
210
+ <td>2.345</td>
211
+ <td>2.415</td>
212
+ </tr>
213
+ </tbody>
214
+ </table>
215
 
 
216
 
217
+ ## Supported Hardware
218
+ Due to the limited types of GPUs available to us, we may not be able to provide highly accurate hardware specifications. We've tested the model on GPUs such as the 3090, 4090, A6000, H800, A100, and even the 4060 with 8GB of VRAM (suitable for deploying quantized 3B model and 1.2B model). We are very grateful for the feedback and contributions from the open-source community, who have also successfully run the model on [50-series GPUs](https://github.com/Yuliang-Liu/MonkeyOCR/issues/90), [H200](https://github.com/Yuliang-Liu/MonkeyOCR/issues/151), [L20](https://github.com/Yuliang-Liu/MonkeyOCR/issues/133), [V100](https://github.com/Yuliang-Liu/MonkeyOCR/issues/144), [2080 Ti](https://github.com/Yuliang-Liu/MonkeyOCR/pull/1) and [npu](https://github.com/Yuliang-Liu/MonkeyOCR/pull/226/files).
219
 
220
 
221
  ## News
222
+ * ```2025.07.10 ``` 🚀 We release [MonkeyOCR-pro-1.2B](https://huggingface.co/echo840/MonkeyOCR-pro-1.2B), a leaner and faster version model that outperforms our previous 3B version in accuracy, speed, and efficiency.
223
+ * ```2025.06.12 ``` 🚀 The model’s trending on [Hugging Face](https://huggingface.co/models?sort=trending). Thanks for the love!
224
+ * ```2025.06.05 ``` 🚀 We release [MonkeyOCR](https://huggingface.co/echo840/MonkeyOCR), an English and Chinese documents parsing model.
225
 
226
 
227
+ # Quick Start
228
+ ## Locally Install
229
  ### 1. Install MonkeyOCR
230
+ See the [installation guide](https://github.com/Yuliang-Liu/MonkeyOCR/blob/main/docs/install_cuda_pp.md#install-with-cuda-support) to set up your environment.
 
 
 
 
 
 
 
 
 
 
231
  ### 2. Download Model Weights
232
  Download our model from Huggingface.
233
  ```python
234
  pip install huggingface_hub
235
 
236
+ python tools/download_model.py -n MonkeyOCR-pro-3B # or MonkeyOCR
237
  ```
238
  You can also download our model from ModelScope.
239
 
240
  ```python
241
  pip install modelscope
242
 
243
+ python tools/download_model.py -t modelscope -n MonkeyOCR-pro-3B # or MonkeyOCR
244
  ```
245
  ### 3. Inference
246
+ You can parse a file or a directory containing PDFs or images using the following commands:
247
+ ```bash
248
+ # Replace input_path with the path to a PDF or image or directory
249
+
250
+ # End-to-end parsing
251
+ python parse.py input_path
252
+
253
+ # Parse files in a dir with specific group page num
254
+ python parse.py input_path -g 20
255
+
256
+ # Single-task recognition (outputs markdown only)
257
+ python parse.py input_path -t text/formula/table
258
+
259
+ # Parse PDFs in input_path and split results by pages
260
+ python parse.py input_path -s
261
+
262
+ # Specify output directory and model config file
263
+ python parse.py input_path -o ./output -c config.yaml
264
+ ```
265
+
266
+ <details>
267
+ <summary><b>More usage examples</b></summary>
268
+
269
  ```bash
270
+ # Single file processing
271
+ python parse.py input.pdf # Parse single PDF file
272
+ python parse.py input.pdf -o ./output # Parse with custom output dir
273
+ python parse.py input.pdf -s # Parse PDF with page splitting
274
+ python parse.py image.jpg # Parse single image file
275
+
276
+ # Single task recognition
277
+ python parse.py image.jpg -t text # Text recognition from image
278
+ python parse.py image.jpg -t formula # Formula recognition from image
279
+ python parse.py image.jpg -t table # Table recognition from image
280
+ python parse.py document.pdf -t text # Text recognition from all PDF pages
281
+
282
+ # Folder processing (all files individually)
283
+ python parse.py /path/to/folder # Parse all files in folder
284
+ python parse.py /path/to/folder -s # Parse with page splitting
285
+ python parse.py /path/to/folder -t text # Single task recognition for all files
286
+
287
+ # Multi-file grouping (batch processing by page count)
288
+ python parse.py /path/to/folder -g 5 # Group files with max 5 total pages
289
+ python parse.py /path/to/folder -g 10 -s # Group files with page splitting
290
+ python parse.py /path/to/folder -g 8 -t text # Group files for single task recognition
291
+
292
+ # Advanced configurations
293
+ python parse.py input.pdf -c model_configs.yaml # Custom model configuration
294
+ python parse.py /path/to/folder -g 15 -s -o ./out # Group files, split pages, custom output
295
+ python parse.py input.pdf --pred-abandon # Enable predicting abandon elements
296
+ python parse.py /path/to/folder -g 10 -m # Group files and merge text blocks in output
297
  ```
298
 
299
+ </details>
300
+
301
+ <details>
302
+ <summary><b>Output Results</b></summary>
303
+
304
+ MonkeyOCR mainly generates three types of output files:
305
 
306
  1. **Processed Markdown File** (`your.md`): The final parsed document content in markdown format, containing text, formulas, tables, and other structured elements.
307
  2. **Layout Results** (`your_layout.pdf`): The layout results drawed on origin PDF.
 
312
 
313
  These files provide both the final formatted output and detailed intermediate results for further analysis or processing.
314
 
315
+ </details>
316
+
317
  ### 4. Gradio Demo
318
  ```bash
319
+ python demo/demo_gradio.py
 
 
320
  ```
321
+ Once the demo is running, you can access it at http://localhost:7860.
322
+
323
+ ### 5. Fast API
324
+ You can start the MonkeyOCR FastAPI service with the following command:
325
  ```bash
326
+ uvicorn api.main:app --port 8000
 
327
  ```
328
+ Once the API service is running, you can access the API documentation at http://localhost:8000/docs to explore available endpoints.
329
+ > [!TIP]
330
+ > To improve API concurrency performance, consider configuring the inference backend as `lmdeploy_queue` or `vllm_queue`.
331
 
332
+ ## Docker Deployment
333
 
334
+ 1. Navigate to the `docker` directory:
 
 
335
 
336
+ ```bash
337
+ cd docker
338
+ ```
339
 
340
+ 2. **Prerequisite:** Ensure NVIDIA GPU support is available in Docker (via `nvidia-docker2`).
341
+ If GPU support is not enabled, run the following to set up the environment:
 
342
 
343
+ ```bash
344
+ bash env.sh
345
+ ```
346
 
347
+ 3. Build the Docker image:
 
 
348
 
349
+ ```bash
350
+ docker compose build monkeyocr
351
+ ```
352
 
353
+ > [!IMPORTANT]
354
+ >
355
+ > If your GPU is from the 20/30/40-series, V100, L20/L40 or similar, please build the patched Docker image for LMDeploy compatibility:
356
+ >
357
+ > ```bash
358
+ > docker compose build monkeyocr-fix
359
+ > ```
360
+ >
361
+ > Otherwise, you may encounter the following error: `triton.runtime.errors.OutOfResources: out of resource: shared memory`
362
 
363
+ 4. Run the container with the Gradio demo (accessible on port 7860):
364
 
 
365
  ```bash
366
+ docker compose up monkeyocr-demo
 
367
  ```
 
 
 
368
 
369
+ Alternatively, start an interactive development environment:
370
 
371
+ ```bash
372
+ docker compose run --rm monkeyocr-dev
373
+ ```
 
 
374
 
375
+ 5. Run the FastAPI service (accessible on port 7861):
376
+ ```bash
377
+ docker compose up monkeyocr-api
378
+ ```
379
+ Once the API service is running, you can access the API documentation at http://localhost:7861/docs to explore available endpoints.
380
 
381
+ ## Windows Support
382
 
383
+ See the [windows support guide](docs/windows_support.md) for details.
384
 
385
+ ## Quantization
386
 
387
+ This model can be quantized using AWQ. Follow the instructions in the [quantization guide](docs/Quantization.md).
388
 
389
+ ## Benchmark Results
390
 
391
+ Here are the evaluation results of our model on OmniDocBench. MonkeyOCR-3B uses DocLayoutYOLO as the structure detection model, while MonkeyOCR-3B* uses our trained structure detection model with improved Chinese performance.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
392
 
393
+ ### 1. The end-to-end evaluation results of different tasks.
394
 
395
+ <table>
396
+ <thead>
397
+ <tr>
398
+ <th rowspan="2"><strong>Model<br>Type</strong></th>
399
+ <th rowspan="2"><strong>Methods</strong></th>
400
+ <th colspan="2"><strong>Overall<sup>Edit</sup>↓</strong></th>
401
+ <th colspan="2"><strong>Text<sup>Edit</sup>↓</strong></th>
402
+ <th colspan="2"><strong>Formula<sup>Edit</sup>↓</strong></th>
403
+ <th colspan="2"><strong>Table<sup>TEDS</sup>↑</strong></th>
404
+ <th colspan="2"><strong>Table<sup>Edit</sup>↓</strong></th>
405
+ <th colspan="2"><strong>Read Order<sup>Edit</sup>↓</strong></th>
406
+ </tr>
407
+ <tr>
408
+ <th><em>EN</em></th>
409
+ <th><em>ZH</em></th>
410
+ <th><em>EN</em></th>
411
+ <th><em>ZH</em></th>
412
+ <th><em>EN</em></th>
413
+ <th><em>ZH</em></th>
414
+ <th><em>EN</em></th>
415
+ <th><em>ZH</em></th>
416
+ <th><em>EN</em></th>
417
+ <th><em>ZH</em></th>
418
+ <th><em>EN</em></th>
419
+ <th><em>ZH</em></th>
420
+ </tr>
421
+ </thead>
422
+ <tbody>
423
+ <tr>
424
+ <td rowspan="8"><strong>Pipeline<br>Tools</strong></td>
425
+ <td>MinerU</td>
426
+ <td>0.150</td>
427
+ <td>0.357</td>
428
+ <td>0.061</td>
429
+ <td>0.215</td>
430
+ <td>0.278</td>
431
+ <td>0.577</td>
432
+ <td>78.6</td>
433
+ <td>62.1</td>
434
+ <td>0.180</td>
435
+ <td>0.344</td>
436
+ <td>0.079</td>
437
+ <td>0.292</td>
438
+ </tr>
439
+ <tr>
440
+ <td>Marker</td>
441
+ <td>0.336</td>
442
+ <td>0.556</td>
443
+ <td>0.080</td>
444
+ <td>0.315</td>
445
+ <td>0.530</td>
446
+ <td>0.883</td>
447
+ <td>67.6</td>
448
+ <td>49.2</td>
449
+ <td>0.619</td>
450
+ <td>0.685</td>
451
+ <td>0.114</td>
452
+ <td>0.340</td>
453
+ </tr>
454
+ <tr>
455
+ <td>Mathpix</td>
456
+ <td>0.191</td>
457
+ <td>0.365</td>
458
+ <td>0.105</td>
459
+ <td>0.384</td>
460
+ <td>0.306</td>
461
+ <td><strong>0.454</strong></td>
462
+ <td>77.0</td>
463
+ <td>67.1</td>
464
+ <td>0.243</td>
465
+ <td>0.320</td>
466
+ <td>0.108</td>
467
+ <td>0.304</td>
468
+ </tr>
469
+ <tr>
470
+ <td>Docling</td>
471
+ <td>0.589</td>
472
+ <td>0.909</td>
473
+ <td>0.416</td>
474
+ <td>0.987</td>
475
+ <td>0.999</td>
476
+ <td>1</td>
477
+ <td>61.3</td>
478
+ <td>25.0</td>
479
+ <td>0.627</td>
480
+ <td>0.810</td>
481
+ <td>0.313</td>
482
+ <td>0.837</td>
483
+ </tr>
484
+ <tr>
485
+ <td>Pix2Text</td>
486
+ <td>0.320</td>
487
+ <td>0.528</td>
488
+ <td>0.138</td>
489
+ <td>0.356</td>
490
+ <td>0.276</td>
491
+ <td>0.611</td>
492
+ <td>73.6</td>
493
+ <td>66.2</td>
494
+ <td>0.584</td>
495
+ <td>0.645</td>
496
+ <td>0.281</td>
497
+ <td>0.499</td>
498
+ </tr>
499
+ <tr>
500
+ <td>Unstructured</td>
501
+ <td>0.586</td>
502
+ <td>0.716</td>
503
+ <td>0.198</td>
504
+ <td>0.481</td>
505
+ <td>0.999</td>
506
+ <td>1</td>
507
+ <td>0</td>
508
+ <td>0.06</td>
509
+ <td>1</td>
510
+ <td>0.998</td>
511
+ <td>0.145</td>
512
+ <td>0.387</td>
513
+ </tr>
514
+ <tr>
515
+ <td>OpenParse</td>
516
+ <td>0.646</td>
517
+ <td>0.814</td>
518
+ <td>0.681</td>
519
+ <td>0.974</td>
520
+ <td>0.996</td>
521
+ <td>1</td>
522
+ <td>64.8</td>
523
+ <td>27.5</td>
524
+ <td>0.284</td>
525
+ <td>0.639</td>
526
+ <td>0.595</td>
527
+ <td>0.641</td>
528
+ </tr>
529
+ <tr>
530
+ <td>PP-StructureV3</td>
531
+ <td>0.145</td>
532
+ <td><strong>0.206</strong></td>
533
+ <td>0.058</td>
534
+ <td><strong>0.088</strong></td>
535
+ <td>0.295</td>
536
+ <td>0.535</td>
537
+ <td>-</td>
538
+ <td>-</td>
539
+ <td>0.159</td>
540
+ <td><strong>0.109</strong></td>
541
+ <td><strong>0.069</strong></td>
542
+ <td><strong>0.091</strong></td>
543
+ </tr>
544
+ <tr>
545
+ <td rowspan="8"><strong>Expert<br>VLMs</strong></td>
546
+ <td>GOT-OCR</td>
547
+ <td>0.287</td>
548
+ <td>0.411</td>
549
+ <td>0.189</td>
550
+ <td>0.315</td>
551
+ <td>0.360</td>
552
+ <td>0.528</td>
553
+ <td>53.2</td>
554
+ <td>47.2</td>
555
+ <td>0.459</td>
556
+ <td>0.520</td>
557
+ <td>0.141</td>
558
+ <td>0.280</td>
559
+ </tr>
560
+ <tr>
561
+ <td>Nougat</td>
562
+ <td>0.452</td>
563
+ <td>0.973</td>
564
+ <td>0.365</td>
565
+ <td>0.998</td>
566
+ <td>0.488</td>
567
+ <td>0.941</td>
568
+ <td>39.9</td>
569
+ <td>0</td>
570
+ <td>0.572</td>
571
+ <td>1.000</td>
572
+ <td>0.382</td>
573
+ <td>0.954</td>
574
+ </tr>
575
+ <tr>
576
+ <td>Mistral OCR</td>
577
+ <td>0.268</td>
578
+ <td>0.439</td>
579
+ <td>0.072</td>
580
+ <td>0.325</td>
581
+ <td>0.318</td>
582
+ <td>0.495</td>
583
+ <td>75.8</td>
584
+ <td>63.6</td>
585
+ <td>0.600</td>
586
+ <td>0.650</td>
587
+ <td>0.083</td>
588
+ <td>0.284</td>
589
+ </tr>
590
+ <tr>
591
+ <td>OLMOCR-sglang</td>
592
+ <td>0.326</td>
593
+ <td>0.469</td>
594
+ <td>0.097</td>
595
+ <td>0.293</td>
596
+ <td>0.455</td>
597
+ <td>0.655</td>
598
+ <td>68.1</td>
599
+ <td>61.3</td>
600
+ <td>0.608<td>0.652</td>
601
+ <td>0.145</td>
602
+ <td>0.277</td>
603
+ </tr>
604
+ <tr>
605
+ <td>SmolDocling-256M</td>
606
+ <td>0.493</td>
607
+ <td>0.816</td>
608
+ <td>0.262</td>
609
+ <td>0.838</td>
610
+ <td>0.753</td>
611
+ <td>0.997</td>
612
+ <td>44.9</td>
613
+ <td>16.5</td>
614
+ <td>0.729</td>
615
+ <td>0.907</td>
616
+ <td>0.227</td>
617
+ <td>0.522</td>
618
+ </tr>
619
+ <tr>
620
+ <td>Dolphin</td>
621
+ <td>0.206</td>
622
+ <td>0.306</td>
623
+ <td>0.107</td>
624
+ <td>0.197</td>
625
+ <td>0.447</td>
626
+ <td>0.580</td>
627
+ <td>77.3</td>
628
+ <td>67.2</td>
629
+ <td>0.180</td>
630
+ <td>0.285</td>
631
+ <td>0.091</td>
632
+ <td>0.162</td>
633
+ </tr>
634
+ <tr>
635
+ <td>MinerU 2</td>
636
+ <td>0.139</td>
637
+ <td>0.240</td>
638
+ <td><strong>0.047</strong></td>
639
+ <td>0.109</td>
640
+ <td>0.297</td>
641
+ <td>0.536</td>
642
+ <td><strong>82.5</strong></td>
643
+ <td>79.0</td>
644
+ <td>0.141</td>
645
+ <td>0.195</td>
646
+ <td><strong>0.069</strong></td>
647
+ <td>0.118</td>
648
+ </tr>
649
+ <tr>
650
+ <td>OCRFlux</td>
651
+
652
+ <td>0.195</td>
653
+ <td>0.281</td>
654
+ <td>0.064</td>
655
+ <td>0.183</td>
656
+ <td>0.379</td>
657
+ <td>0.613</td>
658
+ <td>71.6</td>
659
+ <td>81.3</td>
660
+ <td>0.253</td>
661
+ <td>0.139</td>
662
+ <td>0.086</td>
663
+ <td>0.187</td>
664
+
665
+
666
+ </tr>
667
+ <tr>
668
+ <td rowspan="3"><strong>General<br>VLMs</strong></td>
669
+ <td>GPT4o</td>
670
+ <td>0.233</td>
671
+ <td>0.399</td>
672
+ <td>0.144</td>
673
+ <td>0.409</td>
674
+ <td>0.425</td>
675
+ <td>0.606</td>
676
+ <td>72.0</td>
677
+ <td>62.9</td>
678
+ <td>0.234</td>
679
+ <td>0.329</td>
680
+ <td>0.128</td>
681
+ <td>0.251</td>
682
+ </tr>
683
+ <tr>
684
+ <td>Qwen2.5-VL-7B</td>
685
+ <td>0.312</td>
686
+ <td>0.406</td>
687
+ <td>0.157</td>
688
+ <td>0.228</td>
689
+ <td>0.351</td>
690
+ <td>0.574</td>
691
+ <td>76.4</td>
692
+ <td>72.2</td>
693
+ <td>0.588</td>
694
+ <td>0.619</td>
695
+ <td>0.149</td>
696
+ <td>0.203</td>
697
+ </tr>
698
+ <tr>
699
+ <td>InternVL3-8B</td>
700
+ <td>0.314</td>
701
+ <td>0.383</td>
702
+ <td>0.134</td>
703
+ <td>0.218</td>
704
+ <td>0.417</td>
705
+ <td>0.563</td>
706
+ <td>66.1</td>
707
+ <td>73.1</td>
708
+ <td>0.586</td>
709
+ <td>0.564</td>
710
+ <td>0.118</td>
711
+ <td>0.186</td>
712
+ </tr>
713
+ <tr>
714
+ <td rowspan="4"><strong>Mix</strong></td>
715
+ <td><strong>MonkeyOCR-3B <a href="https://huggingface.co/echo840/MonkeyOCR/blob/main/Structure/doclayout_yolo_docstructbench_imgsz1280_2501.pt">[Weight]</a></strong></td>
716
+ <td>0.140</td>
717
+ <td>0.297</td>
718
+ <td>0.058</td>
719
+ <td>0.185</td>
720
+ <td>0.238</td>
721
+ <td>0.506</td>
722
+ <td>80.2</td>
723
+ <td>77.7</td>
724
+ <td>0.170</td>
725
+ <td>0.253</td>
726
+ <td>0.093</td>
727
+ <td>0.244</td>
728
+ </tr>
729
+ <tr>
730
+ <td><strong>MonkeyOCR-3B* <a href="https://huggingface.co/echo840/MonkeyOCR/blob/main/Structure/layout_zh.pt">[Weight]</a></strong></td>
731
+ <td>0.154</td>
732
+ <td>0.277</td>
733
+ <td>0.073</td>
734
+ <td>0.134</td>
735
+ <td>0.255</td>
736
+ <td>0.529</td>
737
+ <td>78.2</td>
738
+ <td>76.2</td>
739
+ <td>0.182</td>
740
+ <td>0.262</td>
741
+ <td>0.105</td>
742
+ <td>0.183</td>
743
+ </tr>
744
+ <tr>
745
+ <td><strong>MonkeyOCR-pro-3B <a href="https://huggingface.co/echo840/MonkeyOCR-pro-3B">[Weight]</a></strong></td>
746
+ <td><strong>0.138</strong></td>
747
+ <td><strong>0.206</strong></td>
748
+ <td>0.067</td>
749
+ <td>0.107</td>
750
+ <td><strong>0.246</strong></td>
751
+ <td><strong>0.421</strong></td>
752
+ <td>81.5</td>
753
+ <td><strong>87.5</strong></td>
754
+ <td><strong>0.139</strong></td>
755
+ <td>0.111</td>
756
+ <td>0.100</td>
757
+ <td>0.185</td>
758
+ </tr>
759
+ <tr>
760
+ <td><strong>MonkeyOCR-pro-1.2B <a href="https://huggingface.co/echo840/MonkeyOCR-pro-1.2B">[Weight]</a></strong></td>
761
+ <td>0.153</td>
762
+ <td>0.223</td>
763
+ <td>0.066</td>
764
+ <td>0.123</td>
765
+ <td>0.272</td>
766
+ <td>0.449</td>
767
+ <td>76.5</td>
768
+ <td>83.7</td>
769
+ <td>0.176</td>
770
+ <td>0.131</td>
771
+ <td>0.097</td>
772
+ <td>0.187</td>
773
+ </tr>
774
+ </tbody>
775
+ </table>
776
 
777
 
778
  ### 2. The end-to-end text recognition performance across 9 PDF page types.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
779
 
780
+ <table>
781
+ <thead>
782
+ <tr>
783
+ <th><strong>Model<br>Type</strong></th>
784
+ <th><strong>Models</strong></th>
785
+ <th><strong>Book</strong></th>
786
+ <th><strong>Slides</strong></th>
787
+ <th><strong>Financial<br>Report</strong></th>
788
+ <th><strong>Textbook</strong></th>
789
+ <th><strong>Exam<br>Paper</strong></th>
790
+ <th><strong>Magazine</strong></th>
791
+ <th><strong>Academic<br>Papers</strong></th>
792
+ <th><strong>Notes</strong></th>
793
+ <th><strong>Newspaper</strong></th>
794
+ <th><strong>Overall</strong></th>
795
+ </tr>
796
+ </thead>
797
+ <tbody>
798
+ <tr>
799
+ <td rowspan="3"><strong>Pipeline<br>Tools</strong></td>
800
+ <td>MinerU</td>
801
+ <td>0.055</td>
802
+ <td>0.124</td>
803
+ <td><u>0.033</u></td>
804
+ <td>0.102</td>
805
+ <td>0.159</td>
806
+ <td><strong>0.072</strong></td>
807
+ <td><u>0.025</u></td>
808
+ <td>0.984</td>
809
+ <td>0.171</td>
810
+ <td>0.206</td>
811
+ </tr>
812
+ <tr>
813
+ <td>Marker</td>
814
+ <td>0.074</td>
815
+ <td>0.340</td>
816
+ <td>0.089</td>
817
+ <td>0.319</td>
818
+ <td>0.452</td>
819
+ <td>0.153</td>
820
+ <td>0.059</td>
821
+ <td>0.651</td>
822
+ <td>0.192</td>
823
+ <td>0.274</td>
824
+ </tr>
825
+ <tr>
826
+ <td>Mathpix</td>
827
+ <td>0.131</td>
828
+ <td>0.220</td>
829
+ <td>0.202</td>
830
+ <td>0.216</td>
831
+ <td>0.278</td>
832
+ <td>0.147</td>
833
+ <td>0.091</td>
834
+ <td>0.634</td>
835
+ <td>0.690</td>
836
+ <td>0.300</td>
837
+ </tr>
838
+ <tr>
839
+ <td rowspan="4"><strong>Expert<br>VLMs</strong></td>
840
+ <td>GOT-OCR</td>
841
+ <td>0.111</td>
842
+ <td>0.222</td>
843
+ <td>0.067</td>
844
+ <td>0.132</td>
845
+ <td>0.204</td>
846
+ <td>0.198</td>
847
+ <td>0.179</td>
848
+ <td>0.388</td>
849
+ <td>0.771</td>
850
+ <td>0.267</td>
851
+ </tr>
852
+ <tr>
853
+ <td>Nougat</td>
854
+ <td>0.734</td>
855
+ <td>0.958</td>
856
+ <td>1.000</td>
857
+ <td>0.820</td>
858
+ <td>0.930</td>
859
+ <td>0.830</td>
860
+ <td>0.214</td>
861
+ <td>0.991</td>
862
+ <td>0.871</td>
863
+ <td>0.806</td>
864
+ </tr>
865
+ <tr>
866
+ <td>Dolphin</td>
867
+ <td>0.091</td>
868
+ <td>0.131</td>
869
+ <td>0.057</td>
870
+ <td>0.146</td>
871
+ <td>0.231</td>
872
+ <td>0.121</td>
873
+ <td>0.074</td>
874
+ <td>0.363</td>
875
+ <td>0.307</td>
876
+ <td>0.177</td>
877
+ </tr>
878
+ <tr>
879
+ <td>OCRFlux</td>
880
+ <td>0.068</td>
881
+ <td>0.125</td>
882
+ <td>0.092</td>
883
+ <td>0.102</td>
884
+ <td>0.119</td>
885
+ <td>0.083</td>
886
+ <td>0.047</td>
887
+ <td>0.223</td>
888
+ <td>0.536</td>
889
+ <td>0.149</td>
890
+ </tr>
891
+ <tr>
892
+ <td rowspan="3"><strong>General<br>VLMs</strong></td>
893
+ <td>GPT4o</td>
894
+ <td>0.157</td>
895
+ <td>0.163</td>
896
+ <td>0.348</td>
897
+ <td>0.187</td>
898
+ <td>0.281</td>
899
+ <td>0.173</td>
900
+ <td>0.146</td>
901
+ <td>0.607</td>
902
+ <td>0.751</td>
903
+ <td>0.316</td>
904
+ </tr>
905
+ <tr>
906
+ <td>Qwen2.5-VL-7B</td>
907
+ <td>0.148</td>
908
+ <td><strong>0.053</strong></td>
909
+ <td>0.111</td>
910
+ <td>0.137</td>
911
+ <td>0.189</td>
912
+ <td>0.117</td>
913
+ <td>0.134</td>
914
+ <td>0.204</td>
915
+ <td>0.706</td>
916
+ <td>0.205</td>
917
+ </tr>
918
+ <tr>
919
+ <td>InternVL3-8B</td>
920
+ <td>0.163</td>
921
+ <td><u>0.056</u></td>
922
+ <td>0.107</td>
923
+ <td>0.109</td>
924
+ <td>0.129</td>
925
+ <td>0.100</td>
926
+ <td>0.159</td>
927
+ <td><strong>0.150</strong></td>
928
+ <td>0.681</td>
929
+ <td>0.188</td>
930
+ </tr>
931
+ <tr>
932
+ <td rowspan="4"><strong>Mix</strong></td>
933
+ <td><strong>MonkeyOCR-3B <a href="https://huggingface.co/echo840/MonkeyOCR/blob/main/Structure/doclayout_yolo_docstructbench_imgsz1280_2501.pt">[Weight]</a></strong></td>
934
+ <td><strong>0.046</strong></td>
935
+ <td>0.120</td>
936
+ <td><strong>0.024</strong></td>
937
+ <td>0.100</td>
938
+ <td>0.129</td>
939
+ <td>0.086</td>
940
+ <td><strong>0.024</strong></td>
941
+ <td>0.643</td>
942
+ <td><u>0.131</u></td>
943
+ <td>0.155</td>
944
+ </tr>
945
+ <tr>
946
+ <td><strong>MonkeyOCR-3B* <a href="https://huggingface.co/echo840/MonkeyOCR/blob/main/Structure/layout_zh.pt">[Weight]</a></strong></td>
947
+ <td><u>0.054</u></td>
948
+ <td>0.203</td>
949
+ <td>0.038</td>
950
+ <td>0.112</td>
951
+ <td>0.138</td>
952
+ <td>0.111</td>
953
+ <td>0.032</td>
954
+ <td>0.194</td>
955
+ <td>0.136</td>
956
+ <td>0.120</td>
957
+ </tr>
958
+ <tr>
959
+ <td><strong>MonkeyOCR-pro-3B <a href="https://huggingface.co/echo840/MonkeyOCR-pro-3B">[Weight]</a></strong></td>
960
+ <td>0.084</td>
961
+ <td>0.129</td>
962
+ <td>0.060</td>
963
+ <td><strong>0.090</strong></td>
964
+ <td><strong>0.107</strong></td>
965
+ <td><u>0.073</u></td>
966
+ <td>0.050</td>
967
+ <td><u>0.171</u></td>
968
+ <td><strong>0.107</strong></td>
969
+ <td><strong>0.100</strong></td>
970
+ </tr>
971
+ <tr>
972
+ <td><strong>MonkeyOCR-pro-1.2B <a href="https://huggingface.co/echo840/MonkeyOCR-pro-1.2B">[Weight]</a></strong></td>
973
+ <td>0.087</td>
974
+ <td>0.142</td>
975
+ <td>0.059</td>
976
+ <td><u>0.093</u></td>
977
+ <td><u>0.115</u></td>
978
+ <td>0.085</td>
979
+ <td>0.045</td>
980
+ <td>0.226</td>
981
+ <td>0.122</td>
982
+ <td><u>0.112</u></td>
983
+ </tr>
984
+ </tbody>
985
+ </table>
986
 
987
+ ### 3. The evaluation results of olmOCR-bench.
988
+
989
+ <table>
990
+ <thead>
991
+ <tr>
992
+ <th>Model</th>
993
+ <th>ArXiv</th>
994
+ <th>Old Scans<br>Math</th>
995
+ <th>Tables</th>
996
+ <th>Old Scans</th>
997
+ <th>Headers and<br>Footers</th>
998
+ <th>Multi<br>column</th>
999
+ <th>Long Tiny<br>Text</th>
1000
+ <th>Base</th>
1001
+ <th>Overall</th>
1002
+ </tr>
1003
+ </thead>
1004
+ <tbody>
1005
+ <tr>
1006
+ <td>GOT OCR</td>
1007
+ <td>52.7</td>
1008
+ <td>52.0</td>
1009
+ <td>0.2</td>
1010
+ <td>22.1</td>
1011
+ <td>93.6</td>
1012
+ <td>42.0</td>
1013
+ <td>29.9</td>
1014
+ <td>94.0</td>
1015
+ <td>48.3 ± 1.1</td>
1016
+ </tr>
1017
+ <tr>
1018
+ <td>Marker</td>
1019
+ <td>76.0</td>
1020
+ <td>57.9</td>
1021
+ <td>57.6</td>
1022
+ <td>27.8</td>
1023
+ <td>84.9</td>
1024
+ <td>72.9</td>
1025
+ <td>84.6</td>
1026
+ <td><strong>99.1</strong></td>
1027
+ <td>70.1 ± 1.1</td>
1028
+ </tr>
1029
+ <tr>
1030
+ <td>MinerU</td>
1031
+ <td>75.4</td>
1032
+ <td>47.4</td>
1033
+ <td>60.9</td>
1034
+ <td>17.3</td>
1035
+ <td><strong>96.6</strong></td>
1036
+ <td>59.0</td>
1037
+ <td>39.1</td>
1038
+ <td>96.6</td>
1039
+ <td>61.5 ± 1.1</td>
1040
+ </tr>
1041
+ <tr>
1042
+ <td>Mistral OCR</td>
1043
+ <td>77.2</td>
1044
+ <td>67.5</td>
1045
+ <td>60.6</td>
1046
+ <td>29.3</td>
1047
+ <td>93.6</td>
1048
+ <td>71.3</td>
1049
+ <td>77.1</td>
1050
+ <td>99.4</td>
1051
+ <td>72.0 ± 1.1</td>
1052
+ </tr>
1053
+ <tr>
1054
+ <td>Nanonets OCR</td>
1055
+ <td>67.0</td>
1056
+ <td>68.6</td>
1057
+ <td><strong>77.7</strong></td>
1058
+ <td>39.5</td>
1059
+ <td>40.7</td>
1060
+ <td>69.9</td>
1061
+ <td>53.4</td>
1062
+ <td>99.3</td>
1063
+ <td>64.5 ± 1.1</td>
1064
+ </tr>
1065
+ <tr>
1066
+ <td>GPT-4o<br>(No Anchor)</td>
1067
+ <td>51.5</td>
1068
+ <td><strong>75.5</strong></td>
1069
+ <td>69.1</td>
1070
+ <td>40.9</td>
1071
+ <td>94.2</td>
1072
+ <td>68.9</td>
1073
+ <td>54.1</td>
1074
+ <td>96.7</td>
1075
+ <td>68.9 ± 1.1</td>
1076
+ </tr>
1077
+ <tr>
1078
+ <td>GPT-4o<br>(Anchored)</td>
1079
+ <td>53.5</td>
1080
+ <td>74.5</td>
1081
+ <td>70.0</td>
1082
+ <td>40.7</td>
1083
+ <td>93.8</td>
1084
+ <td>69.3</td>
1085
+ <td>60.6</td>
1086
+ <td>96.8</td>
1087
+ <td>69.9 ± 1.1</td>
1088
+ </tr>
1089
+ <tr>
1090
+ <td>Gemini Flash 2<br>(No Anchor)</td>
1091
+ <td>32.1</td>
1092
+ <td>56.3</td>
1093
+ <td>61.4</td>
1094
+ <td>27.8</td>
1095
+ <td>48.0</td>
1096
+ <td>58.7</td>
1097
+ <td><strong>84.4</strong></td>
1098
+ <td>94.0</td>
1099
+ <td>57.8 ± 1.1</td>
1100
+ </tr>
1101
+ <tr>
1102
+ <td>Gemini Flash 2<br>(Anchored)</td>
1103
+ <td>54.5</td>
1104
+ <td>56.1</td>
1105
+ <td>72.1</td>
1106
+ <td>34.2</td>
1107
+ <td>64.7</td>
1108
+ <td>61.5</td>
1109
+ <td>71.5</td>
1110
+ <td>95.6</td>
1111
+ <td>63.8 ± 1.2</td>
1112
+ </tr>
1113
+ <tr>
1114
+ <td>Qwen 2 VL<br>(No Anchor)</td>
1115
+ <td>19.7</td>
1116
+ <td>31.7</td>
1117
+ <td>24.2</td>
1118
+ <td>17.1</td>
1119
+ <td>88.9</td>
1120
+ <td>8.3</td>
1121
+ <td>6.8</td>
1122
+ <td>55.5</td>
1123
+ <td>31.5 ± 0.9</td>
1124
+ </tr>
1125
+ <tr>
1126
+ <td>Qwen 2.5 VL<br>(No Anchor)</td>
1127
+ <td>63.1</td>
1128
+ <td>65.7</td>
1129
+ <td>67.3</td>
1130
+ <td>38.6</td>
1131
+ <td>73.6</td>
1132
+ <td>68.3</td>
1133
+ <td>49.1</td>
1134
+ <td>98.3</td>
1135
+ <td>65.5 ± 1.2</td>
1136
+ </tr>
1137
+ <tr>
1138
+ <td>olmOCR v0.1.75<br>(No Anchor)</td>
1139
+ <td>71.5</td>
1140
+ <td>71.4</td>
1141
+ <td>71.4</td>
1142
+ <td><strong>42.8</strong></td>
1143
+ <td>94.1</td>
1144
+ <td>77.7</td>
1145
+ <td>71.0</td>
1146
+ <td>97.8</td>
1147
+ <td>74.7 ± 1.1</td>
1148
+ </tr>
1149
+ <tr>
1150
+ <td>olmOCR v0.1.75<br>(Anchored)</td>
1151
+ <td>74.9</td>
1152
+ <td>71.2</td>
1153
+ <td>71.0</td>
1154
+ <td>42.2</td>
1155
+ <td>94.5</td>
1156
+ <td><strong>78.3</strong></td>
1157
+ <td>73.3</td>
1158
+ <td>98.3</td>
1159
+ <td>75.5 ± 1.0</td>
1160
+ </tr>
1161
+ <tr>
1162
+ <td>MonkeyOCR-pro-3B <a href="https://huggingface.co/echo840/MonkeyOCR-pro-3B">[Weight]</a></td>
1163
+ <td><strong>83.8</strong></td>
1164
+ <td>68.8</td>
1165
+ <td>74.6</td>
1166
+ <td>36.1</td>
1167
+ <td>91.2</td>
1168
+ <td>76.6</td>
1169
+ <td>80.1</td>
1170
+ <td>95.3</td>
1171
+ <td><strong>75.8 ± 1.0</strong></td>
1172
+ </tr>
1173
+ <tr>
1174
+ <td>MonkeyOCR-pro-1.2B <a href="https://huggingface.co/echo840/MonkeyOCR-pro-1.2B">[Weight]</a></td>
1175
+ <td>80.5</td>
1176
+ <td>62.9</td>
1177
+ <td>71.1</td>
1178
+ <td>32.9</td>
1179
+ <td>92.2</td>
1180
+ <td>68.3</td>
1181
+ <td>74.0</td>
1182
+ <td>92.6</td>
1183
+ <td>71.8 ± 1.1</td>
1184
+ </tr>
1185
+ </tbody>
1186
+ </table>
1187
 
1188
  ## Visualization Demo
1189
 
1190
+ Get a Quick Hands-On Experience with Our Demo: http://vlrlabmonkey.xyz:7685 (The latest model is available for selection)
1191
 
1192
  > Our demo is simple and easy to use:
1193
  >
 
1230
 
1231
 
1232
  ## Acknowledgments
1233
+ We would like to thank [MinerU](https://github.com/opendatalab/MinerU), [DocLayout-YOLO](https://github.com/opendatalab/DocLayout-YOLO), [PyMuPDF](https://github.com/pymupdf/PyMuPDF), [layoutreader](https://github.com/ppaanngggg/layoutreader), [Qwen2.5-VL](https://github.com/QwenLM/Qwen2.5-VL), [LMDeploy](https://github.com/InternLM/lmdeploy), [PP-StructureV3](https://github.com/PaddlePaddle/PaddleOCR), [PP-DocLayout_plus-L](https://huggingface.co/PaddlePaddle/PP-DocLayout_plus-L), and [InternVL3](https://github.com/OpenGVLab/InternVL) for providing base code and models, as well as their contributions to this field. We also thank [M6Doc](https://github.com/HCIILAB/M6Doc), [DocLayNet](https://github.com/DS4SD/DocLayNet), [CDLA](https://github.com/buptlihang/CDLA), [D4LA](https://github.com/AlibabaResearch/AdvancedLiterateMachinery), [DocGenome](https://github.com/Alpha-Innovator/DocGenome), [PubTabNet](https://github.com/ibm-aur-nlp/PubTabNet), and [UniMER-1M](https://github.com/opendatalab/UniMERNet) for providing valuable datasets. We also thank everyone who contributed to this open-source effort.
1234
 
1235
+ ## Limitation
1236
+ Currently, MonkeyOCR do not yet fully support for photographed text, handwritten content, Traditional Chinese characters, or multilingual text. We plan to consider adding support for these features in future public releases. Additionally, our model is deployed on a single GPU, so if too many users upload files at the same time, issues like “This application is currently busy” may occur. The processing time shown on the demo page does not reflect computation time alone—it also includes result uploading and other overhead. During periods of high traffic, this time may be longer. The inference speeds of MonkeyOCR, MinerU, and Qwen2.5 VL-7B were measured on an H800 GPU.
1237
 
1238
  ## Copyright
1239
+ Please don’t hesitate to share your valuable feedback — it’s a key motivation that drives us to continuously improve our framework. Note: Our model is intended for academic research and non-commercial use only. If you are interested in faster (smaller) or stronger one, please contact us at [email protected] or [email protected].