Update README.md
Browse files
README.md
CHANGED
@@ -16,6 +16,7 @@ MonkeyOCR: Document Parsing with a Structure-Recognition-Relation Triplet Paradi
|
|
16 |
[](https://huggingface.co/echo840/MonkeyOCR)
|
17 |
[](https://github.com/Yuliang-Liu/MonkeyOCR/issues?q=is%3Aopen+is%3Aissue)
|
18 |
[](https://github.com/Yuliang-Liu/MonkeyOCR/issues?q=is%3Aissue+is%3Aclosed)
|
|
|
19 |
[](https://github.com/Yuliang-Liu/MonkeyOCR)
|
20 |
</div>
|
21 |
|
@@ -23,9 +24,10 @@ MonkeyOCR: Document Parsing with a Structure-Recognition-Relation Triplet Paradi
|
|
23 |
> **MonkeyOCR: Document Parsing with a Structure-Recognition-Relation Triplet Paradigm**<br>
|
24 |
> Zhang Li, Yuliang Liu, Qiang Liu, Zhiyin Ma, Ziyang Zhang, Shuo Zhang, Zidun Guo, Jiarui Zhang, Xinyu Wang, Xiang Bai <br>
|
25 |
[](https://arxiv.org/abs/2506.05218)
|
26 |
-
[](
|
27 |
[](https://huggingface.co/echo840/MonkeyOCR)
|
28 |
[](https://modelscope.cn/models/l1731396519/MonkeyOCR)
|
|
|
29 |
[](http://vlrlabmonkey.xyz:7685/)
|
30 |
|
31 |
|
@@ -33,66 +35,273 @@ MonkeyOCR: Document Parsing with a Structure-Recognition-Relation Triplet Paradi
|
|
33 |
## Introduction
|
34 |
MonkeyOCR adopts a Structure-Recognition-Relation (SRR) triplet paradigm, which simplifies the multi-tool pipeline of modular approaches while avoiding the inefficiency of using large multimodal models for full-page document processing.
|
35 |
|
36 |
-
1.
|
37 |
-
2.
|
38 |
-
3.
|
39 |
-
|
40 |
-
|
41 |
-
|
42 |
-
|
43 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
44 |
|
45 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
46 |
|
47 |
-
https://youtu.be/T9oaqp-IaZ0
|
48 |
|
49 |
-
|
|
|
50 |
|
51 |
|
52 |
## News
|
53 |
-
* ```2025.
|
|
|
|
|
54 |
|
55 |
|
56 |
-
|
57 |
-
|
58 |
### 1. Install MonkeyOCR
|
59 |
-
|
60 |
-
conda create -n MonkeyOCR python=3.10
|
61 |
-
conda activate MonkeyOCR
|
62 |
-
|
63 |
-
git clone https://github.com/Yuliang-Liu/MonkeyOCR.git
|
64 |
-
cd MonkeyOCR
|
65 |
-
|
66 |
-
# Install pytorch, see https://pytorch.org/get-started/previous-versions/ for your cuda version
|
67 |
-
pip install torch==2.5.1 torchvision==0.20.1 torchaudio==2.5.1 --index-url https://download.pytorch.org/whl/cu124
|
68 |
-
pip install -e .
|
69 |
-
```
|
70 |
### 2. Download Model Weights
|
71 |
Download our model from Huggingface.
|
72 |
```python
|
73 |
pip install huggingface_hub
|
74 |
|
75 |
-
python tools/download_model.py
|
76 |
```
|
77 |
You can also download our model from ModelScope.
|
78 |
|
79 |
```python
|
80 |
pip install modelscope
|
81 |
|
82 |
-
python tools/download_model.py -t modelscope
|
83 |
```
|
84 |
### 3. Inference
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
85 |
```bash
|
86 |
-
#
|
87 |
-
python parse.py
|
88 |
-
#
|
89 |
-
|
90 |
-
|
91 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
92 |
```
|
93 |
|
94 |
-
|
95 |
-
|
|
|
|
|
|
|
|
|
96 |
|
97 |
1. **Processed Markdown File** (`your.md`): The final parsed document content in markdown format, containing text, formulas, tables, and other structured elements.
|
98 |
2. **Layout Results** (`your_layout.pdf`): The layout results drawed on origin PDF.
|
@@ -103,562 +312,882 @@ MonkeyOCR generates three types of output files:
|
|
103 |
|
104 |
These files provide both the final formatted output and detailed intermediate results for further analysis or processing.
|
105 |
|
|
|
|
|
106 |
### 4. Gradio Demo
|
107 |
```bash
|
108 |
-
|
109 |
-
pip install gradio==5.23.3
|
110 |
-
pip install pdf2image==1.17.0
|
111 |
```
|
|
|
|
|
|
|
|
|
112 |
```bash
|
113 |
-
|
114 |
-
python demo/demo_gradio.py
|
115 |
```
|
116 |
-
|
|
|
|
|
117 |
|
118 |
-
|
119 |
|
120 |
-
|
121 |
-
triton.runtime.errors.OutOfResources: out of resource: shared memory
|
122 |
-
```
|
123 |
|
124 |
-
|
|
|
|
|
125 |
|
126 |
-
|
127 |
-
|
128 |
-
```
|
129 |
|
130 |
-
|
131 |
-
|
|
|
132 |
|
133 |
-
|
134 |
-
python tools/lmdeploy_patcher.py restore
|
135 |
-
```
|
136 |
|
137 |
-
|
|
|
|
|
138 |
|
139 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
140 |
|
141 |
-
|
142 |
|
143 |
-
1. Install required dependency (if not already installed):
|
144 |
```bash
|
145 |
-
|
146 |
-
pip install flash-attn==2.7.4.post1 --no-build-isolation
|
147 |
```
|
148 |
-
2. Open the `model_configs.yaml` file
|
149 |
-
3. Set `chat_config.backend` to `transformers`
|
150 |
-
4. Adjust the `batch_size` according to your GPU's memory capacity to ensure stable performance
|
151 |
|
152 |
-
|
153 |
|
154 |
-
```
|
155 |
-
|
156 |
-
|
157 |
-
batch_size: 10 # Adjust based on your available GPU memory
|
158 |
-
```
|
159 |
|
|
|
|
|
|
|
|
|
|
|
160 |
|
161 |
-
##
|
162 |
|
|
|
163 |
|
164 |
-
|
165 |
|
|
|
166 |
|
167 |
-
|
168 |
|
169 |
-
|
170 |
-
<thead>
|
171 |
-
<tr>
|
172 |
-
<th rowspan="2">Model Type</th>
|
173 |
-
<th rowspan="2">Methods</th>
|
174 |
-
<th colspan="2">Overall Edit↓</th>
|
175 |
-
<th colspan="2">Text Edit↓</th>
|
176 |
-
<th colspan="2">Formula Edit↓</th>
|
177 |
-
<th colspan="2">Formula CDM↑</th>
|
178 |
-
<th colspan="2">Table TEDS↑</th>
|
179 |
-
<th colspan="2">Table Edit↓</th>
|
180 |
-
<th colspan="2">Read Order Edit↓</th>
|
181 |
-
</tr>
|
182 |
-
<tr>
|
183 |
-
<th>EN</th>
|
184 |
-
<th>ZH</th>
|
185 |
-
<th>EN</th>
|
186 |
-
<th>ZH</th>
|
187 |
-
<th>EN</th>
|
188 |
-
<th>ZH</th>
|
189 |
-
<th>EN</th>
|
190 |
-
<th>ZH</th>
|
191 |
-
<th>EN</th>
|
192 |
-
<th>ZH</th>
|
193 |
-
<th>EN</th>
|
194 |
-
<th>ZH</th>
|
195 |
-
<th>EN</th>
|
196 |
-
<th>ZH</th>
|
197 |
-
</tr>
|
198 |
-
</thead>
|
199 |
-
<tbody>
|
200 |
-
<tr>
|
201 |
-
<td rowspan="7">Pipeline Tools</td>
|
202 |
-
<td>MinerU</td>
|
203 |
-
<td>0.150</td>
|
204 |
-
<td>0.357</td>
|
205 |
-
<td>0.061</td>
|
206 |
-
<td>0.215</td>
|
207 |
-
<td>0.278</td>
|
208 |
-
<td>0.577</td>
|
209 |
-
<td>57.3</td>
|
210 |
-
<td>42.9</td>
|
211 |
-
<td>78.6</td>
|
212 |
-
<td>62.1</td>
|
213 |
-
<td>0.180</td>
|
214 |
-
<td>0.344</td>
|
215 |
-
<td><strong>0.079</strong></td>
|
216 |
-
<td>0.292</td>
|
217 |
-
</tr>
|
218 |
-
<tr>
|
219 |
-
<td>Marker</td>
|
220 |
-
<td>0.336</td>
|
221 |
-
<td>0.556</td>
|
222 |
-
<td>0.080</td>
|
223 |
-
<td>0.315</td>
|
224 |
-
<td>0.530</td>
|
225 |
-
<td>0.883</td>
|
226 |
-
<td>17.6</td>
|
227 |
-
<td>11.7</td>
|
228 |
-
<td>67.6</td>
|
229 |
-
<td>49.2</td>
|
230 |
-
<td>0.619</td>
|
231 |
-
<td>0.685</td>
|
232 |
-
<td>0.114</td>
|
233 |
-
<td>0.340</td>
|
234 |
-
</tr>
|
235 |
-
<tr>
|
236 |
-
<td>Mathpix</td>
|
237 |
-
<td>0.191</td>
|
238 |
-
<td>0.365</td>
|
239 |
-
<td>0.105</td>
|
240 |
-
<td>0.384</td>
|
241 |
-
<td>0.306</td>
|
242 |
-
<td><strong>0.454</strong></td>
|
243 |
-
<td>62.7</td>
|
244 |
-
<td><strong>62.1</strong></td>
|
245 |
-
<td>77.0</td>
|
246 |
-
<td>67.1</td>
|
247 |
-
<td>0.243</td>
|
248 |
-
<td>0.320</td>
|
249 |
-
<td>0.108</td>
|
250 |
-
<td>0.304</td>
|
251 |
-
</tr>
|
252 |
-
<tr>
|
253 |
-
<td>Docling</td>
|
254 |
-
<td>0.589</td>
|
255 |
-
<td>0.909</td>
|
256 |
-
<td>0.416</td>
|
257 |
-
<td>0.987</td>
|
258 |
-
<td>0.999</td>
|
259 |
-
<td>1</td>
|
260 |
-
<td>-</td>
|
261 |
-
<td>-</td>
|
262 |
-
<td>61.3</td>
|
263 |
-
<td>25.0</td>
|
264 |
-
<td>0.627</td>
|
265 |
-
<td>0.810</td>
|
266 |
-
<td>0.313</td>
|
267 |
-
<td>0.837</td>
|
268 |
-
</tr>
|
269 |
-
<tr>
|
270 |
-
<td>Pix2Text</td>
|
271 |
-
<td>0.320</td>
|
272 |
-
<td>0.528</td>
|
273 |
-
<td>0.138</td>
|
274 |
-
<td>0.356</td>
|
275 |
-
<td>0.276</td>
|
276 |
-
<td>0.611</td>
|
277 |
-
<td>78.4</td>
|
278 |
-
<td>39.6</td>
|
279 |
-
<td>73.6</td>
|
280 |
-
<td>66.2</td>
|
281 |
-
<td>0.584</td>
|
282 |
-
<td>0.645</td>
|
283 |
-
<td>0.281</td>
|
284 |
-
<td>0.499</td>
|
285 |
-
</tr>
|
286 |
-
<tr>
|
287 |
-
<td>Unstructured</td>
|
288 |
-
<td>0.586</td>
|
289 |
-
<td>0.716</td>
|
290 |
-
<td>0.198</td>
|
291 |
-
<td>0.481</td>
|
292 |
-
<td>0.999</td>
|
293 |
-
<td>1</td>
|
294 |
-
<td>-</td>
|
295 |
-
<td>-</td>
|
296 |
-
<td>0</td>
|
297 |
-
<td>0.06</td>
|
298 |
-
<td>1</td>
|
299 |
-
<td>0.998</td>
|
300 |
-
<td>0.145</td>
|
301 |
-
<td>0.387</td>
|
302 |
-
</tr>
|
303 |
-
<tr>
|
304 |
-
<td>OpenParse</td>
|
305 |
-
<td>0.646</td>
|
306 |
-
<td>0.814</td>
|
307 |
-
<td>0.681</td>
|
308 |
-
<td>0.974</td>
|
309 |
-
<td>0.996</td>
|
310 |
-
<td>1</td>
|
311 |
-
<td>0.11</td>
|
312 |
-
<td>0</td>
|
313 |
-
<td>64.8</td>
|
314 |
-
<td>27.5</td>
|
315 |
-
<td>0.284</td>
|
316 |
-
<td>0.639</td>
|
317 |
-
<td>0.595</td>
|
318 |
-
<td>0.641</td>
|
319 |
-
</tr>
|
320 |
-
<tr>
|
321 |
-
<td rowspan="5">Expert VLMs</td>
|
322 |
-
<td>GOT-OCR</td>
|
323 |
-
<td>0.287</td>
|
324 |
-
<td>0.411</td>
|
325 |
-
<td>0.189</td>
|
326 |
-
<td>0.315</td>
|
327 |
-
<td>0.360</td>
|
328 |
-
<td>0.528</td>
|
329 |
-
<td>74.3</td>
|
330 |
-
<td>45.3</td>
|
331 |
-
<td>53.2</td>
|
332 |
-
<td>47.2</td>
|
333 |
-
<td>0.459</td>
|
334 |
-
<td>0.520</td>
|
335 |
-
<td>0.141</td>
|
336 |
-
<td>0.280</td>
|
337 |
-
</tr>
|
338 |
-
<tr>
|
339 |
-
<td>Nougat</td>
|
340 |
-
<td>0.452</td>
|
341 |
-
<td>0.973</td>
|
342 |
-
<td>0.365</td>
|
343 |
-
<td>0.998</td>
|
344 |
-
<td>0.488</td>
|
345 |
-
<td>0.941</td>
|
346 |
-
<td>15.1</td>
|
347 |
-
<td>16.8</td>
|
348 |
-
<td>39.9</td>
|
349 |
-
<td>0</td>
|
350 |
-
<td>0.572</td>
|
351 |
-
<td>1.000</td>
|
352 |
-
<td>0.382</td>
|
353 |
-
<td>0.954</td>
|
354 |
-
</tr>
|
355 |
-
<tr>
|
356 |
-
<td>Mistral OCR</td>
|
357 |
-
<td>0.268</td>
|
358 |
-
<td>0.439</td>
|
359 |
-
<td>0.072</td>
|
360 |
-
<td>0.325</td>
|
361 |
-
<td>0.318</td>
|
362 |
-
<td>0.495</td>
|
363 |
-
<td>64.6</td>
|
364 |
-
<td>45.9</td>
|
365 |
-
<td>75.8</td>
|
366 |
-
<td>63.6</td>
|
367 |
-
<td>0.600</td>
|
368 |
-
<td>0.650</td>
|
369 |
-
<td>0.083</td>
|
370 |
-
<td>0.284</td>
|
371 |
-
</tr>
|
372 |
-
<tr>
|
373 |
-
<td>OLMOCR-sglang</td>
|
374 |
-
<td>0.326</td>
|
375 |
-
<td>0.469</td>
|
376 |
-
<td>0.097</td>
|
377 |
-
<td>0.293</td>
|
378 |
-
<td>0.455</td>
|
379 |
-
<td>0.655</td>
|
380 |
-
<td>74.3</td>
|
381 |
-
<td>43.2</td>
|
382 |
-
<td>68.1</td>
|
383 |
-
<td>61.3</td>
|
384 |
-
<td>0.608</td>
|
385 |
-
<td>0.652</td>
|
386 |
-
<td>0.145</td>
|
387 |
-
<td>0.277</td>
|
388 |
-
</tr>
|
389 |
-
<tr>
|
390 |
-
<td>SmolDocling-256M</td>
|
391 |
-
<td>0.493</td>
|
392 |
-
<td>0.816</td>
|
393 |
-
<td>0.262</td>
|
394 |
-
<td>0.838</td>
|
395 |
-
<td>0.753</td>
|
396 |
-
<td>0.997</td>
|
397 |
-
<td>32.1</td>
|
398 |
-
<td>0.55</td>
|
399 |
-
<td>44.9</td>
|
400 |
-
<td>16.5</td>
|
401 |
-
<td>0.729</td>
|
402 |
-
<td>0.907</td>
|
403 |
-
<td>0.227</td>
|
404 |
-
<td>0.522</td>
|
405 |
-
</tr>
|
406 |
-
<tr>
|
407 |
-
<td rowspan="3">General VLMs</td>
|
408 |
-
<td>GPT4o</td>
|
409 |
-
<td>0.233</td>
|
410 |
-
<td>0.399</td>
|
411 |
-
<td>0.144</td>
|
412 |
-
<td>0.409</td>
|
413 |
-
<td>0.425</td>
|
414 |
-
<td>0.606</td>
|
415 |
-
<td>72.8</td>
|
416 |
-
<td>42.8</td>
|
417 |
-
<td>72.0</td>
|
418 |
-
<td>62.9</td>
|
419 |
-
<td>0.234</td>
|
420 |
-
<td>0.329</td>
|
421 |
-
<td>0.128</td>
|
422 |
-
<td>0.251</td>
|
423 |
-
</tr>
|
424 |
-
<tr>
|
425 |
-
<td>Qwen2.5-VL-7B</td>
|
426 |
-
<td>0.312</td>
|
427 |
-
<td>0.406</td>
|
428 |
-
<td>0.157</td>
|
429 |
-
<td>0.228</td>
|
430 |
-
<td>0.351</td>
|
431 |
-
<td>0.574</td>
|
432 |
-
<td><strong>79.0</strong></td>
|
433 |
-
<td>50.2</td>
|
434 |
-
<td>76.4</td>
|
435 |
-
<td>72.2</td>
|
436 |
-
<td>0.588</td>
|
437 |
-
<td>0.619</td>
|
438 |
-
<td>0.149</td>
|
439 |
-
<td>0.203</td>
|
440 |
-
</tr>
|
441 |
-
<tr>
|
442 |
-
<td>InternVL3-8B</td>
|
443 |
-
<td>0.314</td>
|
444 |
-
<td>0.383</td>
|
445 |
-
<td>0.134</td>
|
446 |
-
<td>0.218</td>
|
447 |
-
<td>0.417</td>
|
448 |
-
<td>0.563</td>
|
449 |
-
<td>78.3</td>
|
450 |
-
<td>49.3</td>
|
451 |
-
<td>66.1</td>
|
452 |
-
<td>73.1</td>
|
453 |
-
<td>0.586</td>
|
454 |
-
<td>0.564</td>
|
455 |
-
<td>0.118</td>
|
456 |
-
<td>0.186</td>
|
457 |
-
</tr>
|
458 |
-
<tr>
|
459 |
-
<td rowspan="2">Mix</td>
|
460 |
-
<td>MonkeyOCR-3B <a href="https://huggingface.co/echo840/MonkeyOCR/blob/main/Structure/doclayout_yolo_docstructbench_imgsz1280_2501.pt">[Weight]</a></td>
|
461 |
-
<td><strong>0.140</strong></td>
|
462 |
-
<td>0.297</td>
|
463 |
-
<td><strong>0.058</strong></td>
|
464 |
-
<td>0.185</td>
|
465 |
-
<td><strong>0.238</strong></td>
|
466 |
-
<td>0.506</td>
|
467 |
-
<td>78.7</td>
|
468 |
-
<td>51.4</td>
|
469 |
-
<td><strong>80.2</strong></td>
|
470 |
-
<td><strong>77.7</strong></td>
|
471 |
-
<td><strong>0.170</strong></td>
|
472 |
-
<td><strong>0.253</strong></td>
|
473 |
-
<td>0.093</td>
|
474 |
-
<td>0.244</td>
|
475 |
-
</tr>
|
476 |
-
<tr>
|
477 |
-
<td>MonkeyOCR-3B* <a href="https://huggingface.co/echo840/MonkeyOCR/blob/main/Structure/layout_zh.pt">[Weight]</a></td>
|
478 |
-
<td>0.154</td>
|
479 |
-
<td><strong>0.277</strong></td>
|
480 |
-
<td>0.073</td>
|
481 |
-
<td><strong>0.134</strong></td>
|
482 |
-
<td>0.255</td>
|
483 |
-
<td>0.529</td>
|
484 |
-
<td>78.5</td>
|
485 |
-
<td>50.8</td>
|
486 |
-
<td>78.2</td>
|
487 |
-
<td>76.2</td>
|
488 |
-
<td>0.182</td>
|
489 |
-
<td>0.262</td>
|
490 |
-
<td>0.105</td>
|
491 |
-
<td><strong>0.183</strong></td>
|
492 |
-
</tr>
|
493 |
-
</tbody>
|
494 |
-
</table>
|
495 |
|
|
|
496 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
497 |
|
498 |
|
499 |
### 2. The end-to-end text recognition performance across 9 PDF page types.
|
500 |
-
<table style="width: 100%; border-collapse: collapse; text-align: center;">
|
501 |
-
<thead>
|
502 |
-
<tr style="border-bottom: 2px solid #000;">
|
503 |
-
<th><b>Model Type</b></th>
|
504 |
-
<th><b>Models</b></th>
|
505 |
-
<th><b>Book</b></th>
|
506 |
-
<th><b>Slides</b></th>
|
507 |
-
<th><b>Financial Report</b></th>
|
508 |
-
<th><b>Textbook</b></th>
|
509 |
-
<th><b>Exam Paper</b></th>
|
510 |
-
<th><b>Magazine</b></th>
|
511 |
-
<th><b>Academic Papers</b></th>
|
512 |
-
<th><b>Notes</b></th>
|
513 |
-
<th><b>Newspaper</b></th>
|
514 |
-
<th><b>Overall</b></th>
|
515 |
-
</tr>
|
516 |
-
</thead>
|
517 |
-
<tbody>
|
518 |
-
<tr>
|
519 |
-
<td rowspan="3"><b>Pipeline Tools</b></td>
|
520 |
-
<td>MinerU</td>
|
521 |
-
<td><u>0.055</u></td>
|
522 |
-
<td>0.124</td>
|
523 |
-
<td><u>0.033</u></td>
|
524 |
-
<td><u>0.102</u></td>
|
525 |
-
<td><u>0.159</u></td>
|
526 |
-
<td><b>0.072</b></td>
|
527 |
-
<td><u>0.025</u></td>
|
528 |
-
<td>0.984</td>
|
529 |
-
<td>0.171</td>
|
530 |
-
<td>0.206</td>
|
531 |
-
</tr>
|
532 |
-
<tr>
|
533 |
-
<td>Marker</td>
|
534 |
-
<td>0.074</td>
|
535 |
-
<td>0.340</td>
|
536 |
-
<td>0.089</td>
|
537 |
-
<td>0.319</td>
|
538 |
-
<td>0.452</td>
|
539 |
-
<td>0.153</td>
|
540 |
-
<td>0.059</td>
|
541 |
-
<td>0.651</td>
|
542 |
-
<td>0.192</td>
|
543 |
-
<td>0.274</td>
|
544 |
-
</tr>
|
545 |
-
<tr>
|
546 |
-
<td>Mathpix</td>
|
547 |
-
<td>0.131</td>
|
548 |
-
<td>0.220</td>
|
549 |
-
<td>0.202</td>
|
550 |
-
<td>0.216</td>
|
551 |
-
<td>0.278</td>
|
552 |
-
<td>0.147</td>
|
553 |
-
<td>0.091</td>
|
554 |
-
<td>0.634</td>
|
555 |
-
<td>0.690</td>
|
556 |
-
<td>0.300</td>
|
557 |
-
</tr>
|
558 |
-
<tr>
|
559 |
-
<td rowspan="2"><b>Expert VLMs</b></td>
|
560 |
-
<td>GOT-OCR</td>
|
561 |
-
<td>0.111</td>
|
562 |
-
<td>0.222</td>
|
563 |
-
<td>0.067</td>
|
564 |
-
<td>0.132</td>
|
565 |
-
<td>0.204</td>
|
566 |
-
<td>0.198</td>
|
567 |
-
<td>0.179</td>
|
568 |
-
<td>0.388</td>
|
569 |
-
<td>0.771</td>
|
570 |
-
<td>0.267</td>
|
571 |
-
</tr>
|
572 |
-
<tr>
|
573 |
-
<td>Nougat</td>
|
574 |
-
<td>0.734</td>
|
575 |
-
<td>0.958</td>
|
576 |
-
<td>1.000</td>
|
577 |
-
<td>0.820</td>
|
578 |
-
<td>0.930</td>
|
579 |
-
<td>0.830</td>
|
580 |
-
<td>0.214</td>
|
581 |
-
<td>0.991</td>
|
582 |
-
<td>0.871</td>
|
583 |
-
<td>0.806</td>
|
584 |
-
</tr>
|
585 |
-
<tr>
|
586 |
-
<td rowspan="3"><b>General VLMs</b></td>
|
587 |
-
<td>GPT4o</td>
|
588 |
-
<td>0.157</td>
|
589 |
-
<td>0.163</td>
|
590 |
-
<td>0.348</td>
|
591 |
-
<td>0.187</td>
|
592 |
-
<td>0.281</td>
|
593 |
-
<td>0.173</td>
|
594 |
-
<td>0.146</td>
|
595 |
-
<td>0.607</td>
|
596 |
-
<td>0.751</td>
|
597 |
-
<td>0.316</td>
|
598 |
-
</tr>
|
599 |
-
<tr>
|
600 |
-
<td>Qwen2.5-VL-7B</td>
|
601 |
-
<td>0.148</td>
|
602 |
-
<td><b>0.053</b></td>
|
603 |
-
<td>0.111</td>
|
604 |
-
<td>0.137</td>
|
605 |
-
<td>0.189</td>
|
606 |
-
<td>0.117</td>
|
607 |
-
<td>0.134</td>
|
608 |
-
<td>0.204</td>
|
609 |
-
<td>0.706</td>
|
610 |
-
<td>0.205</td>
|
611 |
-
</tr>
|
612 |
-
<tr>
|
613 |
-
<td>InternVL3-8B</td>
|
614 |
-
<td>0.163</td>
|
615 |
-
<td><u>0.056</u></td>
|
616 |
-
<td>0.107</td>
|
617 |
-
<td>0.109</td>
|
618 |
-
<td><b>0.129</b></td>
|
619 |
-
<td>0.100</td>
|
620 |
-
<td>0.159</td>
|
621 |
-
<td><b>0.150</b></td>
|
622 |
-
<td>0.681</td>
|
623 |
-
<td>0.188</td>
|
624 |
-
</tr>
|
625 |
-
<tr>
|
626 |
-
<td rowspan="2"><b>Mix</b></td>
|
627 |
-
<td>MonkeyOCR-3B <a href="https://huggingface.co/echo840/MonkeyOCR/blob/main/Structure/doclayout_yolo_docstructbench_imgsz1280_2501.pt">[Weight]</a></td>
|
628 |
-
<td><b>0.046</b></td>
|
629 |
-
<td>0.120</td>
|
630 |
-
<td><b>0.024</b></td>
|
631 |
-
<td><b>0.100</b></td>
|
632 |
-
<td><b>0.129</b></td>
|
633 |
-
<td><u>0.086</u></td>
|
634 |
-
<td><b>0.024</b></td>
|
635 |
-
<td>0.643</td>
|
636 |
-
<td><b>0.131</b></td>
|
637 |
-
<td><u>0.155</u></td>
|
638 |
-
</tr>
|
639 |
-
<tr>
|
640 |
-
<td>MonkeyOCR-3B* <a href="https://huggingface.co/echo840/MonkeyOCR/blob/main/Structure/layout_zh.pt">[Weight]</a></td>
|
641 |
-
<td>0.054</td>
|
642 |
-
<td>0.203</td>
|
643 |
-
<td>0.038</td>
|
644 |
-
<td>0.112</td>
|
645 |
-
<td>0.138</td>
|
646 |
-
<td>0.111</td>
|
647 |
-
<td>0.032</td>
|
648 |
-
<td><u>0.194</u></td>
|
649 |
-
<td><u>0.136</u></td>
|
650 |
-
<td><b>0.120</b></td>
|
651 |
-
</tr>
|
652 |
-
</tbody>
|
653 |
-
</table>
|
654 |
|
655 |
-
|
656 |
-
<
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
657 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
658 |
|
659 |
## Visualization Demo
|
660 |
|
661 |
-
Get a Quick Hands-On Experience with Our Demo: http://vlrlabmonkey.xyz:7685
|
662 |
|
663 |
> Our demo is simple and easy to use:
|
664 |
>
|
@@ -701,8 +1230,10 @@ If you wish to refer to the baseline results published here, please use the foll
|
|
701 |
|
702 |
|
703 |
## Acknowledgments
|
704 |
-
We would like to thank [MinerU](https://github.com/opendatalab/MinerU), [DocLayout-YOLO](https://github.com/opendatalab/DocLayout-YOLO), [PyMuPDF](https://github.com/pymupdf/PyMuPDF), [layoutreader](https://github.com/ppaanngggg/layoutreader), [Qwen2.5-VL](https://github.com/QwenLM/Qwen2.5-VL), [LMDeploy](https://github.com/InternLM/lmdeploy), and [InternVL3](https://github.com/OpenGVLab/InternVL) for providing base code and models, as well as their contributions to this field. We also thank [M6Doc](https://github.com/HCIILAB/M6Doc), [DocLayNet](https://github.com/DS4SD/DocLayNet), [CDLA](https://github.com/buptlihang/CDLA), [D4LA](https://github.com/AlibabaResearch/AdvancedLiterateMachinery), [DocGenome](https://github.com/Alpha-Innovator/DocGenome), [PubTabNet](https://github.com/ibm-aur-nlp/PubTabNet), and [UniMER-1M](https://github.com/opendatalab/UniMERNet) for providing valuable datasets.
|
705 |
|
|
|
|
|
706 |
|
707 |
## Copyright
|
708 |
-
Please don’t hesitate to share your valuable feedback — it’s a key motivation that drives us to continuously improve our framework.
|
|
|
16 |
[](https://huggingface.co/echo840/MonkeyOCR)
|
17 |
[](https://github.com/Yuliang-Liu/MonkeyOCR/issues?q=is%3Aopen+is%3Aissue)
|
18 |
[](https://github.com/Yuliang-Liu/MonkeyOCR/issues?q=is%3Aissue+is%3Aclosed)
|
19 |
+
[](https://github.com/Yuliang-Liu/MonkeyOCR/blob/main/LICENSE.txt)
|
20 |
[](https://github.com/Yuliang-Liu/MonkeyOCR)
|
21 |
</div>
|
22 |
|
|
|
24 |
> **MonkeyOCR: Document Parsing with a Structure-Recognition-Relation Triplet Paradigm**<br>
|
25 |
> Zhang Li, Yuliang Liu, Qiang Liu, Zhiyin Ma, Ziyang Zhang, Shuo Zhang, Zidun Guo, Jiarui Zhang, Xinyu Wang, Xiang Bai <br>
|
26 |
[](https://arxiv.org/abs/2506.05218)
|
27 |
+
[](README.md)
|
28 |
[](https://huggingface.co/echo840/MonkeyOCR)
|
29 |
[](https://modelscope.cn/models/l1731396519/MonkeyOCR)
|
30 |
+
[](https://openbayes.com/console/public/tutorials/91ESrGvEvBq)
|
31 |
[](http://vlrlabmonkey.xyz:7685/)
|
32 |
|
33 |
|
|
|
35 |
## Introduction
|
36 |
MonkeyOCR adopts a Structure-Recognition-Relation (SRR) triplet paradigm, which simplifies the multi-tool pipeline of modular approaches while avoiding the inefficiency of using large multimodal models for full-page document processing.
|
37 |
|
38 |
+
1. MonkeyOCR-pro-1.2B surpasses MonkeyOCR-3B by 7.4% on Chinese documents.
|
39 |
+
2. MonkeyOCR-pro-1.2B delivers approximately a 36% speed improvement over MonkeyOCR-pro-3B, with approximately 1.6% drop in performance.
|
40 |
+
3. On olmOCR-Bench, MonkeyOCR-pro-1.2B outperforms Nanonets-OCR-3B by 7.3%.
|
41 |
+
4. On OmniDocBench, MonkeyOCR-pro-3B achieves the best overall performance on both English and Chinese documents, outperforming even closed-source and extra-large open-source VLMs such as Gemini 2.0-Flash, Gemini 2.5-Pro, Qwen2.5-VL-72B, GPT-4o, and InternVL3-78B.
|
42 |
+
|
43 |
+
See detailed results below.
|
44 |
+
|
45 |
+
### Comparing MonkeyOCR with closed-source and extra large open-source VLMs.
|
46 |
+
<a href="https://zimgs.com/i/EKhkhY"><img src="https://v1.ax1x.com/2025/07/15/EKhkhY.png" alt="EKhkhY.png" border="0" /></a>
|
47 |
+
|
48 |
+
## Inference Speed (Pages/s) on Different GPUs and [PDF](https://drive.google.com/drive/folders/1geumlJmVY7UUKdr8324sYZ0FHSAElh7m?usp=sharing) Page Counts
|
49 |
+
|
50 |
+
<table>
|
51 |
+
<thead>
|
52 |
+
<tr align='center'>
|
53 |
+
<th>Model</th>
|
54 |
+
<th>GPU</th>
|
55 |
+
<th>50 Pages</th>
|
56 |
+
<th>100 Pages</th>
|
57 |
+
<th>300 Pages</th>
|
58 |
+
<th>500 Pages</th>
|
59 |
+
<th>1000 Pages</th>
|
60 |
+
</tr>
|
61 |
+
</thead>
|
62 |
+
<tbody>
|
63 |
+
<tr align='center'>
|
64 |
+
<td rowspan='4'>MonkeyOCR-pro-3B</td>
|
65 |
+
<td>3090</td>
|
66 |
+
<td>0.492</td>
|
67 |
+
<td>0.484</td>
|
68 |
+
<td>0.497</td>
|
69 |
+
<td>0.492</td>
|
70 |
+
<td>0.496</td>
|
71 |
+
</tr>
|
72 |
+
<tr align='center'>
|
73 |
+
<td>A6000</td>
|
74 |
+
<td>0.585</td>
|
75 |
+
<td>0.587</td>
|
76 |
+
<td>0.609</td>
|
77 |
+
<td>0.598</td>
|
78 |
+
<td>0.608</td>
|
79 |
+
</tr>
|
80 |
+
<tr align='center'>
|
81 |
+
<td>H800</td>
|
82 |
+
<td>0.923</td>
|
83 |
+
<td>0.768</td>
|
84 |
+
<td>0.897</td>
|
85 |
+
<td>0.930</td>
|
86 |
+
<td>0.891</td>
|
87 |
+
</tr>
|
88 |
+
<tr align='center'>
|
89 |
+
<td>4090</td>
|
90 |
+
<td>0.972</td>
|
91 |
+
<td>0.969</td>
|
92 |
+
<td>1.006</td>
|
93 |
+
<td>0.986</td>
|
94 |
+
<td>1.006</td>
|
95 |
+
</tr>
|
96 |
+
<tr align='center'>
|
97 |
+
<td rowspan='4'>MonkeyOCR-pro-1.2B</td>
|
98 |
+
<td>3090</td>
|
99 |
+
<td>0.615</td>
|
100 |
+
<td>0.660</td>
|
101 |
+
<td>0.677</td>
|
102 |
+
<td>0.687</td>
|
103 |
+
<td>0.683</td>
|
104 |
+
</tr>
|
105 |
+
<tr align='center'>
|
106 |
+
<td>A6000</td>
|
107 |
+
<td>0.709</td>
|
108 |
+
<td>0.786</td>
|
109 |
+
<td>0.825</td>
|
110 |
+
<td>0.829</td>
|
111 |
+
<td>0.825</td>
|
112 |
+
</tr>
|
113 |
+
<tr align='center'>
|
114 |
+
<td>H800</td>
|
115 |
+
<td>0.965</td>
|
116 |
+
<td>1.082</td>
|
117 |
+
<td>1.101</td>
|
118 |
+
<td>1.145</td>
|
119 |
+
<td>1.015</td>
|
120 |
+
</tr>
|
121 |
+
<tr align='center'>
|
122 |
+
<td>4090</td>
|
123 |
+
<td>1.194</td>
|
124 |
+
<td>1.314</td>
|
125 |
+
<td>1.436</td>
|
126 |
+
<td>1.442</td>
|
127 |
+
<td>1.434</td>
|
128 |
+
</tr>
|
129 |
+
</tbody>
|
130 |
+
</table>
|
131 |
|
132 |
+
## VLM OCR Speed (Pages/s) on Different GPUs and [PDF](https://drive.google.com/drive/folders/1geumlJmVY7UUKdr8324sYZ0FHSAElh7m?usp=sharing) Page Counts
|
133 |
+
|
134 |
+
<table>
|
135 |
+
<thead>
|
136 |
+
<tr align='center'>
|
137 |
+
<th>Model</th>
|
138 |
+
<th>GPU</th>
|
139 |
+
<th>50 Pages</th>
|
140 |
+
<th>100 Pages</th>
|
141 |
+
<th>300 Pages</th>
|
142 |
+
<th>500 Pages</th>
|
143 |
+
<th>1000 Pages</th>
|
144 |
+
</tr>
|
145 |
+
</thead>
|
146 |
+
<tbody>
|
147 |
+
<tr align='center'>
|
148 |
+
<td rowspan='4'>MonkeyOCR-pro-3B</td>
|
149 |
+
<td>3090</td>
|
150 |
+
<td>0.705</td>
|
151 |
+
<td>0.680</td>
|
152 |
+
<td>0.711</td>
|
153 |
+
<td>0.700</td>
|
154 |
+
<td>0.724</td>
|
155 |
+
</tr>
|
156 |
+
<tr align='center'>
|
157 |
+
<td>A6000</td>
|
158 |
+
<td>0.885</td>
|
159 |
+
<td>0.860</td>
|
160 |
+
<td>0.915</td>
|
161 |
+
<td>0.892</td>
|
162 |
+
<td>0.934</td>
|
163 |
+
</tr>
|
164 |
+
<tr align='center'>
|
165 |
+
<td>H800</td>
|
166 |
+
<td>1.371</td>
|
167 |
+
<td>1.135</td>
|
168 |
+
<td>1.339</td>
|
169 |
+
<td>1.433</td>
|
170 |
+
<td>1.509</td>
|
171 |
+
</tr>
|
172 |
+
<tr align='center'>
|
173 |
+
<td>4090</td>
|
174 |
+
<td>1.321</td>
|
175 |
+
<td>1.300</td>
|
176 |
+
<td>1.384</td>
|
177 |
+
<td>1.343</td>
|
178 |
+
<td>1.410</td>
|
179 |
+
</tr>
|
180 |
+
<tr align='center'>
|
181 |
+
<td rowspan='4'>MonkeyOCR-pro-1.2B</td>
|
182 |
+
<td>3090</td>
|
183 |
+
<td>0.919</td>
|
184 |
+
<td>1.086</td>
|
185 |
+
<td>1.166</td>
|
186 |
+
<td>1.182</td>
|
187 |
+
<td>1.199</td>
|
188 |
+
</tr>
|
189 |
+
<tr align='center'>
|
190 |
+
<td>A6000</td>
|
191 |
+
<td>1.177</td>
|
192 |
+
<td>1.361</td>
|
193 |
+
<td>1.506</td>
|
194 |
+
<td>1.525</td>
|
195 |
+
<td>1.569</td>
|
196 |
+
</tr>
|
197 |
+
<tr align='center'>
|
198 |
+
<td>H800</td>
|
199 |
+
<td>1.466</td>
|
200 |
+
<td>1.719</td>
|
201 |
+
<td>1.763</td>
|
202 |
+
<td>1.875</td>
|
203 |
+
<td>1.650</td>
|
204 |
+
</tr>
|
205 |
+
<tr align='center'>
|
206 |
+
<td>4090</td>
|
207 |
+
<td>1.759</td>
|
208 |
+
<td>1.987</td>
|
209 |
+
<td>2.260</td>
|
210 |
+
<td>2.345</td>
|
211 |
+
<td>2.415</td>
|
212 |
+
</tr>
|
213 |
+
</tbody>
|
214 |
+
</table>
|
215 |
|
|
|
216 |
|
217 |
+
## Supported Hardware
|
218 |
+
Due to the limited types of GPUs available to us, we may not be able to provide highly accurate hardware specifications. We've tested the model on GPUs such as the 3090, 4090, A6000, H800, A100, and even the 4060 with 8GB of VRAM (suitable for deploying quantized 3B model and 1.2B model). We are very grateful for the feedback and contributions from the open-source community, who have also successfully run the model on [50-series GPUs](https://github.com/Yuliang-Liu/MonkeyOCR/issues/90), [H200](https://github.com/Yuliang-Liu/MonkeyOCR/issues/151), [L20](https://github.com/Yuliang-Liu/MonkeyOCR/issues/133), [V100](https://github.com/Yuliang-Liu/MonkeyOCR/issues/144), [2080 Ti](https://github.com/Yuliang-Liu/MonkeyOCR/pull/1) and [npu](https://github.com/Yuliang-Liu/MonkeyOCR/pull/226/files).
|
219 |
|
220 |
|
221 |
## News
|
222 |
+
* ```2025.07.10 ``` 🚀 We release [MonkeyOCR-pro-1.2B](https://huggingface.co/echo840/MonkeyOCR-pro-1.2B), — a leaner and faster version model that outperforms our previous 3B version in accuracy, speed, and efficiency.
|
223 |
+
* ```2025.06.12 ``` 🚀 The model’s trending on [Hugging Face](https://huggingface.co/models?sort=trending). Thanks for the love!
|
224 |
+
* ```2025.06.05 ``` 🚀 We release [MonkeyOCR](https://huggingface.co/echo840/MonkeyOCR), an English and Chinese documents parsing model.
|
225 |
|
226 |
|
227 |
+
# Quick Start
|
228 |
+
## Locally Install
|
229 |
### 1. Install MonkeyOCR
|
230 |
+
See the [installation guide](https://github.com/Yuliang-Liu/MonkeyOCR/blob/main/docs/install_cuda_pp.md#install-with-cuda-support) to set up your environment.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
231 |
### 2. Download Model Weights
|
232 |
Download our model from Huggingface.
|
233 |
```python
|
234 |
pip install huggingface_hub
|
235 |
|
236 |
+
python tools/download_model.py -n MonkeyOCR-pro-3B # or MonkeyOCR
|
237 |
```
|
238 |
You can also download our model from ModelScope.
|
239 |
|
240 |
```python
|
241 |
pip install modelscope
|
242 |
|
243 |
+
python tools/download_model.py -t modelscope -n MonkeyOCR-pro-3B # or MonkeyOCR
|
244 |
```
|
245 |
### 3. Inference
|
246 |
+
You can parse a file or a directory containing PDFs or images using the following commands:
|
247 |
+
```bash
|
248 |
+
# Replace input_path with the path to a PDF or image or directory
|
249 |
+
|
250 |
+
# End-to-end parsing
|
251 |
+
python parse.py input_path
|
252 |
+
|
253 |
+
# Parse files in a dir with specific group page num
|
254 |
+
python parse.py input_path -g 20
|
255 |
+
|
256 |
+
# Single-task recognition (outputs markdown only)
|
257 |
+
python parse.py input_path -t text/formula/table
|
258 |
+
|
259 |
+
# Parse PDFs in input_path and split results by pages
|
260 |
+
python parse.py input_path -s
|
261 |
+
|
262 |
+
# Specify output directory and model config file
|
263 |
+
python parse.py input_path -o ./output -c config.yaml
|
264 |
+
```
|
265 |
+
|
266 |
+
<details>
|
267 |
+
<summary><b>More usage examples</b></summary>
|
268 |
+
|
269 |
```bash
|
270 |
+
# Single file processing
|
271 |
+
python parse.py input.pdf # Parse single PDF file
|
272 |
+
python parse.py input.pdf -o ./output # Parse with custom output dir
|
273 |
+
python parse.py input.pdf -s # Parse PDF with page splitting
|
274 |
+
python parse.py image.jpg # Parse single image file
|
275 |
+
|
276 |
+
# Single task recognition
|
277 |
+
python parse.py image.jpg -t text # Text recognition from image
|
278 |
+
python parse.py image.jpg -t formula # Formula recognition from image
|
279 |
+
python parse.py image.jpg -t table # Table recognition from image
|
280 |
+
python parse.py document.pdf -t text # Text recognition from all PDF pages
|
281 |
+
|
282 |
+
# Folder processing (all files individually)
|
283 |
+
python parse.py /path/to/folder # Parse all files in folder
|
284 |
+
python parse.py /path/to/folder -s # Parse with page splitting
|
285 |
+
python parse.py /path/to/folder -t text # Single task recognition for all files
|
286 |
+
|
287 |
+
# Multi-file grouping (batch processing by page count)
|
288 |
+
python parse.py /path/to/folder -g 5 # Group files with max 5 total pages
|
289 |
+
python parse.py /path/to/folder -g 10 -s # Group files with page splitting
|
290 |
+
python parse.py /path/to/folder -g 8 -t text # Group files for single task recognition
|
291 |
+
|
292 |
+
# Advanced configurations
|
293 |
+
python parse.py input.pdf -c model_configs.yaml # Custom model configuration
|
294 |
+
python parse.py /path/to/folder -g 15 -s -o ./out # Group files, split pages, custom output
|
295 |
+
python parse.py input.pdf --pred-abandon # Enable predicting abandon elements
|
296 |
+
python parse.py /path/to/folder -g 10 -m # Group files and merge text blocks in output
|
297 |
```
|
298 |
|
299 |
+
</details>
|
300 |
+
|
301 |
+
<details>
|
302 |
+
<summary><b>Output Results</b></summary>
|
303 |
+
|
304 |
+
MonkeyOCR mainly generates three types of output files:
|
305 |
|
306 |
1. **Processed Markdown File** (`your.md`): The final parsed document content in markdown format, containing text, formulas, tables, and other structured elements.
|
307 |
2. **Layout Results** (`your_layout.pdf`): The layout results drawed on origin PDF.
|
|
|
312 |
|
313 |
These files provide both the final formatted output and detailed intermediate results for further analysis or processing.
|
314 |
|
315 |
+
</details>
|
316 |
+
|
317 |
### 4. Gradio Demo
|
318 |
```bash
|
319 |
+
python demo/demo_gradio.py
|
|
|
|
|
320 |
```
|
321 |
+
Once the demo is running, you can access it at http://localhost:7860.
|
322 |
+
|
323 |
+
### 5. Fast API
|
324 |
+
You can start the MonkeyOCR FastAPI service with the following command:
|
325 |
```bash
|
326 |
+
uvicorn api.main:app --port 8000
|
|
|
327 |
```
|
328 |
+
Once the API service is running, you can access the API documentation at http://localhost:8000/docs to explore available endpoints.
|
329 |
+
> [!TIP]
|
330 |
+
> To improve API concurrency performance, consider configuring the inference backend as `lmdeploy_queue` or `vllm_queue`.
|
331 |
|
332 |
+
## Docker Deployment
|
333 |
|
334 |
+
1. Navigate to the `docker` directory:
|
|
|
|
|
335 |
|
336 |
+
```bash
|
337 |
+
cd docker
|
338 |
+
```
|
339 |
|
340 |
+
2. **Prerequisite:** Ensure NVIDIA GPU support is available in Docker (via `nvidia-docker2`).
|
341 |
+
If GPU support is not enabled, run the following to set up the environment:
|
|
|
342 |
|
343 |
+
```bash
|
344 |
+
bash env.sh
|
345 |
+
```
|
346 |
|
347 |
+
3. Build the Docker image:
|
|
|
|
|
348 |
|
349 |
+
```bash
|
350 |
+
docker compose build monkeyocr
|
351 |
+
```
|
352 |
|
353 |
+
> [!IMPORTANT]
|
354 |
+
>
|
355 |
+
> If your GPU is from the 20/30/40-series, V100, L20/L40 or similar, please build the patched Docker image for LMDeploy compatibility:
|
356 |
+
>
|
357 |
+
> ```bash
|
358 |
+
> docker compose build monkeyocr-fix
|
359 |
+
> ```
|
360 |
+
>
|
361 |
+
> Otherwise, you may encounter the following error: `triton.runtime.errors.OutOfResources: out of resource: shared memory`
|
362 |
|
363 |
+
4. Run the container with the Gradio demo (accessible on port 7860):
|
364 |
|
|
|
365 |
```bash
|
366 |
+
docker compose up monkeyocr-demo
|
|
|
367 |
```
|
|
|
|
|
|
|
368 |
|
369 |
+
Alternatively, start an interactive development environment:
|
370 |
|
371 |
+
```bash
|
372 |
+
docker compose run --rm monkeyocr-dev
|
373 |
+
```
|
|
|
|
|
374 |
|
375 |
+
5. Run the FastAPI service (accessible on port 7861):
|
376 |
+
```bash
|
377 |
+
docker compose up monkeyocr-api
|
378 |
+
```
|
379 |
+
Once the API service is running, you can access the API documentation at http://localhost:7861/docs to explore available endpoints.
|
380 |
|
381 |
+
## Windows Support
|
382 |
|
383 |
+
See the [windows support guide](docs/windows_support.md) for details.
|
384 |
|
385 |
+
## Quantization
|
386 |
|
387 |
+
This model can be quantized using AWQ. Follow the instructions in the [quantization guide](docs/Quantization.md).
|
388 |
|
389 |
+
## Benchmark Results
|
390 |
|
391 |
+
Here are the evaluation results of our model on OmniDocBench. MonkeyOCR-3B uses DocLayoutYOLO as the structure detection model, while MonkeyOCR-3B* uses our trained structure detection model with improved Chinese performance.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
392 |
|
393 |
+
### 1. The end-to-end evaluation results of different tasks.
|
394 |
|
395 |
+
<table>
|
396 |
+
<thead>
|
397 |
+
<tr>
|
398 |
+
<th rowspan="2"><strong>Model<br>Type</strong></th>
|
399 |
+
<th rowspan="2"><strong>Methods</strong></th>
|
400 |
+
<th colspan="2"><strong>Overall<sup>Edit</sup>↓</strong></th>
|
401 |
+
<th colspan="2"><strong>Text<sup>Edit</sup>↓</strong></th>
|
402 |
+
<th colspan="2"><strong>Formula<sup>Edit</sup>↓</strong></th>
|
403 |
+
<th colspan="2"><strong>Table<sup>TEDS</sup>↑</strong></th>
|
404 |
+
<th colspan="2"><strong>Table<sup>Edit</sup>↓</strong></th>
|
405 |
+
<th colspan="2"><strong>Read Order<sup>Edit</sup>↓</strong></th>
|
406 |
+
</tr>
|
407 |
+
<tr>
|
408 |
+
<th><em>EN</em></th>
|
409 |
+
<th><em>ZH</em></th>
|
410 |
+
<th><em>EN</em></th>
|
411 |
+
<th><em>ZH</em></th>
|
412 |
+
<th><em>EN</em></th>
|
413 |
+
<th><em>ZH</em></th>
|
414 |
+
<th><em>EN</em></th>
|
415 |
+
<th><em>ZH</em></th>
|
416 |
+
<th><em>EN</em></th>
|
417 |
+
<th><em>ZH</em></th>
|
418 |
+
<th><em>EN</em></th>
|
419 |
+
<th><em>ZH</em></th>
|
420 |
+
</tr>
|
421 |
+
</thead>
|
422 |
+
<tbody>
|
423 |
+
<tr>
|
424 |
+
<td rowspan="8"><strong>Pipeline<br>Tools</strong></td>
|
425 |
+
<td>MinerU</td>
|
426 |
+
<td>0.150</td>
|
427 |
+
<td>0.357</td>
|
428 |
+
<td>0.061</td>
|
429 |
+
<td>0.215</td>
|
430 |
+
<td>0.278</td>
|
431 |
+
<td>0.577</td>
|
432 |
+
<td>78.6</td>
|
433 |
+
<td>62.1</td>
|
434 |
+
<td>0.180</td>
|
435 |
+
<td>0.344</td>
|
436 |
+
<td>0.079</td>
|
437 |
+
<td>0.292</td>
|
438 |
+
</tr>
|
439 |
+
<tr>
|
440 |
+
<td>Marker</td>
|
441 |
+
<td>0.336</td>
|
442 |
+
<td>0.556</td>
|
443 |
+
<td>0.080</td>
|
444 |
+
<td>0.315</td>
|
445 |
+
<td>0.530</td>
|
446 |
+
<td>0.883</td>
|
447 |
+
<td>67.6</td>
|
448 |
+
<td>49.2</td>
|
449 |
+
<td>0.619</td>
|
450 |
+
<td>0.685</td>
|
451 |
+
<td>0.114</td>
|
452 |
+
<td>0.340</td>
|
453 |
+
</tr>
|
454 |
+
<tr>
|
455 |
+
<td>Mathpix</td>
|
456 |
+
<td>0.191</td>
|
457 |
+
<td>0.365</td>
|
458 |
+
<td>0.105</td>
|
459 |
+
<td>0.384</td>
|
460 |
+
<td>0.306</td>
|
461 |
+
<td><strong>0.454</strong></td>
|
462 |
+
<td>77.0</td>
|
463 |
+
<td>67.1</td>
|
464 |
+
<td>0.243</td>
|
465 |
+
<td>0.320</td>
|
466 |
+
<td>0.108</td>
|
467 |
+
<td>0.304</td>
|
468 |
+
</tr>
|
469 |
+
<tr>
|
470 |
+
<td>Docling</td>
|
471 |
+
<td>0.589</td>
|
472 |
+
<td>0.909</td>
|
473 |
+
<td>0.416</td>
|
474 |
+
<td>0.987</td>
|
475 |
+
<td>0.999</td>
|
476 |
+
<td>1</td>
|
477 |
+
<td>61.3</td>
|
478 |
+
<td>25.0</td>
|
479 |
+
<td>0.627</td>
|
480 |
+
<td>0.810</td>
|
481 |
+
<td>0.313</td>
|
482 |
+
<td>0.837</td>
|
483 |
+
</tr>
|
484 |
+
<tr>
|
485 |
+
<td>Pix2Text</td>
|
486 |
+
<td>0.320</td>
|
487 |
+
<td>0.528</td>
|
488 |
+
<td>0.138</td>
|
489 |
+
<td>0.356</td>
|
490 |
+
<td>0.276</td>
|
491 |
+
<td>0.611</td>
|
492 |
+
<td>73.6</td>
|
493 |
+
<td>66.2</td>
|
494 |
+
<td>0.584</td>
|
495 |
+
<td>0.645</td>
|
496 |
+
<td>0.281</td>
|
497 |
+
<td>0.499</td>
|
498 |
+
</tr>
|
499 |
+
<tr>
|
500 |
+
<td>Unstructured</td>
|
501 |
+
<td>0.586</td>
|
502 |
+
<td>0.716</td>
|
503 |
+
<td>0.198</td>
|
504 |
+
<td>0.481</td>
|
505 |
+
<td>0.999</td>
|
506 |
+
<td>1</td>
|
507 |
+
<td>0</td>
|
508 |
+
<td>0.06</td>
|
509 |
+
<td>1</td>
|
510 |
+
<td>0.998</td>
|
511 |
+
<td>0.145</td>
|
512 |
+
<td>0.387</td>
|
513 |
+
</tr>
|
514 |
+
<tr>
|
515 |
+
<td>OpenParse</td>
|
516 |
+
<td>0.646</td>
|
517 |
+
<td>0.814</td>
|
518 |
+
<td>0.681</td>
|
519 |
+
<td>0.974</td>
|
520 |
+
<td>0.996</td>
|
521 |
+
<td>1</td>
|
522 |
+
<td>64.8</td>
|
523 |
+
<td>27.5</td>
|
524 |
+
<td>0.284</td>
|
525 |
+
<td>0.639</td>
|
526 |
+
<td>0.595</td>
|
527 |
+
<td>0.641</td>
|
528 |
+
</tr>
|
529 |
+
<tr>
|
530 |
+
<td>PP-StructureV3</td>
|
531 |
+
<td>0.145</td>
|
532 |
+
<td><strong>0.206</strong></td>
|
533 |
+
<td>0.058</td>
|
534 |
+
<td><strong>0.088</strong></td>
|
535 |
+
<td>0.295</td>
|
536 |
+
<td>0.535</td>
|
537 |
+
<td>-</td>
|
538 |
+
<td>-</td>
|
539 |
+
<td>0.159</td>
|
540 |
+
<td><strong>0.109</strong></td>
|
541 |
+
<td><strong>0.069</strong></td>
|
542 |
+
<td><strong>0.091</strong></td>
|
543 |
+
</tr>
|
544 |
+
<tr>
|
545 |
+
<td rowspan="8"><strong>Expert<br>VLMs</strong></td>
|
546 |
+
<td>GOT-OCR</td>
|
547 |
+
<td>0.287</td>
|
548 |
+
<td>0.411</td>
|
549 |
+
<td>0.189</td>
|
550 |
+
<td>0.315</td>
|
551 |
+
<td>0.360</td>
|
552 |
+
<td>0.528</td>
|
553 |
+
<td>53.2</td>
|
554 |
+
<td>47.2</td>
|
555 |
+
<td>0.459</td>
|
556 |
+
<td>0.520</td>
|
557 |
+
<td>0.141</td>
|
558 |
+
<td>0.280</td>
|
559 |
+
</tr>
|
560 |
+
<tr>
|
561 |
+
<td>Nougat</td>
|
562 |
+
<td>0.452</td>
|
563 |
+
<td>0.973</td>
|
564 |
+
<td>0.365</td>
|
565 |
+
<td>0.998</td>
|
566 |
+
<td>0.488</td>
|
567 |
+
<td>0.941</td>
|
568 |
+
<td>39.9</td>
|
569 |
+
<td>0</td>
|
570 |
+
<td>0.572</td>
|
571 |
+
<td>1.000</td>
|
572 |
+
<td>0.382</td>
|
573 |
+
<td>0.954</td>
|
574 |
+
</tr>
|
575 |
+
<tr>
|
576 |
+
<td>Mistral OCR</td>
|
577 |
+
<td>0.268</td>
|
578 |
+
<td>0.439</td>
|
579 |
+
<td>0.072</td>
|
580 |
+
<td>0.325</td>
|
581 |
+
<td>0.318</td>
|
582 |
+
<td>0.495</td>
|
583 |
+
<td>75.8</td>
|
584 |
+
<td>63.6</td>
|
585 |
+
<td>0.600</td>
|
586 |
+
<td>0.650</td>
|
587 |
+
<td>0.083</td>
|
588 |
+
<td>0.284</td>
|
589 |
+
</tr>
|
590 |
+
<tr>
|
591 |
+
<td>OLMOCR-sglang</td>
|
592 |
+
<td>0.326</td>
|
593 |
+
<td>0.469</td>
|
594 |
+
<td>0.097</td>
|
595 |
+
<td>0.293</td>
|
596 |
+
<td>0.455</td>
|
597 |
+
<td>0.655</td>
|
598 |
+
<td>68.1</td>
|
599 |
+
<td>61.3</td>
|
600 |
+
<td>0.608<td>0.652</td>
|
601 |
+
<td>0.145</td>
|
602 |
+
<td>0.277</td>
|
603 |
+
</tr>
|
604 |
+
<tr>
|
605 |
+
<td>SmolDocling-256M</td>
|
606 |
+
<td>0.493</td>
|
607 |
+
<td>0.816</td>
|
608 |
+
<td>0.262</td>
|
609 |
+
<td>0.838</td>
|
610 |
+
<td>0.753</td>
|
611 |
+
<td>0.997</td>
|
612 |
+
<td>44.9</td>
|
613 |
+
<td>16.5</td>
|
614 |
+
<td>0.729</td>
|
615 |
+
<td>0.907</td>
|
616 |
+
<td>0.227</td>
|
617 |
+
<td>0.522</td>
|
618 |
+
</tr>
|
619 |
+
<tr>
|
620 |
+
<td>Dolphin</td>
|
621 |
+
<td>0.206</td>
|
622 |
+
<td>0.306</td>
|
623 |
+
<td>0.107</td>
|
624 |
+
<td>0.197</td>
|
625 |
+
<td>0.447</td>
|
626 |
+
<td>0.580</td>
|
627 |
+
<td>77.3</td>
|
628 |
+
<td>67.2</td>
|
629 |
+
<td>0.180</td>
|
630 |
+
<td>0.285</td>
|
631 |
+
<td>0.091</td>
|
632 |
+
<td>0.162</td>
|
633 |
+
</tr>
|
634 |
+
<tr>
|
635 |
+
<td>MinerU 2</td>
|
636 |
+
<td>0.139</td>
|
637 |
+
<td>0.240</td>
|
638 |
+
<td><strong>0.047</strong></td>
|
639 |
+
<td>0.109</td>
|
640 |
+
<td>0.297</td>
|
641 |
+
<td>0.536</td>
|
642 |
+
<td><strong>82.5</strong></td>
|
643 |
+
<td>79.0</td>
|
644 |
+
<td>0.141</td>
|
645 |
+
<td>0.195</td>
|
646 |
+
<td><strong>0.069</strong></td>
|
647 |
+
<td>0.118</td>
|
648 |
+
</tr>
|
649 |
+
<tr>
|
650 |
+
<td>OCRFlux</td>
|
651 |
+
|
652 |
+
<td>0.195</td>
|
653 |
+
<td>0.281</td>
|
654 |
+
<td>0.064</td>
|
655 |
+
<td>0.183</td>
|
656 |
+
<td>0.379</td>
|
657 |
+
<td>0.613</td>
|
658 |
+
<td>71.6</td>
|
659 |
+
<td>81.3</td>
|
660 |
+
<td>0.253</td>
|
661 |
+
<td>0.139</td>
|
662 |
+
<td>0.086</td>
|
663 |
+
<td>0.187</td>
|
664 |
+
|
665 |
+
|
666 |
+
</tr>
|
667 |
+
<tr>
|
668 |
+
<td rowspan="3"><strong>General<br>VLMs</strong></td>
|
669 |
+
<td>GPT4o</td>
|
670 |
+
<td>0.233</td>
|
671 |
+
<td>0.399</td>
|
672 |
+
<td>0.144</td>
|
673 |
+
<td>0.409</td>
|
674 |
+
<td>0.425</td>
|
675 |
+
<td>0.606</td>
|
676 |
+
<td>72.0</td>
|
677 |
+
<td>62.9</td>
|
678 |
+
<td>0.234</td>
|
679 |
+
<td>0.329</td>
|
680 |
+
<td>0.128</td>
|
681 |
+
<td>0.251</td>
|
682 |
+
</tr>
|
683 |
+
<tr>
|
684 |
+
<td>Qwen2.5-VL-7B</td>
|
685 |
+
<td>0.312</td>
|
686 |
+
<td>0.406</td>
|
687 |
+
<td>0.157</td>
|
688 |
+
<td>0.228</td>
|
689 |
+
<td>0.351</td>
|
690 |
+
<td>0.574</td>
|
691 |
+
<td>76.4</td>
|
692 |
+
<td>72.2</td>
|
693 |
+
<td>0.588</td>
|
694 |
+
<td>0.619</td>
|
695 |
+
<td>0.149</td>
|
696 |
+
<td>0.203</td>
|
697 |
+
</tr>
|
698 |
+
<tr>
|
699 |
+
<td>InternVL3-8B</td>
|
700 |
+
<td>0.314</td>
|
701 |
+
<td>0.383</td>
|
702 |
+
<td>0.134</td>
|
703 |
+
<td>0.218</td>
|
704 |
+
<td>0.417</td>
|
705 |
+
<td>0.563</td>
|
706 |
+
<td>66.1</td>
|
707 |
+
<td>73.1</td>
|
708 |
+
<td>0.586</td>
|
709 |
+
<td>0.564</td>
|
710 |
+
<td>0.118</td>
|
711 |
+
<td>0.186</td>
|
712 |
+
</tr>
|
713 |
+
<tr>
|
714 |
+
<td rowspan="4"><strong>Mix</strong></td>
|
715 |
+
<td><strong>MonkeyOCR-3B <a href="https://huggingface.co/echo840/MonkeyOCR/blob/main/Structure/doclayout_yolo_docstructbench_imgsz1280_2501.pt">[Weight]</a></strong></td>
|
716 |
+
<td>0.140</td>
|
717 |
+
<td>0.297</td>
|
718 |
+
<td>0.058</td>
|
719 |
+
<td>0.185</td>
|
720 |
+
<td>0.238</td>
|
721 |
+
<td>0.506</td>
|
722 |
+
<td>80.2</td>
|
723 |
+
<td>77.7</td>
|
724 |
+
<td>0.170</td>
|
725 |
+
<td>0.253</td>
|
726 |
+
<td>0.093</td>
|
727 |
+
<td>0.244</td>
|
728 |
+
</tr>
|
729 |
+
<tr>
|
730 |
+
<td><strong>MonkeyOCR-3B* <a href="https://huggingface.co/echo840/MonkeyOCR/blob/main/Structure/layout_zh.pt">[Weight]</a></strong></td>
|
731 |
+
<td>0.154</td>
|
732 |
+
<td>0.277</td>
|
733 |
+
<td>0.073</td>
|
734 |
+
<td>0.134</td>
|
735 |
+
<td>0.255</td>
|
736 |
+
<td>0.529</td>
|
737 |
+
<td>78.2</td>
|
738 |
+
<td>76.2</td>
|
739 |
+
<td>0.182</td>
|
740 |
+
<td>0.262</td>
|
741 |
+
<td>0.105</td>
|
742 |
+
<td>0.183</td>
|
743 |
+
</tr>
|
744 |
+
<tr>
|
745 |
+
<td><strong>MonkeyOCR-pro-3B <a href="https://huggingface.co/echo840/MonkeyOCR-pro-3B">[Weight]</a></strong></td>
|
746 |
+
<td><strong>0.138</strong></td>
|
747 |
+
<td><strong>0.206</strong></td>
|
748 |
+
<td>0.067</td>
|
749 |
+
<td>0.107</td>
|
750 |
+
<td><strong>0.246</strong></td>
|
751 |
+
<td><strong>0.421</strong></td>
|
752 |
+
<td>81.5</td>
|
753 |
+
<td><strong>87.5</strong></td>
|
754 |
+
<td><strong>0.139</strong></td>
|
755 |
+
<td>0.111</td>
|
756 |
+
<td>0.100</td>
|
757 |
+
<td>0.185</td>
|
758 |
+
</tr>
|
759 |
+
<tr>
|
760 |
+
<td><strong>MonkeyOCR-pro-1.2B <a href="https://huggingface.co/echo840/MonkeyOCR-pro-1.2B">[Weight]</a></strong></td>
|
761 |
+
<td>0.153</td>
|
762 |
+
<td>0.223</td>
|
763 |
+
<td>0.066</td>
|
764 |
+
<td>0.123</td>
|
765 |
+
<td>0.272</td>
|
766 |
+
<td>0.449</td>
|
767 |
+
<td>76.5</td>
|
768 |
+
<td>83.7</td>
|
769 |
+
<td>0.176</td>
|
770 |
+
<td>0.131</td>
|
771 |
+
<td>0.097</td>
|
772 |
+
<td>0.187</td>
|
773 |
+
</tr>
|
774 |
+
</tbody>
|
775 |
+
</table>
|
776 |
|
777 |
|
778 |
### 2. The end-to-end text recognition performance across 9 PDF page types.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
779 |
|
780 |
+
<table>
|
781 |
+
<thead>
|
782 |
+
<tr>
|
783 |
+
<th><strong>Model<br>Type</strong></th>
|
784 |
+
<th><strong>Models</strong></th>
|
785 |
+
<th><strong>Book</strong></th>
|
786 |
+
<th><strong>Slides</strong></th>
|
787 |
+
<th><strong>Financial<br>Report</strong></th>
|
788 |
+
<th><strong>Textbook</strong></th>
|
789 |
+
<th><strong>Exam<br>Paper</strong></th>
|
790 |
+
<th><strong>Magazine</strong></th>
|
791 |
+
<th><strong>Academic<br>Papers</strong></th>
|
792 |
+
<th><strong>Notes</strong></th>
|
793 |
+
<th><strong>Newspaper</strong></th>
|
794 |
+
<th><strong>Overall</strong></th>
|
795 |
+
</tr>
|
796 |
+
</thead>
|
797 |
+
<tbody>
|
798 |
+
<tr>
|
799 |
+
<td rowspan="3"><strong>Pipeline<br>Tools</strong></td>
|
800 |
+
<td>MinerU</td>
|
801 |
+
<td>0.055</td>
|
802 |
+
<td>0.124</td>
|
803 |
+
<td><u>0.033</u></td>
|
804 |
+
<td>0.102</td>
|
805 |
+
<td>0.159</td>
|
806 |
+
<td><strong>0.072</strong></td>
|
807 |
+
<td><u>0.025</u></td>
|
808 |
+
<td>0.984</td>
|
809 |
+
<td>0.171</td>
|
810 |
+
<td>0.206</td>
|
811 |
+
</tr>
|
812 |
+
<tr>
|
813 |
+
<td>Marker</td>
|
814 |
+
<td>0.074</td>
|
815 |
+
<td>0.340</td>
|
816 |
+
<td>0.089</td>
|
817 |
+
<td>0.319</td>
|
818 |
+
<td>0.452</td>
|
819 |
+
<td>0.153</td>
|
820 |
+
<td>0.059</td>
|
821 |
+
<td>0.651</td>
|
822 |
+
<td>0.192</td>
|
823 |
+
<td>0.274</td>
|
824 |
+
</tr>
|
825 |
+
<tr>
|
826 |
+
<td>Mathpix</td>
|
827 |
+
<td>0.131</td>
|
828 |
+
<td>0.220</td>
|
829 |
+
<td>0.202</td>
|
830 |
+
<td>0.216</td>
|
831 |
+
<td>0.278</td>
|
832 |
+
<td>0.147</td>
|
833 |
+
<td>0.091</td>
|
834 |
+
<td>0.634</td>
|
835 |
+
<td>0.690</td>
|
836 |
+
<td>0.300</td>
|
837 |
+
</tr>
|
838 |
+
<tr>
|
839 |
+
<td rowspan="4"><strong>Expert<br>VLMs</strong></td>
|
840 |
+
<td>GOT-OCR</td>
|
841 |
+
<td>0.111</td>
|
842 |
+
<td>0.222</td>
|
843 |
+
<td>0.067</td>
|
844 |
+
<td>0.132</td>
|
845 |
+
<td>0.204</td>
|
846 |
+
<td>0.198</td>
|
847 |
+
<td>0.179</td>
|
848 |
+
<td>0.388</td>
|
849 |
+
<td>0.771</td>
|
850 |
+
<td>0.267</td>
|
851 |
+
</tr>
|
852 |
+
<tr>
|
853 |
+
<td>Nougat</td>
|
854 |
+
<td>0.734</td>
|
855 |
+
<td>0.958</td>
|
856 |
+
<td>1.000</td>
|
857 |
+
<td>0.820</td>
|
858 |
+
<td>0.930</td>
|
859 |
+
<td>0.830</td>
|
860 |
+
<td>0.214</td>
|
861 |
+
<td>0.991</td>
|
862 |
+
<td>0.871</td>
|
863 |
+
<td>0.806</td>
|
864 |
+
</tr>
|
865 |
+
<tr>
|
866 |
+
<td>Dolphin</td>
|
867 |
+
<td>0.091</td>
|
868 |
+
<td>0.131</td>
|
869 |
+
<td>0.057</td>
|
870 |
+
<td>0.146</td>
|
871 |
+
<td>0.231</td>
|
872 |
+
<td>0.121</td>
|
873 |
+
<td>0.074</td>
|
874 |
+
<td>0.363</td>
|
875 |
+
<td>0.307</td>
|
876 |
+
<td>0.177</td>
|
877 |
+
</tr>
|
878 |
+
<tr>
|
879 |
+
<td>OCRFlux</td>
|
880 |
+
<td>0.068</td>
|
881 |
+
<td>0.125</td>
|
882 |
+
<td>0.092</td>
|
883 |
+
<td>0.102</td>
|
884 |
+
<td>0.119</td>
|
885 |
+
<td>0.083</td>
|
886 |
+
<td>0.047</td>
|
887 |
+
<td>0.223</td>
|
888 |
+
<td>0.536</td>
|
889 |
+
<td>0.149</td>
|
890 |
+
</tr>
|
891 |
+
<tr>
|
892 |
+
<td rowspan="3"><strong>General<br>VLMs</strong></td>
|
893 |
+
<td>GPT4o</td>
|
894 |
+
<td>0.157</td>
|
895 |
+
<td>0.163</td>
|
896 |
+
<td>0.348</td>
|
897 |
+
<td>0.187</td>
|
898 |
+
<td>0.281</td>
|
899 |
+
<td>0.173</td>
|
900 |
+
<td>0.146</td>
|
901 |
+
<td>0.607</td>
|
902 |
+
<td>0.751</td>
|
903 |
+
<td>0.316</td>
|
904 |
+
</tr>
|
905 |
+
<tr>
|
906 |
+
<td>Qwen2.5-VL-7B</td>
|
907 |
+
<td>0.148</td>
|
908 |
+
<td><strong>0.053</strong></td>
|
909 |
+
<td>0.111</td>
|
910 |
+
<td>0.137</td>
|
911 |
+
<td>0.189</td>
|
912 |
+
<td>0.117</td>
|
913 |
+
<td>0.134</td>
|
914 |
+
<td>0.204</td>
|
915 |
+
<td>0.706</td>
|
916 |
+
<td>0.205</td>
|
917 |
+
</tr>
|
918 |
+
<tr>
|
919 |
+
<td>InternVL3-8B</td>
|
920 |
+
<td>0.163</td>
|
921 |
+
<td><u>0.056</u></td>
|
922 |
+
<td>0.107</td>
|
923 |
+
<td>0.109</td>
|
924 |
+
<td>0.129</td>
|
925 |
+
<td>0.100</td>
|
926 |
+
<td>0.159</td>
|
927 |
+
<td><strong>0.150</strong></td>
|
928 |
+
<td>0.681</td>
|
929 |
+
<td>0.188</td>
|
930 |
+
</tr>
|
931 |
+
<tr>
|
932 |
+
<td rowspan="4"><strong>Mix</strong></td>
|
933 |
+
<td><strong>MonkeyOCR-3B <a href="https://huggingface.co/echo840/MonkeyOCR/blob/main/Structure/doclayout_yolo_docstructbench_imgsz1280_2501.pt">[Weight]</a></strong></td>
|
934 |
+
<td><strong>0.046</strong></td>
|
935 |
+
<td>0.120</td>
|
936 |
+
<td><strong>0.024</strong></td>
|
937 |
+
<td>0.100</td>
|
938 |
+
<td>0.129</td>
|
939 |
+
<td>0.086</td>
|
940 |
+
<td><strong>0.024</strong></td>
|
941 |
+
<td>0.643</td>
|
942 |
+
<td><u>0.131</u></td>
|
943 |
+
<td>0.155</td>
|
944 |
+
</tr>
|
945 |
+
<tr>
|
946 |
+
<td><strong>MonkeyOCR-3B* <a href="https://huggingface.co/echo840/MonkeyOCR/blob/main/Structure/layout_zh.pt">[Weight]</a></strong></td>
|
947 |
+
<td><u>0.054</u></td>
|
948 |
+
<td>0.203</td>
|
949 |
+
<td>0.038</td>
|
950 |
+
<td>0.112</td>
|
951 |
+
<td>0.138</td>
|
952 |
+
<td>0.111</td>
|
953 |
+
<td>0.032</td>
|
954 |
+
<td>0.194</td>
|
955 |
+
<td>0.136</td>
|
956 |
+
<td>0.120</td>
|
957 |
+
</tr>
|
958 |
+
<tr>
|
959 |
+
<td><strong>MonkeyOCR-pro-3B <a href="https://huggingface.co/echo840/MonkeyOCR-pro-3B">[Weight]</a></strong></td>
|
960 |
+
<td>0.084</td>
|
961 |
+
<td>0.129</td>
|
962 |
+
<td>0.060</td>
|
963 |
+
<td><strong>0.090</strong></td>
|
964 |
+
<td><strong>0.107</strong></td>
|
965 |
+
<td><u>0.073</u></td>
|
966 |
+
<td>0.050</td>
|
967 |
+
<td><u>0.171</u></td>
|
968 |
+
<td><strong>0.107</strong></td>
|
969 |
+
<td><strong>0.100</strong></td>
|
970 |
+
</tr>
|
971 |
+
<tr>
|
972 |
+
<td><strong>MonkeyOCR-pro-1.2B <a href="https://huggingface.co/echo840/MonkeyOCR-pro-1.2B">[Weight]</a></strong></td>
|
973 |
+
<td>0.087</td>
|
974 |
+
<td>0.142</td>
|
975 |
+
<td>0.059</td>
|
976 |
+
<td><u>0.093</u></td>
|
977 |
+
<td><u>0.115</u></td>
|
978 |
+
<td>0.085</td>
|
979 |
+
<td>0.045</td>
|
980 |
+
<td>0.226</td>
|
981 |
+
<td>0.122</td>
|
982 |
+
<td><u>0.112</u></td>
|
983 |
+
</tr>
|
984 |
+
</tbody>
|
985 |
+
</table>
|
986 |
|
987 |
+
### 3. The evaluation results of olmOCR-bench.
|
988 |
+
|
989 |
+
<table>
|
990 |
+
<thead>
|
991 |
+
<tr>
|
992 |
+
<th>Model</th>
|
993 |
+
<th>ArXiv</th>
|
994 |
+
<th>Old Scans<br>Math</th>
|
995 |
+
<th>Tables</th>
|
996 |
+
<th>Old Scans</th>
|
997 |
+
<th>Headers and<br>Footers</th>
|
998 |
+
<th>Multi<br>column</th>
|
999 |
+
<th>Long Tiny<br>Text</th>
|
1000 |
+
<th>Base</th>
|
1001 |
+
<th>Overall</th>
|
1002 |
+
</tr>
|
1003 |
+
</thead>
|
1004 |
+
<tbody>
|
1005 |
+
<tr>
|
1006 |
+
<td>GOT OCR</td>
|
1007 |
+
<td>52.7</td>
|
1008 |
+
<td>52.0</td>
|
1009 |
+
<td>0.2</td>
|
1010 |
+
<td>22.1</td>
|
1011 |
+
<td>93.6</td>
|
1012 |
+
<td>42.0</td>
|
1013 |
+
<td>29.9</td>
|
1014 |
+
<td>94.0</td>
|
1015 |
+
<td>48.3 ± 1.1</td>
|
1016 |
+
</tr>
|
1017 |
+
<tr>
|
1018 |
+
<td>Marker</td>
|
1019 |
+
<td>76.0</td>
|
1020 |
+
<td>57.9</td>
|
1021 |
+
<td>57.6</td>
|
1022 |
+
<td>27.8</td>
|
1023 |
+
<td>84.9</td>
|
1024 |
+
<td>72.9</td>
|
1025 |
+
<td>84.6</td>
|
1026 |
+
<td><strong>99.1</strong></td>
|
1027 |
+
<td>70.1 ± 1.1</td>
|
1028 |
+
</tr>
|
1029 |
+
<tr>
|
1030 |
+
<td>MinerU</td>
|
1031 |
+
<td>75.4</td>
|
1032 |
+
<td>47.4</td>
|
1033 |
+
<td>60.9</td>
|
1034 |
+
<td>17.3</td>
|
1035 |
+
<td><strong>96.6</strong></td>
|
1036 |
+
<td>59.0</td>
|
1037 |
+
<td>39.1</td>
|
1038 |
+
<td>96.6</td>
|
1039 |
+
<td>61.5 ± 1.1</td>
|
1040 |
+
</tr>
|
1041 |
+
<tr>
|
1042 |
+
<td>Mistral OCR</td>
|
1043 |
+
<td>77.2</td>
|
1044 |
+
<td>67.5</td>
|
1045 |
+
<td>60.6</td>
|
1046 |
+
<td>29.3</td>
|
1047 |
+
<td>93.6</td>
|
1048 |
+
<td>71.3</td>
|
1049 |
+
<td>77.1</td>
|
1050 |
+
<td>99.4</td>
|
1051 |
+
<td>72.0 ± 1.1</td>
|
1052 |
+
</tr>
|
1053 |
+
<tr>
|
1054 |
+
<td>Nanonets OCR</td>
|
1055 |
+
<td>67.0</td>
|
1056 |
+
<td>68.6</td>
|
1057 |
+
<td><strong>77.7</strong></td>
|
1058 |
+
<td>39.5</td>
|
1059 |
+
<td>40.7</td>
|
1060 |
+
<td>69.9</td>
|
1061 |
+
<td>53.4</td>
|
1062 |
+
<td>99.3</td>
|
1063 |
+
<td>64.5 ± 1.1</td>
|
1064 |
+
</tr>
|
1065 |
+
<tr>
|
1066 |
+
<td>GPT-4o<br>(No Anchor)</td>
|
1067 |
+
<td>51.5</td>
|
1068 |
+
<td><strong>75.5</strong></td>
|
1069 |
+
<td>69.1</td>
|
1070 |
+
<td>40.9</td>
|
1071 |
+
<td>94.2</td>
|
1072 |
+
<td>68.9</td>
|
1073 |
+
<td>54.1</td>
|
1074 |
+
<td>96.7</td>
|
1075 |
+
<td>68.9 ± 1.1</td>
|
1076 |
+
</tr>
|
1077 |
+
<tr>
|
1078 |
+
<td>GPT-4o<br>(Anchored)</td>
|
1079 |
+
<td>53.5</td>
|
1080 |
+
<td>74.5</td>
|
1081 |
+
<td>70.0</td>
|
1082 |
+
<td>40.7</td>
|
1083 |
+
<td>93.8</td>
|
1084 |
+
<td>69.3</td>
|
1085 |
+
<td>60.6</td>
|
1086 |
+
<td>96.8</td>
|
1087 |
+
<td>69.9 ± 1.1</td>
|
1088 |
+
</tr>
|
1089 |
+
<tr>
|
1090 |
+
<td>Gemini Flash 2<br>(No Anchor)</td>
|
1091 |
+
<td>32.1</td>
|
1092 |
+
<td>56.3</td>
|
1093 |
+
<td>61.4</td>
|
1094 |
+
<td>27.8</td>
|
1095 |
+
<td>48.0</td>
|
1096 |
+
<td>58.7</td>
|
1097 |
+
<td><strong>84.4</strong></td>
|
1098 |
+
<td>94.0</td>
|
1099 |
+
<td>57.8 ± 1.1</td>
|
1100 |
+
</tr>
|
1101 |
+
<tr>
|
1102 |
+
<td>Gemini Flash 2<br>(Anchored)</td>
|
1103 |
+
<td>54.5</td>
|
1104 |
+
<td>56.1</td>
|
1105 |
+
<td>72.1</td>
|
1106 |
+
<td>34.2</td>
|
1107 |
+
<td>64.7</td>
|
1108 |
+
<td>61.5</td>
|
1109 |
+
<td>71.5</td>
|
1110 |
+
<td>95.6</td>
|
1111 |
+
<td>63.8 ± 1.2</td>
|
1112 |
+
</tr>
|
1113 |
+
<tr>
|
1114 |
+
<td>Qwen 2 VL<br>(No Anchor)</td>
|
1115 |
+
<td>19.7</td>
|
1116 |
+
<td>31.7</td>
|
1117 |
+
<td>24.2</td>
|
1118 |
+
<td>17.1</td>
|
1119 |
+
<td>88.9</td>
|
1120 |
+
<td>8.3</td>
|
1121 |
+
<td>6.8</td>
|
1122 |
+
<td>55.5</td>
|
1123 |
+
<td>31.5 ± 0.9</td>
|
1124 |
+
</tr>
|
1125 |
+
<tr>
|
1126 |
+
<td>Qwen 2.5 VL<br>(No Anchor)</td>
|
1127 |
+
<td>63.1</td>
|
1128 |
+
<td>65.7</td>
|
1129 |
+
<td>67.3</td>
|
1130 |
+
<td>38.6</td>
|
1131 |
+
<td>73.6</td>
|
1132 |
+
<td>68.3</td>
|
1133 |
+
<td>49.1</td>
|
1134 |
+
<td>98.3</td>
|
1135 |
+
<td>65.5 ± 1.2</td>
|
1136 |
+
</tr>
|
1137 |
+
<tr>
|
1138 |
+
<td>olmOCR v0.1.75<br>(No Anchor)</td>
|
1139 |
+
<td>71.5</td>
|
1140 |
+
<td>71.4</td>
|
1141 |
+
<td>71.4</td>
|
1142 |
+
<td><strong>42.8</strong></td>
|
1143 |
+
<td>94.1</td>
|
1144 |
+
<td>77.7</td>
|
1145 |
+
<td>71.0</td>
|
1146 |
+
<td>97.8</td>
|
1147 |
+
<td>74.7 ± 1.1</td>
|
1148 |
+
</tr>
|
1149 |
+
<tr>
|
1150 |
+
<td>olmOCR v0.1.75<br>(Anchored)</td>
|
1151 |
+
<td>74.9</td>
|
1152 |
+
<td>71.2</td>
|
1153 |
+
<td>71.0</td>
|
1154 |
+
<td>42.2</td>
|
1155 |
+
<td>94.5</td>
|
1156 |
+
<td><strong>78.3</strong></td>
|
1157 |
+
<td>73.3</td>
|
1158 |
+
<td>98.3</td>
|
1159 |
+
<td>75.5 ± 1.0</td>
|
1160 |
+
</tr>
|
1161 |
+
<tr>
|
1162 |
+
<td>MonkeyOCR-pro-3B <a href="https://huggingface.co/echo840/MonkeyOCR-pro-3B">[Weight]</a></td>
|
1163 |
+
<td><strong>83.8</strong></td>
|
1164 |
+
<td>68.8</td>
|
1165 |
+
<td>74.6</td>
|
1166 |
+
<td>36.1</td>
|
1167 |
+
<td>91.2</td>
|
1168 |
+
<td>76.6</td>
|
1169 |
+
<td>80.1</td>
|
1170 |
+
<td>95.3</td>
|
1171 |
+
<td><strong>75.8 ± 1.0</strong></td>
|
1172 |
+
</tr>
|
1173 |
+
<tr>
|
1174 |
+
<td>MonkeyOCR-pro-1.2B <a href="https://huggingface.co/echo840/MonkeyOCR-pro-1.2B">[Weight]</a></td>
|
1175 |
+
<td>80.5</td>
|
1176 |
+
<td>62.9</td>
|
1177 |
+
<td>71.1</td>
|
1178 |
+
<td>32.9</td>
|
1179 |
+
<td>92.2</td>
|
1180 |
+
<td>68.3</td>
|
1181 |
+
<td>74.0</td>
|
1182 |
+
<td>92.6</td>
|
1183 |
+
<td>71.8 ± 1.1</td>
|
1184 |
+
</tr>
|
1185 |
+
</tbody>
|
1186 |
+
</table>
|
1187 |
|
1188 |
## Visualization Demo
|
1189 |
|
1190 |
+
Get a Quick Hands-On Experience with Our Demo: http://vlrlabmonkey.xyz:7685 (The latest model is available for selection)
|
1191 |
|
1192 |
> Our demo is simple and easy to use:
|
1193 |
>
|
|
|
1230 |
|
1231 |
|
1232 |
## Acknowledgments
|
1233 |
+
We would like to thank [MinerU](https://github.com/opendatalab/MinerU), [DocLayout-YOLO](https://github.com/opendatalab/DocLayout-YOLO), [PyMuPDF](https://github.com/pymupdf/PyMuPDF), [layoutreader](https://github.com/ppaanngggg/layoutreader), [Qwen2.5-VL](https://github.com/QwenLM/Qwen2.5-VL), [LMDeploy](https://github.com/InternLM/lmdeploy), [PP-StructureV3](https://github.com/PaddlePaddle/PaddleOCR), [PP-DocLayout_plus-L](https://huggingface.co/PaddlePaddle/PP-DocLayout_plus-L), and [InternVL3](https://github.com/OpenGVLab/InternVL) for providing base code and models, as well as their contributions to this field. We also thank [M6Doc](https://github.com/HCIILAB/M6Doc), [DocLayNet](https://github.com/DS4SD/DocLayNet), [CDLA](https://github.com/buptlihang/CDLA), [D4LA](https://github.com/AlibabaResearch/AdvancedLiterateMachinery), [DocGenome](https://github.com/Alpha-Innovator/DocGenome), [PubTabNet](https://github.com/ibm-aur-nlp/PubTabNet), and [UniMER-1M](https://github.com/opendatalab/UniMERNet) for providing valuable datasets. We also thank everyone who contributed to this open-source effort.
|
1234 |
|
1235 |
+
## Limitation
|
1236 |
+
Currently, MonkeyOCR do not yet fully support for photographed text, handwritten content, Traditional Chinese characters, or multilingual text. We plan to consider adding support for these features in future public releases. Additionally, our model is deployed on a single GPU, so if too many users upload files at the same time, issues like “This application is currently busy” may occur. The processing time shown on the demo page does not reflect computation time alone—it also includes result uploading and other overhead. During periods of high traffic, this time may be longer. The inference speeds of MonkeyOCR, MinerU, and Qwen2.5 VL-7B were measured on an H800 GPU.
|
1237 |
|
1238 |
## Copyright
|
1239 |
+
Please don’t hesitate to share your valuable feedback — it’s a key motivation that drives us to continuously improve our framework. Note: Our model is intended for academic research and non-commercial use only. If you are interested in faster (smaller) or stronger one, please contact us at [email protected] or [email protected].
|