diff --git a/docs/version3.x/pipeline_usage/PP-ChatOCRv4.en.md b/docs/version3.x/pipeline_usage/PP-ChatOCRv4.en.md index c1f4fedc95..5255c09313 100644 --- a/docs/version3.x/pipeline_usage/PP-ChatOCRv4.en.md +++ b/docs/version3.x/pipeline_usage/PP-ChatOCRv4.en.md @@ -8,12 +8,23 @@ PP-ChatOCRv4-doc is a unique document and image intelligent analysis solution fr -The Document Scene Information Extraction v4 pipeline includes modules for **Layout Region Detection**, **Table Structure Recognition**, **Table Classification**, **Table Cell Localization**, **Text Detection**, **Text Recognition**, **Seal Text Detection**, **Text Image Rectification**, and **Document Image Orientation Classification**. +The PP-ChatOCRv4 pipeline includes the following 9 modules. Each module can be trained and inferred independently and includes multiple models. For more details, please click on the respective module to view the documentation. -If you prioritize model accuracy, choose a model with higher accuracy. If you prioritize inference speed, select a model with faster inference. If you prioritize model storage size, choose a model with a smaller storage size. Benchmarks for some models are as follows: +- [Document Image Orientation Classification Module](../module_usage/doc_img_orientation_classification.en.md) (Optional) +- [Text Image Unwarping Module](../module_usage/text_image_unwarping.en.md) (Optional) +- [Layout Detection Module](../module_usage/layout_detection.en.md) +- [Table Structure Recognition Module](../module_usage/table_structure_recognition.en.md) (Optional) +- [Text Detection Module](../module_usage/text_detection.en.md) +- [Text Recognition Module](../module_usage/text_recognition.en.md) +- [Text Line Orientation Classification Module](../module_usage/textline_orientation_classification.en.md)(Optional) +- [Formula Recognition Module](../module_usage/formula_recognition.en.md) (Optional) +- [Seal Text Detection Module](../module_usage/seal_text_detection.en.md) (Optional) -
👉Model List Details -

Document Image Orientation Classification Module (Optional):

+ +In this pipeline, you can choose the model to use based on the benchmark data below. + +
+Document Image Orientation Classification Module (Optional): @@ -37,7 +48,10 @@ The Document Scene Information Extraction v4 pipeline includes modules for **Lay
-

Text Image Unwarp Module (Optional):

+
+ +
+Text Image Unwarp Module (Optional): @@ -61,37 +75,12 @@ The Document Scene Information Extraction v4 pipeline includes modules for **Lay
-

Table Structure Recognition Module Models:

- - - - - - - - - - - - - - - - - - - - - - - - - - - -
ModelModel Download LinkAccuracy (%)GPU Inference Time (ms)
[Normal Mode / High-Performance Mode]		CPU Inference Time (ms)
[Normal Mode / High-Performance Mode]		Model Storage Size (MB)Description
SLANetInference Model/Training Model59.5223.96 / 21.75- / 43.126.9SLANet is a table structure recognition model developed by Baidu PaddleX Team. The model significantly improves the accuracy and inference speed of table structure recognition by adopting a CPU-friendly lightweight backbone network PP-LCNet, a high-low-level feature fusion module CSP-PAN, and a feature decoding module SLA Head that aligns structural and positional information.
SLANet_plusInference Model/Training Model63.6923.43 / 22.16- / 41.806.9SLANet_plus is an enhanced version of SLANet, the table structure recognition model developed by Baidu PaddleX Team. Compared to SLANet, SLANet_plus significantly improves the recognition ability for wireless and complex tables and reduces the model's sensitivity to the accuracy of table positioning, enabling more accurate recognition even with offset table positioning.
+
-

Layout Detection Module Models:

+ +
+Layout Detection Module Model: +* The layout detection model includes 20 common categories: document title, paragraph title, text, page number, abstract, table, references, footnotes, header, footer, algorithm, formula, formula number, image, table, seal, figure_table title, chart, and sidebar text and lists of references @@ -105,8 +94,62 @@ The Document Scene Information Extraction v4 pipeline includes modules for **Lay + + + + + + + + + + +
PP-DocLayout_plus-LInference Model/Training Model83.253.03 / 17.23634.62 / 378.32126.01A higher-precision layout area localization model trained on a self-built dataset containing Chinese and English papers, PPT, multi-layout magazines, contracts, books, exams, ancient books and research reports using RT-DETR-L
+ + +* The layout detection model includes 1 category: Block: + + + + + + + + + + + + + + + + + + + + + + + +
ModelModel Download LinkmAP(0.5) (%)GPU Inference Time (ms)
[Normal Mode / High-Performance Mode]		CPU Inference Time (ms)
[Normal Mode / High-Performance Mode]		Model Storage Size (MB)Introduction
PP-DocBlockLayoutInference Model/Training Model95.934.60 / 28.54506.43 / 256.83123.92A layout block localization model trained on a self-built dataset containing Chinese and English papers, PPT, multi-layout magazines, contracts, books, exams, ancient books and research reports using RT-DETR-L
+ + +* The layout detection model includes 23 common categories: document title, paragraph title, text, page number, abstract, table of contents, references, footnotes, header, footer, algorithm, formula, formula number, image, figure caption, table, table caption, seal, figure title, figure, header image, footer image, and sidebar text + + + + + + + + + + + + + - + @@ -115,7 +158,7 @@ The Document Scene Information Extraction v4 pipeline includes modules for **Lay - + @@ -124,7 +167,7 @@ The Document Scene Information Extraction v4 pipeline includes modules for **Lay - + @@ -133,9 +176,10 @@ The Document Scene Information Extraction v4 pipeline includes modules for **Lay
ModelDownload LinkmAP(0.5) (%)GPU Inference Time (ms)
[Standard Mode / High Performance Mode]		CPU Inference Time (ms)
[Standard Mode / High Performance Mode]		Model Storage Size (MB)Description
PP-DocLayout-LInference Model/Training ModelInference Model/Pretrained Model 90.4 33.59 / 33.59 503.01 / 251.08
PP-DocLayout-MInference Model/Training ModelInference Model/Pretrained Model 75.2 13.03 / 4.72 43.39 / 24.44
PP-DocLayout-SInference Model/Training ModelInference Model/Pretrained Model 70.9 11.54 / 3.86 18.53 / 6.29
-Note: The evaluation dataset for the above precision metrics is a self-built layout area detection dataset by PaddleOCR, containing 500 common document-type images of Chinese and English papers, magazines, contracts, books, exams, and research reports. GPU inference time is based on an NVIDIA Tesla T4 machine with FP32 precision. CPU inference speed is based on an Intel(R) Xeon(R) Gold 5117 CPU @ 2.00GHz with 8 threads and FP32 precision. -> ❗ The above list includes the 3 core models that are key supported by the text recognition module. The module actually supports a total of 11 full models, including several predefined models with different categories. The complete model list is as follows: +> ❗ The above list includes the 4 core models that are key supported by the text recognition module. The module actually supports a total of 12 full models, including several predefined models with different categories. The complete model list is as follows: + +
👉 Details of Model List * Table Layout Detection Model @@ -160,7 +204,6 @@ The Document Scene Information Extraction v4 pipeline includes modules for **Lay
A high-efficiency layout area localization model trained on a self-built dataset using PicoDet-1x, capable of detecting table regions.
-Note: The evaluation dataset for the above precision metrics is a self-built layout table area detection dataset by PaddleOCR, containing 7835 Chinese and English document images with tables. GPU inference time is based on an NVIDIA Tesla T4 machine with FP32 precision. CPU inference speed is based on an Intel(R) Xeon(R) Gold 5117 CPU @ 2.00GHz with 8 threads and FP32 precision. * 3-Class Layout Detection Model, including Table, Image, and Stamp @@ -193,6 +236,10 @@ The Document Scene Information Extraction v4 pipeline includes modules for **Lay +
22.6 A balanced efficiency and precision layout area localization model trained on a self-built dataset of Chinese and English papers, magazines, and research reports using PicoDet-L.
+ +

Table Classification Module Models:

+ @@ -203,7 +250,6 @@ The Document Scene Information Extraction v4 pipeline includes modules for **Lay
RT-DETR-H_layout_3cls Inference Model/Training ModelA high-precision layout area localization model trained on a self-built dataset of Chinese and English papers, magazines, and research reports using RT-DETR-H.
-Note: The evaluation dataset for the above precision metrics is a self-built layout area detection dataset by PaddleOCR, containing 1154 common document images of Chinese and English papers, magazines, and research reports. GPU inference time is based on an NVIDIA Tesla T4 machine with FP32 precision. CPU inference speed is based on an Intel(R) Xeon(R) Gold 5117 CPU @ 2.00GHz with 8 threads and FP32 precision. * 5-Class English Document Area Detection Model, including Text, Title, Table, Image, and List @@ -228,7 +274,6 @@ The Document Scene Information Extraction v4 pipeline includes modules for **Lay
A high-efficiency English document layout area localization model trained on the PubLayNet dataset using PicoDet-1x.
-Note: The evaluation dataset for the above precision metrics is the [PubLayNet](https://developer.ibm.com/exchanges/data/all/publaynet/) dataset, containing 11245 English document images. GPU inference time is based on an NVIDIA Tesla T4 machine with FP32 precision. CPU inference speed is based on an Intel(R) Xeon(R) Gold 5117 CPU @ 2.00GHz with 8 threads and FP32 precision. * 17-Class Area Detection Model, including 17 common layout categories: Paragraph Title, Image, Text, Number, Abstract, Content, Figure Caption, Formula, Table, Table Caption, References, Document Title, Footnote, Header, Algorithm, Footer, and Stamp @@ -270,10 +315,44 @@ The Document Scene Information Extraction v4 pipeline includes modules for **Lay -
470.2 A high-precision layout area localization model trained on a self-built dataset of Chinese and English papers, magazines, and research reports using RT-DETR-H.
+
+
-

Text Detection Module Models:

+
+Table Structure Recognition Module Models (Optional): + + + + + + + + + + + + + + + + + + + + + + + + + + + +
ModelModel Download LinkAccuracy (%)GPU Inference Time (ms)
[Normal Mode / High-Performance Mode]		CPU Inference Time (ms)
[Normal Mode / High-Performance Mode]		Model Storage Size (MB)Description
SLANetInference Model/Training Model59.5223.96 / 21.75- / 43.126.9SLANet is a table structure recognition model developed by Baidu PaddleX Team. The model significantly improves the accuracy and inference speed of table structure recognition by adopting a CPU-friendly lightweight backbone network PP-LCNet, a high-low-level feature fusion module CSP-PAN, and a feature decoding module SLA Head that aligns structural and positional information.
SLANet_plusInference Model/Training Model63.6923.43 / 22.16- / 41.806.9SLANet_plus is an enhanced version of SLANet, the table structure recognition model developed by Baidu PaddleX Team. Compared to SLANet, SLANet_plus significantly improves the recognition ability for wireless and complex tables and reduces the model's sensitivity to the accuracy of table positioning, enabling more accurate recognition even with offset table positioning.
+
+ +
+Text Detection Module Models @@ -324,8 +403,10 @@ The Document Scene Information Extraction v4 pipeline includes modules for **Lay
+
-

Text Recognition Module Models:

+
+Text Recognition Module Models @@ -556,13 +637,6 @@ The RepSVTR text recognition model is a mobile-oriented text recognition model b
ModelModel Download Links
- - - - - - - * Multilingual Recognition Models @@ -674,7 +748,10 @@ devanagari_PP-OCRv3_mobile_rec_infer.tar">Inference Model/Inference Model/Inference Model/Training Model + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + - - - - - + + + -
85.9143.501311.84 / 1311.84- / 8288.071530UniMERNet is a formula recognition model developed by Shanghai AI Lab. It uses Donut Swin as the encoder and MBartDecoder as the decoder. The model is trained on a dataset of one million samples, including simple formulas, complex formulas, scanned formulas, and handwritten formulas, significantly improving the recognition accuracy of real-world formulas.
PP-FormulaNet-SInference Model/Training Model87.0045.71182.25 / 182.25- / 254.39224PP-FormulaNet is an advanced formula recognition model developed by the Baidu PaddlePaddle Vision Team. The PP-FormulaNet-S version uses PP-HGNetV2-B4 as its backbone network. Through parallel masking and model distillation techniques, it significantly improves inference speed while maintaining high recognition accuracy, making it suitable for applications requiring fast inference. The PP-FormulaNet-L version, on the other hand, uses Vary_VIT_B as its backbone network and is trained on a large-scale formula dataset, showing significant improvements in recognizing complex formulas compared to PP-FormulaNet-S.
PP-FormulaNet-LInference Model/Training Model90.3645.781482.03 / 1482.03- / 3131.54695
PP-FormulaNet_plus-SInference Model/Training Model88.7153.32179.20 / 179.20- / 260.99248PP-FormulaNet_plus is an enhanced version of the formula recognition model developed by the Baidu PaddlePaddle Vision Team, building upon the original PP-FormulaNet. Compared to the original version, PP-FormulaNet_plus utilizes a more diverse formula dataset during training, including sources such as Chinese dissertations, professional books, textbooks, exam papers, and mathematics journals. This expansion significantly improves the model’s recognition capabilities. Among the models, PP-FormulaNet_plus-M and PP-FormulaNet_plus-L have added support for Chinese formulas and increased the maximum number of predicted tokens for formulas from 1,024 to 2,560, greatly enhancing the recognition performance for complex formulas. Meanwhile, the PP-FormulaNet_plus-S model focuses on improving the recognition of English formulas. With these improvements, the PP-FormulaNet_plus series models perform exceptionally well in handling complex and diverse formula recognition tasks.
PP-FormulaNet_plus-MInference Model/Training Model91.4589.761040.27 / 1040.27- / 1615.80592
PP-FormulaNet_plus-LInference Model/Training Model92.2290.641476.07 / 1476.07- / 3125.58698
LaTeX_OCR_rec Inference Model/Training Model0.88210.082340.0174.5539.96 1088.89 / 1088.89 - / - 99LaTeX-OCR is a formula recognition algorithm based on an autoregressive large model. It uses Hybrid ViT as the backbone network and a transformer as the decoder, significantly improving the accuracy of formula recognition.
+
-

Seal Text Detection Module Models:

+
+Seal Text Detection Module Models (Optional): @@ -759,8 +891,10 @@ devanagari_PP-OCRv3_mobile_rec_infer.tar">Inference Model/Inference Model/ -PP-ChatOCRv4 产线中包含版面区域检测模块、表格结构识别模块、表格分类模块、表格单元格定位模块、文本检测模块、文本识别模块、印章文本检测模块、文本图像矫正模块、文档图像方向分类模块。 PP-ChatOCRv4 产线中包含以下9个模块。每个模块均可独立进行训练和推理，并包含多个模型。有关详细信息，请点击相应模块以查看文档。 @@ -665,30 +664,85 @@ devanagari_PP-OCRv3_mobile_rec_infer.tar">推理模型/推理模型/训练模型 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + - - - - - + + + -
85.9143.501311.84 / 1311.84- / 8288.071530UniMERNet是由上海AI Lab研发的一款公式识别模型。该模型采用Donut Swin作为编码器，MBartDecoder作为解码器，并通过在包含简单公式、复杂公式、扫描捕捉公式和手写公式在内的一百万数据集上进行训练，大幅提升了模型对真实场景公式的识别准确率
PP-FormulaNet-S推理模型/训练模型87.0045.71182.25 / 182.25- / 254.39224PP-FormulaNet 是由百度飞桨视觉团队开发的一款先进的公式识别模型，支持5万个常见LateX源码词汇的识别。PP-FormulaNet-S 版本采用了 PP-HGNetV2-B4 作为其骨干网络，通过并行掩码和模型蒸馏等技术，大幅提升了模型的推理速度，同时保持了较高的识别精度，适用于简单印刷公式、跨行简单印刷公式等场景。而 PP-FormulaNet-L 版本则基于 Vary_VIT_B 作为骨干网络，并在大规模公式数据集上进行了深入训练，在复杂公式的识别方面，相较于PP-FormulaNet-S表现出显著的提升，适用于简单印刷公式、复杂印刷公式、手写公式等场景。
PP-FormulaNet-L推理模型/训练模型90.3645.781482.03 / 1482.03- / 3131.54695
PP-FormulaNet_plus-S推理模型/训练模型88.7153.32179.20 / 179.20- / 260.99248PP-FormulaNet_plus 是百度飞桨视觉团队在 PP-FormulaNet 的基础上开发的增强版公式识别模型。与原版相比，PP-FormulaNet_plus 在训练中使用了更为丰富的公式数据集，包括中文学位论文、专业书籍、教材试卷以及数学期刊等多种来源。这一扩展显著提升了模型的识别能力。 + +其中，PP-FormulaNet_plus-M 和 PP-FormulaNet_plus-L 模型新增了对中文公式的支持，并将公式的最大预测 token 数从 1024 扩大至 2560，大幅提升了对复杂公式的识别性能。同时，PP-FormulaNet_plus-S 模型则专注于增强英文公式的识别能力。通过这些改进，PP-FormulaNet_plus 系列模型在处理复杂多样的公式识别任务时表现更加出色。
PP-FormulaNet_plus-M推理模型/训练模型91.4589.761040.27 / 1040.27- / 1615.80592
PP-FormulaNet_plus-L推理模型/训练模型92.2290.641476.07 / 1476.07- / 3125.58698
LaTeX_OCR_rec 推理模型/训练模型0.88210.082340.0174.5539.96 1088.89 / 1088.89 - / - 99LaTeX-OCR是一种基于自回归大模型的公式识别算法，通过采用 Hybrid ViT 作为骨干网络，transformer作为解码器，显著提升了公式识别的准确性。