#  Deep learning for table detection and structure recognition: A survey

Mahmoud Kasem  Abdelrahman Abdallah  Alexander Berendeyev  Ebrahem Elkady
Mahmoud Abdalla  Mohamed Mahmoud  Mohamed Hamada  Daniyar Nurseitov  Islam
Taj-Eddin

###### Abstract

Tables are everywhere, from scientific journals, papers, websites, and
newspapers all the way to items we buy at the supermarket. Detecting them is
thus of utmost importance to automatically understanding the content of a
document. The performance of table detection has substantially increased
thanks to the rapid development of deep learning networks. The goals of this
survey are to provide a profound comprehension of the major developments in
the field of Table Detection, offer insight into the different methodologies,
and provide a systematic taxonomy of the different approaches. Furthermore, we
provide an analysis of both classic and new applications in the field. Lastly,
the datasets and source code of the existing models are organized to provide
the reader with a compass on this vast literature. Finally, we go over the
architecture of utilizing various object detection and table structure
recognition methods to create an effective and efficient system, as well as a
set of development trends to keep up with state-of-the-art algorithms and
future research. We have also set up a public GitHub repository where we will
be updating the most recent publications, open data, and source code. The
GitHub repository is available at https://github.com/abdoelsayed2016/table-
detection-structure-recognition.

###### keywords:

Convolutional neural networks , deep learning , Document processing , table
detection , table structure recognition.

[inst1]organization=Information Technology Department, FCI,addressline=Assiut
University, city=Assiut, postcode=71515, state=Assiut, country=Egypt

[inst2] organization=Department of Information System, International IT
University,,city=Almaty, postcode=050000, country=Kazakhstan

[inst3] organization=KazMunayGas Engineering LLP,city=Nur-Sultan,
postcode=010000, country=Kazakhstan

[inst4] organization=Satbayev University,city=Almaty, postcode=050013,
country=Kazakhstan

[inst5] organization=Information Technology Institute(ITI),city=Alexandria,
postcode=5310002, country=Egypt

[inst6] organization=College of Electrical and Computer Engineering, Chungbuk
National University,city=Cheongju, postcode=28644, country=South Korea

##  1 Introduction

Textbooks, lists, formulae, graphs, tables, and other elements are common in
documents. Most papers, in particular, contain several sorts of tables.
Tables, as a significant part of papers, may convey more information in fewer
words and allow readers to quickly explore, compare, and comprehend the
content. Table detection and structure identification are crucial tasks in
image analysis because they allow retrieving vital information from tables in
a digital format. Because of the document’s type and the variety of document
layouts, detecting and extracting images or document tables is tough.
Researchers have previously used heuristic techniques to recognize tables or
to break pages into many parts for table extraction. Few studies have focused
on table structure recognition in documents following table detection.

The layout and content analysis of documents are used to detect tables. Tables
come in a number of layouts and formats. As a result, creating a general
method for table detection and table structure recognition is quite difficult.
Table detection is regarded as a difficult subject in scientific circles. A
large number of studies have been conducted in this sector, although the
majority of them have limitations. Existing commercial and open-source
document analysis algorithms, such as Tesseract, are unable to fully detect
table areas from document images. [1].

Machine learning and deep learning have been proven to be very effective in
computer vision research. On computer vision tasks such as picture
classification, object detection, object position estimation, learning, and so
on, deep convolutional neural networks (types of feed-forward artificial
neural systems) have outperformed alternative learning models. The
effectiveness of Convolutional Neural Networks (CNNs) in object identification
is based on their ability to learn substantial mid-level visual properties
rather than the hand-crafted low-level representations that are often utilized
in particular approaches to image categorization. The object is defined by its
major characteristics, which include shape, size, color, texture, and other
characteristics. To identify such an item, a picture must clearly reveal the
object’s presence and, moreover, its position [2].

Object detection may thus be described as a method of locating real-world
items in photographs. Detection is closely connected to categorization because
it includes determining the existence and location of a certain item in an
image. There are many items that may be identified in a picture, including
automobiles, buildings, tables, human faces, and so on. Deep learning
approaches, such as deep neural networks, region-based convolutional neural
networks, and deeply convolutional neural networks, can improve object
identification precision, and efficacy.

In recent years, a variety of remarkable and creative strategies have been
used to improve deep learning model detection accuracy and solve complex
challenges encountered during the training and testing process of deep
learning object recognition models. Modification of the activation function of
deep CNNs [3], Transfer learning [4, 5], cancer diagnosis, detection [6, 7,
8], and classification[9], and medical question answers[10, 11], as well as
software engineering applications such as optimizing the time and schedule of
software projects[12, 13], Intrusion Detection in IoT [14, 15] and handwritten
recognition for various languages[16, 17, 18, 19]., and inventive ways in the
combined selection of the activation function and the optimization system for
the proposed deep learning model are among these unique strategies. Among the
various variables and initiatives that have contributed to the rapid
advancement of table detection algorithms, the development of deep
convolutional neural networks and GPU computational capacity should be
credited. Deep learning models are now widely used in many aspects of computer
vision, including general table detection[20, 21, 22, 23, 24]. Table
structures, on the other hand, receive far less attention, and the table
structure is typically characterized by the rows and columns of a table [25,
26, 27].

Figure 1 shows a basic pipeline comparison of deep learning techniques and
conventional approaches for the task of understanding tables. Traditional
table recognition techniques either can’t handle varied datasets well enough
or need extra metadata from PDF files. Extensive pre- and post-processing were
also used in the majority of early approaches to improve the effectiveness of
conventional table recognition systems. However, deep learning algorithms
retrieve features using neural networks, primarily convolutional neural
networks [21], instead of manually created features. Object detection or
segmentation networks then try to differentiate the tabular portion that is
further broken down and recognized in a document image.

This survey examines deep learning-based table detection and classification
architectures in depth. While current evaluations are comprehensive [28, 29],
the majority of them do not address recent advancements in the field.

The following are the paper’s main contributions:

  1. 1.

We provide a brief history of Table Datasets and the differences between them.

  2. 2.

The paper examines important table detection methods, as well as the evolution
of these methods over time.

  3. 3.

We give a thorough analysis of table structure recognition in-depth.

  4. 4.

We provide Table Classification methods and compare these methods. There was
no study that provided a broad summary of these issues that we could identify.

  5. 5.

Experiments Result on some datasets of Table detection

(a) Traditional Table Detection approaches

(b) Deep Learning approaches for Table Detection

Figure 1: Table analysis pipeline comparison of conventional and deep learning
methods. While convolutional networks are used in deep learning techniques,
classical approaches primarily perform feature extraction through image
processing techniques. Deep learning methods for interpreting tables are more
generalizable and independent of data than conventional approaches.

###  1.1 Comparison with Previous Reviews

For many years, the issue with table analysis has been widely acknowledged.
Figure 2 shows the upward trend in publications during the previous eight
years, this analysis values were derived from Scopus. There have been notable
table detection and table classification surveys published. There are
outstanding studies on the subject of table detection in these surveys [28,
29]. There have been few recent surveys that specifically address the subject
of table detection and classification. B. Coüasnon [30] released another
review on table recognition and forms. The review gives a quick rundown of the
most recent techniques at the time, S. Khusro [31] released the most recent
review on the identification and extraction of tables in PDF documents the
following year, according to our knowledge. Deep learning enables
computational models to learn fantastically complex, subtle, and abstract
representations, resulting in significant advancements in a wide range of
problems such as visual recognition, object detection, speech recognition,
natural language processing, and medical image analysis. In contrast, despite
the fact that various deep learning-based algorithms for table identification
have been presented, we are unaware of any recent thorough survey. For further
advancement in table detection, a detailed review and explanation of prior
work are required, especially for researchers new to the topic.

Figure 2: shows an illustration of an expanding trend in the area of table
analysis. This information was gathered by looking through the annual reports
on table detection and table identification from the years 2015 to 2022, this
analysis values were derived from Scopus.

###  1.2 Scope

The quantity of studies on deep learning-based table detection is staggering.
They are so numerous that any complete examination of the state of the art
would be beyond the scope of any acceptable length paper. As a result,
selection criteria must be established, and we have narrowed our attention to
the best journal and conference articles.

The main goal of this paper is to provide a comprehensive survey of deep
learning-based table detection and classification techniques, as well as some
taxonomy, a high-level perspective, and organization, based on popular
datasets, evaluation metrics, context modeling, and detection proposal
methods. Our goal is for our classification to make it easier for readers to
comprehend the similarities and differences across a wide range of tactics.
The suggested taxonomy provides a framework for researchers to comprehend
existing research and highlight open research problems for the future.

##  2 Major Challenges

###  2.1 Object detection Challenges

Developing a general-purpose algorithm that fulfills two competing criteria of
high quality/accuracy and great efficiency is ideal for object detection.
High-quality detection must accurately localize and recognize objects in
images or video frames, allowing for distinction of a wide range of object
categories in the real world and localization and recognition of object
instances from the same category, despite intra-class appearance variations,
for high robustness. High efficiency necessitates that the full detection
process is completed in real-time while maintaining reasonable memory and
storage requirements.

###  2.2 Table detection Challenges

Although a trained segmentation model can accurately locate tables,
conventional machine learning techniques have flaws in the structural
identification of tables. A major issue is a large number of things in such a
little space. As a result, the network misses out on critical visual cues that
may aid in the detection and recognition of tables [20]. As physical rules are
available, intersections of horizontal and vertical lines are computed to
recognize table formations. The Hough transform is a prominent approach in
computer vision that aids in the detection of lines in document scans [32].
Length, rotation, and average darkness of a line are utilized to filter out
false positives and determine if the line is, in fact, a table line [33]. The
intersections of the remaining horizontal and vertical lines are computed
after the Hough lines have been filtered. Table cells are created based on the
crossings.

##  3 A Quick Overview of Deep Learning

From image classification and video processing to speech recognition and
natural language understanding, deep learning has transformed a wide range of
machine learning activities. Given the incredible rate of change[34], there is
a plethora of current survey studies on deep learning [35, 36, 37, 38, 39, 40,
41, 42, 43, 44, 45] , medical image analysis applications [38], natural
language processing [42], and speech recognition systems [44].

Convolutional neural networks (CNNs), the most common deep learning model, can
use the fundamental properties of actual signals: translation invariance,
local connection, and compositional hierarchies. A typical CNN comprises a
hierarchical structure and numerous layers for learning data representations
at different levels of abstraction [36]. We start with a convolution

$\begin{split}x^{l-1}*w^{l}\end{split}$ (1)

between a feature map from the previous layer l-1 and an input feature map
$x^{l-1}$ , convolved using a 2D convolutional kernel (or filter or weights)
$w^{l}$. This convolution is seen as a series of layers that have been
subjected to a nonlinear process $\sigma$, such that

$\begin{split}x^{l}_{j}=\sigma\Bigg{(}\sum_{i=1}^{N^{l-1}}x^{l-1}_{i}*w^{l}_{i,j}+b^{l}_{j}\Bigg{)}\end{split}$
(2)

with a bias term $b^{l}_{j}$ and a convolution between the $N^{l-1}$ input
feature maps $x^{l-1}_{i}$ and the matching kernel $w^{l}_{i,j}$ . For each
element, the element-wise nonlinear function $\sigma(.)$ is commonly a
rectified linear unit (ReLU) for each element,

$\begin{split}\sigma(x)=max\\{x,0\\}\end{split}$ (3)

Finally, pooling is the process of downsampling and upsampling feature maps.
Deep convolution neural networks(DCNNs) are CNNs with a large number of
layers, often known as ”deep” networks . A CNN’s most basic layers consist of
a series of feature maps, each of which operates as a neuron. A set of weights
$w_{i,j}$ connects each neuron in a convolutional layer to feature maps from
the preceding layer (essentially a set of 2D filters). Whereas convolutional
and pooling layers make up the early CNN layers, the subsequent layers are
usually completely connected. The input picture is repeatedly convolved from
earlier to later layers, and the receptive field or region of support grows
with each layer. In general, the first CNN layers extract low-level
characteristics (such as edges), whereas subsequent layers extract more
generic features of increasing complexity. [35, 46, 47, 36].

DCNNs have a hierarchical structure that allows them to learn data
representations at numerous levels of abstraction, the ability to learn highly
complicated functions, and the ability to learn feature representations
directly and automatically from data with minimum domain expertise. The
availability of huge size labeled datasets and GPUs with extremely high
computational capabilities is what has made DCNNs so successful.

Despite the enormous achievements, there are still acknowledged flaws. There
is a critical need for labeled training data as well as expensive
computational resources, and selecting proper learning parameters and network
designs still requires substantial expertise and experience. Trained networks
are difficult to comprehend, lack resistance to degradations and many DCNNs
have been proven to be vulnerable to assaults [37], all of which restrict
their applicability in real-world applications.

##  4 Datasets and Evaluation Metrics

###  4.1 Datasets

This section will describe datasets that are available and have been most
commonly used for table detection, table structure recognition, and
classification tasks.

####  4.1.1 ICDAR 2013

ICDAR2013 dataset[48] referred to in served as the competition’s official
practice dataset. Rather than focusing on a specific subset of documents,
authors have always intended to evaluate systems as broadly as possible, and
this dataset, as well as the actual competition dataset, were generated by
systematically collecting PDFs from a Google search in order to make the
selection as objective as possible. they are limited to two governmental
sources with the additional search terms site:europa.eu and site:*.gov in
order to obtain documents whose publications are known to be in the public
domain. The ICDAR2013 dataset contains 150 tables, 75 of which are in 27 EU
excerpts and 75 of which are in 40 US Government excerpts. Table regions are
rectangular areas of a page whose coordinates define them. Multiple regions
can be included in the same table because a table can span multiple pages.
Table detection or location and table structure recognition are the two sub-
tasks of ICDAR2013. The task of table structure recognition compares methods
for determining table cell structure given precise location information.
Figure 3 presents a few examples from this dataset.

####  4.1.2 ICDAR 2017 POD

Dataset [49] was published for the ICDAR2017 Page Object Detection (POD)
competition. This dataset is frequently used to test different methods of
table detection. Compared to the ICDAR2013 table dataset, this dataset is
significantly larger. There are 2417 total images in it, including figures,
tables, and formulae. The dataset is commonly split into 1600 photos which
have 731 tabular areas for training, and the remaining 817 images which have
350 tabular regions for testing. Figure 4 illustrates two examples of this
dataset.

####  4.1.3 ICDAR2019

ICDAR2019 [50] proposed a dataset for table detection (TRACK A) and table
recognition (TRACK B). The dataset is divided into two types, historical and
modern datasets. It contains 1600 images for training and 839 images for
testing. The historical type contains 1200 images in tracks A and B for
training and 499 images for testing. The modern type contains 600 images in
tracks A and B for training and 340 images for testing. Document images
containing one or more tables are provided for TRACK A. TRACK B has two sub-
tracks: the first (B.1) provides the table region, and only the table
structure recognition is required. The second sub-track (B.2) contains no
prior knowledge. That is, both table region detection and table structure
recognition must be performed. For the annotation of the dataset, a similar
notation was derived from the ICDAR 2013 [48] Table Competition format, and
the structures were stored in a single XML file. Each table element
corresponds to a table with a single Coords element with a points attribute
indicating the coordinates of the bounding polygon with N vertices. Each table
element contains a list of cell elements as well. The attributes start-row,
start-col, end-row, and end-col denote the position of each cell element in
the table. The cell element’s Coords denote the coordinates of the bounding
polygon of this cell box, and the content is the text within this cell. Figure
5 presents a few examples from this dataset.

(a)

(b)

Figure 3: Examples of images in ICDAR 2013

(a)

(b)

Figure 4: Examples of images in ICDAR 2017

(a)

(b)

Figure 5: Examples of images in ICDAR 2019

####  4.1.4 TabStructDB

TabStructDB is a different publicly available image-based table structure
recognition dataset that was promoted by SA Siddiqui [51]. The well-known
ICDAR 2017 page object detection dataset, which contains pictures annotated
with structural details, was used to curate this dataset. Figure 6 illustrates
two examples of this dataset.

(a)

(b)

Figure 6: Examples of images in TabStructDB

####  4.1.5 TABLE2LATEX-450K

TABLE2LATEX-450K [52] is another sizable dataset that was released at the most
recent ICDAR conference. The dataset includes 450,000 annotated tables and the
associated pictures. This enormous dataset was created by crawling through all
of the LaTeX source documents and ArXiv publications from 1991 to 2016. The
high-quality labeled dataset is obtained via source code extraction and
further refining. Figure 7 presents a few examples from this dataset.

(a)

(b)

Figure 7: Examples of images in TABLE2LATEX-450K

####  4.1.6 RVL-CDIP (SUBSET)

A well-known dataset in the world of document analysis is the RVL-CDIP
(Ryerson Vision Lab Complex Document Information Processing) [53]. 400,000
photos are included, evenly spread over 16 classifications. For the purpose of
detecting tables, P Riba [54] makes subset dataset by annotating the 518
invoices in the RVL-CDIP dataset. The dataset has been made available to the
general public. For testing table identification methods especially created
for invoice document pictures, this subset of the real RVL-CDIP dataset [53]
is a significant contribution. Figure 8 presents a few examples from this
dataset.

(a)

(b)

Figure 8: Examples of images in RVL-CDIP (SUBSET)

####  4.1.7 IIIT-AR-13K

IIT-AR-13K is a brand-new dataset that is introduced by A Mondal [55]. The
yearly reports that were publicly available and written in English and other
languages were collected to create this dataset. The biggest manually
annotated dataset for the problem of graphical page object recognition,
according to the authors, has been released. Annotations for figures, natural
imagery, logos, and signatures are included in the dataset in addition to the
tables. For numerous tasks of page object detection, the authors have included
the train, validation, and test splits. In order to train for table detection,
11,000 samples are employed, while 2000 and 3000 samples are allotted for
validation and testing, respectively. Figure 9 illustrates two examples of
this dataset.

(a)

(b)

Figure 9: Examples of images in IIIT-AR-13K

####  4.1.8 CamCap

CamCap is a collection of camera-captured photos suggested by W Seo [56]. Only
85 photos are present (38 tables on curved surfaces having 1295 cells and 47
tables on planar surfaces consisting of 1162 cells). For the sake of detecting
tables and identifying their structures, the suggested dataset is accessible
to the general public. This dataset is a significant addition to evaluating
the reliability of table identification techniques on camera-captured document
pictures. Figure 10 illustrates two examples of this dataset.

(a)

(b)

Figure 10: Examples of images in CamCap

####  4.1.9 UNLV Table

The UNLV Table dataset[57] includes 2889 pages of scanned document images
gathered from various sources (magazines, newspapers, business letters, annual
reports, etc). Scanned images are available in bitonal, grayscale, and fax
formats, with resolutions ranging from 200 to 300 DPI. There is ground truth
data in addition to the original dataset, which contains manually marked
zones; zone types are provided in text format. Figure 11 illustrates two
examples of this dataset.

####  4.1.10 UW-3 Table

The UW-3 Table dataset [58] contains 1600 skew-corrected English document
images with manually edited entity bounding box ground-truth. Page frames,
text and non-text zones, text lines, and words are all surrounded by these
bounding boxes. Each zone’s type (text, math, table, half-tone, etc.) is also
indicated. There are approximately 120 document images with at least one
marked table zone. The UNLV and UW-3 Table dataset taught a user how to use
the T-Truth tool and asked him to prepare ground truth for the target images
in the above dataset. Each image’s ground truth is stored in an XML. Another
expert manually validated the ground truths using the T-Truth tool’s preview
edit mode, and incorrect ground truths were corrected. These iterations were
repeated several times to ensure that the ground truth was correct. authors
discovered that the majority of errors occur when labeling column spanning
cells where the column boundaries coincide with the word boundaries. Problems
can also arise when there are multiple interpretations of a table structure,
as described by Nagy[59], and domain knowledge is required to correctly label
the table structure. Figure 12 illustrates two examples of this dataset.

(a)

(b)

Figure 11: Examples of images in UNLV

(a)

(b)

Figure 12: Examples of images in UW3

####  4.1.11 Marmot

The Marmot dataset[60] is considered the first large dataset in table
detection. it contains 2000 PDF pages with ground truth data. This labeling
task was completed by 15 people. To reduce subjectivity, a unified labeling
standard was established, and each ground-truth file is double-checked. The
dataset’s size is still growing. The e-document pages in the dataset have a
wide range of language types, page layouts, and table styles. First, it is
made up of roughly equal parts Chinese and English pages. The Chinese pages
were chosen from over 120 e-Books with diverse subject areas from Founder
Apabi’s digital library, with no more than 15 pages chosen from each book. The
English pages were retrieved from the web. Over 1500 conference and journal
papers from 1970 to 2011 were crawled, covering a wide range of topics. The
Chinese e-Book pages are mostly one column, whereas the English pages are
printed in both one and two columns. This dataset includes a wide range of
table types, from ruled tables to partially and non-ruled tables, horizontal
tables to vertical tables, inside-column tables to span-column tables, and so
on. A few samples from this dataset are shown in Figure 13.

(a)

(b)

Figure 13: Examples of images in Marmot

####  4.1.12 TableBank

The TableBank [61] proposed a novel weak supervision approach for
automatically creating the dataset, which is orders of magnitude larger than
existing human-labeled datasets for table analysis. Unlike the traditional
weakly supervised training set, this approach can generate not only large
amounts of but also high-quality training data. There are many electronic
documents available on the internet nowadays, such as Microsoft Word (.docx)
and Latex (.tex) files. By definition, these online documents contain mark-up
tags for tables in their source code. Intuitively, these source codes
manipulate by adding bounding boxes within each document using the mark-up
language. The Office XML code for Word documents can be modified to identify
the borderline of each table. The code for Latex documents can also be
modified to recognize table bounding boxes. This method generates high-quality
labeled data for a wide range of domains, including business documents,
official filings, research papers, and so on, which is extremely useful for
large-scale table analysis tasks. The TableBank dataset is made up of 417,234
high-quality labeled tables and their original documents from a variety of
domains. A few samples from this dataset are shown in Figure 14.

(a)

(b)

Figure 14: Examples of images in TableBank

####  4.1.13 DeepFigures

DeepFigures [62] uses no human assistance to generate high-quality training
labels for the task of figure extraction in a huge number of scientific
papers. authors do this by locating figures and captions in the rasterized PDF
using supplementary data from two big web collections of scientific articles
(PubMed and arXiv). authors provide the resulting dataset of approximately 5.5
million tables and figures induced labels to facilitate the development of
modern data-driven approaches for this task, which is 4,000 times larger than
the previous largest figure extraction dataset and has an average precision of
96.8%. Samples from this dataset are shown in Figure 15.

Figure 15: Examples of images in DeepFigures

####  4.1.14 PubTables-1M

PubTables-1M [63] contains nearly one million tables from scientific articles,
supports multiple input modalities and contains detailed header and location
information for table structures, making it useful for a wide variety of
modeling approaches. It also addresses a significant source of ground truth
inconsistency observed in prior datasets called over-segmentation, using a
novel canonicalization procedure. We demonstrate that these improvements lead
to a significant increase in training performance and a more reliable estimate
of model performance at evaluation for table structure recognition. Further,
authors show that transformer-based object detection models trained on
PubTables-1M produce excellent results for all three tasks of detection,
structure recognition, and functional analysis without the need for any
special customization for these tasks. Figure 16 illustrates two examples of
this dataset.

(a)

(b)

Figure 16: Examples of images in PubTables-1M

####  4.1.15 SciTSR

SciTSR [64] presents a large-scale table structure recognition dataset derived
from scientific articles that comprise 15,000 tables from PDF files and their
related structural labels. Figure 17 illustrates two examples of this dataset.

(a)

(b)

Figure 17: Examples of images in SciTSR

####  4.1.16 FinTabNet

FinTabNet [65] introduces GTE, a vision-guided systematic framework for
combined table detection and cell structured identification that can be
constructed on top of any object detection model. Create a new penalty based
on the natural cell containment constraint of tables with GTE-Table to train
their table network with the help of cell location predictions. GTE-Cell is a
novel hierarchical cell detection network that uses table layouts to detect
cells. Build a technique for automatically labeling table and cell structures
in existing texts to create a huge corpus of training and test data for a low
cost. FinTabNet is a collection of real-world and complicated scientific and
financial datasets with thorough table structure annotations to aid in
structure identification training and testing. Figure 18 illustrates two
examples of this dataset.

(a)

(b)

Figure 18: Examples of images in FinTabNet

####  4.1.17 PubTabNet

PubTabNet [66] one of the biggest openly accessible table recognition
collection, including 568k table pictures and structured HTML representations.
PubTabNet is built automatically by comparing the XML and PDF formats of
scientific publications in the PubMed CentralTM Open Access Subset (PMCOA).
authors also suggest an attention-based encoder-dual-decoder (EDD)
architecture for converting table graphics to HTML code. A structure decoder
is included in the model, which reconstructs the table structure and assists
the cell decoder in recognizing cell content. Furthermore, authors also
propose a new Tree-Edit-Distance-based Similarity (TEDS) metric for table
recognition that better captures multi-hop cell misalignment and OCR errors
than the existing metric. Figure 19 illustrates two examples of this dataset.

(a)

(b)

Figure 19: Examples of images in PubTabNet

####  4.1.18 TNCR

TNCR [67] a new table collection containing images of varied quality gathered
from free access websites The TNCR dataset may be used to recognize tables in
scanned document pictures and classify them into five categories. TNCR has
roughly 6621 photos and 9428 captioned tables. To build numerous robust
baselines, this work used state-of-the-art deep learning-based approaches for
table detection. On the TNCR dataset, Deformable DERT with Resnet-50 Backbone
Network delivers the best results compared to other methods, with an accuracy
of 86.7%, recall of 89.6%, and f1 score of 88.1%. A few samples from this
dataset are shown in Figure 20.

(a)

(b)

Figure 20: Examples of images in TNCR

####  4.1.19 SynthTabNet

To correct an imbalance in the earlier datasets, A Nassar [68] proposes
SynthTabNet, a synthetically created dataset with a variety of appearance
styles and complexity. The authors have created four synthetic datasets, each
with 150k samples. The most common words from PubTabNet and FinTabNet as well
as randomly produced text make up the corpora used to create the table
content. The first two synthetic datasets have been adjusted to closely
resemble the look of the real datasets while incorporating more intricate
table structures. The third one adopts a colorful style with strong contrast,
while the final one has tables with little content. Last but not least, The
authors have integrated all synthetic datasets into a single, 600k-example
synthetic dataset. A few samples from this dataset are shown in Figure 21.

(a)

(b)

Figure 21: Examples of images in SynthTabNet

Table 1 presents a comparison between some of the popular datasets of table
detection and structure recognition.

Table 1: The table illustrates a quantitative comparison between some famous
datasets in table detection.

Dataset Total pages Total Tables Table detection Table Structure
Classification Scanned ICDAR2013 462 150 ✓ ✓ ✗ ✓ ICDAR2017-POD 2,417 - ✓ ✗ ✗ ✓
TabStructDB 2.4k - ✗ ✓ ✗ ✓ TABLE2LATEX-450K - 450,000 ✗ ✓ ✗ ✓ RVL-CDIP
(SUBSET) 518 - ✓ ✗ ✗ ✓ IIIT-AR-13K 13K - ✓ ✗ ✗ ✓ CamCap 85 - ✓ ✓ ✗ ✗ UNLV 2889
- ✓ ✓ ✗ ✓ UW-3 dataset 1600 - ✓ ✓ ✗ ✓ Marmot 2000 - ✓ ✗ ✗ ✓ TableBank -
417,234 ✓ ✗ ✗ ✓ ICDAR2019 - 2000 ✓ ✓ ✗ ✓ DeepFigures - 5.5 million ✓ ✗ ✗ ✓
PubTables-1M 460,589 1 million ✓ ✓ ✗ ✓ SciTSR - 15,000 ✗ ✓ ✗ ✗ FinTabNet
89,646 112,887 ✓ ✓ ✗ ✗ PubTabNet - 568k ✗ ✓ ✗ ✓ TNCR 6621 9428 ✓ ✗ ✓ ✓
SynthTabNet 600k - ✓ ✓ ✓ ✓

###  4.2 Metrics

Table detectors use multiple criteria to measure the performance of the
detectors viz., frames per second (FPS), precision, and recall. However, mean
Average Precision (mAP) is the most common evaluation metric. Precision is
derived from Intersection over Union (IoU), which is the ratio of the area of
overlap and the area of union between the ground truth and the predicted
bounding box. A threshold is set to determine if the detection is correct. If
the IoU is more than the threshold, it is classified as True Positive while an
IoU below it is classified as False Positive. If the model fails to detect an
object present in the ground truth, it is termed a False Negative. Precision
measures the percentage of correct predictions while recall measures the
correct predictions with respect to the ground truth.

Average Precision (AP) $\displaystyle=\frac{\textrm{True Positive
(TP)}}{(\textrm{True Positive (TP)}+\textrm{False Positive (FP)})}$ (4)
$\displaystyle=\frac{TruePositive}{AllObservations}$

Average Recall (AR) $\displaystyle=\frac{\textrm{True Positive
(TP)}}{(\textrm{True Positive (TP)}+\textrm{False Negative (FN) })}$ (5)
$\displaystyle=\frac{TruePositive}{AllGroundTruth}$

F1-score
$\displaystyle=\frac{2*(\textrm{AP}*\textrm{AR})}{(\textrm{AP}+\textrm{AR})}$
(6)

Based on the above equation, average precision is computed separately for each
class. To compare performance between the detectors, the mean of average
precision of all classes, called mean average precision (mAP) is used, which
acts as a single metric for final evaluation.

IOU is a metric that finds the difference between ground truth annotations and
predicted bounding boxes. This metric is used in most state of art object
detection algorithms. In object detection, the model predicts multiple
bounding boxes for each object and based on the confidence scores of each
bounding box it removes unnecessary boxes based on their threshold value. We
need to declare the threshold value based on our requirements.

IOU $\displaystyle=\frac{\textrm{Area of union }}{\textrm{area of
intersection}}$ (7)

##  5 Table detection and structure recognition Models

Table detection has been studied for an extended period of time. Researchers
used different methods that can be categorized as follows:

  1. 1.

heuristic-based methods

  2. 2.

machine learning-based methods

  3. 3.

deep learning-based methods

Primarily heuristic-based methods were mainly used in the 1990s, 2000s, and
early 2010. They employed different visual cues like lines, keywords, space
features, etc. to detect tables.

P. Pyreddy et al. [69] proposed an approach of detecting tables using
character alignment, holes, and gaps. Wang et al. [70]. used a statistical
approach to detect table lines depending on the distance between consecutive
words. Grouped horizontal consecutive words together with vertical adjacent
lines were employed to propose table entity candidates. Jahan et al. [71]
presented a method that uses local thresholds for word spacing and line height
for detecting table regions.

Itonori [72] proposed a rule-based approach that led to the text-block
arrangement and ruled line position to localize the table in the documents.
Chandran and Kasturi [73] developed another table detection approach based on
vertical and horizontal lines. Wonkyo Seo et al. [56] used junctions
(intersection of the horizontal and vertical line) detection with further
processing.

Hassan et al. [74] locate and segment tables by analyzing spatial features of
text blocks. Ruffolo et al. [75] introduced PDF-TREX, a heuristic bottom-up
approach for table recognition in single-column PDF documents. It uses the
spatial features of page elements to align and group them into paragraphs and
tables. Nurminen [76] proposed a set of heuristics to locate subsequent text
boxes with common alignments and assign them the probability of being a table.

Fang et al. [77] used the table header as a starting point to detect the table
region and decompose its elements. Harit et al. [78] proposed a technique for
table detection based on the identification of unique table start and trailer
patterns. Tupaj et al. [79] proposed an OCR based table detection technique.
The system searches for sequences of table-like lines based on the keywords

The above methods work relatively well on documents with uniform layouts.
However, heuristic rules need to be tweaked to a wider variety of tables and
are not really suited for generic solutions. Therefore, machine learning
approaches started to be employed to solve the table detection problem.

Machine learning-based methods were common around the 2000s and the 2010s.

Kieninger et al. [80] applied an unsupervised learning approach by clustering
word segments. Cesarini et al. [81] used a modified XY tree supervised
learning approach. Fan et al.[82] uses both supervised and unsupervised
approaches to table detection in PDF documents. Wang and Hu [83] applied
Decision tree and SVM classifiers to layout, content type and word group
features. T. Kasar et al. [84] used the junction detection and then passed the
information to the SVM classifier. Silva et al. [85] applied joint probability
distribution over sequential observations of visual page elements (Hidden
Markov Models) to merge potential table lines into tables. Klampfl et al. [86]
compare two unsupervised table recognition methods from digital scientific
articles. Docstrum algorithm [87] applies KNN to aggregate structures into
lines and then uses perpendicular distance and angle between lines to combine
them into text blocks. It must be noted that this algorithm was devised in
1993, earlier than other methods mentioned in this section.

F Shafait [88] proposes a useful method for table recognition that performs
well on documents with a range of layouts, including business reports, news
stories, and magazine pages. The Tesseract OCR engine offers an open-source
implementation of the algorithm.

As neural networks gained interest, researchers started to apply them to
document layout analysis tasks. Initially, they were used at simpler tasks
like table detection. Later on, as more complex architectures were developed,
more work was put into table columns and overall structure recognition.

Hao et al. [24] employed CNN to detect whether a certain region proposal is a
table or not. Azka Gilani et al. [22] proposed a Faster R-CNN-based model to
make up for the limitations of Hao et al. [24] and other prior methodologies.

Sebastian Schreiber et al. [20] were the first to perform table detection and
structure recognition using Faster RCNN. He et al. [89], used FCN for semantic
page segmentation. S. Arif et al. [90] attempted to improve the accuracy of
Faster R-CNN by using semantic color-coding of text. Reza et al. [91] used a
combination of GAN-based architecture for table detection. Agarwal et al. [92]
used a multistage extension of Mask R-CNN with a dual backbone for detecting
tables.

Recently transformer-based models were applied to document layout analysis,
Smock, Brandon et al. [63] applied Carion et al.[93] DEtection TRansformer
framework, a transformer encoder-decoder architecture, to their table dataset
for both table detection and structure recognition tasks. Xu et al. [94]
proposed a self-supervised pre-trained Document Image Transformer model using
large-scale unlabeled text images for document analysis, including table
detection

###  5.1 Table detection Models

In this section, we examine the deep learning methods used for document image
table detection. We have divided the methods into several deep learning ideas
for the benefit of our readers’ convenience. Table 2 lists all the object
identification-based table detection strategies. It also discusses various
deep learning-based methods that have been used in these methods.

A Gilani [22] has shown how to recognize tables using deep learning. Document
pictures are pre-processed initially in the suggested technique. These photos
are then sent into a Region Proposal Network for table detection, which is
followed by a fully connected neural network. The suggested approach works
with great precision on a variety of document pictures, including documents,
research papers, and periodicals, with various layouts.

D Prasad [95] presents an automatic table detection approach for interpreting
tabular data in document pictures, which primarily entails addressing two
issues: table detection and table structure recognition. Using a single
Convolution Neural Network (CNN) model, provide an enhanced deep learning-
based end-to-end solution for handling both table detection and structure
recognition challenges. CascadeTabNet is a Cascade mask Region-based CNN High-
Resolution Network (Cascade mask R-CNN HRNet)-based model that simultaneously
identifies table areas and recognizes structural body cells from those tables.

SS Paliwal [96] presents TableNet which is a new end-to-end deep learning
model for both table detection and structure recognition. To divide the table
and column areas, the model uses the dependency between the twin objectives of
table detection and table structure recognition. Then, from the discovered
tabular sub-regions, semantic rule-based row extraction is performed.

Y Huang [97] describes a table detecting algorithm based on the YOLO
principle. The authors offer various adaptive improvements to YOLOv3,
including an anchor optimization technique and two post-processing methods, to
account for the significant differences between document objects and real
objects. also employ k-means clustering for anchor optimization to create
anchors that are more suited for tables than natural objects, making it easier
for our model to find the exact placements of tables. The additional
whitespaces and noisy page objects are deleted from the projected results
during the post-processing procedure.

L Hao [24] offers a new method for detecting tables in PDF documents that are
based on convolutional neural networks, one of the most widely used deep
learning models. The suggested method begins by selecting some table-like
areas using some vague constraints, then building and refining convolutional
networks to identify whether the selected areas are tables or not.
Furthermore, the convolutional networks immediately extract and use the visual
aspects of table sections, while the non-visual information contained in
original PDF documents is also taken into account to aid in better detection
outcomes.

SA Siddiqui [98] provide a novel strategy for detecting tables in documents.
The approach given here takes advantage of data’s potential to recognize
tables with any arrangement. however, the given method works directly on
photos, making it universally applicable to any format. The proposed method
uses a unique mix of deformable CNN and speedier R-CNN/FPN. Because tables
might be present at variable sizes and transformations, traditional CNN has a
fixed receptive field, which makes table recognition difficult (orientation).
Deformable convolution bases its receptive field on the input, allowing it to
shape it to match the input. The network can accommodate tables of any layout
because of this customization of the receptive field.

N Sun [99] presents a corner-finding approach for faster R-CNN-based table
detection. The Faster R-CNN network is first used to achieve coarse table
identification and corner location. then, coordinate matching is used to group
those corners that belong to the same table. Untrustworthy edges are filtered
at the same time. Finally, the matching corner group fine-tunes and adjusts
the table borders. At the pixel level, the suggested technique enhances table
boundary finding precision.

I Kavasidis[100] propose a method for detecting tables and charts using a
combination of deep CNNs, graphical models, and saliency ideas. M Holeček
[101] presented the concept of table understanding utilizing graph
convolutions in structured documents like bills, extending the applicability
of graph neural networks. A PDF document is used in the planned research as
well. The job of line item table detection and information extraction are
combined in this study to tackle the problem of table detection. Any word may
be quickly identified as a line item or not using the line item technique.
Following word classification, the tabular region may be easily identified
since, in contrast to other text sections on bills, table lines are able to
distinguish themselves rather effectively.

Á Casado-García [102] Uses object detection techniques, The authors have shown
that fine-tuning from a closer domain improves the performance of table
detection after conducting a thorough examination. The authors have utilized
Mask R-CNN, YOLO, SSD, and Retina Net in conjunction with object detection
algorithms. Two basic datasets are chosen to be used in this investigation,
TableBank and PascalVOC.

X Zheng [103] provides Global Table Extractor (GTE), a method for jointly
detecting tables and recognizing cell structures that can be implemented on
top of any object detection model. To train their table network with the help
of cell placement predictions, the authors develop GTE-Table, which introduces
a new penalty based on the inherent cell confinement limitation of tables. A
novel hierarchical cell identification network called GTE-Cell makes use of
table styles. Additionally, in order to quickly and inexpensively build a
sizable corpus of training and test data, authors develop a method to
automatically classify table and cell structures in preexisting texts.

Y Li [104] provides a new network to produce the layout elements for table
text and to enhance the performance of less ruled table identification. The
Generative Adversarial Networks(GAN) and this feature generator model are
comparable. The authors mandate that the feature generator model extract
comparable features for both heavily governed and loosely ruled tables.

DD Nguyen [105] introduces TableSegNet, a fully convolutional network with a
compact design that concurrently separates and detects tables. TableSegNet
uses a shallower path to discover table locations in high resolution and a
deeper path to detect table areas in low resolution, splitting the found
regions into separate tables. TableSegNet employs convolution blocks with
broad kernel sizes throughout the feature extraction process and an additional
table-border class in the main output to increase the detection and separation
capabilities.

D Zhang [106] suggests a YOLO-table-based table detection methodology. To
enhance the network’s capacity to learn the spatial arrangement aspects of
tables, the authors incorporate involution into the network’s core, and the
authors create a simple Feature Pyramid Network to increase model efficacy.
This research also suggests a table-based enhancement technique.

Table 2: A comparison of the benefits and drawbacks of several deep learning-
based table detection methods

Literature Method Benefits Drawbacks A Gilani[22] Faster R-CNN 1) On scanned
document pictures, this is the first deep learning-based table detection
method. 2) The object detection technique is made easier by converting RGB
pixels to distance measures. There are additional phases in the pre-processing
process. S Schreiber[20] transfer learning methods \+ Faster R-CNN end-to-end
strategy for detecting tables and table structures that is straightforward and
efficient When compared to other state-of-the-art techniques, it is less
accurate. SA Siddiqui [98] Deformable CNN + Faster R-CNN Deformable
convolutional neural networks’ dynamic receptive field aids in the
reconfiguration of multiple tabular boundries. When compared to standard
convolutions, deformable convolutions are computationally demanding. SS
Paliwal [96] Networks with fully convolutions 1) First attempt at combining a
single solution to handle both the problem of table detection and structure
recognition. 2) A comprehensive method for structure recognition and detection
in document pictures. This approach only functions on column detection when
used for table structure extraction. P Riba [54] OCR-based Graph NN that makes
use of textual characteristics The suggested technique makes use of more data
than only spatial attributes. 1) No comparisons to other state-of- the-art
strategies. 2) Additional annotations are needed using this strategy in
addition to the tabular data. N Sun [99] Faster R-CNN + Locate corners 1)
Better outcomes are obtained using a novel technique. 2) Faster R-CNN is used
to identify not just tables, but also the corners of tabular borders 1) It is
necessary to do postprocessing operations such as corner refining. 2) Because
of the additional detections, the computation is more involved. I Kavasidis
[100] combination of deep CNNs, graphical models, and saliency 1) Dilated
convolutions rather than conventional convolutions are used. 2) Using this
technique, saliency detection is performed in place of table detection. To
provide equivalent results, many processing stages are necessary. M Holeček
[101] Graph NN + line item identification Method This approach yields
encouraging outcomes when used to layout-intensive documents like invoices and
PDFs. 1) Limited baseline approach without comparisons to other state-of-the-
art techniques 2) No publicly accessible table datasets are used for the
evaluation of the approach. Y Huang [97] YOLO In comparison, a quicker and
more effective strategy The suggested methodology relies on data-driven post-
processing methods. Y Li [104] Generative Adversarial Networks(GAN) For ruling
and less ruled tables, the GAN-based strategy drives the network to extract
comparable characteristics. In document images with different tabular layouts,
the generator-based model is susceptible. M Li [61] Faster R-CNN This method
demonstrates how a basic Faster R-CNN can yield excellent results when used
with a huge dataset like TableBank. Just a simple Faster-RCNN implementation D
Prasad [95] Cascade mask Region-based CNN High-Resolution Network-based model
The study shows how iterative transfer learning may be used to transform
pictures, which can lessen the need on huge datasets. The same as[22], There
are additional phases in the pre-processing process. Á Casado-García [102]
Liken fine-tuning + Mask R-CNN, RetinaNet , SSD and YOLO Describe the
advantages of using object detection networks in conjunction with domain-
specific fine-tuning techniques for table detection. Closed domain fine-tuning
is still insufficient to get state-of-the-art solutions. M Agarwal [92]
multistage extension of Mask R-CNN with a dual backbone 1) A comprehensive
object detection-based frame- work utilizing a composite backbone to deliver
state- of-the-art outcomes 2) Extensive tests on benchmark datasets for table
detection that are openly accessible. The technique is computationally
expensive since it uses a composite backbone in addition to deformable
convolutions. X Zheng [103] Global Table Extractor (GTE) which is general
method for object detection 1) The problem of table detection is benefited by
the extra piece-wise constraint loss introduced. 2) a complete method that is
compatible with all object detection frameworks. Annotations for cellular
borders are necessary since the process of table detection depends on cell
detection.

###  5.2 Table Structure Recognition Models

In order to recognize table structures in document images, deep learning
approaches are reviewed in this part. We divided the methods into discrete
deep-learning principles for the benefit of our readers. Table 3,4 lists all
methods for recognizing table structures based on object detection, as well as
their benefits and drawbacks. It also discusses various deep learning-based
methods that have been used in these methods.

A Zucker [107] presents CluSTi, a Clustering approach for recognizing the
Structure of Tables in invoice scanned images, as an effective way. CluSTi
makes three contributions. To begin, it uses a clustering approach to
eliminate high noise from the table pictures. Second, it uses state-of-the-art
text recognition to extract all text boxes. Finally, CluSTi organizes the text
boxes into the correct rows and columns using a horizontal and vertical
clustering technique with optimum parameters. Z Zhang [108] present Split,
Embed, and Merge (SEM) is a table structure recognizer that is accurate. M
Namysl [109] presents a versatile and modular table extraction approach in
this research.

E Koci [110] offers a new method for identifying tables in spreadsheets and
constructing layout areas after determining the layout role of each cell.
Using a graph model, they express the spatial interrelationships between these
areas. On this foundation, they present Remove and Conquer (RAC), a table
recognition algorithm based on a set of carefully selected criteria.

Using the potential of deformable convolutional networks, SA Siddiqui [51]
proposes a unique approach for analyzing tabular patterns in document
pictures. P Riba [54] presents a graph-based technique for recognizing tables
in document pictures in this paper. also employ the location, context, and
content type instead of the raw content (recognized text), thus it’s just a
structural perception technique that’s not reliant on the language or the
quality of the text reading. E Koci [111] use genetic-based techniques for
graph partitioning, to recognize the sections of the graph matching to tables
in the sheet.

SA Siddiqui [112] described the structure recognition issue as the semantic
segmentation issue. To segment the rows and columns, the authors employed
fully convolutional networks. The approach of prediction tiling is introduced,
which lessens the complexity of table structural identification, assuming
consistency in a tabular structure. The author imported pre-trained models
from ImageNet and used the structural models of FCN’s encoder and decoder. The
model creates features of the same size as the original input picture when
given an image.

SA Khan [113] presents a robust deep learning-based solution for extracting
rows and columns from a recognized table in document pictures in this work.
The table pictures are pre-processed before being sent into a bi-directional
Recurrent Neural Network using Gated Recurrent Units (GRU) and a fully-
connected layer with softmax activation in the suggested solution. SF Rashid
[114] provides a new learning-based approach for table content identification
in diverse document pictures. SR Qasim [115] presents a graph network-based
architecture for table recognition as a superior alternative to typical neural
networks. S Raja [116] describes a method for recognizing table structure that
combines cell detection and interaction modules to locate the cells and
forecast their relationships with other detected cells in terms of row and
column. Also, add structural limitations to the loss function for cell
identification as extra differential components. The existing issues with end-
to-end table identification were examined by Y Deng [52], who also highlighted
the need for a larger dataset in this area.

Another study by Y Zou [117] called for the development of an image-based
table structure identification technique using fully convolutional networks.
the shown work divides a table’s rows, columns, and cells. All of the table
components’ estimated bounds are enhanced using connected component analysis.
Based on the placement of the row and column separators, row and column
numbers are then allocated for each cell. In addition, special algorithms are
used to optimize cellular borders.

To identify rows and columns in tables, KA Hashmi [118] suggested a guided
technique for table structure identification. The localization of rows and
columns may be made better, according to this study, by using an anchor
optimization approach. The boundaries of rows and columns are detected in
their proposed work using Mask R-CNN and optimized anchors.

Another effort to segment tabular structures is the ReS2TIM paper by W Xue
[119] which describes the reconstruction of syntactic structures from the
table. Regressing the coordinates for each cell is this model’s main
objective. A network that can identify the neighbors of each cell in a table
is initially built using the new technique. In the study, a distance-based
weighting system is given that will assist the network in overcoming the
training-related class imbalance problem.

C Tensmeyer [120] has presented SPLERGE (Split and Merge), another method
using dilated convolutions. Their strategy entails the use of two distinct
deep learning models, the first of which establishes the grid-like layout of
the table and the second of which determines if further cell spans over many
rows or columns are possible.

A Nassar [68] provide a fresh identification model for table structures. The
latter enhances the most recent encoder-dual-decoder from PubTabNet end-to-end
deep learning model in two important aspects. First, the authors provide a
brand-new table-cell object detection decoder. This allows them to easily
access the content of the table cells in programmatic PDFs without having to
train any proprietary OCR decoders. The authors claim that this architectural
improvement makes table-content extraction more precise and enables them to
work with non-English tables. Second, transformer-based decoders take the
place of LSTM decoders.

S Raja [121] suggests a novel object-detection-based deep model that is
tailored for quick optimization and captures the natural alignments of cells
inside tables. Dense table recognition may still be problematic even with
precise cell detection because multi-row/column spanning cells make it
difficult to capture long-range row/column relationships. Therefore, the
authors also seek to enhance structure recognition by determining a unique
rectilinear graph-based formulation. The author emphasizes the relevance of
empty cells in a table from a semantics standpoint. The authors recommend a
modification to a well-liked assessment criterion to take these cells into
consideration. To stimulate fresh perspectives on the issue, then provide a
moderately large assessment dataset with annotations that are modeled after
human cognition.

X Shen [122] suggested two modules, referred to as Rows Aggregated (RA) and
Columns Aggregated (CA). First, to produce a rough forecast for the rows and
columns and address the issue of high error tolerance, feature slicing and
tiling are applied. Second, the attention maps of the channels are computed to
further obtain the row and column information. In order to complete the rows
segmentation and columns segmentation, the authors employ RA and CA to
construct a semantic segmentation network termed the Rows and Columns
Aggregated Network (RCANet).

C Ma[123] present RobusTabNet, a novel method for recognizing the structure of
tables and detecting their borders from a variety of document pictures. The
authors suggest using CornerNet as a new region proposal network to produce
higher quality table proposals for Faster R-CNN, which has greatly increased
the localization accuracy of Faster R-CNN for table identification. by
utilizing only the minimal ResNet-18 backbone network. Additionally, the
authors suggest a brand-new split-and-merge approach for recognizing table
structures. In this method, each detected table is divided into a grid of
cells using a novel spatial CNN separation line prediction module, and then a
Grid CNN cell merging module is used to recover the spanning cells. Their
table structure recognizer can accurately identify tables with significant
blank areas and geometrically deformed (even curved) tables because the
spatial CNN module can efficiently transmit contextual information throughout
the whole table picture. B Xiao [124] postulates that a complex table
structure may be represented by a graph, where the vertices and edges stand in
for individual cells and the connections between them. Then, the authors
design a conditional attention network and characterize the table structure
identification issue as a cell association classification problem (CATT-Net).

A Jain [125] suggests training a deep network to recognize the spatial
relationships between various word pairs included in the table picture in
order to decipher the table structure. The authors offer an end-to-end
pipeline called TSR-DSAW: TSR through Deep Spatial Association of Words, which
generates a digital representation of a table picture in a structured format
like HTML. The suggested technique starts by utilizing a text-detection
network, such as CRAFT, to identify every word in the input table picture.
Next, using dynamic programming, word pairings are created. These word
pairings are underlined in each individual image and then given to a
DenseNet-121 classifier that has been trained to recognize spatial
correlations like same-row, same-column, same-cell, or none. Finally, The
authors apply post-processing to the classifier output in order to produce the
HTML table structure.

H Li [126] formulate the issue as a cell relation extraction challenge and
provide T2, a cutting-edge two-phase method that successfully extracts table
structures from digitally preserved texts. T2 offers a broad idea known as a
prime connection that accurately represents the direct relationships between
cells. To find complicated table structures, it also builds an alignment graph
and uses a message-passing network.

Table 3: A comparison of the benefits and drawbacks of several deep learning-
based table Structure recognition methods

Literature Method Benefits Drawbacks SF Rashid [114] Uses the geometric pos-
ition of words + A neu- ral network model (autoMLP) No reliance on complex
layout analysis Mechanism. Can be used on the diverse set of documents with
different layouts limitation is in marking columns boundaries due to
variations in the number of words in each column E Koci [110] Encoding of
spatial inter- relations between these regions using a graph rep- resentation,
as well as rules and heuristics 1) Recognition for single-table and multi-
table spreadsheets. 2) No reliance on any assumptions with what regards the
arran- gement of tables Tables with few columns and empty cells are not
handled well. SA Siddiqui [51] deformable CNN \+ Faster R-CNN 1) The use of
deformable convolution can handle various tabular structures. 2) rel- eased a
new dataset that contained table structure data. The tables in the proposed
approach won’t operate co- rrectly if they have a row a- nd column span. SA
Siddiqui [112] Fully CNNs The complexity of the task of identifying t- able
structures is reduced by the proposed prediction tiling approach. 1)
Additional post-processing p- rocesses are necessary when ro- ws or columns
are excessively fragmented. 2) The technique is based on the tabular
structures’ consistency assumption. SR Qasim [115] Graph NN + CNN 1) This
paper also presents a unique, memory- efficient training strategy based on
Monte Ca- rlo. 2) The suggested approach makes use of both textual and spatial
characteristics. The publicly accessible table datasets are not used to test
the system. W Xue [119] Graph NN + weights depending on distance For the cell
relationship network, the class imbalance issue is solved using the distance-
based weighting method. When dealing with sparse tab- les, the approach is
insecure. C Tensmeyer [120] Dilated Convolutions \+ Fully CNN The technique is
effective with both scanned and PDF document images. The post-processing
heuristics determine how the merging p- ortion of the method works. SA Khan
[113] RNN The reduced receptive field of CNNs is solved by the bi-directional
GRU. Pre-processing procedures including binarization, noise reduction, and
morphological modification are ne- cessary. P Riba [54] Graph Neural Networks
approach 1) It is not constrained to rigid tabular lay- outs in terms of
single rows, columns or pr- esence of rule lines. 2) The model is langua- ge
independent 1) The method may have problems when dealing with border cond-
itions. 2) There is a small amount of training data in the RVL-CDIP dataset
and F1, Precision and Re- call metrics are lower than other methods. Y Deng
[52] Encoder decoder net 1) In the work that is given, issues with end- to-end
table recognition are examined. 2) Mad -e a contribution with yet another
sizable data -set in the area of table comprehension. The other publicly
accessible table recognition datasets are not used to assess the suggested
base- line technique. E Koci [111] Graph model + Appl- ication of genetic-
based approaches Requires little to no involvement of domain experts The
accuracy of GE depends on the number of edges. Specifica- lly, we determined
that GE ach- ieves an accuracy of only 19% for multi-table graphs D Prasad
[95] Cascade mask Reg- ionbased CNN High- Resolution Network- based model
Direct regression occurs at cellular bound- aries using an end-to-end method.
Tables with/out ruling lin- es must undergo further post-processing. S Raja
[116] Mask R-CNN + ResNet-101 based Net 1) An additional alignment loss is
sugges- ted for precise cell detection. 2) A train- able top-down for cell
identification and bottom-up for structure recognition coll- ection is
proposed. When cells are empty, the strategy is weak.

Table 4: A comparison of the benefits and drawbacks of several deep learning-
based table Structure recognition methods (continue Table 3)

Literature Method Benefits Drawbacks B Xiao [124] cells’ bounding boxes \+
conditional attention network Only utilizes visual features without any meta-
data 1) Assumes that the coordinates of cells in the table are known. 2)
Difficulties with tables with- out borders Y Zou [117] Fully CNNs 1) Using
linked component analysis enhances the outcomes. 2) In a table, cells are
segmen- ted in addition to the rows and columns. To provide comparison
findings, a small number of post-process- ing procedures utilizing specific
algorithms are necessary. X Zhong [66] Dual decoder with attention-based enc-
oding 1) To assess table recognition techniques, the methodology offers a
unique evaluation metric called TEDS. 2) released a huge table dataset. The
technique cannot be readily compared to other state-of-the- art techniques. KA
Hashmi [118] Utilizing an optimi- zation technique for anchors+ Mask R- CNN
Networks of region proposals converge mo- re quickly and effectively thanks to
optim- ized anchoring. This study relies on the pre liminary pre-processing
phase of clustering the ground truth to find appropriate anchors. A Zucker
[107] Character Region Awareness for Text Detection (CRAFT) and Density-Based
Spatial Clustering of Applications with Noise (DBSCAN) A bottom-up method,
which emphasizes that the table structure is formed by relative pos- itions of
text cells, and not by inherent bou- ndaries Cannot handle spreading rows or
columns well X Zheng [103] Method for object detecting generally An additional
innovative cluster-based tech nique combined with a hierarchical network to
detect tabular forms. Accurately classifying a table is a prerequisite for
final cell structure identification. Z Zhang [108] A combination of fully
convo- lutional network (FCN)+ RoI- Align + the pretrained BERT model + Gated
Recurrent Unit (GRU) decoder Directly operates on table images with no
dependency on meta-information, can pro- cess simple and complex tables
Oversegments tables when space between cells is large, doesn’t handle merged
cells well M Namysl [109] Rule-based algorithms + graph-based table inter-
pretation method 1) Approach allows processing images and digital documents.
2) Processing steps can be adapted separately 1) Support the most frequent ta-
ble formats only. Reliance on the presence of predefined keywords. 2) Prone to
the errors propagated from the upstream components of system. 3) Focus on the
ta- bles with rulings A Nassar [68] End-to-end neural network \+ CNN Backbone
+ tran- sformer based layers 1) Handles different languages without being
trained on them. 2) Predicts tables structure and bounding boxes for the table
content Work with PDF documents A Jain [125] spatial associations + dyna- mic
programming techniques Recognizing complex table structures having multi-span
rows/columns and missing cells Uses OCR to read words from images Not language
agnostic S Raja [121] object detection Better detection of empty cells Fails
for very sparse tables wh- ere most of the cells are empty

##  6 Methodology

In this section, we will extend the methodology and methods used for the TNCR
dataset in our previous work [67]. in the previous work, we described Cascade
R-CNN, Cascade Mask R-CNN, Cascade RPN, Hybrid Task Cascade (HTC), YOLO, and
Deformable DETR. In this section, we will describe additional four
methodologies of using object detection and classification for Faster R-CNN,
Mask R-CNN, HRNets, Resnest, and Dynamic R-CNN.

###  6.1 Faster R-CNN

Faster R-CNN [127] contains two modules: The RPN is a fully-convolutional
network that generates region proposals, and the Fast-RCNN detector takes the
proposal from RPN as input and generates object detection results as seen in
Fig. 22. A feature extraction network, which is often a pretrained CNN, is
employed in a Faster R-CNN object detection network, similar to what we
utilized for its predecessor. Following that, there are two trainable
subnetworks. The first is a Region Proposal Network (RPN), which is used to
produce object proposals as its name indicates, and the second is used to
predict the object’s real class. The RPN that is put after the last
convolutional layer is thus the major differentiator for Faster R-CNN. This
has been taught to generate region proposals without the use of any external
mechanisms such as Selective Search. Then, similar to Fast R-CNN, utilize ROI
pooling, an upstream classifier, and a bounding box regressor.

Figure 22: Faster R-CNN

###  6.2 Mask R-CNN

Mask R-CNN [128] uses R-CNN to effectively detect objects in an image while
also performing object segmentation tasks for each region of interest. As a
result, segmentation runs concurrently with classification and bounding box
regression. The high-level architecture of the Mask R-CNN is shown in Fig. 23.

Figure 23: Mask R-CNN

Mask RCNN is divided into two phases. First, it generates proposals based on
the input image for regions where an object might be present. Second, based on
the first stage proposals, it predicts the object’s class, refines the
bounding box, and creates a mask at the pixel level of the object. The
backbone structure is related to both phases. The concept of Mask R-CNN is
simple: Faster R-CNN outputs a class label and a bounding-box offset for each
candidate object; Mask R-CNN adds a third branch that outputs the object mask.

###  6.3 HRNets

Ke Sun et at,[129, 130] present a novel architecture called High-Resolution
Net, which is capable of maintaining high-resolution representations through
the entire process. The first stage of the HRNet project is to build a high-
resolution subnetwork. The next stage is to add more high-to-low resolutions
subnetworks. The multi-scale fusions are carried out by HRNet through a
parallel multi-resolution network as seen in Fig. 24.

Figure 24: HRNet

Compared to existing widely-used networks [131, 132, 133, 134], HRNet has two
advantages: it can connect high-to-low-resolution subnetworks in parallel, and
it can provide better pose estimation. Most existing fusion schemes combine
low-resolution and high-resolution representations. Instead of doing so, HRNet
performs multiscale fusions to boost both the high-resolution and low-
resolution representations.

###  6.4 Resnest

Resnest [135] is a simple architecture that combines the features of a
multipath network with a channel-wise attention strategy. It allows for the
preservation of independent representations in the meta structure. As in a
multi-path network, a Resnet network module performs a set of transformations
on low-dimensional embeddings and concatenates their outputs. Each
transformation is carried out with a different attention strategy to capture
the interdependencies of a feature map. The key difference between the two is
that the attention strategy is focused on the specific channel and not the
whole network. The Split-Attention block is a computing unit that combines
feature map group and split attention operations. A Split-Attention Block is
depicted in Fig. 25.

Figure 25: Resnest

###  6.5 Dynamic R-CNN

Hongkai Zhang et al, [136] proposes Dynamic RCNN, a simple but effective
method for maximizing the dynamic quality of object detection proposals. It is
made up of two parts: Dynamic Label Assignment and Dynamic SmoothL1 Loss,
which are used for classification and regression, respectively. First, adjust
the IoU threshold for positive/negative samples based on the proposals
distribution in the training procedure to train a better classifier that is
discriminative for high IoU proposals. Set the threshold as the proposal’s IoU
at a certain percentage because this can reflect the overall distribution’s
quality. Change the shape of the regression loss function for regression to
adaptively fit the regression label distribution change and ensure the
contribution of high-quality samples to training.

##  7 Experiments Results

###  7.1 Experiment Settings

The MMdetection [137] library for PyTorch has been used to implement each of
the proposed and tested models. A vast variety of object detection and
instance segmentation methods, as well as associated parts and modules, are
included in the object detection toolkit known as MMDetection. It gradually
transforms into a single platform that includes several widely used detection
techniques and contemporary modules. Three Tesla V100-SXM GPUs with 16 GB of
GPU memory, 16 GB of RAM, two Intel Xeon E-5-2680 CPUs, and four NVIDIA Tesla
k20x GPUs were used in the trials, which were carried out on the Google
Colaboratory platform. With pictures scaled to a constant size of 1300
$\times$ 1500 and a batch size of 16, all models have been trained and
evaluated. The optimizer with a momentum of 0.9, a weight decay of 0.0001, and
a learning rate of 0.02 is known as SGD. The Feature Pyramid Network (FPN)
neck is used by all models.

###  7.2 Results of TNCR dataset

The Faster R-CNN model has achieved good performance in table detection
compared with Cascade-RCNN and Cascade Mask-RCNN in most of the backbones. We
have trained the Faster R-CNN model with L1 Loss [138] with Resnet-50 for
bounding box regression. As shown in Table 5, it achieves f1-score of 0.921.
Resnet-101 backbone achieves the highest F1 score over 50% to 65%,
ResNeXt-101-64x4d achieves the highest F1 score over 70% to 95% and
ResNeXt-101-64x4d achieves the highest F1 score over 50%:95% of 0.786.
Resnet-50 backbone with 1$\times$ Lr schedule achieves the lowest performance
over 50% to 60% IoUs. Also, the Resnet-50 backbone with L1 Los achieves the
lowest performance from 65% to 95% IoUs and also achieves the lowest
performance over 50%:95%.

Table 5: Faster R-CNN

Backbone Lr schd/Losses IoU 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 50%:95%
Resnet-50 L1Loss Precision 0.875 0.872 0.872 0.866 0.858 0.844 0.823 0.782
0.688 0.424 0.649 Recall 0.973 0.972 0.970 0.964 0.956 0.941 0.922 0.890 0.812
0.577 0.775 F1-Score 0.921 0.919 0.918 0.912* 0.904* 0.889* 0.869* 0.832*
0.744* 0.488* 0.706* Resnet-50 1x Precision 0.874 0.872 0.871 0.869 0.863
0.844 0.827 0.783 0.693 0.431 0.653 Recall 0.972 0.969 0.968 0.966 0.959 0.942
0.929 0.895 0.823 0.587 0.779 F1-Score 0.920* 0.917* 0.916* 0.914 0.908 0.890
0.875 0.835 0.752 0.497 0.710 Resnet-101 1x Precision 0.885 0.885 0.882 0.879
0.870 0.867 0.849 0.820 0.763 0.555 0.720 Recall 0.973 0.973 0.971 0.969 0.961
0.956 0.943 0.920 0.870 0.698 0.835 F1-Score 0.926 0.926 0.924 0.921 0.913
0.909 0.893 0.867 0.812 0.618 0.773 ResNeXt-101-32x4d 1x Precision 0.880 0.879
0.877 0.875 0.872 0.866 0.845 0.817 0.760 0.575 0.727 Recall 0.976 0.976 0.975
0.972 0.969 0.962 0.944 0.921 0.871 0.711 0.843 F1-Score 0.925 0.924 0.923
0.920 0.917 0.911 0.891 0.865 0.811 0.635 0.780 ResNeXt-101-64x4d 1x Precision
0.884 0.884 0.880 0.879 0.876 0.871 0.856 0.833 0.780 0.581 0.733 Recall 0.972
0.970 0.969 0.967 0.965 0.961 0.950 .931 0.884 0.724 0.848 F1-Score 0.925
0.925 0.922 0.920 0.918 0.913 0.900 0.879 0.828 0.644 0.786

We implemented Mask R-CNN [128] to use R-CNN for table objects in an image and
also used for performing object segmentation for each ROI. As seen in Table 6,
Mask R-CNN shows good performance in our dataset in precision, recall, and F1
score for all backbones. Resnet-101 backbone has achieved the highest F1 score
of 0.774 over 50%:95% and maintains the highest F1 score at various IoUs.
ResNeXt-101-32x4d achieves the lowest performance over 50% to 95% IoUs and
also achieves an f1 score of 0.512 over 50%:95%. ResNeXt-101-64x4d also
achieves the lowest performance at various IoUs except for 95% IoU.

Table 6: Mask R-CNN

Backbone Lr schd IoU 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 50%:95% Resnet-50
1x Precision 0.877 0.876 0.874 0.871 0.868 0.858 0.834 0.800 0.728 0.506 0.692
Recall 0.973 0.972 0.970 0.967 0.963 0.952 0.932 0.903 0.840 0.651 0.812
F1-Score 0.922 0.921 0.919 0.916 0.913 0.902 0.880 0.848 0.779 0.569 0.747
Resnet-101 1x Precision 0.878 0.877 0.875 0.874 0.869 0.861 0.847 0.812 0.762
0.553 0.716 Recall 0.977 0.976 0.974 0.973 0.966 0.959 0.949 0.918 0.874 0.711
0.844 F1-Score 0.924 0.923 0.921 0.920 0.914 0.907 0.895 0.861 0.814 0.622
0.774 ResNeXt-101-32x4d 1x Precision 0.778 0.777 0.774 0.769 0.759 0.749 0.713
0.651 0.477 0.407 0.434 Recall 0.975 0.974 0.968 0.964 0.952 0.941 0.913 0.856
0.725 0.695 0.626 F1-Score 0.865* 0.864* 0.860* 0.855* 0.844* 0.834* 0.800*
0.739* 0.575* 0.513* 0.512* ResNeXt-101-64x4d 1x Precision 0.778 0.777 0.774
0.769 0.759 0.749 0.713 0.651 0.477 0.417 0.434 Recall 0.975 0.974 0.968 0.964
0.952 0.941 0.913 0.856 0.725 0.705 0.626 F1-Score 0.865 0.864 0.860 0.855
0.844 0.834 0.800 0.739 0.575 0.524 0.512

In the following tables 7, 8, 9, 10, 11 and 12 show the comparative analysis
we have trained HRNets with different methods and each method with different
backbones for object detection and instance segmentation models. In Table 7,
we evaluated and calculated f1-score. It shows that HRNetV2p-W40 achieves
better performance over 50% to 65%. Also, HRNetV2p-W18 achieves better
performance over 70%, 75%, 80%, and 95%. HRNetV2p-W18 achieves an f1 score of
0.842 over 50%:95% IoU. HRNetV2p-W32 backbone achieves the lowest performance
over 50% to 70%, and 90% IoUs and HRNetV2p-W40 achieve the lowest performance
over 75% to 85% and 95%. HRNetV2p-W40 achieves f1 score of 0.841 over 50%:95%
IoU.

Table 7: HRNets - Cascade R-CNN

Backbone Lr schd IoU 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 50%:95%
HRNetV2p-W18 20e Precision 0.894 0.894 0.894 0.892 0.892 0.886 0.880 0.862
0.825 0.712 0.803 Recall 0.962 0.962 0.960 0.960 0.960 0.954 0.950 0.937 0.906
0.813 0.887 F1-Score 0.926 0.926 0.925 0.924 0.924 0.918 0.913 0.897 0.863
0.759 0.842 HRNetV2p-W32 20e Precision 0.895 0.895 0.893 0.893 0.893 0.889
0.881 0.869 0.828 0.717 0.806 Recall 0.955 0.955 0.954 0.954 0.953 0.949 0.943
0.933 0.900 0.810 0.882 F1-Score 0.924* 0.924* 0.922* 0.922* 0.922* 0.918
0.910 0.899 0.862* 0.760 0.842 HRNetV2p-W40 20e Precision 0.893 0.891 0.891
0.891 0.888 0.880 0.871 0.854 0.831 0.705 0.799 Recall 0.967 0.965 0.965 0.964
0.961 0.956 0.948 0.935 0.914 0.811 0.889 F1-Score 0.928 0.926 0.926 0.926
0.923 0.916* 0.907* 0.892* 0.870 0.754* 0.841*

Table 8 shows the performance of the HRNets Faster R-CNN detector with various
backbone structures with combinations of Lr Schedule. The HRNetV2p-W18 with
1$\times$ Lr Schedule backbone shows a low performance compared with other
backbones. it achieves an f1 score of 0.770. It achieves 3.2% less than
HRNetV2p-W18 with 2$\times$ Lr Schedule. HRNetV2p-W40 with 1$\times$ Lr
Schedule backbone achieves better performance over 50% to 85% IoUs and
HRNetV2p-W40 with 2$\times$ Lr Schedule backbone achieves better performance
over 90% and 95% IoUs. HRNetV2p-W18 with 2$\times$ Lr Schedule backbone
achieves an f1 score of 0.802 over 50%:95%. HRNetV2p-W32 with 1$\times$ Lr
Schedule backbone share same performance over 50% to 60%.

Table 8: HRNets - Faster R-CNN

Backbone Lr schd IoU 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 50%:95%
HRNetV2p-W18 1x Precision 0.867 0.865 0.863 0.859 0.853 0.845 0.827 0.806
0.750 0.556 0.711 Recall 0.972 0.970 0.968 0.964 0.959 0.952 0.940 0.915 0.869
0.711 0.842 F1-Score 0.916* 0.914* 0.912* 0.908* 0.902* 0.895* 0.879* 0.857*
0.805* 0.624* 0.770* HRNetV2p-W18 2x Precision 0.876 0.873 0.872 0.869 0.867
0.857 0.845 0.817 0.776 0.628 0.752 Recall 0.962 0.960 0.958 0.955 0.953 0.946
0.937 0.910 0.874 0.759 0.860 F1-Score 0.916 0.914 0.912 0.909 0.907 0.899
0.888 0.860 0.822 0.687 0.802 HRNetV2p-W32 1x Precision 0.877 0.876 0.874
0.869 0.862 0.859 0.839 0.822 0.759 0.579 0.728 Recall 0.969 0.968 0.967 0.963
0.957 0.954 0.939 0.922 0.870 0.728 0.849 F1-Score 0.920 0.919 0.918 0.913
0.907 0.904 0.886 0.869 0.810 0.645 0.783 HRNetV2p-W32 2x Precision 0.877
0.877 0.877 0.874 0.869 0.864 0.847 0.820 0.785 0.592 0.735 Recall 0.964 0.964
0.963 0.960 0.956 0.951 0.939 0.918 0.886 0.734 0.849 F1-Score 0.918 0.918
0.917 0.914 0.910 0.905 0.890 0.866 0.832 0.655 0.787 HRNetV2p-W40 1x
Precision 0.875 0.874 0.873 0.872 0.868 0.862 0.851 0.827 0.779 0.612 0.743
Recall 0.970 0.969 0.968 0.967 0.964 0.958 0.949 0.930 0.888 0.753 0.862
F1-Score 0.920 0.919 0.918 0.917 0.913 0.907 0.897 0.875 0.829 0.675 0.798
HRNetV2p-W40 2x Precision 0.880 0.880 0.877 0.877 0.873 0.861 0.852 0.834
0.802 0.629 0.754 Recall 0.957 0.957 0.954 0.954 0.951 0.943 0.935 0.918 0.890
0.755 0.856 F1-Score 0.916 0.916 0.913 0.913 0.910 0.900 0.891 0.873 0.843
0.686 0.801

Table 9 shows the performance of the HRNets HTC method with the same Lr
Schedule. HRNetV2p-W40 backbone suffers from overfitting through the dataset.
HRNetV2p-W18 achieves f1 score of 0.840, precision of 0.901 and recall 0.788
over 50%:95%. HRNetV2p-W18 achieves better performance over various IoUs.
HRNetV2p-W32 shows less performance compare with HRNetV2p-W18 with 8.3% over
50%:95% for the f1 score.

Table 9: HRNets - HTC

Backbone Lr schd IoU 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 50%:95%
HRNetV2p-W18 20e Precision 0.885 0.885 0.883 0.882 0.881 0.875 0.862 0.849
0.808 0.691 0.788 Recall 0.987 0.987 0.984 0.984 0.982 0.976 0.966 0.954 0.915
0.816 0.901 F1-Score 0.933 0.933 0.930 0.930 0.928 0.922 0.911 0.898 0.858
0.748 0.840 HRNetV2p-W32 20e Precision 0.851 0.851 0.849 0.846 0.843 0.834
0.816 0.792 0.737 0.516 0.684 Recall 0.985 0.985 0.984 0.981 0.976 0.968 0.951
0.929 0.885 0.710 0.848 F1-Score 0.913* 0.913* 0.911* 0.908* 0.904* 0.896*
0.878* 0.855* 0.804* 0.597* 0.757*

Table 10 shows the performance of HRNets HTC method with the same Lr Schedule.
HRNetV2p-W32 has achieved the highest f1 score of 0.871 over 50%:95% and
continues to achieve the highest F1 score at various IoUs. HRNetV2p-W32 shows
good performance compare with HRNetV2p-W18 with 12% over 50%:95% for f1 score.
HRNetV2p-W40 with 1$\times$ and 2$\times$ Lr Schedule backbones suffer from
overfitting through the dataset. Table 11 shows the performance of HRNets
Cascade Mask R-CNN method. HRNetV2p-W32 and HRNetV2p-W40 backbones suffer from
overfitting through the dataset. HRNetV2p-W18 achieve f1 score of 0.903 over
50%:95%.

Table 10: HRNets - Mask R-CNN

Backbone Lr schd IoU 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 50%:95%
HRNetV2p-W18 1x Precision 0.848 0.845 0.840 0.839 0.835 0.829 0.817 0.793
0.736 0.521 0.684 Recall 0.971 0.969 0.966 0.964 0.960 0.956 0.947 0.928 0.876
0.698 0.834 F1-Score 0.905* 0.902* 0.898* 0.897* 0.893* 0.887* 0.877* 0.855*
0.799* 0.596* 0.751* HRNetV2p-W32 1x Precision 0.859 0.857 0.857 0.857 0.852
0.848 0.833 0.816 0.764 0.585 0.816 Recall 0.971 0.969 0.969 0.969 0.965 0.960
0.947 0.934 0.889 0.744 0.934 F1-Score 0.911 0.909 0.909 0.909 0.904 0.900
0.886 0.871 0.821 0.654 0.871

Table 11: HRNets - Cascade Mask R-CNN

Backbone Lr schd IoU 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 50%:95%
HRNetV2p-W18 20e Precision 0.888 0.887 0.887 0.886 0.885 0.884 0.872 0.858
0.828 0.732 0.810 Recall 0.970 0.970 0.970 0.967 0.967 0.965 0.955 0.942 0.918
0.836 0.903 F1-Score 0.927 0.926 0.926 0.924 0.924 0.922 0.911 0.898 0.870
0.780 0.903

Table 12 shows the performance of HRNets with Fully Convolutional One-Stage
(FCOS) Object Detection. HRNets FCOS achieves less performance compared with
other models. HRNetV2p-W18 with 2$\times$ Lr Schedule achieve an increment of
25% f1 score from HRNetV2p-W18 with 1$\times$ Lr Schedule and HRNetV2p-W18
with 2$\times$ Lr Schedule achieve an increment of 22.5% f1 score from
HRNetV2p-W32. HRNetV2p-W18 with 2$\times$ Lr Schedule achieve an f1 score of
0.648.

Table 12: HRNets - FCOS

Backbone Lr schd IoU 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 50%:95%
HRNetV2p-W18 1x Precision 0.511 0.507 0.498 0.485 0.467 0.441 0.405 0.328
0.222 0.086 0.298 Recall 0.959 0.946 0.930 0.910 0.885 0.844 0.798 0.697 0.517
0.244 0.601 F1-Score 0.666* 0.660* 0.648* 0.632* 0.611* 0.579* 0.537* 0.446*
0.310* 0.127* 0.398* HRNetV2p-W18 2x Precision 0.790 0.788 0.782 0.779 0.770
0.759 0.729 0.691 0.596 0.335 0.563 Recall 0.983 0.978 0.972 0.969 0.959 0.947
0.917 0.878 0.786 0.545 0.764 F1-Score 0.875 0.872 0.866 0.863 0.854 0.842
0.812 0.773 0.677 0.414 0.648 HRNetV2p-W32 1x Precision 0.566 0.561 0.555
0.539 0.528 0.504 0.469 0.400 0.275 0.086 0.326 Recall 0.970 0.964 0.956 0.928
0.906 0.868 0.818 0.730 0.571 0.241 0.605 F1-Score 0.714 0.709 0.702 0.681
0.667 0.637 0.596 0.516 0.371 0.126 0.423

In the following tables 13 and 14 show the comparative analysis, we have
trained ResNeSt with Cascade R-CNN and Faster R-CNN methods, and each method
with different backbones(Resnest-50, Resnest-101). For Cascade R-CNN S-101
backbone achieve an f1 score of 0.845 over 50%:95% IoU and also achieves the
highest performance compare with the S-50 backbone and the Faster R-CNN
method. The Faster R-CNN S-101 backbone achieves an f1 score of 0.748 over
50%:95% IoU. Cascade R-CNN S-101 backbone has an increment of 9.2% over
50%:95% for the f1 score.

Dynamic RCNN proposes by [136], it is a simple but effective method for
maximizing the dynamic quality of object detection proposals. Table 15 show
the Dynamic RCNN with Resnet-50 achieves an f1 score of 0.628, the precision
of 0.561, recall of 0.714 over 50%:95%.

Table 13: Resnest - Cascade R-CNN

Backbone Lr schd IoU 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 50%:95% S-50 1x
Precision 0.895 0.894 0.891 0.885 0.881 0.875 0.870 0.854 0.808 0.659 0.777
Recall 0.977 0.976 0.974 0.969 0.965 0.959 0.954 0.940 0.903 0.784 0.880
F1-Score 0.934 0.933 0.930 0.925 0.921 0.915 0.910 0.894 0.852 0.716 0.825
S-101 1x Precision 0.905 0.903 0.902 0.899 0.893 0.891 0.884 0.876 0.826 0.693
0.799 Recall 0.985 0.984 0.983 0.979 0.976 0.972 0.965 0.958 0.917 0.811 0.898
F1-Score 0.943 0.941 0.940 0.937 0.932 0.929 0.922 0.915 0.869 0.747 0.845

Table 14: Resnest - Faster R-CNN

Backbone Lr schd IoU 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 50%:95% S-50 1x
Precision 0.884 0.884 0.880 0.879 0.872 0.861 0.844 0.809 0.709 0.429 0.656
Recall 0.970 0.970 0.968 0.967 0.961 0.951 0.935 0.906 0.824 0.597 0.784
F1-Score 0.925 0.925 0.921 0.920 0.914 0.903 0.887 0.854 0.762 0.499 0.714
S-101 1x Precision 0.893 0.893 0.890 0.888 0.879 0.876 0.862 0.823 0.747 0.495
0.694 Recall 0.981 0.979 0.977 0.975 0.967 0.963 0.950 0.921 0.861 0.645 0.813
F1-Score 0.934 0.934 0.931 0.929 0.920 0.917 0.903 0.869 0.799 0.560 0.748

Table 15: Dynamic R-CNN

Backbone Lr schd IoU 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 50%:95% Resnet-50
1x Precision 0.855 0.854 0.853 0.849 0.839 0.823 0.802 0.764 0.646 0.267 0.561
Recall 0.978 0.977 0.975 0.971 0.963 0.943 0.925 0.888 0.793 0.451 0.714
F1-Score 0.912 0.911 0.909 0.905 0.896 0.878 0.859 0.821 0.711 0.335 0.628

Table 16: Table detection

Approach Dataset Method IoU Year 50% 55% 60% 65% 70% 75% 80% 85% 90% 95%
50%:95% Tesseract [88] UNLV Tab-stop Detection Precision - - - - - - - - 86.00
- - 2010 Recall - - - - - - - - 79.00 - - F1-Score - - - - - - - - 82.35 - - A
Gilani[22] UNLV Faster R-CNN Precision - - - - - - - - 82.30 - - 2017 Recall -
- - - - - - - 90.67 - - F1-Score - - - - - - - - 86.29 - - SA Siddiqui[98]
UNLV Deformable CNN \+ Faster R-CNN Precision 78.6 - - - - - - - - - - 2018
Recall 74.9 - - - - - - - - - - F1-Score 76.7 - - - - - - - - - - Á Casado-
García[102] UNLV YOLO Precision - - 93.0 - 92.0 - 83.0 - 48.0 - - 2020 Recall
- - 95.0 - 94.0 - 85.0 - 49.0 - - F1-Score - - 94.0 - 93.0 - 84.0 - 49.0 - - M
Agarwal [92] UNLV Cascade mask R-CNN Precision 96.0 - 94.4 - 91.5 - 82.6 -
61.8 - - 2018 Recall 77.0 - 75.8 - 73.4 - 66.3 - 49.6 - - F1-Score 86.5 - 85.1
- 82.5 - 74.4 - 55.7 - - S Schreiber[20] ICDAR2013 Mask R-CNN Precision 97.40
- - - - - - - - - - 2017 Recall 96.15 - - - - - - - - - - F1-Score 96.77 - - -
- - - - - - - SA Siddiqui[51] ICDAR2013 Deformable CNN Precision 99.6 - - - -
- - - - - - 2018 Recall 99.6 - - - - - - - - - - F1-Score 99.6 - - - - - - - -
- - I Kavasidis[100] ICDAR2013 Semantic Image Segmentation Precision 97.5 - -
- - - - - - - - 2019 Recall 98.1 - - - - - - - - - - F1-Score 97.8 - - - - - -
- - - - Y Huang[97] ICDAR2013 YOLO Precision 100 - 98.6 - - - 89.2 - - - -
2019 Recall 94.9 - 93.6 - - - 84.6 - - - - F1-Score 97.3 - 96.1 - - - 86.8 - -
- - SS Paliwal[96] ICDAR2013 fully convolutions Precision 96.97 - - - - - - -
- - - 2019 Recall 96.28 - - - - - - - - - - F1-Score 96.62 - - - - - - - - - -
Á Casado-García[102] ICDAR2013 Mask R-CNN Precision - - 70.0 - 70.0 - 70.0 -
47.0 - - 2020 Recall - - 97.0 - 97.0 - 97.0 - 65.0 - - F1-Score - - 81.0 -
81.0 - 81.0 - 54.0 - - D Prasad[95] ICDAR2013 Cascade mask R-CNN HRNet
Precision 100 - - - - - - - - - - 2020 Recall 100 - - - - - - - - - - F1-Score
100 - - - - - - - - - - M Li[61] ICDAR2013 Faster R-CNN Precision 96.58 - - -
- - - - - - - 2020 Recall 95.94 - - - - - - - - - - F1-Score 96.25 - - - - - -
- - - - M Agarwal [92] ICDAR2013 Cascade mask R-CNN Precision 100.0 - 100.0 -
98.7 - 94.2 - 66.0 - - 2021 Recall 100.0 - 100.0 - 98.7 - 94.2 - 66.0 - -
F1-Score 100.0 - 100.0 - 98.7 - 94.2 - 66.0 - - X Zheng[103] ICDAR2013 object
detection networks Precision 98.97 - - - - - - - - - - 2021 Recall 99.77 - - -
- - - - - - - F1-Score 99.31 - - - - - - - - - - SA Siddiqui[51] ICDAR2017
Deformable CNN Precision - - 96.5 - - - 96.7 - - - - 2018 Recall - - 97.1 - -
- 93.7 - - - - F1-Score - - 96.8 - - - 95.2 - - - - Y Huang[97] ICDAR2017 YOLO
Precision - - 97.8 - - - 97.5 - - - - 2019 Recall - - 97.2 - - - 96.8 - - - -
F1-Score - - 97.5 - - - 97.1 - - - -

Table 17: Table detection (Continue Table 16)

Approach Dataset Method IoU Year 50% 55% 60% 65% 70% 75% 80% 85% 90% 95%
50%:95% Y Li[104] ICDAR2017 Generative Adversarial Networks(GAN) Precision - -
94.4 - - - 90.3 - - - - 2019 Recall - - 94.4 - - - 90.3 - - - - F1-Score - -
94.4 - - - 90.3 - - - - N Sun [99] ICDAR2017 Faster R-CNN Precision - - - - -
- 94.3 - - - - 2019 Recall - - - - - - 95.6 - - - - F1-Score - - - - - - 94.9
- - - - Á Casado-García[102] ICDAR2017 RetinaNet Precision - - 92.0 - 92.0 -
89.0 - 79.0 - - 2020 Recall - - 87.0 - 87.0 - 84.0 - 75.0 - - F1-Score - -
89.0 - 89.0 - 86.0 - 77.0 - - M Agarwal [92] ICDAR2017 Cascade mask R-CNN
Precision - - 96.9 - - - - - - - - 2021 Recall - - 89.9 - - - - - - - -
F1-Score - - 93.4 - - - - - - - - D Prasad[95] ICDAR2019 Cascade mask R-CNN
HRNet Precision - - - - - - - - - - - 2020 Recall - - - - - - - - - - -
F1-Score - - 94.3 - 93.4 - 92.5 - 90.1 - - M Agarwal [92] ICDAR2019 Cascade
mask R-CNN Precision 98.7 - 98.0 - 97.7 - 97.1 - 93.4 - - 2021 Recall 94.6 -
93.9 - 93.6 - 93.0 - 89.5 - - F1-Score 96.6 - 95.9 - 95.6 - 95.0 - 91.5 - - X
Zheng[103] ICDAR2019 object detection networks Precision - - - - - - 96.0 -
90.0 - - 2021 Recall - - - - - - 95.0 - 89.0 - - F1-Score - - - - - - 95.5 -
95.5 - - SA Siddiqui[98] Mormot Deformable CNN Precision 84.9 - - - - - - - -
- - 2018 Recall 94.6 - - - - - - - - - - F1-Score 89.5 - - - - - - - - - - M
Agarwal [92] TableBank Cascade mask R-CNN Precision 93.4 - 99.5 - - - - - - -
- 2021 Recall 92.4 - 97.8 - - - - - - - - F1-Score 92.9 - 98.6 - - - - - - - -
P Riba [54] RVL-CDIP Graph NN Precision 15.2 - - - - - - - - - - 2019 Recall
36.5 - - - - - - - - - - F1-Score 21.5 - - - - - - - - - -

Table 18: Table Structure Recognition

Approach Dataset Method IoU Year 50% 55% 60% 65% 70% 75% 80% 85% 90% 95%
50%:95% S Schreiber[20] ICDAR2013 Fully CNN Precision 95.93 - - - - - - - - -
- 2017 Recall 87.36 - - - - - - - - - - F1-Score 91.44 - - - - - - - - - - SA
Siddiqui[51] ICDAR2013 Deformable CNN Precision 93.19 - - - - - - - - - - 2019
Recall 93.08 - - - - - - - - - - F1-Score 92.98 - - - - - - - - - - W Xue[119]
ICDAR2013 Graph NN + weights depending on distance Precision 92.6 - - - - - -
- - - - 2019 Recall 44.7 - - - - - - - - - - F1-Score 60.3 - - - - - - - - - -
SS Paliwal[96] ICDAR2013 fully CNN Precision 92.15 - - - - - - - - - - 2019
Recall 89.87 - - - - - - - - - - F1-Score 90.98 - - - - - - - - - - SA
Khan[113] ICDAR2013 Bi-directional RNN Precision 96.92 - - - - - - - - - -
2019 Recall 90.12 - - - - - - - - - - F1-Score 93.39 - - - - - - - - - - C
Tensmeyer[120] ICDAR2013 Dilated Convolutions \+ Fully CNN Precision 95.8 - -
- - - - - - - - 2019 Recall 94.6 - - - - - - - - - - F1-Score 95.2 - - - - - -
- - - - Z Chi[64] ICDAR2013 Fully CNN Precision 88.5 - - - - - - - - - - 2019
Recall 86.0 - - - - - - - - - - F1-Score 87.2 - - - - - - - - - - Á Casado-
García[102] ICDAR2013 Mask R-CNN Precision - - 70.0 - 70.0 - 70.0 - 47.0 - -
2020 Recall - - 97.0 - 97.0 - 97.0 - 65.0 - - F1-Score - - 81.0 - 81.0 - 81.0
- 54.0 - - S Raja[116] ICDAR2013 Object Detection Methods Precision 92.7 - - -
- - - - - - - 2020 Recall 91.1 - - - - - - - - - - F1-Score 91.9 - - - - - - -
- - - KA Hashmi[118] ICDAR2013 Object Detection Methods Precision 95.37 - - -
- - - - - - - 2021 Recall 95.56 - - - - - - - - - - F1-Score 95.46 - - - - - -
- - - - D Prasad[95] ICDAR2019 Object Detection Methods Precision - - - - - -
- - - - - 2020 Recall - - - - - - - - - - - F1-Score - - 43.8 - 35.4 - 19.0 -
3.6 - - Y Zou[117] ICDAR2019 Fully CNN Precision - - 18.79 - - - 1.71 - - - -
2021 Recall - - 10.07 - - - 0.92 - - - - F1-Score - - 13.11 - - - 1.19 - - - -
X Zheng[103] ICDAR2019 Object Detection Methods Precision - - - - - - - - - -
- 2021 Recall - - - - - - - - - - - F1-Score 54.8 - 38.5 - - - - - - - -

Table 19:  Open source code for most of the studies articles in Table
Detection

Article Model Year Framework Link Z Chi [64] SciTSR 2019 Pytorch
https://github.com/Academic-Hammer/SciTSR D Prasad [95] CascadeTabNet 2020
Pytorch https://github.com/DevashishPrasad/CascadeTabNet Á Casado-García [102]
- 2020 mxnet https://github.com/holms-ur/fine-tuning M Li [95] TableBank 2020
Pytorch, Detectron2 https://github.com/doc-analysis/TableBank S Raja S Raja
[116] TabStructNet 2020 tensorflow
https://github.com/sachinraja13/TabStructNet.git X Zhong [66] PubTabNet 2020 -
https://github.com/ibm-aur-nlp/PubTabNet M Agarwal [92] CDeC-Net 2021 PyTorch
https://github.com/mdv3101/CDeCNet

###  7.3 Result of different datasets

####  7.3.1 Table Detection Evaluations

To identify the tabular region in the document image and regress the
coordinates of a bounding box that has been designated as a tabular region is
the task of table detection.Tables 16,17 demonstrates how several table
detection techniques that have been thoroughly researched compare in terms of
performance. The ICDAR-2013, ICDAR-2017-POD, ICDAR-2019, and UNLV datasets are
typically used to assess the effectiveness of table detection algorithms.

Additionally described in Tables 16,17 is the Intersection Over Union (IOU)
criterion used to determine precision and recall. The most accurate results
across all relevant datasets are underlined. It is important to note that
several of the approaches did not specify the IOU threshold value but instead
compared their findings to those of other approaches that did. So, for those
procedures, we have taken into consideration the same threshold value.

####  7.3.2 Table Recognition Evaluations

Table recognition entails both segmenting the structure of tables (the task of
structural segmentation is assessed based on how precisely the rows or columns
of the tables are split) and extracting the data from the cells. We will
review the evaluations of the few previously discussed approaches in this
part.

Table 18 provides a summary of the findings. It is important to note that the
different datasets and evaluation measures used in these procedures mean that
the provided methodologies are not directly comparable to one another.

###  7.4 Open source code

Several open source frameworks for creating generic deep learning models, most
of which are written in Python, are available online, including TensorFlow,
Keras, PyTorch, and MXNet.The open-source projects for table detection and
structure recognition are summarized in Table 19. Many of the authors have
also made open-source implementations of their proposed models available.
TensorFlow and PyTorch are the most often utilized frameworks in these open
source projects.

##  8 Conclusion

In the field of document analysis, table analysis is a significant and
extensively researched problem. The challenge of interpreting tables has been
dramatically transformed and new standards have been set thanks to the use of
deep learning ideas.

As we said at the paper’s main contribution’s paragraph at the Introduction
section, we have addressed several current processes that have advanced the
process of information extraction from tables in document pictures by
implementing deep learning concepts. We have discussed methods that use deep
learning to detect, identify, and classify tables. We have also shown the most
and least well-known techniques that have been used to detect and identify
tables, respectively.

As we did at section 7, all of the datasets that are publicly accessible and
their access details have been compiled. On numerous datasets, we have
presented a thorough performance comparison of the methodologies that have
been addressed. On well-known datasets that are freely accessible to the
public, state-of-the-art algorithms for table detection have produced results
that are almost flawless. Once the tabular region has been identified, the
work of structurally segmenting tables and then recognizing them follows.

We conclude that both of these areas still have opportunities for development.

## References

  * [1] J. Hu, R. S. Kashi, D. Lopresti, G. T. Wilfong, Evaluating the performance of table processing algorithms, International Journal on Document Analysis and Recognition 4 (3) (2002) 140–153.
  * [2] P. Dollár, R. Appel, S. Belongie, P. Perona, Fast feature pyramids for object detection, IEEE transactions on pattern analysis and machine intelligence 36 (8) (2014) 1532–1545.
  * [3] J. Yang, G. Yang, Modified convolutional neural network based on dropout and the stochastic gradient descent optimizer, Algorithms 11 (3) (2018) 28.
  * [4] S. Li, W. Liu, G. Xiao, Detection of srew nut images based on deep transfer learning network, in: 2019 Chinese Automation Congress (CAC), IEEE, 2019, pp. 951–955.
  * [5] K. L. Masita, A. N. Hasan, S. Paul, Pedestrian detection using r-cnn object detector, in: 2018 IEEE Latin American Conference on Computational Intelligence (LA-CCI), IEEE, 2018, pp. 1–6.
  * [6] Z. Hu, J. Tang, Z. Wang, K. Zhang, L. Zhang, Q. Sun, Deep learning for image-based cancer detection and diagnosis- a survey, Pattern Recognition 83 (2018) 134–149.
  * [7] J. Redmon, S. Divvala, R. Girshick, A. Farhadi, You only look once: Unified, real-time object detection, in: Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 779–788.
  * [8] A. Abdallah, A. Berendeyev, I. Nuradin, D. Nurseitov, Tncr:table net detection and classification dataset, Neurocomputing 473 (2022) 79–97.  doi:10.1016/j.neucom.2021.11.101.
URL https://www.sciencedirect.com/science/article/pii/S0925231221018142

  * [9] R. Fakoor, F. Ladhak, A. Nazi, M. Huber, Using deep learning to enhance cancer diagnosis and classification, in: Proceedings of the international conference on machine learning, Vol. 28, ACM, New York, USA, 2013, pp. 3937–3949.
  * [10] S. Minaee, Z. Liu, Automatic question-answering using a deep similarity neural network, in: 2017 IEEE Global Conference on Signal and Information Processing (GlobalSIP), IEEE, 2017, pp. 923–927.
  * [11] A. Abdallah, M. Kasem, M. A. Hamada, S. Sdeek, Automated question-answer medical model based on deep learning technology, in: Proceedings of the 6th International Conference on Engineering & MIS 2020, 2020, pp. 1–8.
  * [12] A. Arpteg, B. Brinne, L. Crnkovic-Friis, J. Bosch, Software engineering challenges of deep learning, in: 2018 44th Euromicro Conference on Software Engineering and Advanced Applications (SEAA), IEEE, 2018, pp. 50–59.
  * [13] M. A. Hamada, A. Abdallah, M. Kasem, M. Abokhalil, Neural network estimation model to optimize timing and schedule of software projects, in: 2021 IEEE International Conference on Smart Information Systems and Technologies (SIST), IEEE, 2021, pp. 1–7.
  * [14] M. Mahmoud, M. Kasem, A. Abdallah, H. S. Kang, Ae-lstm: Autoencoder with lstm-based intrusion detection in iot, in: 2022 International Telecommunications Conference (ITC-Egypt), IEEE, 2022, pp. 1–6.
  * [15] W. Xu, J. Jang-Jaccard, A. Singh, Y. Wei, F. Sabrina, Improving performance of autoencoder-based network anomaly detection on nsl-kdd dataset, IEEE Access 9 (2021) 140136–140146.
  * [16] S. A. Mahmoud, I. Ahmad, W. G. Al-Khatib, M. Alshayeb, M. T. Parvez, V. Märgner, G. A. Fink, Khatt: An open arabic offline handwritten text database, Pattern Recognition 47 (3) (2014) 1096–1112.
  * [17] D. Nurseitov, K. Bostanbekov, D. Kurmankhojayev, A. Alimova, A. Abdallah, R. Tolegenov, Handwritten kazakh and russian (hkr) database for text recognition, Multimedia Tools and Applications 80 (21) (2021) 33075–33097.
  * [18] N. Toiganbayeva, M. Kasem, G. Abdimanap, K. Bostanbekov, A. Abdallah, A. Alimova, D. Nurseitov, Kohtd: Kazakh offline handwritten text dataset, Signal Processing: Image Communication 108 (2022) 116827.
  * [19] A. Fischer, C. Y. Suen, V. Frinken, K. Riesen, H. Bunke, A fast matching algorithm for graph-based handwriting recognition, in: International Workshop on Graph-Based Representations in Pattern Recognition, Springer, 2013, pp. 194–203.
  * [20] S. Schreiber, S. Agne, I. Wolf, A. Dengel, S. Ahmed, Deepdesrt: Deep learning for detection and structure recognition of tables in document images, in: 2017 14th IAPR international conference on document analysis and recognition (ICDAR), Vol. 1, IEEE, 2017, pp. 1162–1167.
  * [21] M. Traquair, E. Kara, B. Kantarci, S. Khan, Deep learning for the detection of tabular information from electronic component datasheets, in: 2019 IEEE Symposium on Computers and Communications (ISCC), IEEE, 2019, pp. 1–6.
  * [22] A. Gilani, S. R. Qasim, I. Malik, F. Shafait, Table detection using deep learning, in: 2017 14th IAPR international conference on document analysis and recognition (ICDAR), Vol. 1, IEEE, 2017, pp. 771–776.
  * [23] D. N. Tran, T. A. Tran, A. Oh, S. H. Kim, I. S. Na, Table detection from document image using vertical arrangement of text blocks, International Journal of Contents 11 (4) (2015) 77–85.
  * [24] L. Hao, L. Gao, X. Yi, Z. Tang, A table detection method for pdf documents based on convolutional neural networks, in: 2016 12th IAPR Workshop on Document Analysis Systems (DAS), IEEE, 2016, pp. 287–292.
  * [25] S. Mao, A. Rosenfeld, T. Kanungo, Document structure analysis algorithms: a literature survey, Document recognition and retrieval X 5010 (2003) 197–207.
  * [26] E. Kara, M. Traquair, M. Simsek, B. Kantarci, S. Khan, Holistic design for deep learning-based discovery of tabular structures in datasheet images, Engineering Applications of Artificial Intelligence 90 (2020) 103551.
  * [27] M. Sarkar, M. Aggarwal, A. Jain, H. Gupta, B. Krishnamurthy, Document structure extraction using prior based high resolution hierarchical semantic segmentation, in: European Conference on Computer Vision, Springer, 2020, pp. 649–666.
  * [28] R. Zanibbi, D. Blostein, J. R. Cordy, A survey of table recognition, Document Analysis and Recognition 7 (1) (2004) 1–16.
  * [29] D. W. Embley, M. Hurst, D. Lopresti, G. Nagy, Table-processing paradigms: a research survey, International Journal of Document Analysis and Recognition (IJDAR) 8 (2) (2006) 66–86.
  * [30] B. Coüasnon, A. Lemaitre, Recognition of tables and forms (2014).
  * [31] S. Khusro, A. Latif, I. Ullah, On methods and tools of table detection, extraction and annotation in pdf documents, Journal of Information Science 41 (1) (2015) 41–57.
  * [32] R. Szeliski, Computer vision: algorithms and applications, Springer Science & Business Media, 2010.
  * [33] B. C. G. Lee, Line detection in binary document scans: a case study with the international tracing service archives, in: 2017 IEEE International Conference on Big Data (Big Data), IEEE, 2017, pp. 2256–2261.
  * [34] L. Liu, W. Ouyang, X. Wang, P. Fieguth, J. Chen, X. Liu, M. Pietikäinen, Deep learning for generic object detection: A survey, International journal of computer vision 128 (2) (2020) 261–318.
  * [35] Y. Bengio, A. Courville, P. Vincent, Representation learning: A review and new perspectives, IEEE transactions on pattern analysis and machine intelligence 35 (8) (2013) 1798–1828.
  * [36] Y. LeCun, Y. Bengio, G. Hinton, et al., Deep learning. nature, 521 (7553), 436-444, Google Scholar Google Scholar Cross Ref Cross Ref (2015).
  * [37] I. Goodfellow, Y. Bengio, A. Courville, Deep learning, MIT press, 2016.
  * [38] G. Litjens, T. Kooi, B. E. Bejnordi, A. A. A. Setio, F. Ciompi, M. Ghafoorian, J. A. Van Der Laak, B. Van Ginneken, C. I. Sánchez, A survey on deep learning in medical image analysis, Medical image analysis 42 (2017) 60–88.
  * [39] X. X. Zhu, D. Tuia, L. Mou, G.-S. Xia, L. Zhang, F. Xu, F. Fraundorfer, Deep learning in remote sensing: A comprehensive review and list of resources, IEEE Geoscience and Remote Sensing Magazine 5 (4) (2017) 8–36.
  * [40] J. Gu, Z. Wang, J. Kuen, L. Ma, A. Shahroudy, B. Shuai, T. Liu, X. Wang, G. Wang, J. Cai, et al., Recent advances in convolutional neural networks, Pattern Recognition 77 (2018) 354–377.
  * [41] S. Pouyanfar, S. Sadiq, Y. Yan, H. Tian, Y. Tao, M. P. Reyes, M.-L. Shyu, S.-C. Chen, S. S. Iyengar, A survey on deep learning: Algorithms, techniques, and applications, ACM Computing Surveys (CSUR) 51 (5) (2018) 1–36.
  * [42] T. Young, D. Hazarika, S. Poria, E. Cambria, Recent trends in deep learning based natural language processing, ieee Computational intelligenCe magazine 13 (3) (2018) 55–75.
  * [43] J. Zhou, G. Cui, S. Hu, Z. Zhang, C. Yang, Z. Liu, L. Wang, C. Li, M. Sun, Graph neural networks: A review of methods and applications, AI Open 1 (2020) 57–81.
  * [44] Z. Zhang, J. Geiger, J. Pohjalainen, A. E.-D. Mousa, W. Jin, B. Schuller, Deep learning for environmentally robust speech recognition: An overview of recent developments, ACM Transactions on Intelligent Systems and Technology (TIST) 9 (5) (2018) 1–28.
  * [45] Z. Wu, S. Pan, F. Chen, G. Long, C. Zhang, S. Y. Philip, A comprehensive survey on graph neural networks, IEEE transactions on neural networks and learning systems 32 (1) (2020) 4–24.
  * [46] M. D. Zeiler, R. Fergus, Visualizing and understanding convolutional networks, in: European conference on computer vision, Springer, 2014, pp. 818–833.
  * [47] M. Oquab, L. Bottou, I. Laptev, J. Sivic, Learning and transferring mid-level image representations using convolutional neural networks, in: Proceedings of the IEEE conference on computer vision and pattern recognition, 2014, pp. 1717–1724.
  * [48] M. Göbel, T. Hassan, E. Oro, G. Orsi, Icdar 2013 table competition, in: 2013 12th International Conference on Document Analysis and Recognition, IEEE, 2013, pp. 1449–1453.
  * [49] L. Gao, X. Yi, Z. Jiang, L. Hao, Z. Tang, Icdar2017 competition on page object detection, in: 2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR), Vol. 1, IEEE, 2017, pp. 1417–1422.
  * [50] L. Gao, Y. Huang, H. Déjean, J.-L. Meunier, Q. Yan, Y. Fang, F. Kleber, E. Lang, Icdar 2019 competition on table detection and recognition (ctdar), in: 2019 International Conference on Document Analysis and Recognition (ICDAR), IEEE, 2019, pp. 1510–1515.
  * [51] S. A. Siddiqui, I. A. Fateh, S. T. R. Rizvi, A. Dengel, S. Ahmed, Deeptabstr: deep learning based table structure recognition, in: 2019 International Conference on Document Analysis and Recognition (ICDAR), IEEE, 2019, pp. 1403–1409.
  * [52] Y. Deng, D. Rosenberg, G. Mann, Challenges in end-to-end neural scientific table recognition, in: 2019 International Conference on Document Analysis and Recognition (ICDAR), IEEE, 2019, pp. 894–901.
  * [53] A. W. Harley, A. Ufkes, K. G. Derpanis, Evaluation of deep convolutional nets for document image classification and retrieval, in: 2015 13th International Conference on Document Analysis and Recognition (ICDAR), IEEE, 2015, pp. 991–995.
  * [54] P. Riba, A. Dutta, L. Goldmann, A. Fornés, O. Ramos, J. Lladós, Table detection in invoice documents by graph neural networks, in: 2019 International Conference on Document Analysis and Recognition (ICDAR), IEEE, 2019, pp. 122–127.
  * [55] A. Mondal, P. Lipps, C. Jawahar, Iiit-ar-13k: a new dataset for graphical object detection in documents, in: International Workshop on Document Analysis Systems, Springer, 2020, pp. 216–230.
  * [56] W. Seo, H. I. Koo, N. I. Cho, Junction-based table detection in camera-captured document images, International Journal on Document Analysis and Recognition (IJDAR) 18 (1) (2015) 47–57.
  * [57] A. Shahab, F. Shafait, T. Kieninger, A. Dengel, An open approach towards the benchmarking of table structure recognition systems, in: Proceedings of the 9th IAPR International Workshop on Document Analysis Systems, 2010, pp. 113–120.
  * [58] I. T. Phillips, User’s reference manual for the uw english/technical document image database iii, UW-III English/technical document image database manual (1996).
  * [59] J. Hu, R. Kashi, D. Lopresti, G. Nagy, G. Wilfong, Why table ground-truthing is hard, in: Proceedings of Sixth International Conference on Document Analysis and Recognition, IEEE, 2001, pp. 129–133.
  * [60] J. Fang, X. Tao, Z. Tang, R. Qiu, Y. Liu, Dataset, ground-truth and performance metrics for table detection evaluation, in: 2012 10th IAPR International Workshop on Document Analysis Systems, IEEE, 2012, pp. 445–449.
  * [61] M. Li, L. Cui, S. Huang, F. Wei, M. Zhou, Z. Li, Tablebank: Table benchmark for image-based table detection and recognition, in: Proceedings of the 12th Language Resources and Evaluation Conference, 2020, pp. 1918–1925.
  * [62] N. Siegel, N. Lourie, R. Power, W. Ammar, Extracting scientific figures with distantly supervised neural networks, in: Proceedings of the 18th ACM/IEEE on joint conference on digital libraries, 2018, pp. 223–232.
  * [63] B. Smock, R. Pesala, R. Abraham, W. Redmond, Pubtables-1m: Towards comprehensive table extraction from unstructured documents, arXiv preprint arXiv:2110.00061 (2021).
  * [64] Z. Chi, H. Huang, H.-D. Xu, H. Yu, W. Yin, X.-L. Mao, Complicated table structure recognition, arXiv preprint arXiv:1908.04729 (2019).
  * [65] X. Zheng, D. Burdick, L. Popa, P. Zhong, N. X. R. Wang, Global table extractor (gte): A framework for joint table identification and cell structure recognition using visual context, Winter Conference for Applications in Computer Vision (WACV) (2021).
  * [66] X. Zhong, E. ShafieiBavani, A. Jimeno Yepes, Image-based table recognition: data, model, and evaluation, in: European Conference on Computer Vision, Springer, 2020, pp. 564–580.
  * [67] A. Abdallah, A. Berendeyev, I. Nuradin, D. Nurseitov, Tncr: Table net detection and classification dataset, Neurocomputing 473 (2022) 79–97.
  * [68] A. Nassar, N. Livathinos, M. Lysak, P. Staar, Tableformer: Table structure understanding with transformers, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 4614–4623.
  * [69] P. Pyreddy, W. Croft, Tinti: A system for retrieval in text tables title2 (1997).
  * [70] Y. Wangt, I. T. Phillipst, R. Haralick, Automatic table ground truth generation and a background-analysis-based table structure extraction method, in: Proceedings of Sixth International Conference on Document Analysis and Recognition, IEEE, 2001, pp. 528–532.
  * [71] M. A. Jahan, R. G. Ragel, Locating tables in scanned documents for reconstructing and republishing, in: 7th International Conference on Information and Automation for Sustainability, IEEE, 2014, pp. 1–6.
  * [72] K. Itonori, Table structure recognition based on textblock arrangement and ruled line position, in: Proceedings of 2nd International Conference on Document Analysis and Recognition (ICDAR’93), IEEE, 1993, pp. 765–768.
  * [73] S. Chandran, R. Kasturi, Structural recognition of tabulated data, in: Proceedings of 2nd International Conference on Document Analysis and Recognition (ICDAR’93), IEEE, 1993, pp. 516–519.
  * [74] T. Hassan, R. Baumgartner, Table recognition and understanding from pdf files, in: Ninth International Conference on Document Analysis and Recognition (ICDAR 2007), Vol. 2, IEEE, 2007, pp. 1143–1147.
  * [75] E. Oro, M. Ruffolo, Trex: An approach for recognizing and extracting tables from pdf documents, in: 2009 10th International Conference on Document Analysis and Recognition, IEEE, 2009, pp. 906–910.
  * [76] A. Nurminen, Algorithmic extraction of data in tables in pdf documents, Master’s thesis (2013).
  * [77] J. Fang, P. Mitra, Z. Tang, C. L. Giles, Table header detection and classification, in: Twenty-Sixth AAAI Conference on Artificial Intelligence, 2012\.
  * [78] G. Harit, A. Bansal, Table detection in document images using header and trailer patterns, in: Proceedings of the Eighth Indian Conference on Computer Vision, Graphics and Image Processing, 2012, pp. 1–8.
  * [79] S. Tupaj, Z. Shi, C. H. Chang, H. Alam, Extracting tabular information from text files, EECS Department, Tufts University, Medford, USA 1 (1996).
  * [80] T. Kieninger, A. Dengel, The t-recs table recognition and analysis system, in: International Workshop on Document Analysis Systems, Springer, 1998, pp. 255–270.
  * [81] F. Cesarini, S. Marinai, L. Sarti, G. Soda, Trainable table location in document images, in: Object recognition supported by user interaction for service robots, Vol. 3, IEEE, 2002, pp. 236–240.
  * [82] M. Fan, D. S. Kim, Table region detection on large-scale pdf files without labeled data, CoRR, abs/1506.08891 (2015).
  * [83] Y. Wang, J. Hu, A machine learning based approach for table detection on the web, in: Proceedings of the 11th international conference on World Wide Web, 2002, pp. 242–250.
  * [84] T. Kasar, P. Barlas, S. Adam, C. Chatelain, T. Paquet, Learning to detect tables in scanned document images using line information, in: 2013 12th International Conference on Document Analysis and Recognition, IEEE, 2013, pp. 1185–1189.
  * [85] A. C. e Silva, Learning rich hidden markov models in document analysis: Table location, in: 2009 10th International Conference on Document Analysis and Recognition, IEEE, 2009, pp. 843–847.
  * [86] S. Klampfl, K. Jack, R. Kern, A comparison of two unsupervised table recognition methods from digital scientific articles, D-Lib Magazine 20 (11) (2014) 7.
  * [87] L. O’Gorman, The document spectrum for page layout analysis, IEEE Transactions on pattern analysis and machine intelligence 15 (11) (1993) 1162–1173.
  * [88] F. Shafait, R. Smith, Table detection in heterogeneous documents, in: Proceedings of the 9th IAPR International Workshop on Document Analysis Systems, 2010, pp. 65–72.
  * [89] D. He, S. Cohen, B. Price, D. Kifer, C. L. Giles, Multi-scale multi-task fcn for semantic page segmentation and table detection, in: 2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR), Vol. 1, IEEE, 2017, pp. 254–261.
  * [90] S. Arif, F. Shafait, Table detection in document images using foreground and background features, in: 2018 Digital Image Computing: Techniques and Applications (DICTA), IEEE, 2018, pp. 1–8.
  * [91] M. M. Reza, S. S. Bukhari, M. Jenckel, A. Dengel, Table localization and segmentation using gan and cnn, in: 2019 International Conference on Document Analysis and Recognition Workshops (ICDARW), Vol. 5, IEEE, 2019, pp. 152–157.
  * [92] M. Agarwal, A. Mondal, C. Jawahar, Cdec-net: Composite deformable cascade network for table detection in document images, in: 2020 25th International Conference on Pattern Recognition (ICPR), IEEE, 2021, pp. 9491–9498.
  * [93] N. Carion, F. Massa, G. Synnaeve, N. Usunier, A. Kirillov, S. Zagoruyko, End-to-end object detection with transformers, in: European conference on computer vision, Springer, 2020, pp. 213–229.
  * [94] J. Li, Y. Xu, T. Lv, L. Cui, C. Zhang, F. Wei, Dit: Self-supervised pre-training for document image transformer, arXiv preprint arXiv:2203.02378 (2022).
  * [95] D. Prasad, A. Gadpal, K. Kapadni, M. Visave, K. Sultanpure, Cascadetabnet: An approach for end to end table detection and structure recognition from image-based documents, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops, 2020, pp. 572–573.
  * [96] S. S. Paliwal, D. Vishwanath, R. Rahul, M. Sharma, L. Vig, Tablenet: Deep learning model for end-to-end table detection and tabular data extraction from scanned document images, in: 2019 International Conference on Document Analysis and Recognition (ICDAR), IEEE, 2019, pp. 128–133.
  * [97] Y. Huang, Q. Yan, Y. Li, Y. Chen, X. Wang, L. Gao, Z. Tang, A yolo-based table detection method, in: 2019 International Conference on Document Analysis and Recognition (ICDAR), IEEE, 2019, pp. 813–818.
  * [98] S. A. Siddiqui, M. I. Malik, S. Agne, A. Dengel, S. Ahmed, Decnt: Deep deformable cnn for table detection, IEEE access 6 (2018) 74151–74161.
  * [99] N. Sun, Y. Zhu, X. Hu, Faster r-cnn based table detection combining corner locating, in: 2019 International Conference on Document Analysis and Recognition (ICDAR), IEEE, 2019, pp. 1314–1319.
  * [100] I. Kavasidis, C. Pino, S. Palazzo, F. Rundo, D. Giordano, P. Messina, C. Spampinato, A saliency-based convolutional neural network for table and chart detection in digitized documents, in: International conference on image analysis and processing, Springer, 2019, pp. 292–302.
  * [101] M. Holeček, A. Hoskovec, P. Baudiš, P. Klinger, Table understanding in structured documents, in: 2019 International Conference on Document Analysis and Recognition Workshops (ICDARW), Vol. 5, IEEE, 2019, pp. 158–164.
  * [102] Á. Casado-García, C. Domínguez, J. Heras, E. Mata, V. Pascual, The benefits of close-domain fine-tuning for table detection in document images, in: International workshop on document analysis systems, Springer, 2020, pp. 199–215.
  * [103] X. Zheng, D. Burdick, L. Popa, X. Zhong, N. X. R. Wang, Global table extractor (gte): A framework for joint table identification and cell structure recognition using visual context, in: Proceedings of the IEEE/CVF winter conference on applications of computer vision, 2021, pp. 697–706.
  * [104] Y. Li, L. Gao, Z. Tang, Q. Yan, Y. Huang, A gan-based feature generator for table detection, in: 2019 International Conference on Document Analysis and Recognition (ICDAR), IEEE, 2019, pp. 763–768.
  * [105] D.-D. Nguyen, Tablesegnet: a fully convolutional network for table detection and segmentation in document images, International Journal on Document Analysis and Recognition (IJDAR) 25 (1) (2022) 1–14.
  * [106] D. Zhang, R. Mao, R. Guo, Y. Jiang, J. Zhu, Yolo-table: disclosure document table detection with involution, International Journal on Document Analysis and Recognition (IJDAR) (2022) 1–14.
  * [107] A. Zucker, Y. Belkada, H. Vu, V. N. Nguyen, Clusti: Clustering method for table structure recognition in scanned images, Mobile Networks and Applications 26 (4) (2021) 1765–1776.
  * [108] Z. Zhang, J. Zhang, J. Du, F. Wang, Split, embed and merge: An accurate table structure recognizer, Pattern Recognition 126 (2022) 108565.
  * [109] M. Namysl, A. M. Esser, S. Behnke, J. Köhler, Flexible table recognition and semantic interpretation system., in: VISIGRAPP (4: VISAPP), 2022, pp. 27–37.
  * [110] E. Koci, M. Thiele, W. Lehner, O. Romero, Table recognition in spreadsheets via a graph representation, in: 2018 13th IAPR International Workshop on Document Analysis Systems (DAS), IEEE, 2018, pp. 139–144.
  * [111] E. Koci, M. Thiele, O. Romero, W. Lehner, A genetic-based search for adaptive table recognition in spreadsheets, in: 2019 International Conference on Document Analysis and Recognition (ICDAR), IEEE, 2019, pp. 1274–1279.
  * [112] S. A. Siddiqui, P. I. Khan, A. Dengel, S. Ahmed, Rethinking semantic segmentation for table structure recognition in documents, in: 2019 International Conference on Document Analysis and Recognition (ICDAR), IEEE, 2019, pp. 1397–1402.
  * [113] S. A. Khan, S. M. D. Khalid, M. A. Shahzad, F. Shafait, Table structure extraction with bi-directional gated recurrent unit networks, in: 2019 International Conference on Document Analysis and Recognition (ICDAR), IEEE, 2019, pp. 1366–1371.
  * [114] S. F. Rashid, A. Akmal, M. Adnan, A. A. Aslam, A. Dengel, Table recognition in heterogeneous documents using machine learning, in: 2017 14th IAPR International conference on document analysis and recognition (ICDAR), Vol. 1, IEEE, 2017, pp. 777–782.
  * [115] S. R. Qasim, H. Mahmood, F. Shafait, Rethinking table recognition using graph neural networks, in: 2019 International Conference on Document Analysis and Recognition (ICDAR), IEEE, 2019, pp. 142–147.
  * [116] S. Raja, A. Mondal, C. Jawahar, Table structure recognition using top-down and bottom-up cues, in: European Conference on Computer Vision, Springer, 2020, pp. 70–86.
  * [117] Y. Zou, J. Ma, A deep semantic segmentation model for image-based table structure recognition, in: 2020 15th IEEE International Conference on Signal Processing (ICSP), Vol. 1, IEEE, 2020, pp. 274–280.
  * [118] K. A. Hashmi, D. Stricker, M. Liwicki, M. N. Afzal, M. Z. Afzal, Guided table structure recognition through anchor optimization, IEEE Access 9 (2021) 113521–113534.
  * [119] W. Xue, Q. Li, D. Tao, Res2tim: Reconstruct syntactic structures from table images, in: 2019 International Conference on Document Analysis and Recognition (ICDAR), IEEE, 2019, pp. 749–755.
  * [120] C. Tensmeyer, V. I. Morariu, B. Price, S. Cohen, T. Martinez, Deep splitting and merging for table structure decomposition, in: 2019 International Conference on Document Analysis and Recognition (ICDAR), IEEE, 2019, pp. 114–121.
  * [121] S. Raja, A. Mondal, C. Jawahar, Visual understanding of complex table structures from document images, in: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2022, pp. 2299–2308.
  * [122] X. Shen, L. Kong, Y. Bao, Y. Zhou, W. Liu, Rcanet: A rows and columns aggregated network for table structure recognition, in: 2022 3rd Information Communication Technologies Conference (ICTC), IEEE, 2022, pp. 112–116.
  * [123] C. Ma, W. Lin, L. Sun, Q. Huo, Robust table detection and structure recognition from heterogeneous document images, arXiv preprint arXiv:2203.09056 (2022).
  * [124] B. Xiao, M. Simsek, B. Kantarci, A. A. Alkheir, Table structure recognition with conditional attention, arXiv preprint arXiv:2203.03819 (2022).
  * [125] A. Jain, S. Paliwal, M. Sharma, L. Vig, Tsr-dsaw: Table structure recognition via deep spatial association of words, arXiv preprint arXiv:2203.06873 (2022).
  * [126] H. Li, L. Zeng, W. Zhang, J. Zhang, J. Fan, M. Zhang, A two-phase approach for recognizing tables with complex structures, in: International Conference on Database Systems for Advanced Applications, Springer, 2022, pp. 587–595.
  * [127] S. Ren, K. He, R. Girshick, J. Sun, Faster r-cnn: Towards real-time object detection with region proposal networks, arXiv preprint arXiv:1506.01497 (2015).
  * [128] K. He, G. Gkioxari, P. Dollar, R. Girshick, Mask r-cnn, 2017 IEEE International Conference on Computer Vision (ICCV) (Oct 2017).
  * [129] K. Sun, B. Xiao, D. Liu, J. Wang, Deep high-resolution representation learning for human pose estimation, in: CVPR, 2019.
  * [130] K. Sun, Y. Zhao, B. Jiang, T. Cheng, B. Xiao, D. Liu, Y. Mu, X. Wang, W. Liu, J. Wang, High-resolution representations for labeling pixels and regions, CoRR abs/1904.04514 (2019).
  * [131] A. Newell, K. Yang, J. Deng, Stacked hourglass networks for human pose estimation, in: European conference on computer vision, Springer, 2016, pp. 483–499.
  * [132] E. Insafutdinov, L. Pishchulin, B. Andres, M. Andriluka, B. Schiele, Deepercut: A deeper, stronger, and faster multi-person pose estimation model, in: European Conference on Computer Vision, Springer, 2016, pp. 34–50.
  * [133] B. Xiao, H. Wu, Y. Wei, Simple baselines for human pose estimation and tracking, in: Proceedings of the European conference on computer vision (ECCV), 2018, pp. 466–481.