# instance segmentation and teeth classification in panoramic x-rays

Devichand Budagam
Department of Computer Science and Engineering
Indian Institute of Technology Kharagpur
Kharagpur, India
<EMAIL_ADDRESS>
& Ayush Kumar
Department of Humanities and Social Sciences
Indian Institute of Technology Kharagpur
Kharagpur, India
<EMAIL_ADDRESS>
& Sayan Ghosh
Department of Chemical Engineering
Indian Institute of Technology Kharagpur
Kharagpur, India
<EMAIL_ADDRESS>
& Anuj Shrivastav
Department of Architecture and Regional Planning
Indian Institute of Technology Kharagpur
Kharagpur, India
<EMAIL_ADDRESS>
& Azamat Zhanatuly Imanbayev
School of IT and Engineering
Kazakh-British Technical University
Almaty, Kazakhstan
<EMAIL_ADDRESS>
& Iskander Rafailovich Akhmetov
School of IT and Engineering
Kazakh-British Technical University
Almaty, Kazakhstan
<EMAIL_ADDRESS>& Dmitrii Kaplun
Artificial Intelligence Research Institute
China University of Mining and Technology
Xuzhou, China
Department of Automation and Control Processes
St. Petersburg Electrotechnical University ”LETI”
St. Petersburg, Russia
<EMAIL_ADDRESS>
& Sergey Antonov
Department of Automation and Control Processes
St. Petersburg Electrotechnical University ”LETI”
St. Petersburg, Russia
<EMAIL_ADDRESS>
& Artem Rychenkov
Department of Information Systems
St. Petersburg Electrotechnical University ”LETI”
St. Petersburg, Russia
<EMAIL_ADDRESS>
& Gleb Cyganov
Faculty of Digital Transformation
ITMO University
St. Petersburg, Russia
<EMAIL_ADDRESS>
& Aleksandr Sinitca
Centre for Digital Telecommunication Technologies
St. Petersburg Electrotechnical University ”LETI”
St. Petersburg, Russia
<EMAIL_ADDRESS>

###### Abstract

Teeth segmentation and recognition are critical in various dental applications
and dental diagnosis. Automatic and accurate segmentation approaches have been
made possible by integrating deep learning models. Although teeth segmentation
has been studied in the past, only some techniques were able to effectively
classify and segment teeth simultaneously. This article offers a pipeline of
two deep learning models, U-Net and YOLOv8, which results in BB-UNet, a new
architecture for the classification and segmentation of teeth on panoramic
X-rays that is efficient and reliable. We have improved the quality and
reliability of teeth segmentation by utilising the YOLOv8 and U-Net
capabilities. The proposed networks have been evaluated using the mean average
precision (mAP) and dice coefficient for YOLOv8 and BB-UNet, respectively. We
have achieved a 3% increase in mAP score for teeth classification compared to
existing methods, and a 10-15% increase in dice coefficient for teeth
segmentation compared to U-Net across different categories of teeth. A new
Dental dataset was created based on UFBA-UESC dataset with Bounding-Box and
Polygon annotations of 425 dental panoramic X-rays. The findings of this
research pave the way for a wider adoption of object detection models in the
field of dental diagnosis.

_Keywords_ Teeth Segmentation $\cdot$ Teeth Classification $\cdot$ Panoramic
X-Rays $\cdot$ YOLOv8 $\cdot$ U-Net $\cdot$ BB-UNet

##  1 Introduction

The increasing demand for dental care and dentists is driven by population
growth, increasing life expectancy, and a heightened focus on oral health.
Advanced technologies are necessary to improve diagnostic quality, treatment
planning, and patient care. In this evolving dental field, deep learning
architectures have emerged as a promising solution [1, 2, 3, 4] despite
certification difficulties.

The FDI World Dental Federation notation [5] shown in Fig.1 is a dental
notation system used to identify and label teeth uniquely. It is a standard
method used worldwide to communicate tooth information in a consistent and
universally understood way. The FDI notation system assigns a two-digit number
to each tooth in the mouth and is widely used by dental professionals.

Teeth segmentation and classification, which involves accurately delineating
individual teeth, consists of classifying each image pixel into an object of
interest and numbering the teeth from dental images, which is crucial for
various dental applications. The traditional manual method of segmenting and
annotating teeth can be quite labor-intensive and time-consuming; unclear
images can make it difficult for a specialist to evaluate them. To this end,
automated teeth segmentation using deep learning techniques has become an
effective alternative due to its success in various complex computer vision
tasks. However, due to the high variability, low contrast, and high amplitude
noise in panoramic X-rays, automated teeth segmentation and classification
pose many challenges in recognizing and locating each tooth and classifying it
precisely. They can even mislead in the formulation of the diagnosis.

So far, most of the research on teeth segmentation is based on several
unsupervised pixel-wise segmentation approaches [6] mainly studied using
intraoral images (periapical X-rays or bitewing X-rays). The article by G.
Jader et al. [6] experimented with small data sets for most of the methods
discussed and evaluated them with very few changes to the dataset, using a
dataset of 1500 panoramic X-ray images called the UFBA-UESC dental image
dataset. They concluded that these segmentation methods failed to effectively
isolate the teeth from neighboring teeth in the mouth. Very few authors
performed experiments on panoramic X-ray images, threshold-based [7, 8, 9],
region-based [10], cluster-based [11], boundary-based [12, 13, 14] and one-
class segmentation [15, 16, 17]. In the paper on deep instance segmentation of
teeth on panoramic radiographs [18] the authors investigated teeth
segmentation on the UFBA-UESC dental images dataset using a Mask R-CNN [19]
for instance segmentation task, but all teeth were classified into a single
category ignoring independent tooth recognition. Silva et al.[20] proposed
different deep learning architectures PANet, HTC, ResNeSt, and Mask R-CNN, in
[21], and they used Mask R-CNN with different segmentation heads pointRend
[22] and FCN [23] and obtained excellent results.

Figure 1: FDI Teeth Numbering Notation

This paper proposes a different method to perform teeth classification and
instance segmentation by exploiting spatial prior information on U-Net [24]
using a one-stage object detection model, YOLO [25] rather than performing the
task with a two-stage objection detection model such as Mask R-CNN as in
previously mentioned methods. U-Net architecture can accurately segment the
image and locate tooth positions, enabling us to efficiently segment the
teeth. Integration of prior knowledge, such as the location or shape of the
objects, can improve the performance of CNNs. Integration of prior knowledge
in CNNs has provided good results in the paper Zotti et al. [26]. Therefore,
we impose the teeth location information with the help of a boundary box prior
to the skip connections of U-Net [27].

##  2 Materials and Method

###  2.1 Dataset Construction

In dental image analysis, where prediction accuracy is paramount, our
preparatory work focuses on carefully selecting a representative dataset and
processing it for subsequent model training. Our foundation was the UFBA-UESC
Dental Images Dataset introduced by G. Jader et al.[6], an extensive
collection of anonymized panoramic X-ray dental images of high variability. It
comprises 1500 images of dimensions 512$\times$512$\times$3, classified into
10 distinct categories. These categories represented different types of dental
cases, with standard 32 teeth, with dental appliances, and with dental
restorations. The dataset also uses images where the number of teeth is less
than 32 if they are extracted and more if the jaw has an abnormal mutation.
This diversity mirrored the real-world variations in dental scans due to
factors such as dental anomalies or missing teeth. The structure of the open
data set UFBA-UESC is presented in table 1.

Table 1: Description of the UFBA-UESC dataset Category | 32 Teeth | Restoration | Dental appliance | Images | Used Images
---|---|---|---|---|---
1 | $\checkmark$ | $\checkmark$ | $\checkmark$ | 73 | 24
2 | $\checkmark$ | $\checkmark$ |  | 220 | 72
3 | $\checkmark$ |  | $\checkmark$ | 45 | 15
4 | $\checkmark$ |  |  | 140 | 32
5 |  | Images containing dental implant |  | 120 | 37
6 |  | Images containing more than 32 teeth |  | 170 | 30
7 |  | $\checkmark$ | $\checkmark$ | 115 | 33
8 |  | $\checkmark$ |  | 457 | 140
9 |  |  | $\checkmark$ | 45 | 7
10 |  |  |  | 115 | 35
Total |  |  |  | 1500 | 425

Due to the lack of image labels necessary for training models in the presented
dataset, we initially labelled 425 images from the set. We collected two sets
of basic annotations, for instance segmentation and object detection
respectively. We selected these 425 images randomly from all categories and
maintained same distribution of images across all categories as in the
original dataset. For manual labelling, we chose the semi-automated annotation
tool Roboflow [28] for bounding box annotations needed for the objection
detection task. We also used another annotation tool, Apeer [29], to help us
create individual segmentation masks for each of the 32 teeth in the images in
the dataset. These binary masks provided additional information by focusing on
the fine contours and boundaries of the teeth. We converted the resulting
segmented polygons into binary maps of size 512$\times$512$\times$32\. This
comprehensive approach to annotation was pivotal in ensuring our model’s
success in dental image analysis. An example of annotations is shown in Fig.
2.

Figure 2: (a) Annotated polygon mask. (b) Annotated bounding Box scan. (c)
Binary mask of polygon-based annotation

From Fig. 3, we can see that the dataset is not biased towards any particular
class of teeth and maintains variability across all classes of teeth and all
categories of panoramic X-ray images, which makes the dataset suitable for
training teeth detection or segmentation models. It is possible to use this
data set extensively for teeth segmentation and detection, as it is one of the
largest publicly available datasets.

(a)

(b)

Figure 3: (a) Class distribution among 32 teeth (b) Count of each tooth

###  2.2 Model Architecture

####  2.2.1  YOLO Architecture

At the moment, many different models that solve the object detection problem
have been developed. However, the two most commonly used approaches for object
detection tasks include single-stage and two-stage neural network
architectures. One-stage approaches have an advantage over two-stage
approaches in that the former produces outcomes considerably more quickly and
efficiently. YOLO is one of the most popular single-stage models to solve the
problem of object detection tasks on images. Recently, many different versions
of the YOLO model have been released by various developers. Such models
include YOLO, YOLOv4, YOLOv5, YOLOv7, YOLOv8, and others.

Figure 4:  YOLOv8 Architecture [30]

YOLOv8 is the latest version of the YOLO model. The architecture of this model
is shown in Fig.4 and includes new enhancements that provide superior
detection accuracy while maintaining high speed and efficiency. The YOLOv8
backbone network architecture is based on the previous version of YOLOv5. The
critical difference between this model and the old version is that the authors
of YOLOv8 abandoned predefined anchors, simplifying the model’s architecture
and eliminating the need for additional tuning of hyperparameters. The YOLOv8
model also includes several essential modifications.

  * •

CSPDarknet53 Feature Extractor: YOLOv8 uses CSPDarknet53, a variant of the
Darknet architecture, as the feature extractor. This component comprises
convolutional layers, batch normalization, and SiLU activation functions. The
notable difference is that YOLOv8 replaces the original 6x6 convolutional
layer with a 3x3 convolutional layer to improve feature extraction.

  * •

Module C2f: YOLOv8 introduces the C2f module to combine high-level features
with contextual information efficiently. This is achieved by concatenating
bottleneck block outputs, consisting of two 3x3 convolutions with residual
connections. This architectural change aims to improve the presentation of
features. In the previous version of YOLOv5, instead of C2f, the C3 block,
which contains one more convolution layer than C2f module, was used.
Considering that C2f is used eight times throughout the entire architecture,
this change is a significant advantage in choosing a new architecture.

  * •

Detection head: YOLOv8 uses an anchor-free detection strategy, eliminating the
need for predefined anchor fields and directly predicting the centres of
objects. A significant improvement to the model detection head is the use of a
different activation function. Objectness estimation in the output layer uses
a sigmoid activation function representing the probability of an object being
present in the bounding box. YOLOv8 uses a softmax function for class
probabilities that indicates the probability of an object belonging to each
class. To optimize the model, YOLOv8 uses CIoU (complete intersection through
a union) and DFL (dynamic focal loss) loss functions for bounding box
regression and binary cross-entropy for classification. These loss functions
effectively improve object detection, especially for small objects.

Thanks to all these enhancements, the model performed better than earlier
object detection models, which makes it appropriate for classifying teeth
which are difficult to classify because of numerous teeth and minute details
found in panoramic X-rays.

####  2.2.2 BB-UNet Architecture

Our methodology is built upon the U-Net framework, a highly regarded model for
the semantic segmentation of medical images. U-Net is selected for its
efficiency in delivering robust results, even with limited training data. The
architecture includes encoder layers, which extract contextual information
from images, and decoder layers, focused on identifying patterns and reverting
image maps to their original dimensions. We use skip connections to merge both
contextual and positional information effectively. Despite U-Net’s proficiency
in integrating global features, it struggles to incorporate specific location
data. To overcome this, we have integrated location-based constraints into the
learning process, specifically using binary maps of bounding boxes, as per
methods introduced in [27], previously in articles [26] and [31] have
implemented constrained neural networks for medical image analysis for
achieving better performance. This integration enhances the model’s ability to
consider location information and improves performance.

Incorporating prior knowledge via bounding boxes is achieved through the use
of BB-Convolution layers added to the skip connections. A BB-Convolution layer
consists of 2D max-pooling, followed by dual convolution layers, and
culminates with a sigmoid activation layer. Different teeth are identified
using bounding boxes, fed in through a 32 multi-channel binary map into the
BB-Convolution layer. This process produces a feature map that assists the
network in more accurately pinpointing the location of each tooth. These BB-
Convolution layers are incorporated into each skip connection, as demonstrated
in Fig.5. Our model is structured into four phases, each containing encoder
and decoder elements. The encoder layers gather contextual information and
features within the images. In contrast, the decoder layers focus on pattern
identification and restoring the image maps to their initial size. Within the
encoder phases, two convolution blocks are followed by max-pooling layers
(with a stride of 2) to decrease feature resolution. The decoder phase
includes a transposed convolution block followed by two convolution blocks. At
every stage, the output from the BB-Convolution layer is element-wise
multiplied with the encoder’s output features before being concatenated with
the output from the decoder’s transposed convolution layers.

Figure 5: BB-UNet Architecture

The final layer of our architecture consists of a $1\times 1$ convolution,
ending with a softmax activation that generates a pixel-by-pixel probability
map.

We have decided to use a regularized variant of Dice loss to optimize the
parameters of the BB-UNet during the model’s training. Dice loss is a widely
used metric in medical segmentation and computer vision tasks for calculating
the similarity between two images and is defined as:

$Loss=\frac{1}{N}\sum_{n=0}^{N}\frac{2\sum_{i=1}^{M}P_{n}(i)\hat{P_{n}}(i)}{\sum_{i=1}^{M}P_{n}(i)^{2}+\sum_{i=1}^{M}\hat{P_{n}}(i)^{2}}+\lambda\frac{1}{N}\sum_{n=0}^{N}\sum_{i=0}^{M}(P_{n}(i)-\hat{P_{n}}(i))^{2}$
(1)

Where N is the number of class labels, and M is the number of pixels in each
channel of the image.$P_{n}(i)$ and $\hat{P_{n}}(i)$ are the pixel values in
the predicted map and ground truth label respectively.Here $\lambda$, being a
regularisation constant.The latter component of the loss function plays a
significant role in preventing overfitting along with the spatial dropout
layers introduced in the BB-UNet architecture.

###  2.3 Evaluation Metrics

We have used several metrics to evaluate the quality of the proposed method.
We calculate accuracy, recall, precision, and mAP:

  * •

_Accuracy_ calculates the ratio of correctly predicted instances to the total
number of instances.

  * •

_Recall_ measures the ability of the model to capture and correctly identify
all relevant instances of a particular class.

  * •

_Precision_ measures the accuracy of the positive predictions made by the
model. It indicates the proportion of true positives among all instances
predicted as positive.

  * •

_mAP_ calculates the precision-recall area’s average under the curve for
multiple classes at different confidence thresholds, providing a comprehensive
evaluation of model performance.

  * •

_AP50_ calculates the precision-recall area’s average under the curve for
multiple classes at a confidence thresholds of 0.5.

These metrics can be formulated in terms of the confusion matrix. If we have
$N$ classes and a confusion matrix $M$, then Equations 2, 3, 4 and 5 can be
used.

$Accuracy=\frac{\sum_{i=1}^{N}M_{ii}}{\sum_{i=1}^{N}\sum_{j=1}^{N}M_{ij}}$ (2)

$Precision_{i}=\frac{M_{ii}}{\sum_{j=1}^{N}M_{ji}}$ (3)

$Recall_{i}=\frac{M_{ii}}{\sum_{j=1}^{N}M_{ij}}$ (4)

$mAP=\frac{1}{N}\sum_{i=1}^{N}AP_{i}=\frac{1}{N}\sum_{i=1}^{N}\sum_{j=1}^{n}(Recall_{j}-Recall_{j-1})Precision_{j}$
(5)

Where $M_{ij}$ is the corresponding element of a confusion matrix, $M$ and n
is the number of thresholds. mAP is used as the primary metric for teeth
classification. In contrast, these metrics cannot provide a robust assessment
of instance segmentation of teeth; the dice coefficient is used as the primary
metric for instance segmentation of teeth. The dice coefficient is a measure
of the similarity between two sets, and it is used to quantify the agreement
between the predicted segmentation masks and the ground truth masks. The dice
coefficient is defined as:

$\text{Dice
Coefficient}=\frac{1}{N}\sum_{n=0}^{N}\frac{2\sum_{i=1}^{M}P_{n}(i)\hat{P_{n}}(i)}{\sum_{i=1}^{M}P_{n}(i)^{2}+\sum_{i=1}^{M}\hat{P_{n}}(i)^{2}}$
(6)

where N is the number of class labels and M is number of pixels in each
channel of the image.$P_{n}(i)$ and $\hat{P_{n}}(i)$ are the pixel values in
predicted map and the ground truth label, respectively.

Figure 6: Pipeline of the method

###  2.4 Model Pipeline and Training

We propose a pipeline comprising a two-step process of YOLOv8 and BB-UNet
architectures as shown in Fig.6. The first step involves training the YOLOv8
model to classify the teeth, locate the teeth, and extract the bounding boxes
of each located tooth. The latter step is training the BB-UNet, for instance
segmentation of teeth.

For training YOLOv8, the images were resized to $640\times 640$, and histogram
equalization was applied to improve the contrast of the images. From the
originally built dataset, 894 training images and 128 validation images were
created using two augmentation approaches: random cropping with a range of 0%
to 20% and brightness adjustment with a range of 0% to 10%. Throughout the
training, we processed the dataset in batches, each containing ten images. To
avoid overfitting, we applied dropout with a rate of 0.6 as a regularization
technique. Additionally, to ensure efficient learning during the training, SGD
optimizer with a learning rate of 0.005 was used. Fig.7 shows the variation in
training losses, that is, box loss, class loss, DFL loss and mAP score on
validation dataset varies over a span of 30 epochs for YOLOv8 training.

(a)

(b)

Figure 7: (a) Training loss curves for YOLOV8 (b) AP50 and mAP curves for
YOLOv8

For training BB-UNet, the bounding box information from YOLOv8 was used to
generate $512\times 512\times 32$ bounding box binary maps which served as
inputs to BB-Convolution layers. CLAHE [32](Contrast Limited Adaptive
Histogram Equalisation) with a contrast limit of 0.02 was applied to enhance
the details of the image. Horizontal and vertical flipping were used as data
augmentation techniques, finally generating training data of 340 images and
test data of 85 images. The model was trained with a batch size of 2 tensors
of size $512\times 512\times 35$, where 32 channels correspond to binary maps
of bounding boxes and the remaining three channels correspond to the original
image which served as inputs to the BB-UNet. Fig.8 shows the variation in
training loss, validation loss, and dice coefficient on validation dataset
over 60 epochs for BB-UNet training.

(a)

(b)

Figure 8: (a) Training and Validation loss curves for BB-UNet (b) Dice
coefficient curves for BB-UNet

A dropout rate of 0.12 and regularisation weight, $\lambda$ of 0.1, were used
to prevent overfitting, a momentum of 0.99, and a learning rate of 0.0003 with
Adam optimizer were used for the training process. The learning rate was
halved if validation loss did not improve over a period of 5 epochs. A 4-fold
cross-validation method is used to train the model on the training data, and
the average dice coefficient over the 4-fold cross-validation is $0.82\pm
0.006$. A Nvidia A100 GPU with a 12-core CPU and 120GB RAM was used for all
training and evaluation processes.

##  3 Experiments and Results

###  3.1 Quantitative Analysis

The YOLOv8 model was evaluated on the dataset containing 425 images and 11591
instances of teeth and achieved a precision of 94.3, a recall of 92.3, and a
mAP of 72.9 and a mean average precision at 50 (AP50) of 94. 6% in all classes
of teeth. The results indicated a striking balance between precision and
recall and achieved an excellent mAP. These results suggest the effectiveness
of the model in accurately localising and recognising teeth in the dataset.
YOLOv8 performed better than Mask R-CNN, having a mAP of 70.5 [20] and other
supervised and unsupervised methods[6] and provided more accurate results.
This demonstrates that a single-stage object detection algorithm can
outperform a two-stage object detection method like Mask R-CNN. Due to the
intricate intersection of the box region with other teeth in the image, the
teeth belonging to the incisors have the lowest mAP score of all the tooth
classes, approximately 60.00. YOLOv8 was able to localize the molar teeth
precisely with a mAP of nearly 80.00 even though they have quite complex
shapes, which shows the effectiveness of YOLOv8.

The confusion matrix in Fig.9 for YOLOv8 predictions for a confidence
threshold of 0.5 and IoU threshold of 0.5, where the last row shows the number
of instances of tooth being unclassified indicates that YOLOv8 was able to
accurately recognize the instances of every tooth with very few
misclassifications but was unable to recognize some instances of every tooth
above the confidence threshold; this indicates lack of knowledge of instances
of the tooth in some images of the dataset.

For evaluation of BB-UNet and U-Net, two test datasets were prepared from the
original dataset; Test Dataset 1: contains 85 images belonging to all
categories, Test Dataset 2: contains 72 images belonging to all categories
except cateogory-5 and category-6; i.e, images containing dental implants and
images having more than 32 teeth.Fig.10(a) shows that, when evaluated on Test
Dataset 1 for all tooth kinds, BB-UNet performed better than U-Net and
consistently achieved a dice coefficient of nearly 0.84 for all kinds of
teeth, with U-Net having an overall dice coefficient of 0.70 and BB-UNet with
an overall dice coefficient of 0.84. Because molars and premolars typically
have complex geometries with more than two roots, U-Net had trouble precisely
locating them. However, BB-UNet was able to do so because prior knowledge was
incorporated into the model architecture. For all tooth kinds, the dice
coefficient has increased by 10% to 15% when compared to U-Net, indicating the
significance of prior knowledge in the instance segmentation of teeth. The
primary limitation of BB-UNet was its incapacity to identify teeth in images
with dental implants and images with more than 32 teeth because of the
intricacy of the images and the presence of other dental instruments degraded
the quality of prior knowledge and BB-UNet’s performance. Fig.10(b) compares
BB-UNet’s performance on Test Dataset 1 and Test Dataset 2, having an overall
dice coefficient of 0.84 and 0.86, respectively. This indicates that BB-UNet’s
performance was poor in locating teeth in category-5 and category-6 images,
resulting in scattered segmentation masks.

Figure 9: Confusion matrix for YOLOv8 predictions

(a)

(b)

Figure 10: (a)A comparison between BB-UNet and U-Net’s segmentation results(b)
Segmentation outcomes of Test dataset 1 and Test dataset 2 are compared

###  3.2 Qualitative Analysis

The YOLOv8 model’s performance analysis demonstrates accurate predictions for
images lacking fillings and dental implants. However, challenges arise when
encountering additional teeth, fillings, or implants. Notably, the scarcity of
annotated data encompassing teeth with fillings or implants within the dataset
contributes to this issue. Within Figure 11(a), Figure 11(b), the model
presents precise predictions; conversely, Figure 11(c), Figure 11(d) exhibits
missing tooth predictions, notably in instances of fillings, implants, or
degraded teeth, posing difficulty for accurate detection.

(a)

(b)

(c)

(d)

Figure 11: Classification results and corresponding heatmaps of YOLOv8

Comparatively, the BB-UNet segmentation method notably outperforms the
standard U-Net segmentation regarding quality. Figure 12 highlights the
superior pixel-level classification of teeth achieved by BB-UNet, especially
in complex tooth structures like molars and premolars, where U-Net encounters
challenges. Nonetheless, in scenarios where bounding box predictions are
absent, as illustrated in Figure 13, BB-UNet’s performance aligns with U-Net
due to the absence of prior knowledge.

The disparity in performance between models underscores the impact of dataset
diversity on model proficiency, particularly in scenarios involving intricate
tooth conditions like fillings, implants, or varying tooth degradation levels.
Moreover, the dependence of BB-UNet’s performance on bounding box predictions
emphasizes the importance of comprehensive data annotation for robust
segmentation outcomes.

(a) Panoramic X-Ray

(b) Ground Truth

(c) BB-UNet

(d) U-Net

Figure 12: Superior Segmentation results of BB-UNet over U-Net

(a) Panoramic X-Ray

(b) Ground Truth

(c) BB-UNet

(d) U-Net

Figure 13: Poor segmentation results of BB-UNet due to missing prior knowldege

##  4 Discussion

Table 2 compares the obtained results of the YOLOv8 architecture with those of
other popular methods on classification of dental X-rays. YOLOv8 outperformed
Mask R-CNN, HTC, and ResNetSt proposed by Silva et al.[20] but has a lower
AP50 score than these model architectures. Mask R-CNN + FCN and Mask R-CNN +
pointRend proposed by Pinheiro et al.[21] have better performance than YOLOv8
concerning mAP score but have less AP50 score compared to YOLOv8. PANet is the
only model outperforming YOLOv8 in both mAP and AP50 scores. Most of these
mentioned methods have utilized transfer learning for initializing the weights
of the model architecture and have been evaluated on a smaller test dataset
that does not contain complex panoramic X-rays belonging to category-5 and
category-6, compared to YOLOv8 which was evaluated on an extensive dataset,
including many complexities and was able to achieve superior performance for
classification of teeth.

Model Architecture | mAP | AP50
---|---|---
Mask R-CNN[20] | 70.5 | 97.2
Mask R-CNN + FCN[21] | 74.1 | 92.8
Mask R-CNN + pointRend[21] | 75.3 | 94.4
PANet[20] | 74.0 | 99.7
HTC[20] | 71.1 | 97.3
ResNeSt[20] | 72.1 | 96.8
YOLOv8 | 72.9 | 94.6
Table 2: Analysis of Teeth Classification Results Using Different Models

We also provide an evaluation of the BB-UNet + YOLOv8 relative to other
popular methods in Table 3. BB-UNet + YOLOv8 significantly outperforms U-Net
in terms of dice coefficient. Mask R-CNN and U-Net + Mask R-CNN proposed by R.
Nader et al.[33] have a better dice coefficient than BB-UNet + YOLOv8.
However, these architectures have utilized transfer learning to initialize
weights for Mask R-CNN architecture. Our model achieved comparable performance
with other model architectures even with a more challenging dataset. The
performance of the BB-UNet majorly depends on the prior knowledge provided by
the YOLOv8 model and can be improved by including more images with dental
implants and images having more than 32 teeth in the dataset. Bharati et
al.[34] demostrated that YOLO infers more quickly than Mask R-CNN, which makes
it appropriate for dentistry applications, BB-UNet + YOLOv8 was able to obtain
comparable performance with other model designs at a higher speed of
inference.

Model | Dice Coefficient(%)
---|---
Architecture | Incisors | Canines | Premolars | Molars
U-Net | 73.29 | 69.92 | 67.62 | 64.98
Mask R-CNN [33] | 89.56 | 89.45 | 88.70 | 87.55
U-Net + Mask R-CNN [33] | 91.55 | 91.00 | 90.00 | 88.58
BB-UNet + YOLOv8 ( Test Dataset 1) | 85.81 | 84.91 | 84.89 | 84.40
BB-UNet + YOLOv8 ( Test Dataset 2) | 85.71 | 86.64 | 86.22 | 86.03
Table 3: Analysis of Teeth Instance Segmentation Results Using Different
Models

##  5 Conclusion

Our study revealed promising advancements in segmentation performance by
incorporating the YOLOv8 output onto the U-Net model’s skip connection.
However, despite these strides, several areas for enhancement were identified.
The YOLOv8 model encountered challenges in accurately predicting numerous
labels belonging to the images of categories 5 and 6 within the dataset
adversely affecting overall model performance. Addressing these imbalances by
expanding the dataset size rather than augmentation strategies could bolster
performance. Our conclusion suggests that enlarging the dataset and refining
the object detection model hold the potential for achieving superior
segmentation and classification outcomes in future research work.

##  6 Dataset Availability

The dataset and the source code for the pipelines are available at
https://drive.digiratory.ru/d/s/wEIroe9cokkVRFvlJWO7S5rhynA2NVZu/lWzaszgSZ1t--
n8kdVdugbjhTUGyuqA_-NLSgvkOD8Ao or on request from the authors.

## References

  * [1] Titus KL Schleyer, Thankam P Thyvalikakath, Heiko Spallek, Miguel H Torres-Urquidy, Pedro Hernandez, and Jeannie Yuhaniak.  Clinical computing in general dentistry.  Journal of the American Medical Informatics Association, 13(3):344–352, 2006.
  * [2] Lubaina T Arsiwala-Scheppach, Akhilanand Chaurasia, Anne Müller, Joachim Krois, and Falk Schwendicke.  Machine learning in dentistry: a scoping review.  Journal of Clinical Medicine, 12(3):937, 2023.
  * [3] Nuha Junaid, Niha Khan, Naseer Ahmed, Maria Shakoor Abbasi, Gotam Das, Afsheen Maqsood, Abdul Razzaq Ahmed, Anand Marya, Mohammad Khursheed Alam, and Artak Heboyan.  Development, application, and performance of artificial intelligence in cephalometric landmark identification and diagnosis: a systematic review.  In Healthcare, volume 10, page 2454. MDPI, 2022.
  * [4] Eun-Hye Kim, Seunghoon Kim, Hyun-Joo Kim, Hyoung-oh Jeong, Jaewoong Lee, Jinho Jang, Ji-Young Joo, Yerang Shin, Jihoon Kang, Ae Kyung Park, et al.  Prediction of chronic periodontitis severity using machine learning models based on salivary bacterial copy number.  Frontiers in Cellular and Infection Microbiology, 10:571515, 2020.
  * [5] FDI World Dental Federation.  Fdi notation.  http://www.fdiworldental.org/.  Accessed: 2023-09-12.
  * [6] Gil Jader, Luciano Oliveira, and Matheus Melo Pithon.  Automatic segmenting teeth in x-ray images: Trends, a novel data set, benchmarking and future perspectives.  ArXiv, abs/1802.03086, 2018.
  * [7] Aqsa Ajaz and D. Kathirvelu.  Dental biometrics: Computer aided human identification system using the dental panoramic radiographs.  In 2013 International Conference on Communication and Signal Processing, pages 717–721, 2013.
  * [8] Muhamad Rizal Mohamed razali, Nazatul Sabariah Ahmad, Zulkifly Mohd Zaki, and Waidah Ismail.  Region of adaptive threshold segmentation between mean, median and otsu threshold for dental age assessment.  In 2014 International Conference on Computer, Communications, and Control Technology (I4CT), pages 353–356, 2014.
  * [9] Rarasmaya Indraswari, Agus Zainal Arifin, Dini Adni Navastara, and Naser Jawas.  Teeth segmentation on dental panoramic radiographs using decimation-free directional filter bank thresholding and multistage adaptive thresholding.  In 2015 International Conference on Information & Communication Technology and Systems (ICTS), pages 49–54, 2015.
  * [10] Guilherme Monteiro Tosoni, F. Walker, Nedra Joyner, John V. Tsimikas, and Alan G Lurie.  Recursive hierarchical segmentation analysis (rhseg) of bone mineral changes on digital panoramic images.  Oral Surgery Oral Medicine Oral Pathology Oral Radiology and Endodontology, 101, 2006.
  * [11] Mutasem Khalil Sari Alsmadi.  A hybrid fuzzy c-means and neutrosophic for jaw lesions segmentation.  Ain Shams Engineering Journal, 2016.
  * [12] Muhamad Rizal Mohamed Razali, Nazatul Sabariah Ahmad, Rozita Hassan, Zulkifly Mohd Zaki, and Waidah Ismail.  Sobel and canny edges segmentations for the dental age assessment.  In 2014 International Conference on Computer Assisted System in Health, pages 62–66, 2014.
  * [13] Md Mosaddik Hasan, Waidah Ismail, Rozita Hassan, and Atsuo Yoshitaka.  Automatic segmentation of jaw from panoramic dental x-ray images using gvf snakes.  In 2016 World Automation Congress (WAC), pages 1–6, 2016.
  * [14] Lucie Grajciarova, Magdalena Kasparova, Soroush Kakawand, Aleš Procházka, and Tatjana Dostalova.  Study of edge detection task in dental panoramic x-ray images.  Dento maxillo facial radiology, 42, 05 2013.
  * [15] Thorbjørn Louring Koch, Mathias Perslev, Christian Igel, and Sami Sebastian Brandt.  Accurate segmentation of dental panoramic radiographs with u-nets.  In 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019), pages 15–19, 2019.
  * [16] Yue Zhao, Pengcheng Li, Chenqiang Gao, Yang Liu, Qiaoyi Chen, Feng Yang, and Deyu Meng.  Tsasnet: Tooth segmentation on dental panoramic x-ray images by two-stage attention segmentation network.  Knowl. Based Syst., 206:106338, 2020.
  * [17] Qiaoyi Chen, Yue Zhao, Yang Liu, Yongqing Sun, Chongshi Yang, Pengcheng Li, Lingming Zhang, and Chenqiang Gao.  Mslpnet: multi-scale location perception network for dental panoramic x-ray image segmentation.  Neural Computing and Applications, 33:10277 – 10291, 2021.
  * [18] Gil Jader, Jefferson Fontineli, Marco Ruiz, Kalyf Abdalla, Matheus Pithon, and Luciano Oliveira.  Deep instance segmentation of teeth in panoramic x-ray images.  In 2018 31st SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI), pages 400–407, 2018.
  * [19] Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick.  Mask r-cnn.  In 2017 IEEE International Conference on Computer Vision (ICCV), pages 2980–2988, 2017.
  * [20] Bernardo Silva, Laís Pinheiro, Luciano Oliveira, and Matheus Pithon.  A study on tooth segmentation and numbering using end-to-end deep neural networks.  In 2020 33rd SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI), pages 164–171, 2020.
  * [21] Laís. Pinheiro, Bernardo Silva, Brenda Sobrinho, Fernanda Lima, Patrícia Cury, and Luciano Oliveira.  Numbering permanent and deciduous teeth via deep instance segmentation in panoramic x-rays.  In Letícia Rittner, Eduardo Romero Castro, Natasha Lepore, Jorge Brieva, Marius G. Linguraru, and Adam Walker, editors, 17th International Symposium on Medical Information Processing and Analysis, volume 12088 of Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series, page 120880C, December 2021.
  * [22] Alexander Kirillov, Yuxin Wu, Kaiming He, and Ross B. Girshick.  Pointrend: Image segmentation as rendering.  CoRR, abs/1912.08193, 2019.
  * [23] Jonathan Long, Evan Shelhamer, and Trevor Darrell.  Fully convolutional networks for semantic segmentation.  CoRR, abs/1411.4038, 2014.
  * [24] Olaf Ronneberger, Philipp Fischer, and Thomas Brox.  U-Net: Convolutional networks for biomedical image segmentation.  In Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pages 234–241. Springer, 2015.
  * [25] Joseph Redmon, Santosh Divvala, Ross Girshick, and Ali Farhadi.  You only look once: Unified, real-time object detection.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 779–788, 2016.
  * [26] Clement Zotti, Zhiming Luo, Alain Lalande, and Pierre-Marc Jodoin.  Convolutional neural network with shape prior applied to cardiac mri segmentation.  IEEE Journal of Biomedical and Health Informatics, 23(3):1119–1128, 2019.
  * [27] Rosana El Jurdi, Caroline Petitjean, Paul Honeine, and Fahed Abdallah.  Bb-unet: U-net with bounding box prior.  IEEE Journal of Selected Topics in Signal Processing, 14:1189–1198, 2020.
  * [28] B. Dwyer, J. Nelson, and J. Solawetz.  Roboflow (version 1.0) [software].  https://roboflow.com/.  Accessed: 2023-20-05.
  * [29] David Dang, Mu Le, Thomas Irmer, Oguzhan Angay, Bernhard Fichtl, and Bernhard Schwarz.  (2021).apeer: an interactive cloud platform for microscopists to easily deploy deep learning.  https://www.apeer.com/home/.  Accessed: 2023-12-06.
  * [30] Ultralytics.  Rangeking yolov8 architecture.  https://github.com/ultralytics/ultralytics/issues/189.  Accessed: 2023-10-14.
  * [31] Ozan Oktay, Enzo Ferrante, Konstantinos Kamnitsas, Mattias Heinrich, Wenjia Bai, Jose Caballero, Stuart A. Cook, Antonio de Marvao, Timothy Dawes, Declan P. O‘Regan, Bernhard Kainz, Ben Glocker, and Daniel Rueckert.  Anatomically constrained neural networks (acnns): Application to cardiac image enhancement and segmentation.  IEEE Transactions on Medical Imaging, 37(2):384–395, 2018.
  * [32] Karel Zuiderveld.  Contrast limited adaptive histogram equalization.  Graphics gems, pages 474–485, 1994.
  * [33] Rafic Nader, Andrey Smorodin, Natalia De La Fournière, Yves Amouriq, and Florent Autrusseau.  Automatic teeth segmentation on panoramic x-rays using deep neural networks.  In 2022 26th International Conference on Pattern Recognition (ICPR), pages 4299–4305, 2022.
  * [34] Puja Bharati and Ankita Pramanik.  Deep learning techniques—r-cnn to mask r-cnn: A survey.  Computational Intelligence in Pattern Recognition, 2019.


# The groups $\Gamma_{n}^{4}$, braids, and $3$-manifolds

V.O.Manturov111This text is a result of discussion with L.H.Kauffman,,
E.A.Mudraya, and N.D.Shaposhnik. I am deeply indebted to them for creating
some parts of this text , I.M.Nikonov

(April 2023)

###### Abstract

We introduce a family of groups $\Gamma_{n}^{k}$ for integer parameters $n>k.$
These groups originate from discussion of braid groups on $2$-surfaces. On the
other hand, they turn out to be related to 3-manifolds (in particular, they
lead to new relationships between braids and manifolds), triangulations (ideal
triangulations) cluster algebras, dynamics of moving points, quivers,
hyperbolic structures, tropical geometry, and, probably, many other areas
still to be discovered.

Among crucial reason of this importance of groups $\Gamma_{n}^{4}$ we mention
the Ptolemy relation, Pentagon relation, cluster algebra, Stasheff polytope.

Keywords: Delaunay triangulation, braids, 3-manifolds, Turaev–Viro invariants,
recoupling theory, spine

MSC 2020: 20F36, 57K20, 57K31, 13F60

##  1 Introduction

In [7], the author introduced a family of groups $\Gamma_{n}^{4}$ depending on
an integer parameter $n>5$. These groups naturally describe motions of points
on the plane (or a $2$-surface) by means of Voronoï tiling and Delaunay
triangulations. As points pass through non-generic moments, generators of
$\Gamma_{n}^{k}$ appear: they correspond to flips of triangulations (Fig. 1).

Figure 1: A flip

The relations of the group $\Gamma$ are arranged in such a way that isotopic
braids give rise to equal elements of the group $\Gamma$. In other words,
there is a homomorphism from the pure braid group on $n$ strands to the group
$\Gamma_{n}^{4}$. It seems likely that the kernel of the map
$PB_{n}\to\Gamma_{n}^{4}$ consists of full twists (centre of the pure braid
group), cf. [15]. Nevertheless, the map is very far from being surjective:
there are many words in the alphabet of $\Gamma$ which do not originate from
any braids. Understanding the full depth of the groups $\Gamma_{n}^{4}$ (in
particular, topological) is a subject of special investigation.

In [12], some presentations of the braid groups by means of $\Gamma$ are
obtained.

The most interesting relation of the group $\Gamma_{n}^{4}$ is the pentagon
relation. Graphically it looks as shown in Fig. 3 right. This relation says
that one can apply five consecutive flips to a given triangulation of a
pentagon and return to the initial triangulation.

The pentagon relation appears in various branches of mathematics under various
names. Just to mention a few of them: cluster algebras, triangulation of
polytopes, Teichmüller spaces, Associahedra (Stasheff polytope), recoupling
theory for $3$-manifolds (as Biedenharn–Elliot identity).

This naturally suggests that all those parts of mathematics should be related
to each other. Some of them indeed are: say, in [5], the authors found very
deep connections between possible triangulations of $2$-surfaces, cluster
algebras and … View Figure 2.

Figure 2: Cluster algebras connections and applications [1]

However, it seems that the authors overlooked the group structure. Once one
gets this group structure, one immediately gets invariants of braids as shown
in [12]. Hence, potentially all structures listed above should have something
to do with braid groups (not only classical braid groups for the plane but
also braid groups for $2$-surfaces).

One exception is the groupoid structure [2] related to cluster algebras and
Teichmüller spaces. This relation seems quite natural since Teichmüller spaces
are related to mapping class groups and braid groups. The relations between
braid groups these groupoids will be discussed elsewhere.

On the other hand, from the point of view of a low-dimensional topologist, it
is worth mentioning that more or less the same equation appears in another
form (actually, the tensor form) in the study of $3$-manifold invariants
(Turaev–Viro invariants [17, 18]).

The aim of the present paper is to establish a deeper connection between the
subjects listed above.

The two concrete applications we touch on in more detail, are:

  1. 1.

How to use recoupling theory (previously used for constructing $3$-manifold
invariants [6]) to get invariants of braids.

A tiny piece of this research was sketched in [16].

  2. 2.

How to use constructions of braid invariants to get Turaev–Viro like
invariants of $3$-manifolds.

Of course, we don’t pretend to cover all possible relations in this single
paper. In particular, we don’t touch on higher groups $\Gamma_{n}^{k},k\geq
5$, introduced by I.M.Nikonov in the book [12]. These groups describe
triangulations of higher dimensional manifolds; again, similar structures from
other points of view, and again, the group structure was overlooked in many
deep and serious papers.

The paper is organised as follows. In Section 2 we define the groups
$\Gamma_{n}^{4}$ and in Section 3 describe the homomorphism from the braid
group to $\Gamma_{n}^{4}$. In Section 4 we remind Ptolemy relation. Section 6
we describe invariants of braids which come from the recoupling theory. In
Section 7 we remind special spines of $3$-manifolds. In Section 10 we describe
invariants of $3$-manifolds based on Ptolemy relation. Section 12 concludes
the paper with the discussion of possible further applications of the groups
$\Gamma$.

### Acknowledgements

We are extremely grateful to Louis H.Kauffman, Vladimir G.Turaev, Victor
M.Buchstaber, Elizaveta A.Mudraya, Nikita D.Shaposhnik, Zheyan Wan for various
stimulating discussions.

Some relations of the pentagon equation (which also lead to representations of
various braid groups) where pointed out to the first named author by Igor
G.Korepanov. We are extremely grateful to him for pointing it out and for the
joy the first author shared with him when working on this project. Korepanov’s
articles [9, 10, 11] should give rise to many further deep connections of the
group $\Gamma_{n}^{k}$ (not only $\Gamma_{n}^{4}$); these results should
appear elsewhere.

##  2 The groups $\Gamma_{n}^{4}$

The groups $\Gamma_{n}^{4}$ were introduced in [7, 12, 13, 14] with the aim to
give an algebraic description of the space of triangulations with a fixed
number of vertices.

######  Definition 1.

The group $\Gamma_{n}^{4}$ is given by generators
$\\{d_{(ijkl)}\leavevmode\nobreak\ |\leavevmode\nobreak\
\\{i,j,k,l\\}\subset\bar{n},|\\{i,j,k,l\\}|=4\\}$ and relations:

  1. 1.

$d_{(ijkl)}^{2}=1$ for $(i,j,k,l)\subset\bar{n}$,

  2. 2.

$d_{(ijkl)}d_{(stuv)}=d_{(stuv)}d_{(ijkl)}$, for
$|\\{i,j,k,l\\}\cap\\{s,t,u,v\\}|<3$,

  3. 3.

$d_{(ijkl)}d_{(ijlm)}d_{(jklm)}d_{(ijkm)}d_{(iklm)}=1$ for distinct
$i,j,k,l,m$.

  4. 4.

$d_{(ijkl)}=d_{(jilk)}=d_{(klij)}=d_{(kjil)}=d_{(ilkj)}=d_{(jkli)}=d_{(lkji)}=d_{(lijk)}$
for distinct $i,j,k,l$.

Figure 3: Sequences of flips corresponding to relations in $\Gamma_{n}^{4}$

###  2.1 Where do groups $\Gamma$ come from?

We consider a braid: $n$ pairwise distinct points move on the plane (or two-
surface). They form a Voronoï tiling and its dual Delaunay triangulation; as
they move generically they undergo flips we write down generators
corresponding to flips. Thus for each generic braid we get a word.

Relations from groups $\Gamma$ are such that isotopic braids yield equal
words.

Known fact: Pentagon relation yields a braid group action on labeled
triangulation.(every edge is labeled by a Laurent polynomials. As we shall
see, the groups $\Gamma$ lead to invariants of $3$-manifolds.

and the techniques used for constructing invariants of 3-manifolds, can lead
to finite-dimensional representations of $\Gamma$.

##  3 The map from pure braid groups to $\Gamma_{n}^{4}$

In this section we describe the group homomorphism $f_{n}$ from the pure braid
group $PB_{n}$ to $\Gamma_{n}^{4}$ [7]. The topological background for that is
very easy: we consider codimension 1 ‘‘walls’’ which correspond to generators
(flips) and codimension 2 relations (of the group $\Gamma$). Having this, we
construct a map on the level of generators.

Let us use the notation $\bar{n}:=\\{1,\cdots n\\}$.

######  Definition 2.

The pure braid group $PB_{n}$ of $n$ strands is the group given by group
presentation generated by $\\{b_{ij}|i,j\in\bar{n},i<j\\}$ subject to the
following relations:

  1. 1.

$b_{ij}b_{kl}=b_{kl}b_{ij}$ for $i,j,k,l\in\bar{n}$ such that $i<j<k<l$ or
$i<k<l<j$;

  2. 2.

$b_{ij}b_{ik}b_{jk}=b_{ik}b_{jk}b_{ij}=b_{jk}b_{ij}b_{ik}$ for
$i,j,k\in\bar{n}$ such that $i<j<k$;

  3. 3.

$b_{ik}b_{jk}b_{jl}b_{jk}=b_{jk}b_{jl}b_{jk}b_{ik}$ for $i,j,k,l\in\bar{n}$
such that $i<j<k<l$.

The mapping $f_{n}:PB_{n}\rightarrow\Gamma_{n}^{4}$ can be formulated as
follows: Let us denote

$d_{\\{p,q,(r,s)_{s}\\}}=\left\\{\begin{array}[]{cc}d_{(pqrs)}&\text{if}\leavevmode\nobreak\
p<q<s,\\\ d_{(prsq)}&\text{if}\leavevmode\nobreak\ p<s<q,\\\
d_{(rspq)}&\text{if}\leavevmode\nobreak\ s<p<q,\\\
d_{(qprs)}&\text{if}\leavevmode\nobreak\ q<p<s,\\\
d_{(qrsp)}&\text{if}\leavevmode\nobreak\ q<s<p,\\\
d_{(rsqp)}&\text{if}\leavevmode\nobreak\ s<q<p.\end{array}\right.$

######  Remark 1.

Notice that the generator $d_{\\{p,q,(r,s)_{s}\\}}$ corresponds to four points
$P_{p},P_{q},P_{r},P_{s}$ such that they are placed on a circle according to
the order of $p,q,s$ and the point $P_{r}$ is placed close to $P_{s}$ for the
orientation $P_{r}$ to $P_{s}$ to be the counterclockwise orientation, see
Fig. 4. The subscription $s$ of $(r,s)_{s}$ means that the point $P_{s}$ does
not move, but the point $P_{r}$ will move turning around the point $P_{s}$
after this moment. In other words, when we use the notation
$d_{\\{p,q,(r,s)_{s}\\}}$, we are looking that the point $P_{r}$ is ‘‘moving’’
closely to the point $P_{s}$, turning around $P_{s}$. We would like to
highlight that $d_{\\{p,q,(r,s)_{s}\\}}\neq d_{\\{p,q,(s,r)_{s}\\}}$.

Figure 4: For $p<s<q$, $d_{\\{p,q,(r,s)_{s}\\}}=d_{(prsq)}$, but
$d_{\\{p,q,(s,r)_{s}\\}}=d_{(psrq)}$.

Let $b_{ij}\in PB_{n}$, $1\leq i<j\leq n$, be a generator. Consider the
elements

$\displaystyle\gamma_{i,(i,j)}^{I}=\prod_{p=2}^{j-1}\prod_{q=1}^{p-1}d_{\\{p,q,(i,j)_{j}\\}},$
$\displaystyle\gamma_{i,(i,j)}^{II}=\prod_{p=1}^{j-1}\prod_{q=1}^{n-j}d_{\\{(j-p),(j+p),(i,j)_{j}\\}},$
$\displaystyle\gamma_{i,(i,j)}^{III}=\prod_{p=1}^{n-j+1}\prod_{q=0}^{n-p+1}d_{\\{(n-p),(n-q),(i,j)_{j}\\}},$
$\displaystyle\gamma_{i,(i,j)}=\gamma_{i,(i,j)}^{II}\gamma_{i,(i,j)}^{I}\gamma_{i,(i,j)}^{III}.$

Now we define $f_{n}:PB_{n}\rightarrow\Gamma_{n}^{4}$ by

$f_{n}(b_{ij})=\gamma_{i,(i,(i+1))}\cdots\gamma_{i,(i,(j-1))}\gamma_{i,(i,j)}\gamma_{i,(j,i)}\gamma_{i,((j-1),i)}^{-1}\cdots\gamma^{-1}_{i,((i+1),i)},$

for $1\leq i<j\leq n$.

######  Theorem 1 ([7]).

The map $f_{n}:PB_{n}\rightarrow\Gamma_{n}^{4}$, which is defined above, is a
well defined homomorphism.

##  4 The Ptolemy relation and the pentagon relation

For an inscribed quadrilateral with consequent edge-lengths a,b,c,d and two
diagonals x, y one has: $ac+bd=xy$.

Figure 5: An inscribed quadrilateral

Another way to thinking of this relation is as follows. We enumerate the
vertices of the quadrilateral by numbers from $1$ to $4$ and associate
variable $x_{ij}$ with the edge oriented from $i$ to $j$, so that the opposite
edge is assigned $x_{ji}=-x_{ij}$. Then the Ptolemy relation looks like:

$x_{12}x_{34}+x_{23}x_{14}+x_{13}x_{42}=0.$ (1)

This equation is invariant under permutation of indices $(1,2,3,4)$, hence it
is valid also for self-intersecting quadrilaterals.

The signed Ptolemy relation

Figure 6: An inscribed quadrilateral

The label corresponds to an oriented edge. Inverting the edge will change the
sign of the label.

The ‘‘oriented’’ Ptolemy relation defined as above is invariant under any
permutation of indices $1,2,3,4$.

##  5 The Pentagon relation

There are exactly five triangulations of a pentagon with no additional points.

Each of these triangulations contains two non-intersecting (adjacent)
diagonals.

For each triangulation, one can flip one of the diagonals in question.

Performing the five consequent flips, we return to the initial position.

So, schematically, the pentagon relation looks like

This relation was known to specialists in cluster algebras [3, 4].

For us, it is the main relation in groups $\Gamma_{n}^{4}$.

The geometric meaning of this relation from the point of view of the braid
group is the following:

we consider five points almost on the same circle. The Delaunay triangulation
consists of three triangles tiling the pentagon.

As we move these points a little, the Delaunay triangulation undergo five
consecutive flips and since there is no braiding in this process, the element
of the group $\Gamma$ represented by the product of such flips has to be
trivial.

##  6 Recoupling theory

The essence of the recoupling theory [6, 8] is the following.

We draw trivalent graphs (on the plane or on 2-surface) and mark labels by
integer numbers.

Whenever we see number $k$, we can think that there is a bunch of $k$ parallel
strands along this edge. At each vertex each of those strands turns right or
left (Fig. 7).

Figure 7: Strands at a vertex

This naturally yields the two conditions, the parity and the triangle
inequality.

Besides, fix an integer $r\geq 3$ and restrict ourselves to the situation when
the numbers of strands $f(a),f(b),f(c)$ for three edges coming to one vertex
satisfy $f(a)+f(b)+f(c)\leq 2r$. This leads to the notion of an admissible
colouring.

As the time $t$ goes, the tree undergoes flips.

Fix an integer $r\geq 3$. Denote $q=e^{\frac{i\pi}{r}}\in\mathbb{C}$. For any
regular value $t$, consider the set $F_{r}(t)=F_{r}(G(t))$ of admissible
colourings of the edges of the Delaunay triangulation $G(t)$, i.e. maps
$f:E(G(t))\to\mathbb{N}\cup\\{0\\}$ such that for any triangle of $G(t)$ with
edges $a,b,c$ one has

  1. 1.

$f(a)+f(b)+f(c)$ is even;

  2. 2.

$f(a)+f(b)\geq f(c)$, $f(a)+f(c)\geq f(b)$, $f(b)+f(c)\geq f(a)$ ;

  3. 3.

$f(a)+f(b)+f(c)\leq 2r-4$.

Let $V_{r}(t)$ be the linear space with the basis $F_{r}(t)$.

Figure 8: Transformation of Delaunay triangulation

Consider a singular value $t_{k}$. At the moment $t_{k}$ a transformation
(flip) of the triangulation occurs (Fig. 8). By recoupling theorem [6, Theorem
6.2] one can define a linear operator $A_{r}(t_{k}):V_{r}(X(t_{k}-0))\to
V_{r}(X(t_{k}+0))$ whose coefficients are $q-6j$-symbols. More detailed, the
triangulations $G(t_{k}-0)$ and $G(t_{k}+0)$ differ by an edge:
$j=E(G(t_{k}-0))\setminus E(G(t_{k}+0))$, $i=E(G(t_{k}+0))\setminus
E(G(t_{k}-0))$. Let $a,b,c,d$ be the edges incident to same triangles as $i$
or $j$. For any $f\in F_{r}(t_{k}-0)$ considered as a basis element of
$V_{r}(t_{k}-0)$, we set

$A_{r}(t_{k})f=\sum_{g\in F_{r}(t_{k}-0)\colon f=g|_{E(G(t_{k}-0))\cap
E(G(t_{k}+0))}}\left\\{\begin{array}[]{ccc}g(a)&g(b)&g(i)\\\
f(c)&f(d)&f(j)\end{array}\right\\}_{q}\cdot g,$

where $\left\\{\begin{array}[]{ccc}\alpha&\beta&\xi\\\
\gamma&\delta&\eta\end{array}\right\\}_{q}$ is the $q-6j$-symbol (for the
definition see [6, Proposition 11]).

Let $A_{r}(\beta)=\prod_{k=1}^{m}A_{r}(t_{k}):V(X)\to V(X)$ be the composition
of operators $A(t_{k})$.

######  Theorem 2.

The operator $A_{r}(\beta)$ is a pure braid invariant.

###### Proof.

Consider a pure braid isotopy $\beta_{s}$, $s\in[0,1]$. We can assume the
isotopy is generic. This means that the isotopy includes a finite number of
values $s_{l}$ such that the braid $\beta_{s_{l}}$ has a unique value $t^{*}$
such that the set $X_{s}(t^{*})$ contains one of the following singular
configurations

  1. 1.

five points which belong to one circle (or line);

  2. 2.

two quadruples of points which lie on one circle. The circles for the
quadruples are different.

The first case leads to the pentagon relation for Delaunay triangulation
transformations (Fig. 9). This relation induces a pentagon relation for the
operator $A_{r}(\beta)$ which holds by theorem [6, Proposition 10].

Figure 9: Pentagon relation

The second case leads to commutativity relation $A_{1}A_{2}=A_{2}A_{1}$ where
$A_{1}$, $A_{2}$ correspond to flips on the given quadruples of points. The
relation holds by construction of the operators $A_{1}$, $A_{2}$. ∎

Thus, we get a construction analogous to that in [12, Section 16.6] which uses
Ptolemy relation.

######  Example 1.

Consider the point configuration $P_{1}=(0,0)$, $P_{2}=(\frac{1}{3},0)$,
$P_{3}=(-\frac{1}{2},\frac{\sqrt{3}}{2})$,
$P_{3}=(-\frac{1}{2},-\frac{\sqrt{3}}{2})$, $P_{5}=(1,0)$. Consider the pure
braid $\beta$ given by dynamics of $P_{2}$ moving around $P_{1}$:
$P_{2}(t)=(\frac{1}{3}\cos t,\frac{1}{3}\sin t)$, $t\in[0,2\pi]$. The dynamics
produces six different Delaunay triangulations. For $r=4$, each triangulation
has $160$ admissible colourings. The operator $A_{4}(\beta)$ of dimension
$160$ is not identity (it has the eigenvalue $-1$ of multiplicity $20$). Thus,
the invariant $A_{r}$ is not trivial.

##  7 Special spines of $3$-manifolds

Here we shall give the definition of special spine (see [17]).

######  Definition 3.

An _simple polyhedron_ is a finite CW-complex $P$ such that each point $x$ in
$P$ has a neighbourhood homeomorphic to one of the three configurations (see
Figure 10):

  1. 1.

a disc;

  2. 2.

three discs intersecting along a common arc in their boundaries, where the
point $x$ is on the common arc; or

  3. 3.

the _butterfly_ , which is a configuration built from four arcs that meet at
the point $x$, with a face running along each of the six possible pairs of
arcs.

Figure 10: The three types of points in a simple polyhedron $P$.

Each simple polyhedron is naturally stratified. In this stratification each
stratum of dimension $2$ (a $2$-component) is a connected component of the set
of nonsingular points. Strata of dimension $1$ consist of open or closed
triple lines, and dimension $0$ strata are true vertices.

A simple polyhedron $P$ is called _special_ if:

  1. 1.

Each 1-stratum of $P$ is an open 1-cell.

  2. 2.

Each 2-component of $P$ is an open 2-cell.

A _spine_ of a compact connected $3$-manifold with boundary $M$ is a
subpolyhedron $P\subset IntM$ such that $M\setminus P$ is homeomorphic to
$\partial M\times[0,1)$. By a spine of a closed connected $3$-manifold $M$ we
mean a spine of $M\setminus IntB^{3}$ where $B^{3}$ is a $3$-ball in $M$.

##  8 Turaev–Viro invariants

Consider a special spine of a $3$-manifold and associate weights of
butterflies to true vertices. These weight depend on $6$ parameters and they
are called ($q-6j$)-symbols).

Figure 11: The Matveev-Piergallini move

##  9 $(q-6j)$-symbols

Each butterfly has $6$ wings which are naturally split into three pairs of
opposite ones. It is naturally to encode a butterfly by the $1$-frame of the
$3$-simplex.

Hence, a coloured butterfly associates $6$ numbers (or elements of some ring).
Since each butterfly has many symmetries, it is natural to require that these
assignments are invariant under rotations. In other words, we associate
numbers ($(q-6j)$-symbols)

$\left[\begin{array}[]{ccc}a&b&c\\\ d&e&f\end{array}\right]$

to hexuples of integers $(a,b,c,d,e,f)$ in such a way that

$\left[\begin{array}[]{ccc}a&b&c\\\
d&e&f\end{array}\right]=\left[\begin{array}[]{ccc}b&a&c\\\
e&d&f\end{array}\right],\quad\left[\begin{array}[]{ccc}a&b&c\\\
d&e&f\end{array}\right]=\left[\begin{array}[]{ccc}f&e&a\\\
c&b&d\end{array}\right].$

##  10 Constructing invariants of $3$-manifolds from $\Gamma$

Below, we give a hint how to construct the invariant geometrically.

However, there is a tiny gap in the argument.

Motivated by this example, we present an algebraic description of the
invariant.

For each triangle with sides $(a,b,x)$ and two inscribed quadrilaterals
$(a,b,c,d),(a,b,c^{\prime},d^{\prime})$ with diagonals $y,y^{\prime}$
respectively there exists a unique $z$ giving rise to three (self-
intersecting) quadrilaterals.

Solution: completing a pentagon.

Figure 12: Completing quadrilaterals to a pentagon

###  10.1 Ptolemy yields Biedenharn–Elliot

Figure 13:

Associate real-valued variables to (oriented) $2$-cells of a special cell.
With each true vertex we have a butterfly; associate this butterfly’s weight
to it.

Let $S$ be a spine. Consider the commutative algebra

$P(S)=\mathbb{Z}_{2}\langle x\in S_{2}\mid\Pi_{c},c\in S_{0}\rangle$

where $S_{0}$ is the set of vertices of the spine $S$, $S_{2}$ is the set of
$2$-cells of the spine, and $\Pi_{c}$ is the Ptolemy relation (2) at the
vertex $c$. Let $F(S)$ denote the fraction field of the algebra $P(S)$.

Two fields $F_{1}$ and $F_{2}$ of characteristic $2$ are _stable equivalent_
if there exist sets $X_{1},X_{2}$ such that
$F_{1}\otimes\mathbb{Z}_{2}(X_{1})\simeq F_{2}\otimes\mathbb{Z}_{2}(X_{2})$
where $\mathbb{Z}_{2}(X_{1})$ is the field of rational functions over the
generator set $X_{i}$, $i=1,2$.

######  Theorem 3.

Given a 3-manifold $M$, the stable equivalence class of the fraction field
$F(S)$ does not depend on the choice of the spine $S$ of the manifold $M$.

###### Proof.

Let $S$ and $S^{\prime}$ be spines connected by a Matveev–Piergallini move
(Fig.11). Let us show that the algebras $P(S)$ and $P(S^{\prime})$ are stable
equivalent. The correspondence between $2$-cells of the spines $S$ and
$S^{\prime}$ is one-to-one (except the cell $z$) and induces a homomorphism
$\phi\colon F(S)\to F(S^{\prime})$. We need to check that $\phi$ is well
defined and bijective.

The algebras $P(S)$ and $P(S^{\prime})$ have the same relations except those
of the vertices participating in the Matveev–Piergallini move. Let us look at
the those relations. For the algebra $P(S)$ we have

$ad+by+cx=0,\quad ad^{\prime}+by^{\prime}+c^{\prime}x=0,$

hence

$x=\frac{ad^{\prime}+by^{\prime}}{c^{\prime}},\quad
y=\frac{ad+cx}{b}=\frac{acd^{\prime}+ac^{\prime}d+bcy^{\prime}}{bc^{\prime}}$

in the field $F(S)$. For the algebra $P(S^{\prime})$ we have

$az+cy^{\prime}+c^{\prime}y=0,\quad bz+cd^{\prime}+c^{\prime}d=0,\quad
dy^{\prime}+d^{\prime}y+xz=0,$

hence

$\displaystyle z=\frac{cd^{\prime}+c^{\prime}d}{b},$ $\displaystyle
y=\frac{az+cy^{\prime}}{c^{\prime}}=\frac{acd^{\prime}+ac^{\prime}d+bcy^{\prime}}{bc^{\prime}},$
$\displaystyle
x=\frac{dy^{\prime}+d^{\prime}y}{z}=\frac{b(bc^{\prime}dy^{\prime}+ac(d^{\prime})^{2}+ac^{\prime}dd^{\prime}+bcd^{\prime}y^{\prime})}{bc^{\prime}(cd^{\prime}+c^{\prime}d)}=\frac{ad^{\prime}+by^{\prime}}{c^{\prime}}$

in $F(S^{\prime})$. Thus, the expressions for $x$ and $y$ in $F(S)$ and
$F(S^{\prime})$ coincide. That means the map $\phi$ establishes an isomorphism
between the fraction fields.

Figure 14: The second Matveev-Piergallini move

For a second Matveev–Piergallini move (Fig. 14) we have no relations for the
spine $S$ and the relations

$ac+bz+xy=0,\quad ac+b^{\prime}z+xy=0$

for the spine $S^{\prime}$. Then $z=\frac{ac+xy}{b}$ and $b^{\prime}=b$.
Hence, we can identify the fields $F(S)$ and $F(S^{\prime})$.

Figure 15: The bubble move

For a bubble move (Fig. 15) we get two additional variables and no new
relations. Hence, $F(S^{\prime})=F(S)\otimes\mathbb{Z}_{2}(x,y)$.

Thus, the fields $F(S)$ and $F(S^{\prime})$ are stable equivalent. ∎

##  11 Generalisations of the above construction of 3-manifold invariants

In the above construction we dealt with a system of equations with 2-cell
weights acting as variables.

For each hexuple of labels we wrote an equation of the type

$ac+bd=xy.$ (2)

Call it the main equation.

Let us fix some curvature $\kappa$ (positive, negative or zero) and let us fix
some radius $r$ (positive or equal to infinity).

By $(\kappa,r)$-main equation we mean the equation on the lengths of $6$ edges
of the tetrahedron inscribed in the circle of radius $r$ in the 3-space of
constant curvature $\kappa$. For the case $r=0$ we take the equation for an
inscribed quadrilateral with $2$ diagonals.

Hence, the $(0,0)$-main equation is exactly the above main equation
$ac+bd=xy$.

We argue that if we fix $\kappa,r$ and replace the main equation with the
$(\kappa,r)$-main equation, the space of solutions to the equations will still
be an invariant of $3$-manifolds.

Indeed, the only thing we used with the quadrilateral is: whenever we draw two
(inscribed) quadrilaterals sharing the same triangle (refer to the figure)
then we can find exactly one edge length (of the fifth edge of the pentagon)
which gives rise to the three remaining quadrilaterals.

The only thing one has to change in the above argument is just to replace
‘‘inscribed in a circle on the plane’’ with ‘‘inscribed in a sphere of radius
$r$ in the space of constant curvature $\kappa$.’’

##  12 Why are the groups $\Gamma$ ubiquitous?

Many structures in mathematics are of extreme importance not because they just
solve a problem or provide nice invariants (which may also be the case) but
because they uncover close ties between various branches of mathematics. This
allows one to construct a ‘‘universal translator’’ between different languages
different mathematicians speak.

Among such famous objects, we have already mentioned cluster algebras (see
Fig. 2), we just mention Coxeter groups and Associahedra (Stasheff polytopes).
Among other branches of mathematics, the groups $\Gamma$ are related at least
to these three ones.

Among further directions which may appear in further papers, we mention just
two ones:

In topology: passing from braids to knots, passing from 3-manifolds to
4-manifolds etc.

In group theory: deeper understanding the structure of the group $\Gamma$
themselves.

Among all possible presentations and actions of the groups $\Gamma_{n}^{k}$ we
mention just one method that the first named author calls the photography
method:

We take several (say, two) photos of some picture, and if we see that some of
them are sufficient to recover the whole picture, then we carefully write down
the way one picture can be transformed to the other.

The Ptolemy transformation and the generalisation given in the last section
are evidences of that.

Namely, in order to show that Ptolemy yields pentagon, we need not calculate
anything (see [7, 12, 13, 14]), we need not calculate anything: we just say
that if two quadrilaterals sharing three vertices are both inscribed then the
whole pentagon is inscribed, and the data concerning any of its quadrilaterals
can be restored from the data of the two quadrilaterals.

What do we mean by data? In geometry, those can be edge lengths, angles,
dihedral angles, areas, volumes, etc.

But probably other data can come from interesting number theory (say,
quadruples, quintuples etc) and hence yield new invariants of (at least)
braids and 3-manifolds.

We expect that many representations of the groups $\Gamma$ and other groups
(known or still to be discovered) can be explained by using this method.
Probably, many statements and relations the cluster algebra theory also follow
from this method.

## References

  * [1] A. Felikson, Ptolemy Relation and Friends, arXiv:2302.06379.
  * [2] V.V. Fock, L.O. Chekhov, A quantum Teichmüller space, _Theor Math Phys_ 120:3 (1999) 1245–1259.
  * [3] S. Fomin and A. Zelevinsky, Cluster algebras I: Foundations, _J. Amer. Math. Soc._ 15 (2002) 497–529.
  * [4] S. Fomin and A. Zelevinsky, Cluster algebras II: Finite type classification, _Invent. Math._ 154:1 (2003) 63–121.
  * [5] I.M. Gelfand, M.M. Kapranov, A.V. Zelevinsky, _Discriminants, resultants, and multidimensional determinants_ , Springer, 2008.
  * [6] L.H. Kauffman, S. Lins, _Temperley–Lieb Recoupling Theory and Invariants of 3-Manifolds_ , Princeton University Press, 1994.
  * [7] S. Kim, V.O. Manturov, Artin’s braids, braids for three space, and groups $\Gamma_{n}^{4}$ and $G^{k}_{n}$, J. Knot Theory Ramifications 28:10 (2019) 1950063.
  * [8] A.N. Kirillov, N.Y. Reshetikhin, Representations of the algebra $U_{q}(sl_{2})$, $q$-orthogonal polynomials and invariants of links, in _Infinite dimensional Lie algebras and groups_ , _Adv. Ser. In Math. Phys._ 7, World Scientific, Singapore (1988), 285–339.
  * [9] I. G. Korepanov, S. Saito, Finite-dimensional analogues of the string $s\leftrightarrow t$ duality and the pentagon equation, _Theoret. and Math. Phys._ 120:1 (1999), 862–869.
  * [10] I.G. Korepanov, N.M. Sadykov, Pentagon Relations in Direct Sums and Grassmann Algebras, _SIGMA_ 9 (2013) 030.
  * [11] I.G. Korepanov, Two-cocycles give a full nonlinear parameterization of the simplest 3–3 relation, _Lett Math Phys_ 104 (2014) 1235–1261.
  * [12] V.O. Manturov, D. Fedoseev, S. Kim, I. Nikonov, Invariants And Pictures: Low-dimensional Topology And Combinatorial Group Theory, Series On Knots And Everything 66, World Scientific, 2020.
  * [13] V.O. Manturov, D. Fedoseev, I. Nikonov, Manifolds of triangulations, braid groups of manifolds, and the groups $\Gamma_{n}^{k}$ In:_Garanzha, V.A., Kamenski, L., Si, H. (eds) Numerical Geometry, Grid Generation and Scientific Computing. Lecture Notes in Computational Science and Engineering, vol 143._ Springer, Cham., 2021, 13–36.
  * [14] V.O. Manturov, D. Fedoseev, S. Kim, I. Nikonov, On groups $G_{n}^{k}$ and $\Gamma_{n}^{k}$: A study of manifolds, dynamics, and invariants, _Bull. Math. Sci._ 11:2 (2021) 2150004.
  * [15] V.O. Manturov, Realisability of $G^{3}_{n}$, realisability projection, and kernel of the $G^{3}_{n}$-braid presentation, arXiv:2210.13338.
  * [16] V.O. Manturov, I. Nikonov, On an invariant of pure braids, arXiv:2303.04423.
  * [17] S. Matveev, Algorithmic topology and classification of 3-manifolds, Springer-Verlag Berlin Heidelberg, 2007.
  * [18] V. G. Turaev, O. Ya. Viro, State sum invariants of 3-manifolds and quantum $6j$-symbols, _Topology_ 31:4 (1992), 865–902.