Modular Neural Network Approaches for Surgical Image Recognition
Deep learning-based applications have seen a lot of success in recent years.
Text, audio, image, and video have all been explored with great success using
deep learning approaches. The use of convolutional neural networks (CNN) in
computer vision, in particular, has yielded reliable results. In order to
achieve these results, a large amount of data is required. However, the dataset
cannot always be accessible. Moreover, annotating data can be difficult and
time-consuming. Self-training is a semi-supervised approach that managed to
alleviate this problem and achieve state-of-the-art performances. Theoretical
analysis even proved that it may result in a better generalization than a
normal classifier. Another problem neural networks can face is the increasing
complexity of modern problems, requiring a high computational and storage cost.
One way to mitigate this issue, a strategy that has been inspired by human
cognition known as modular learning, can be employed. The principle of the
approach is to decompose a complex problem into simpler sub-tasks. This
approach has several advantages, including faster learning, better
generalization, and enables interpretability.
  In the first part of this paper, we introduce and evaluate different
architectures of modular learning for Dorsal Capsulo-Scapholunate Septum (DCSS)
instability classification. Our experiments have shown that modular learning
improves performances compared to non-modular systems. Moreover, we found that
weighted modular, that is to weight the output using the probabilities from the
gating module, achieved an almost perfect classification. In the second part,
we present our approach for data labeling and segmentation with self-training
applied on shoulder arthroscopy images.
Nosseiba Ben Salem, Younes Bennani, Joseph Karkazan, Abir Barbara, Charles Dacheux, Thomas Gregory
# Modular Neural Network Approaches for Surgical Image Recognition

###### Abstract

Deep learning-based applications have seen a lot of success in recent years. Text, audio, image, and video have all been explored with great success using deep learning approaches. The use of convolutional neural networks (CNN) in computer vision, in particular, has yielded reliable results. In order to achieve these results, a large amount of data is required. However, the dataset cannot always be accessible. Moreover, annotating data can be difficult and time-consuming. Self-training is a semi-supervised approach that managed to alleviate this problem and achieve state-of-the-art performances. Theoretical analysis even proved that it may result in a better generalization than a normal classifier. Another problem neural networks can face is the increasing complexity of modern problems, requiring a high computational and storage cost. One way to mitigate this issue, a strategy that has been inspired by human cognition known as modular learning, can be employed. The principle of the approach is to decompose a complex problem into simpler sub-tasks. This approach has several advantages, including faster learning, better generalization, and enables interpretability. In the first part of this paper, we introduce and evaluate different architectures of modular learning for Dorsal Capsulo-Scapholunate Septum (DCSS) instability classification. Our experiments have shown that modular learning improves performances compared to non-modular systems. Moreover, we found that weighted modular, that is to weight the output using the probabilities from the gating module, achieved an almost perfect classification. In the second part, we present our approach for data labeling and segmentation with self-training applied on shoulder arthroscopy images.

Data Science Machine Learning Modular Neural Networks Self-training Dorsal Capsulo-Scapholunate Septum Shoulder arthroscopy.

## 1 Introduction

Cognitive neuropsychology has been a great inspiration and the key behind the success of data science. Our brains are the most powerful machines, that's why they have been trying to replicate how it works and how it solves a problem into machines. This was the main reason to the idea of neural networks and their ability to handle complex problems. Continuing to explore the human mind, some researchers focused their attention on how our brain is subdivided into independent components. Each subset of neurons specializes in a function, and they interact with each other [12]. Thereby, the beginning of using modular architecture in learning. The main idea is to decompose a complex task into simple independent sub-tasks. It was first introduced in [20] when they demonstrated that dividing processing produces better performance under certain conditions. Later, other researchers revealed that adding a priori knowledge helps the model improve the results compared to non-modular. And in recent research [15] stated that
modularity leads to better generalization, faster learning, possesses scaling properties, and allows interpretability. Therefore, the rising interest among researchers. The modular architecture is a concept, therefore, it exists several strategies to decompose a problem. For instance, models can compete with each other to complete the same task [1], or they can cooperate [18], etc. However, it may not always be evident to determine how and where to subdivide.

Another concept that has shown remarkable results, is the semi-supervised algorithm self-training. With the urging need for a massive number of data to train models and the difficulty or high cost of labeling data, semi-supervised learning became an active research discipline. Self-training alleviates this problem by relying on a small subset of labeled samples and a large subset of unlabeled samples, and still achieves state-of-the-art results [5]. It assumes that the decision boundary should lie in a low-density region to obtain better generalization [7]. Pseudo-labeling, that is annotating the unlabeled samples iteratively, we call them pseudo-labels, and adding them to the training subset, is a common use of self-training.

In this work, we built systems using modular learning and self-training and applied them in two contexts. The first application aims to classify instability in a wrist structure named Dorsal Capsulo-Scapholunate Septum (DCSS). We tested different strategies of modular learning and compared them to the non-modular architecture. In the second application, we tested pseudo-labeling on shoulder arthroscopy images to detect structures using image segmentation.

To this end, we describe a brief overview of the background and investigate the related works on both modular learning and self-training in the first section. Then in the second section, we present our proposed approach for classifying DCSS using modular learning and display the obtained results. Finally, we present the self-training approach for data labeling and segmentation.

## 2 Fundamental background of the proposed approach and related work

In this section, we will introduce some basic background on modular learning and self-training methods and give an overview on the previous approaches in the literature.

### Modular learning

According to neuroscience, our brain is an amalgamation of several modules operating independently and they cooperate and communicate with each other [25]. This decomposition, inspired researchers to build a system composed of different simple modules and then integrate their outputs. Each module is a neural network with learnable parameters and output a probability of prediction. Modules are trained independently and are controlled by a gating module. The gating module is responsible to activate modules according to a certain rule, either set by the user or found automatically. There exist several ways to set the interaction between the different modules. For instance, one can fragment a complex problem into smaller sub-tasks. Each module is responsible for a fragment, and they cooperate between each other to create a complementary network. Or modules can compete between each other by learning to solve the same problem.

Using modular learning, offers major advantages comparing to non-modular system [15]. For instance, using multiple simple networks might significantly reduce the computational time, usually needed to train a complex model. It has been shown that modular systems improves substantially the robustness of neural networks[24]. Combining knowledge from different modules makes the model more robust. It also reduces the model's complexity, thus optimized computation time in the training process. Moreover, the modularity limits the propagation of biases, contrary to a non-modular model. In 1989, Rueckl, [20] began to study the representation and location of objects of "what" and "where" to separate. They used retinal images for classification and revealed that modular learning is more efficient in computation and learned more easily compared to non-modular network under certain circumstances. There exists several strategies to build modular architecture depending on the type of interaction between the structures. For instance, in [2], they use decoupled modules where data is separated and each group is assigned to a module. The final prediction is the maximum probability from all modules. In [18], they used hierarchical network. Others [1], and [3], use ensemble networks where all the modules are identical, and they learn to perform the same task. The final prediction is a majority vote of prediction probability from all the networks.

The most common structure is the Multiple-experts network [11]. It consists of gating module and experts. The gating module controls the activation of the networks and distributes the tasks. The networks do not need to be identical it can be a hybrid system with a variety of techniques [4]. We used multiple experts in our case.
### Self-Training

Supervised learning using neural networks has seen success in several applications. To ensure a better generalization, neural networks require to be trained on a huge amount of labeled data. However, labeling data can be an expensive and time-consuming task and in some cases not accessible. Thus, the rise of a domain called semi-supervised. This domain aims to use both labeled and unlabeled data from the same or similar distribution for a specific problem. There exist several semi-supervised methods. Self-training is a common semi-supervised method. Its increasing interest is due to its better performance compared to supervised learning. This result is somehow surprising, given that we don't add any external information. Theoretical works have proven that self-training engenders a better generalization bound given certain conditions.

Let \(N\) be the number of labeled data and \(M\) the number of unlabeled data. An initial model \(f_{\mathbf{w}^{(0)}}\) is trained from \(\left\{\mathbf{x}_{n},y_{n}\right\}_{n=1}^{N}\). Given the unlabeled data \(\left\{\widetilde{\mathbf{x}}_{m}\right\}_{m=1}^{M}\) The obtained model plays the role of pseudo-labeler by computing \(\hat{y}_{m}=\hat{f}_{\mathbf{w}^{(0)}}\left(\mathbf{x}_{m}\right)\). Let \(k\) be the number of selected predicted labels and let \(\ell\) be the number of iterations. Then \(f_{\mathbf{w}^{(\ell)}}\) is trained from \(\left\{\mathbf{x}_{n},y_{n}\right\}_{n=1}^{N}\cup\left\{\mathbf{x}_{m},\hat{y}_{m} \right\}_{m=1}^{k}\) and is used as pseudo-labeler for the next iteration. Reiterate until reaching stopping criteria. The empirical risk to be minimized is as follows

\[\hat{R}(\mathbf{W}) =\frac{\lambda}{2N}\sum_{n=1}^{N}\left(y_{n}-f\left(\mathbf{W};\mathbf{x} _{n}\right)\right)^{2}\] \[+\frac{\tilde{\lambda}}{2M}\sum_{m=1}^{M}\left(\hat{y}_{m}-f\left( \mathbf{W};\hat{\mathbf{x}}_{m}\right)\right)^{2} \tag{1}\]

where \(\lambda+\tilde{\lambda}=1\).

Similar to modular training, self-training was used decades ago from 1976 to train a model without external knowledge [9]. And in these recent years, researchers have shown an interest in self-training given its ability to train the model with few labeled data and a lot of unlabeled ones. The strategy of choosing pseudo-labels differs from work to another. For instance, [14], selected the data having the maximum prediction probability in the current model tested on denoising auto-encoder,[22] selected the nearest observations in feature space. Others [27], [23], selected the proportion of the most confident data instead of setting a threshold.

The preceding research has supposed all the classes are equals and uses a constant threshold to sort out the most confident unlabeled data to add to the training set. That is what motivated the use of dynamic threshold to prevent ignoring certain unlabeled data. [26], suggested adapting curriculum learning in self-training and called it curriculum pseudo labeling (CPL). Curriculum learning is an approach based on training the model starting with the data easier to learn to the more complex ones. The strategy of CPL is to adapt the threshold for each class and select unlabeled data depending on the model's learning status. They compute performances for different classes and adjust the threshold accordingly. Meaning, if the results in a certain class are not high, the threshold ought to be lower. The authors demonstrated that their method converged faster than fixed threshold methods and achieved good results on state-of-the-art benchmarks of semi-supervised learning. Another use of curriculum labeling in [6] achieved significant results. They observed that the distribution of the maximum probability prediction follows the Pareto distribution, that is, a skewed distribution. The idea is to choose unlabeled data by taking a percentile of the distribution of the maximum probability for pseudo-labeled samples.

## 3 Proposed approaches: Modular learning for classification

### Motivation

The first purpose of this work is to classify images of the Dorsal Capsulo-Scapholunate Septum (DCSS) to identify the stage of scapholunate instability using a modular architecture. DCSS is a structure in the wrist that ensures a scapholunate articulation [16]. A common disease in DCSS is scapholunate instability. It causes severe wrist dysfunction, making it difficult for the patient to perform everyday activities [13]. Identifying the scapholunate instability requires an accurate diagnosis in order to prevent it in the early phases. We used images from wrist arthroscopy and the goal is to determine whether the structure is healthy or if it exists instability in stage 1, stage 2, or stage 3. We exploited the nature of the problem to build a modular system. We can divide the problem into, first, predicting if the structure is healthy or pathological. Then, if the DCSS is pathological, we can then use a second component to discriminate between the three stage.
### System architecture

The system is composed of two modules; the gating module and the discriminative module. The gating module is trained to detect if the DCSS is healthy or not from the arthroscopy image. If the DCSS is healthy, we output its prediction probability \(P(y=\text{`healthy'}|X)\). If the DCSS is pathological, we feed the image to the discriminative module. The latter can be designed in different ways, and the final output can be computed differently. We choose to test three architectures (modular, modular 1 Vs 1, and weighted modular). In the following, we will detail the different cases. Let \(H\) be the event 'healthy', \(\bar{H}\)' pathological' and \(S_{i}\), \(i\in\{1,2,3\}\) the stages of instability.

Modular (All)In this case, the discriminative module is trained to distinguish between the three stages of instability. Given the image predicted by the gating module to be pathological, the discriminative module computes the prediction probability \(P(S_{i}|X,\bar{H}),\quad i=1,\ldots,3\). The discriminative module's prediction is as follows:

\[\arg\max_{i}P(S_{i}|X,\bar{H}) \tag{2}\]

Modular (1 Vs 1)In this case, we built three experts competing with each other. The first module is intended to discriminate between stage 1 and 2, The second for the stage 1 and 3, and, finally, one for the stage 2 and 3. The output is the maximum from each expert. The three modules computes the probabilities in equation 3 with i= {1,2}, {1,3} and {2,3} respectively.

\[P_{m}(S_{i}|X,\bar{H})\times P_{m}(\bar{H}|X)+P_{m}(S_{i}|X,H)\times P_{m}(H| X). \tag{3}\]

The discriminative module's prediction is as follows:

\[\arg\max_{(i,k)}P_{m_{k}}(S_{i}|X),\quad i=1,\ldots,3\quad k=1,\ldots,3. \tag{4}\]

Weighted modularThis system is similar to the first modular case, except for computing the output. \(P(S_{i}|X,\bar{H})\times P(\bar{H}|X)+P(S_{i}|X,H)\times P(H|X),\quad i=1, \ldots,3\). In this case, the gating module assigns a weight to each output and picks the output that yields the highest prediction probability. The advantage of this method is to take into account the error in the gating module. In the first system, we supposed that the gating module is perfect opposing to this architecture. For instance, if the difference between the prediction probability of the gating expert is not large, it means that the module is not confident. By assigning weights to the outputs with the gating module probabilities, we add this information to the prediction. The figure 1 represents the weighted modular architecture. The discriminative module's prediction is as follows:

\[\arg\max_{i}P(S_{i}|X),\quad i=1,\ldots,3. \tag{5}\]

Figure 1: Architecture of weighted modular learning.
#### 3.2.1 Learning phase

In each module, the network is composed of a pretrained model ResNet18 and a fully connected layer. The architecture of ResNet18 is illustrated in figure 2. It is composed of 18 deep layers. Every four layers are similar, and each layer is composed of two blocks of deep residual network. The latter characterizes by adding an identity connection, meaning if the output of the layer is \(F(x)\), the output of the block is \(F(x)+x\) followed by the activation function ReLU. The residual network is an attempt to solve the problem of vanishing gradient, that is, when the gradient reaches zero quickly [10]. And the final layer consists of average pooling followed by a fully connected layer and a softmax.

### Experiments

#### 3.3.1 Dataset

The images used in this work were sampled from arthroscopic videos on Dorsal Capsulo-Scapholunate Septum (DCSS) performed at Avicenne Hospital. The collected data includes 840 photos in total, including 105 healthy DCSS images (12.5%), 173 (20,6%) in stage 1, 187 (22.2%) in stage 2, and 375 in stage 3 (44.64%). Entry from each patient was equally distributed. In order to maintain a balance between the classes, we have expanded the training set using image manipulations such as random rotations, adjusting the brightness, zooming in on the image, etc. After transformation, we were able to obtain 3076 healthy DCSS images, 1111 in stage 1, 1126 in stage 2, and 1279 in stage 3. The image's size was then resized to \(224\times 224\). The figure 3 presents samples from the dataset.

### Setting

We used 80% (6592) of the dataset for training and 20% (252) for testing. We used stochastic gradient descent to optimize the model and a learning rate of \(0.001\). We trained for 10 epochs. We used the same hyperparameters for both models. The first model was trained on all the training datasets labeled healthy, not healthy and the second model was trained on only non-healthy DCSS.

#### 3.4.1 Results and discussion

We evaluated the three types of modular learning using accuracy with an estimation of the confidence interval. The confidence interval was computed as explained in [4]. They assume that the probability distribution of the number of

Figure 3: Visualization examples of DCSS images.

Figure 2: ResNet18 architecture[17].
errors is binomial, so we calculate the interval according to the formula below

\[I_{\alpha}=\frac{P+\frac{Z_{\alpha}^{2}}{2N}\pm Z_{\alpha}\sqrt{\frac{P(1-P)}{N}+ \frac{Z_{\alpha}^{2}}{4N^{2}}}}{1+\frac{Z_{\alpha}^{2}}{N}}, \tag{6}\]

where \(N\) is the number of observations in the test set, \(P\) is the probability that an observation is an error, and, \(\alpha\%\) is the confidence level, usually set to \(95\%\), \(Z_{\alpha}\) is a value given by the normalized centered Gaussian distribution (\(Z_{95\%}=1.96\) and \(Z_{90\%}=2.48\)).

The table 1 summarizes the results obtained. As we can see in the table, modular learning increased the accuracy from \(97.6\%\) to \(98.4\%\) and a tighter confidence interval of \([97.79\%;100\%]\) with an almost perfect gating module. These results demonstrate that adding a priori knowledge from the modular nature of the problem allowed a better generalization and a finer decision boundary than the non-modular architecture. The decomposition of the task makes the problem less complex, thus a bigger probability to converge towards the ground truth function.

The experiments in 1, 2 also showed that weighted modular had better performance than the first case modular architecture with \(98.81\%\) accuracy. This indicates that adding probabilities from the gating module calibrates the results.

We investigated the distribution of the difference between the prediction probability of the different classes in the discriminative module. We can see in figure 4 that the median difference is almost one, which proves that the discriminative module is confident in its decision. We can also notice a high difference between the prediction probability of stage 2 and 3 meaning that the model is able to distinguish between the two stages confidently. However surprisingly, the probabilities between stages 1 and 3 are relatively small. By analyzing the images further, we notice the presence of redness in both stages, which could be the reason for such results.

Since the problem is medical, it is important to explain the choice made by the model. However, neural network is a black box. Therefore, we attempted to evaluate two methods to interpret the results using a heatmap on images. The pixels with the higher contributions to the neural network's prediction are located in hotter areas. The first method is called GradCam (gradient-weighted class activation mapping), which creates visual explanations [21]. Grad-CAM creates a coarse localization map that highlights key areas in the image by using the gradients of any target concept and flowing into the final convolutional layer. This makes it possible to visualize the results of various CNN model layers.

The second approach used for explainability is Sobol attribution method [8]. It relies on sensitivity analysis techniques commonly used in physics and economics, in particular the Sobol indices. The method, not only does it identify the contribution of each image regions to the neural network's decision but also, the sobol indices are able to estimate High-order interactions between the regions in the image and their contributions. To estimate the indices, the authors used Jansen estimator with Quasi-Monte Carlo for sampling. First, they sample masks from the sequence, apply them to the image with a perturbation operator, then obtain prediction scores by forwarding the obtained image to the system. The explanations are then computed using the masks and the prediction scores. In figure 5 and 6, we evaluate displays the visual output of the discriminative model of the two methods. We notice that the GradCam provides a continuous region, and we notice a certain pattern for each class, while the Sobol attribution method gives a finer heatmap. In both cases, both methods were able to locate the rip in the DCSS in Stage 3.

## 4 Proposed approaches: Self-training for data labeling and segmentation

### Motivation

The main objective of this section is to, first, label arthroscopy images using self-training and then detect and identify structures from arthroscopy videos with a modular architecture. We used videos of shoulder arthoscopy. In one second

\begin{table}
\begin{tabular}{|c||c|c|} \hline Architecture & Accuracy (\%) & Confidence interval(\%) \\ \hline Non-modular & 97.6 & [96.52 ; 100] \\ \hline Modular (All) & 98.4 & [97.79 ; 100] \\ \hline Modular (1 Vs 1) & 94.4 & [91.1 ; 100] \\ \hline Weighted modular & 98.81 & [98.4 ; 100] \\ \hline \end{tabular}
\end{table}
Table 1: Average accuracy on different modular architecture
of video, we can obtain from 25 to 30 images per second. Therefore, several minutes in a video can produce several images, thus labeling these data can be difficult and time-consuming. The idea is to select few pictures from each video and apply the self-training process for pseudo-labeling, that is, producing predicted labels from a small set of labeled data and adding them to the training dataset, of the rest of the video. This way, we exploit all the available data we have.

### System architecture

The model for the self-training (i.e. pseudo-labeler) is built using a segmentation network. It takes as input an arthroscopic image and its ground truth mask with 5 classes (background, long biceps tendon, subscapularis tendon, supraspinatus tendon, cartilage (glenoid and humeral)), and it outputs a prediction mask. We have found while training the segmentation task, that the model was able to perform a binary segmentation, meaning identifying the structure versus the background. However, it fails to discriminate between the different classes. That's what motivates the use of

\begin{table}
\begin{tabular}{|l|r|r|r|r|r|} \hline  & \multicolumn{4}{|c|}{Predicted result} & \multicolumn{1}{c|}{Recall (\%)} \\ \hline  & Healthy & Stage 1 & Stage 2 & Stage 3 & \\ \hline Ground truth & Healthy & \(30\) & \(0\) & \(0\) & \(0\) & \(100\) \\ \cline{2-6}  & Stage 1 & \(2\) & \(60\) & \(1\) & \(0\) & \(95.23\) \\ \cline{2-6}  & Stage 2 & \(0\) & \(0\) & \(62\) & \(0\) & \(100\) \\ \cline{2-6}  & Stage 3 & \(0\) & \(0\) & \(0\) & \(97\) & \(100\) \\ \hline Precision (\%) & & \(93.75\) & \(100\) & \(98.41\) & \(100\) & \\ \hline \end{tabular}
\end{table}
Table 2: Evaluation matrix of weighted modular on test set

Figure 4: Distribution of difference in prediction.

Figure 5: Interpretation of model using GradCam.
structure. Then, we remove the predicted background from the initial image and feed it to the second network. The latter is trained to learn to discriminate between the four classes.

The self-training phase, consists in, first, training the two models with the available annotated images. Then, we predict the mask for all the images in the unlabeled set. Afterward, we select only the images that produced a mean prediction probability superior to a threshold. We set another threshold to filter the selected outputs, pixel by pixel, according to their prediction probability. The experiments have shown that adding the second filter to the predicted images increases the accuracy. Let \(\hat{y}_{m}\) be a predicted image from the model. \(\hat{y}_{m_{i,j}}\) represents the pixel at position \((i,j)\). Let \(T\in[0,1]\) be the threshold and the Softmax function is defined as \(S:\mathbb{R}^{K}\rightarrow(0,1)^{K}\) where \(K\geq 1\); \(S(\mathbf{x}_{i})=\frac{e^{x_{i}}}{\sum_{j=1}^{K}e^{x_{j}}}\) for \(i=1,\ldots,K\) and \(\mathbf{x}=(x_{1},\ldots,x_{K})\in\mathbb{R}^{K}\). The image is filtered as follows;

\[\hat{y}_{m_{i,j}}=\begin{cases}\hat{y}_{m_{i,j}},&\text{if }\arg\max_{(i,j)}S( \hat{y}_{m_{i,j}})\geq T\\ 0,&\text{otherwise.}\end{cases} \tag{7}\]

We add the final outputs to the training set and reiterate until reaching the stopping criteria; either all the images are annotated or the algorithm reached the maximum number of iteration.

Let \(P\in[0,1]\) be a threshold, then in each iteration the set of selected images is as following:

\[\hat{I}\coloneqq\{\hat{f}_{\mathbf{w}^{(\ell)}}(\mathbf{x}_{m})\mid\arg\max_{\mathbf{x}_{ m}}S(\hat{f}_{\mathbf{w}^{(\ell)}}(\mathbf{x}_{m}))\geq P\} \tag{8}\]

and the output prediction is defined as follows:

\[\hat{y}_{m}=\delta_{\arg\max_{(i,j)}S(I_{m_{i,j}})\geq T}\hat{I}_{m};\quad 0 \leq m\leq n(\hat{I}), \tag{9}\]

where \(n(\hat{I})\) is the cardinality of the set \(\hat{I}\).

#### 4.2.1 Learning phase

The architecture of each module is inspired by UNet [19]. The choice of this particular pretrained model is motivated, first, by its common use in medical applications. The second key is its ability to perform well with few labeled images [19]. The architecture of UNet is U-shaped. It consists of an encoder (contracting path) and a symmetric decoder (expanding path). The contracting path consists of 2 blocks of \(3\times 3\) convolution followed by the activation function rectified linear unit (ReLU) then a \(2\times 2\) max pooling. These operations are repeated five times. The number of channels doubles in each time (64, 128, 256, 512, 1024). The expanding path consists of the same blocks and ReLU and uses \(2\times 2\) for the up convolution. The final layer is a \(1\times 1\) convolution. The figure 7 displays the UNet architecture.

Figure 6: Interpretation of model using Sobol attribution method.
### Experiments

#### 4.3.1 Dataset

The data of our second study were collected by video recording of shoulder arthroscopies performed at Avicenne Hospital between November 2021 and June 2022. The videos of arthroscopies could be retrieved directly from the arthroscopy video console in mp4 format. From these arthroscopies, a total of 459 images were extracted (121 of long biceps tendon, 41 of supraspinatus tendon, 181 of subscapularis tendon, and 116 of glenoid or humeral cartilage), and saved in png format. The images were annotated by a surgeon using the software "LabelStudio". Labeling was made using polygon annotations. The latter ensures a more precise annotation for irregular shapes. Figure 8 presents examples of shoulder arthroscopy images with their segmentation.

### Setting

Data was split into \(70\%\) train set (321 images), and \(30\%\) test set (138 images). We used \(10\%\) of the training set (32 images) as labeled data to train the first model in self-training and we supposed the \(90\%\) of the training set (289 images) unlabeled data. The final evaluation was made on the test set. The images were resized to \(224\times 224\). We used an Adam optimizer and 15 number of epoch. After experiments, we set the threshold to \(0.7\).

Figure 8: Visualization examples of segmentation on shoulder arthroscopy image. Each color corresponds to the classes accordingly; red: Biceps, blue: Cartilage.green: Subscapularis, yellow: Supraspinatus.

Figure 7: UNet architecture[19].
#### 4.4.1 Results and discussion

We use a threshold on the prediction probability to choose from the unlabeled subset, and the samples to add to the training set (improved accuracy of self-training from 76% to 80%). We varied the threshold to analyze its effect on performance. We reported the results on the test set. As shown in table 3, increasing the threshold improves the accuracy. However, for the threshold \(0.9\), the performance degrades. One reason can be that the strict selection of samples engenders the model to train on the same samples, therefore overfits. The optimal threshold value is \(0.7\) with an accuracy of \(80\%\). We also show that the class Supraspinatus, presented in yellow in figure 8 has the highest accuracy compared to other classes. We found that adding the pixel-by-pixel threshold, explained in 4.2, improved the accuracy (from \(76\%\) to \(80\%\)). We observed the effect of the ratio of the labeled data in the first iteration on the performance in figure 9. We notice that the greater percentage, the higher accuracy. We choose the ratio \(10\%\). Samples of the segmentation predictions on the test set are shown in figure 10.
Conclusion

In this paper, we first introduce a modular framework for Dorsal Capsulo-Scapholunate Septum classification. We have shown that decomposing the problem into a gating module and discriminative module improved the performance. Moreover, including the contribution of the gating module as weights, achieved the best results compared to the different strategies tested. Additionally, we presented a modular self-training approach for data labeling and segmentation. Our results validated the improvement of results by using self-training. However, while modular learning outperformed non-modular systems in the first problem, it was not the case for the second problem. The reason may be related to the strategy of dividing the problem. In the classification problem, we used a priori knowledge on the classification subject, contrary to the segmentation problem.

In conclusion, modular learning is an efficient approach to reduce the complexity of a model and improve model generalization. However, it is crucial to manage when and where to divide a problem. In future work, a different strategy should be developed and further analyzed for modular self-training segmentation. Moreover, propose a new measure of feature selection for interpretability on convolutional neural networks.
