Real-Time Damage Detection in Fiber Lifting Ropes Using Convolutional
  Neural Networks
The health and safety hazards posed by worn crane lifting ropes mandate
periodic inspection for damage. This task is time-consuming, prone to human
error, halts operation, and may result in the premature disposal of ropes.
Therefore, we propose using deep learning and computer vision methods to
automate the process of detecting damaged ropes. Specifically, we present a
novel vision-based system for detecting damage in synthetic fiber rope images
using convolutional neural networks (CNN). We use a camera-based apparatus to
photograph the lifting rope's surface, while in operation, and capture the
progressive wear-and-tear as well as the more significant degradation in the
rope's health state. Experts from Konecranes annotate the collected images in
accordance with the rope's condition; normal or damaged. Then, we pre-process
the images, design a CNN model in a systematic manner, evaluate its detection
and prediction performance, analyze its computational complexity, and compare
it with various other models. Experimental results show the proposed model
outperforms other techniques with 96.4% accuracy, 95.8% precision, 97.2%
recall, 96.5% F1-score, and 99.2% AUC. Besides, they demonstrate the model's
real-time operation, low memory footprint, robustness to various environmental
and operational conditions, and adequacy for deployment in industrial systems.
Tuomas Jalonen, Mohammad Al-Sa'd, Roope Mellanen, Serkan Kiranyaz, Moncef Gabbouj
# Real-Time Damage Detection in Fiber Lifting

###### Abstract

The health and safety hazards posed by worn crane lifting ropes mandate periodic inspection for damage. This task is time-consuming, prone to human error, halts operation, and may result in the premature disposal of ropes. Therefore, we propose using deep learning and computer vision methods to automate the process of detecting damaged ropes. Specifically, we present a novel vision-based system for detecting damage in synthetic fiber rope images using convolutional neural networks (CNN). We use a camera-based apparatus to photograph the lifting rope's surface, while in operation, and capture the progressive wear-and-tear as well as the more significant degradation in the rope's health state. Experts from Konecrames annotate the collected images in accordance with the rope's condition; normal or damaged. Then, we pre-proves the images, design a CNN model in a systematic manner, evaluate its detection and prediction performance, analyze its computational complexity, and compare it with various other models. Experimental results show the proposed model outperforms other techniques with 96.4% accuracy, 95.8% precision, 97.2% recall, 96.5% F1-score, and 99.2% AUC. Besides, they demonstrate the model's real-time operation, low memory footprint, robustness to various environmental and operational conditions, and adequacy for deployment in industrial systems.

 Computer vision, damage detection, deep learning, fiber rope, industrial safety.

## I Introduction

Traditional industries are transitioning to smart manufacturing under the Industry 4.0 paradigm [1, 2]. This transition allows the use of recent advances in artificial intelligence and computer vision to increase productivity and improve manufacturing safety [3, 4]. Nonetheless, lifting heavy payloads is still a major health and safety hazard in many manufacturing environments [5]. For example, cranes such as the one shown in Fig. 1 can move objects weighing several metric tons; hence, inspecting its ropes for damage is paramount to prevent serious accidents, injuries, and additional costs [6]. More specifically, lifting ropes are major points of failure that require periodic inspection and replacement [8]. However, manual inspection procedures are labor intensive, time consuming, subjective, and often require halting the production process [9, 10, 11]. Therefore, we propose a novel real-time damage detection system for synthetic fiber lifting ropes based on deep learning and computer vision techniques.

The main contributions of this paper are:

* Developing the first fiber lifting rope image dataset.
* Proposing a deep learning based system for detecting damage in fiber lifting rope images.
* Designing an industrial solution that achieves high performance with a light memory footprint.

Lifting ropes are commonly manufactured from steel wires or lately using synthetic fibers such as polyethylene [12]. Synthetic lifting ropes have many benefits over traditional steel wire ropes. For example, they demonstrate higher resistance to corrosion, do not require greasing, and are easier to install [13]. Moreover, despite their higher purchase price, synthetic fiber ropes are lightweight which allows utilizing smaller cranes; leading to cost reductions [14]. However, synthetic ropes, just as steel wires, do suffer from wear-and-tear and get damaged over time. Common damage types in fiber ropes are: strand cuts, abrasion, melting, compression damage, pulled strands and inconsistent diameter. In contrast to steel wires which tend to break from the inside [15], synthetic rope damages manifest on the rope's surface and can be visually inspected by an expert [14]. Currently, monitoring the condition of synthetic fiber ropes is performed manually by inspectors following the ISO-9554 standard [9, 10]. Although it is the standard practice, this procedure is cumbersome, discontinuous in time, interrupts operation, and may result in the premature disposal of ropes [11]. Therefore, automatic

Fig. 1: The fiber rope cranes used in this work. Photos are published with permission from Konecrames [7].
damage detection by leveraging the recent advancements in computer vision, image processing, and deep learning techniques is needed [14]. On the one hand, image processing and related feature extraction methods utilize expert knowledge and attempt to characterize damage in rope images similar to the ones identified by expert inspectors [16]. These techniques generally perform well in a controlled environment, but they poorly integrate the varying conditions and operations found in a real-life setting e.g., noise, lighting conditions, oil residue, and dust [17, 18]. On the other hand, deep learning tools discard the notion of hand-crafted features by learning abstractions that maximize the detection of damaged ropes. In fact, they yield discriminatory features without predisposition to the standard markings and can accommodate a wider range of environmental and/or operational conditions [17]. Therefore, deep learning techniques are more suited to detect damage in synthetic fiber rope images compared to engineering-based feature extraction methods.

The construct of damage indicators in fiber ropes was first articulated in [11] where changes in the rope's width and length were found to be important. This particular finding was verified in [19] using computer vision and thermal imaging; however, explicit identification for damaged ropes was not performed. In fact, detecting damage in synthetic fiber rope images has received less attention in the literature compared to steel wire cables. Fortunately, these detection techniques are suitable for adoption due to the similarity between the two problems; they both deal with detecting damaged yarns or strands in rope images. For instance, the health condition of balancing tail ropes was monitored in [20] using a convolutional neural network (CNN). The rope image was captured and then fed to the CNN model to classify its health state as either normal, or if it suffers from one out of eight common damage types. Moreover, a CNN-based approach was designed in [21] to detect surface defects in steel wire rope images. The model classified the acquired images into normal, broken, or damaged, and achieved a 99.7% overall accuracy. The same problem was tackled in [22] using support vector machines trained with texture-based hand-crafted features. The proposed system achieved a 93.3% classification accuracy and it was further improved in [23] to reach 95.9%. Nonetheless, the limited sample size and the reliance on hand-crafted features hampered robustness in noisy environments. This was evidenced in [24] which showed that the model accuracy drops to 80.5% when training/testing with a different dataset. Moreover, the utility of CNNs combined with image processing techniques was shown to increase the accuracy of the model in [22] from 93.3% to 95.5% [18]. This has motivated us to design a CNN-based solution for detecting damage in synthetic fiber rope images. However, our proposed solution will be developed to have both high performance and low computational requirements, allowing for easy integration into industrial systems and efficient deployment [25].

The rest of this paper is organized as follows: section II describes our methodology for building the experimental setup, collecting data, designing the damage detection system, and evaluating its performance. Afterwards, we present and discuss the system's performance and compare it to various other models in section III. Finally, section IV concludes the paper and suggests topics for future research.

## II Methodology

The proposed fiber rope damage detection system is overviewed in Fig. 2 and consists of the following stages:

1. Setup an experimental apparatus with a three-camera circular array to photograph the ropes' surface area.
2. Collect the captured images and label them as normal or damaged according to the ropes' health condition.
3. Preprocess the collected images to enhance contrast and down-sample to reduce computational complexity.
4. Split the pre-processed images into train and test sets in a 5-fold cross-validation fashion.
5. Train and test a classification model on the train and test folds, respectively, and repeat for every data split.
6. Evaluate and analyze the model's fold-averaged performance using various metrics.

The design process undertaken in this work is governed by the following requirements:

* High performance in detecting damaged ropes and robustness to different environmental and operational conditions.
* Lightweight for implementation and deployment.
* Remote sensing by neither interfering with the crane operation nor the rope structure.
* Modularity to facilitate maintenance, upgrades, and compatibility with IoT and edge devices.

The remaining subsections discuss and detail each stage in the proposed system.

### _Experimental setup_

The experimental setup was built and operated by Konecranes and the following experiment was repeated for three different synthetic fiber ropes; see Table I for the ropes' properties. First, the crane illustrated in Fig. 1 was set to continuously lift a payload of 5 metric tons in a controlled setting. The payload lifting height was approximately 5 meters and during a lifting cycle, the crane was stopped at the top and bottom (payload resting on the floor). After that, a circular camera array, comprised of three RGB cameras placed at 120deg apart, was used to capture approximately 13 mm of the lifting rope. The camera framerate was selected such that subsequent rope images have roughly 1/3 spatial overlap, resulting in 20 meters of rope being photographed during a lifting cycle. Finally, the crane lifting and rope imaging steps were repeated for weeks to cover the ropes' lifespan; from new to unusable.

### _Data collection_

The rope imaging experiment generated 4,984,000 high-resolution photos; each being tagged with a timestamp and the rope's imaged position. The raw photos were then screened for duplicates by discarding images that examined the same rope position. In other words, we ensured that images for the same rope position would be distinct by capturing different health
conditions. This is important to avoid cross-contamination in data splits. After that, we selected 143,000 random samples and experts from Konecranes labeled them as _normal_ or _damaged_ according to the lifting rope health condition. Out of those images, 10,000 samples were labeled as damaged. Finally, to avoid data imbalance issues, we formed a balanced dataset containing 20,000 samples; 10,000 images from each class. The collected dataset is available from Konecranes and it was used under license for this study1.

Footnote 1: Please contact Roope Mellanen at roope.mellanen@konecranes.com for data inquiry.

### _Preprocessing and data splitting_

The annotated high-resolution rope images were down-sampled to \(256\times 256\times 3\) pixels. After that, we enhanced the photos' contrast via histogram equalization [26]; see Fig. 3 for a sample, and we standardized the pixel values to range between 0 and 1. Finally, the pre-processed images were randomly divided into five equally sized portions while maintaining class balance (5-fold stratified cross-validation); see stage 4 in Fig. 2. In other words, each split had 16,000 (8,000 damaged and 8,000 normal) and 4,000 (2,000 damaged and 2,000 normal) images for training and testing, respectively. Fig. 4 demonstrates samples of normal and damaged ropes from the acquired dataset. By examining the images, one notes a significant variation in the clarity of the rope's health state and in the severity of the damage. For example, Fig. (a)a conveys a more damaged rope when compared to the one presented in Fig. (e)e. However, the damage can also be minuscule without clear visual indications as presented in Fig. (d)d. Finally, the dirt and oil stains found in most rope images present a challenge for any vision-based tool.

### _Proposed deep learning model_

The collected rope images constitute an over-complete description for the rope as a whole; hence, the problem of damage detection reduces to classifying each image independently. We designed a lightweight CNN architecture to classify the fiber lifting rope images, and we tested different variants to find the best performing model.

The architecture design starts with a convolutional layer (\(3\times 3\) kernel with ReLU activation) to extract preliminary feature maps from the input images. After that, those initial features are passed through a number of blocks each consisting of the following sequential elements: (1) convolutional layer to

Fig. 3: Histogram equalization for an example rope image.

Fig. 2: The proposed vision-based damage detection system for synthetic fiber lifting ropes. The system is comprised of the following stages: (1) experimental setup with a three-camera circular array to capture rope images; (2) collection and annotation of the captured images; (3) preprocessing to enhance quality and down-sampling to reduce complexity; (4) data splitting into 5-fold training and testing sets; (5) training/testing the proposed deep learning model; and (6) evaluating and analyzing the system’s performance and computational complexity.
extract features (\(3\times 3\) kernel with ReLU activation), (2) Max Pooling to down-sample the features (\(2\times 2\) kernel), (3) and dropout to regularize the network by reducing the neurons' interdependent learning (\(0.4\) rate). Finally, the learned abstractions are flattened and passed through a dropout layer (\(0.4\) rate), a fully connected layer (\(20\) nodes), another dropout layer (\(0.2\) rate), and lastly, a binary classification layer with a Softmax activation function. In this work, 16 model variants were generated from this architecture by altering the number of blocks (1, 2, or 3), input image sizes (\(16\times 16\), \(32\times 32\), or \(64\times 64\)), and the input image color state (color or grayscale); see Appendix A for the model variants' structure and Table II for details on the variant that we selected for further analysis and comparison.

The models' training was performed for 200 epochs using an Adam optimizer [27] to minimize the cross-entropy loss regularized by a weight decay to reduce overfitting [28, 29], i.e.:

\[\mathcal{L}=-y\log(\hat{y})-(1-y)\log(1-\hat{y})+\lambda||\mathbf{w}||_{2}^{2 }\,, \tag{1}\]

where \(\mathcal{L}\) denotes the regularized loss, \(y\) and \(\hat{y}\) are the true and predicted labels, respectively, \(\lambda=5\times 10^{-4}\) is the selected \(L_{2}\) regularization rate, and \(\mathbf{w}\) is the network's weight matrix [28]. Moreover, the training batch size was set to 32 and to ensure convergence the learning rate was decayed by [30]:

\[\eta(n)=\begin{cases}10^{-3}&:n\leq 120\\ 10^{-4}&:120<n\leq 150\\ 10^{-5}&:150<n\leq 180\\ 10^{-6}&:180<n\leq 200\end{cases}\,, \tag{2}\]

where \(\eta\) is the learning rate and \(n\) is the epoch number.

This training process was conducted for each model using the train set in each data split (five training sets). Additionally, apart from the generated variants, we also trained the following three baseline models for comparison; Zhou _et al._ (2019) [21, Zhou _et al._ (2021) [18], and Schuler _et al._ (2022) [31]. Detailed description on these models can be found in Appendixes B-D.

### _Performance evaluation and analysis_

The trained models were evaluated using the test set in each data split (five test sets). Their performance was analyzed by various tools and metrics to quantify their fold-averaged detection, prediction, and misclassification outcomes.

Fig. 4: Example images from the acquired dataset show significant variation in the severity and clarity of damages because of dirt and oil stains. The first row (a)-(e) show damaged ropes while the second row (f)-(j) present some healthy samples.
#### Ii-B1 Classification

we quantified the models' classification performance by accuracy, precision, recall, false positive rate (FPR), and the F1-score, i.e.:

\[\text{Accuracy}=\frac{TP+TN}{TP+TN+FP+FN}\,, \tag{3}\]

\[\text{Precision}=\frac{TP}{TP+FP}\,, \tag{4}\]

\[\text{Recall}=\frac{TP}{TP+FN}\,, \tag{5}\]

\[\text{FPR}=\frac{FP}{FP+TN}\,, \tag{6}\]

\[\text{F1-score}=2\left(\frac{\text{Precision}\times\text{Recall}}{\text{ Precision}+\text{Recall}}\right)\,, \tag{7}\]

where \(TP\), \(TN\), \(FP\), and \(FN\) are true positives, true negatives, false positives, and false negatives, respectively (positive/negative denotes a damaged/normal rope).

Moreover, we used the area under the averaged receiver operating curve (AUC) and confusion matrices to fully characterize the classification quality. The AUC was computed by averaging linearly interpolated receiver operating curves.

#### Ii-B2 Prediction

we assessed the models' predictive capacity using Gradient-weighted Class Activation Mapping (Grad-CAM) which uses gradients of the last convolutional layer to measure the relevance of the input image pixels for classification [32]. Specifically, Grad-CAM yields a distribution with high values for pixels that contributed more to the outcome.

Furthermore, we utilized t-Distributed Stochastic Neighbor Embedding (t-SNE); a dimensionality reduction method that clusters similar high dimensional samples and departs dissimilar ones in two- or three-dimensional space [33]. In specific, given an array of learned features \(\mathbf{x}=[\mathbf{x}_{1},\mathbf{x}_{2},\cdots,\mathbf{x}_{N}]\), the similarity between features \(i\) and \(j\) can be measured by:

\[p_{ij}=\frac{p_{j|i}+p_{i|j}}{2N}\,, \tag{8}\]

\[p_{j|i}=\left\{\begin{array}{ll}\frac{\exp(-||\mathbf{x}_{i}-\mathbf{x}_{j} ||^{2}/2\sigma_{i}^{2})}{\sum_{k\neq i}\exp(-||\mathbf{x}_{i}-\mathbf{x}_{k}|| ^{2}/2\sigma_{i}^{2})}&:i\neq j\\ 0&:i=j\end{array}\right., \tag{9}\]

where \(p_{ij}\) is a probabilistic measure for the similarity between \(\mathbf{x}_{i}\) and \(\mathbf{x}_{j}\), \(\sum_{i,j}p_{ij}=1\), \(\sum_{j}p_{j|i}=1\), and \(\sigma_{i}\) is the adaptive Gaussian kernel bandwidth. Now, t-SNE aims to learn the two- or three-dimensional map \(\mathbf{y}=[\mathbf{y}_{1},\mathbf{y}_{2},\cdots,\mathbf{y}_{N}]\) with a probabilistic similarity \(q_{ij}\) that resembles \(p_{ij}\), i.e. [33]:

\[q_{ij}=\frac{(1+||\mathbf{y}_{i}-\mathbf{y}_{j}||^{2})^{-1}}{\sum_{k\neq l}(1+ ||\mathbf{y}_{k}-\mathbf{y}_{l}||^{2})^{-1}}\,. \tag{10}\]

The similarity matching between \(q_{ij}\) and \(p_{ij}\) in t-SNE is maximized by minimizing the Kullback-Leibler divergence of \(p_{ij}\) from \(q_{ij}\) via gradient descent, i.e.:

\[\min_{\mathbf{y}_{i}}\left(\sum_{i\neq j}p_{ij}\log\left(\frac{p_{ij}}{q_{ij} }\right)\right)\,. \tag{11}\]

Both the Grad-CAM and t-SNE help in characterizing the models' predictive power when supplied with new data. In other words, given large enough training samples, if the Grad-CAM and t-SNE show genuine learning and clear separability, one may infer the model adequacy for unseen samples.

#### Ii-B3 Misclassification

visualizing the model's misclassified samples is paramount for interpretability and for outlining performance caps. Moreover, it enables a better understanding for the model's weaknesses and for identifying human errors in annotation. For example, by assuming some error in the labeling process, the performance of a genuine model will be limited, or capped, by the labels' quality.

### _Computational complexity_

The complexity of the models was assessed by their total number of trainable parameters, required input image size, the models' memory size requirement, processing time, and their processing rate (frame rate). We analyzed their computational complexity by Monte-Carlo simulations where we fed the models with 4,000 test samples, predicted their health state (normal or damaged), and repeated the process ten times for validation. Note that this process does not include the imaging, data loading, nor preprocessing stages. It only quantifies the models' inference complexity. We used an Apple MacBook Pro with an ARM-based M1 Pro chip, 10-core CPU, integrated 16-core GPU, 16-core neural engine, and 16 GB of RAM. The experiments' codes were written in Python 3.9.7 using Tensorflow 2.7.0 and are publicly available.

## III Results and discussion

### _Model selection_

The best model, out of the 16 generated variants, was selected based on its ability to balance between precision and recall with minimum computational requirements. By examining the results in Table III, one notes that the model variants CNN9 and CNN15 yield the highest 5-fold averaged precision/recall balance (96.5% average). Besides, they are the top-2 models in terms of accuracy, precision, F1-score, and AUC. Nevertheless, due to the apparent disparity in computational resources (input image sizes: \(32\times 32\times 3\) v.s. \(64\times 64\times 3\)), we opted for model CNN9 and used it for further analysis and comparison. Note that "Proposed CNN" refers to the CNN9 variant in the remaining of this paper.

### _Performance analysis_

Fig. 5 compares the training and testing accuracy/loss curves (averaged over the data splits) of the proposed CNN to the three baseline models. The results show the curves converging successfully after epoch 150, and the learning rate decay, scheduled at epoch 151, has ensured stability by suppressing perturbation. This indicates that the training phase was executed long enough and was not terminated prematurely. Moreover, by examining the difference between the training and testing accuracy/loss curves one notes that the three baseline models exhibit further overfitting when compared to the proposed network, with Schuler _et al._ (2022) being the extreme case, followed by Zhou _et al._ (2019), and lastly, Zhou
et al._ (2021). We argue that this is caused by the unnecessary high number of parameters which hinder generalization.

The models' testing performance for detecting damaged rope images is demonstrated in Table IV and Fig. 6. The measures in Table IV show that the Zhou _et al._ (2021) model has the highest recall at 97.9%, which is the most important metric in order to prevent accidents. However, its precision, at 90.3%, is relatively low suggesting that the model labels most rope images as damaged, making it unpractical for a real-life setting. Moreover, both the Schuler _et al._ (2022) and the proposed models yield similar recall levels (97.1% and 97.2%, respectively). Nevertheless, the proposed CNN results in higher accuracy, precision, and F1-score with 2.3, 4.2, and 2.2 percentage point gains, respectively. In addition, it demonstrates a better precision/recall trade-off, which is reflected by the averaged ROC curve in Fig. 6 along with its AUC value (99.2%). Finally, the results suggest that the Zhou _et al._ (2019) and Schuler _et al._ (2022) models are not the best in any of the six metrics.

### _Computational complexity analysis_

Table V summarizes the complexity analysis results which indicate that the proposed model is the fastest one requiring approximately 19 milliseconds per input image and running in real-time at 54 fps. Nonetheless, it is important to note that Zhou _et al._ (2019) and (2021) models are comparatively fast, but the Schuler _et al._ (2022) operates below real-time at 12 fps. Besides, the listed prediction speeds could be further improved by running them in C++. However, the most notable and important difference is the proposed model's lighter memory footprint. In specific, our model accepts small-sized images and requires less disk space for storage. These advantages can lead to savings in equipment and operational costs, improve latency, and mitigate privacy issues by not using cloud services.

### _The Grad-CAM and t-SNE analysis_

The Grad-CAM and t-SNE results are depicted in Figs. 7 and 8, respectively. In Fig. 7, we generated the Grad-CAM heatmap for two example images showing damaged ropes that were correctly classified by the proposed model. The results show that our CNN model is indeed focusing on the intuitively relevant parts of the input image, which are the broken strands. One also notes that the network does not focus on the ropes' oil and dirt residue as demonstrated in Fig. 6(b). This suggests genuine learning by the model and robustness to environmental

Fig. 5: Comparing the models’ training and testing accuracy/loss curves averaged over the data splits in solid/dotted lines.

Fig. 6: The models’ 5-fold averaged ROC curves alongside their computed AUC values. The corner portion is magnified to ease visualization.
and operational conditions. However, it is important to note that the CNN still slightly focuses on the image background. Moreover, the t-SNE results in Fig. 8 demonstrate good class separation, but they also show a need for verifying some ground-truth labels. In specific, the t-SNE shows few rope images labeled as normal within the damaged rope support.

### _Misclassifications_

Two example prediction errors by our CNN are presented in Fig. 9. The rope in Fig. 9a is labeled as damaged, but predicted as normal, while the one in Fig. 9b is labeled as normal, but predicted as damaged. Such instances pose a challenge for the system because they are clearly in between the two classes; they are slightly worn with a few broken strings, but strictly not damaged, according to our experts. Despite that, these ropes are not likely to break at these spots and they would be classified as damaged after more wear. Moreover, the similarities between the two images suggest

Fig. 8: The proposed CNN t-SNE results showing good separation between the two classes. We used the model’s last layer features (FC[20]) in Table A.1) as input to the t-SNE algorithm.

Fig. 7: The proposed CNN Grad-CAM heatmaps for two correctly classified damaged ropes. The heatmaps indicate the model’s adequacy by focusing on the pixels that are relevant for detecting damage. We used the model’s last convolutional layer features as input to the Grad-Cam algorithm.
possible annotation errors which may prevent the proposed model from reaching its full potential. Nonetheless, human errors are expected, and the model outcome still shows good potential and applicability.

## IV Conclusions

Damaged lifting ropes are a major safety hazard in manufacturing, cargo loading/unloading, and construction because they can lead to serious accidents, injuries, and financial losses. The visual inspection of damage in synthetic lifting ropes is a time-consuming task, interrupts operation, and may result in the premature disposal of ropes. Therefore, combining computer vision and deep learning techniques is intuitive for automation and advancement.

This work presents a novel vision-based deep learning solution for detecting damage in fiber lifting rope images. First, we built a three-camera circular array apparatus to photograph the rope's surface. Afterward, the rope surface images were collected in a database, annotated by experts, preprocessed to improve contrast, and split into train and test sets in a 5-fold cross-validation fashion. Moreover, we systematically designed a CNN-based model to classify damaged rope images, evaluated its detection and prediction performance using various tools, and compared it to three different baseline models. Additionally, we analyzed its computational complexity in terms of processing time and memory footprint. In summary, the results indicated various performance and computational advantages for using the proposed system when compared to similar solutions. Specifically, the system testing yielded 96.4% accuracy, 95.8% precision, 97.2% recall, 96.5% F1-score, 99.2% AUC, and a significant generalization capability. Besides, it runs at 54 fps, occupies 1.7 MB in memory, and requires low-resolution input images; thus, making the proposed system a real-time lightweight solution. The developed system was also found robust to various environmental and operational conditions e.g., oil residue and dust.

The proposed model's main drawback is not determining the rope health state as a whole but assessing each image individually. Besides, the system's output is binary and does not directly communicate the rope's health condition. These limitations propose extending the developed solution in various ways such as: (1) collecting a larger training dataset with different rope sizes and types to improve generalization; (2) investigating other machine learning solutions and techniques; (3) including the cost of the imaging apparatus in the design process, e.g., using cheaper cameras; (4) extending the model's output to multiple classes e.g., normal, worn, and damaged, or to a continuous score indicating the rope's health condition (regression); and (5) incorporating the proposed solution to automate or recommend spare-part ordering.

## Acknowledgment

We would like to thank Konecranes and Juhani Kerovuori for their collaboration on this project.

## Appendix A The proposed model variants

Table A.1 summarizes the proposed CNN model variants' architecture for implementation. The variants were generated by altering the number of blocks, input image sizes, and input image color state.

## Appendix B The Zhou et al. (2019) and (2021) models

The proposed damage detection solution was compared to the Zhou _et al._ (2019) [21] and Zhou _et al._ (2021)2[18] models in terms of performance and computational requirements. These models were originally designed to detect surface damage in steel wire rope images with high performance. Although the rope material is different from our experiments (steel v.s. fiber), these detection models are still suitable for adoption due to the similarity between the two problems; they both deal with detecting damaged yarns or strands in rope images. The Zhou _et al._ (2019) and (2021) architectures accept grayscale input images of size \(64\times 64\) and \(96\times 96\), and produce outputs of size 3 and 2, respectively. In this work, we changed the Zhou _et al._ (2019) output shape to 2 in order to match our problem definition, and we added six dropout layers (\(0.5\) rate) to mitigate overfitting. Moreover, we increased the Zhou _et al._ (2021) original two dropout rates to 0.6 and added three more dropout layers to avoid overfitting.

Footnote 2: The adopted Zhou _et al._ (2021) model is named WRIPDCNN1 in [18].

## Appendix C The Schuler et al. (2022) model

The proposed damage detection solution was also compared to the Schuler _et al._ (2022)3[31] model; a highly efficient CNN-based classifier. The Schuler _et al._ (2022) model was designed in a clever manner to reduce the number of required parameters while maintaining high performance; the accuracy drop was 2% for a 55% reduction in parameters when tested on the CIFAR-10 dataset [31]. Therefore, we opted for this architecture for comparison because it is aligned with our design requirements; high efficiency and performance. The Schuler _et al._ (2022) model accepts colored input images of size \(32\times 32\times 3\), and in this work, we used its original implementation. Nonetheless, we reduced its output size from 10 to 2 in order to match our problem definition, and added dropout layers (\(0.4\) rate) to minimize overfitting.

Fig. 9: Two example misclassification samples by the proposed CNN.
## Appendix D Baseline training and evaluation settings

The baseline models' training followed the same procedure described in section II-D using the train set in each data split (five training sets), but with no weight decay. Besides, their performance was evaluated using the test set in each data split (five test sets) and quantified by the measures described in section II-E. Note that the Schuler model has caused out-of-memory issues when trained with a batch size above 32 using the equipment described in section II-F.
