 all effects applied together. Experiments were performed on a desktop PC with a processor Intel Core i5 760, 8 GB of operating memory, and the operating system Windows 10, and the image-generating algorithm was implemented using the functions of OpenCV 3.4.3.
Processing time was computed only from frames that contained identified markers and followed after another frame with identified markers. This procedure provided better results regarding the performance of algorithms that utilize data from previous frames, because these algorithms are often optimized for such situations. If there were no such frames, the result was missing, as is visible in the graphs.
The first experiment evaluated the basic performance of marker detectors in very good visual conditions to obtain a reference for other results. In this test, fog effect, glow effect, perspective effect, and sensor noise effect were not applied, the intensity of the background was set to 150, the intensities of black and white marker colors were set to 0 and 255, respectively, and the scale of the markers was a variable parameter in this experiment, changing from 5–30 pixels. The intensity of the background was chosen to be a bit lighter that half-tone intensity to resemble real images, but did not influence the results, as observed in preliminary tests.
Each value of scale was tested in 20 videos consisting of 100 frames, and the results (percentage of detected markers and detection time) were averaged and are presented in Figure 8. It is shown that good visibility conditions were not a problem for any of the solutions, although some solutions had problems with specific scales of markers. The ARUco detector started to lose some markers when the scale of the marker was higher than 23 (the same value as the maximum window size of its adaptive thresholding algorithm). When the scale was larger, the thresholded image contained a fictional contour inside the marker border, and in some cases, this contour was chosen as the contour of the marker, which made the detected marker smaller and often not identified. This behavior was emphasized by some image-improving algorithms.
The Normal version of ARUco3 struggled to find small markers. Further tests found that this was caused by the white border around markers, which was too small, but when its size was increased, the number of detected markers reached 100%. The processing times show that the Fast and VideoFast variants of ARUco3 were clearly the fastest algorithms for the detection of markers in these very good visual conditions. Normal version of ARUco3 and ARUco provided the results at reasonable speeds, while the speed of UWARUco and ARUco combined with image-improving algorithms was lower, since these algorithms contained additional processing steps. The speed of AprilTag2 was also reasonably low in this test.
None of the solutions had problems with precision of detection, although this precision decreased with larger markers in the case of ARUco and UWARUco and their variants, because these detectors chose the wrong contours in some cases. However, even in these cases, the corners of markers were found with a precision of units of pixels.
The task here was to simulate bad visibility conditions that cause lower contrast of the image. Glow and perspective effects were not applied; the intensity of the background was again set to 150; and the scale of all markers was set to 15 to prevent ARUco and UWARUco from choosing the wrong contours, as observed in the previous experiment. The sigma of sensor noise σ sn was set to five, which creates images that are similar to real images. The effect of fog was applied, but it was modified for this test. Since this effect changes only the difference between intensities of black and white regions of the marker, this difference became the main parameter of this analysis. Instead of depending on the distance dist or fog density f d, the intensities of black and white were set to specific values, starting at 70 and 230 to simulate a low-fog environment, and moving towards the intensity of the background, ending at 148 and 152 to simulate a high-fog environment. The difference between these intensities therefore started with 160 and ended with four.
The results are presented in Figure 9. In worse visibility conditions, the Fast and VideoFast versions of ARUco3 required more trials to find proper global threshold values, but when they did, they did not lose them, unless the visibility conditions were very bad. The decrease in the number of detected markers that was visible in the graphs was caused by a loss of markers in the first video frames. This experiment also showed that UWARUco was able to find markers in very bad conditions, being the last algorithm that detected markers when the difference between black and white regions became very low. AprilTag2 was also able to find markers in bad conditions, and so could the Fast and VideoFast versions of ARUco3 if they chose a good global threshold, but their results were worse than the results of UWARUco.
Image-improving algorithms did not perform well in this test. The results show that there was no need to improve images for UWARUco, and in the case of ARUco, the improvement in the number of detected markers was disputable, since the number of detected markers often increased in bad visibility conditions, but decreased in good visibility conditions.
The processing time was higher when compared to the times obtained in the reference, mainly due to the presence of sensor noise in the image. The processing time of AprilTag2 increased greatly, which makes this algorithm unusable in real-time solutions. The processing times of ARUco, the Base version of UWARUco, and the Normal and Fast versions of ARUco3 also increased, but less than then the time of AprilTag2. The processing time of the Masked version of UWARUco remained the same as in the reference, so it became faster than the other algorithms, whose processing time increased. The fastest solution of this test was the VideoFast version of ARUco3, but it detected fewer markers.
Here, markers were placed in a foggy environment at different distances from the viewer, which influenced their size and also their contrast, due to the presence of fog. This distance was set between values of 1.0 and 4.0 and changed continuously between successive frames, since abrupt changes could negatively influence marker detectors that use data from previous frames. The scale of the markers started at 30 at a distance of 1.0 and decreased linearly with the distance; the intensity of the background was again fixed at 150; the intensities of black and white marker regions were zero and 255 at zero distance; the glow effect was not present; and the sigma of sensor noise σ sn was again set to five. The density of the fog was set to f d = 1.0, so that the difference in intensities between black and white regions at maximum depth was approximately five, as in the previous test when choosing the worst conditions.
This test had no variable parameter, so only one set of 20 videos with 100 frames was generated. The results are presented in Table 1. The Masked version of UWARUco detected the highest number of markers, while its processing time was comparable with other algorithms, with the exception of the VideoFast version of ARUco3, which was faster, but detected a lower number of markers. The results of image improving algorithms show that in this test, they were able to increase the number of detected markers of ARUco at the cost of higher detection time, some of them placing ARUco at approximately the same level as the Base version of UWARUco. They also slightly improved the results of the Masked version of UWARUco, again at the cost of increased processing time.
Next, the detection of markers in foggy environments that appeared to glow in the image was evaluated. Perspective and fog effects were not applied; the scale of markers was set to 15; and the sigma of sensor noise was set to σ sn = 5. The intensity of the background was set to a low value of 50, so that black marker areas could have higher intensity than the background. The glow effect was simulated with 20 levels, each level increasing the intensity of the black marker areas linearly from 0–100, the intensity of white marker areas from 255–400, the glow alpha α glow from 0.0 – 0.8, and the glow sigma σ glow from 0–60.
As with the previous experiments, each glow effect level was tested with 20 videos consisting of 100 frames. The averaged results are presented in Figure 10. Several marker-detecting algorithms (ARUco, the Fast and VideoFast versions of ARUco3, and AprilTag2) struggled to find markers when the glow was at high levels: the Fast and VideoFast variants of ARUco3 again lost several frames until they found the proper global threshold value. The processing time of all detectors stayed approximately the same without any influence caused by an increased level of glow. Since the number of detected markers was already very high, image-improving algorithms could not increase these values. Some of them (CLAHE and deblur combined with ARUco) decreased the number of detected markers, but many of them just increased the computation time. One exception to this was the combination of ARUco and MBUWWB, which decreased the computation time at glow levels above six, while keeping the number of detected markers very high. This happened because at glow levels higher than six, black marker regions had higher intensity than the background, so most of the artificial sensor noise was removed, which made the thresholded image mostly noise-free.
Finally, all effects affecting synthetically-generated images were incorporated. Several variables were changed at the same time, continuously between successive frames. To focus more on the bad visibility conditions, the values were chosen to obtain a difference between the black and white areas that was lower than 100 and a glow level between 10 and 20: the intensity of background C bg was changed between values of 50 and