# Benchmarks for Industrial Inspection Based on Structured Light

Yuping Ye<EMAIL_ADDRESS>Siyuan Chen Zhan Song Shenzhen Institute of
Advanced Technology, Chinese Academy of Sciences, Shenzhen, China  GC
Inovation Co. Ltd, Shenzhen, China  University of Chinese Academy of Sciences,
Beijing, China  The Chinese University of Hong Kong, Hong Kong

###### Abstract

Robustness and accuracy are two critical metrics for industrial inspection. In
this paper, we propose benchmarks which can evaluate the structured light
method’s performance . Our evaluation metric was learning from a lot of
inspection tasks from the factories. The metric we proposed consists of four
detailed criteria such as flatness, length, height and sphericity. Then we can
judge whether the structured light method/device can be applied to specified
inspection task by our evaluation metric quickly. A structured light device
built for TypeC pin needles inspection performance is evaluated via our
metrics in the final experimental section.

###### keywords:

Strucutre Light, Industrial Inspection, 3D Vision, 3D Reconstruction

††journal: ARXIV

<EMAIL_ADDRESS>Ye<EMAIL_ADDRESS>Chen

<EMAIL_ADDRESS>Song

##  1 Introduction

With the fast development of manufacturing, a lot of heavy and tedious work
can be done by machines automatically, which improves the efficiency
significantly. By comparison, the inspection as the last and crucial step in
making products, especially for 3C products (Computer, Communication, Consumer
Electronic products) still involved much labour. With the development trend of
device miniaturization, more and more inspection work cannot be completed by
simple labour without a specialized and complicated tool. Take the mobile
phone as an instance; mobile phones’ size decreases each year dramatically,
making more stringent requirements for inspection tools. Recently, The famous
contract manufacturer Foxconn Electronics is trying to introduce structured
light based inspection machine for fault detection. As one of the most
essential noncontact optical 3D measurement methods, structure light has been
broadly used in a variety of applications such as virtual reality, animation,
and industrial inspection.

Fig. 1: Several inspection cases from assemble line. a). shows coplanarity
inspection of BGA solder balls, b). shows SIM card tray’s edge height
measurement, c) shows LED’s defect detection d) shows solder paste area
measurement on an small motor , e) shows the screw holes positioning and edge
coplanarity checking etc on mobile phone.

A typical structured light system consists of one camera and one light
emitter(projector/laser emitter). The light emitter project the coded pattern
onto the target object. The camera is then adopted to capture the illuminated
target object; finally, the decoding algorithm is utilized for these images to
obtain geometrical information of the target object. Although structured light
technology has a research history of more than forty years, it is only in
recent years that structured light technology being used to inspect small and
microscale devices. This is mainly caused by high-resolution cameras’
popularity in recent years and the miniaturization trend of devices. Due to
the characteristic that structured light easy to build and integrate into the
detection equipment, it has gained more and more attention from the industry.
Several inspection cases from production line based on structure light
technology are presented in Fig. 1: Left column shows the texture image of the
workpieces need to inspect, the right column shows the 3D model obtain by
structured light technology.

The inspection method/device which deploys on an assembly line must be very
robust and accurate. In scientific research, researchers always pay more
attention to accuracy than robustness and use different metrics to evaluate
their proposed reconstruction methods. These situations make it difficult for
engineers to select a suitable structured light method/device for inspection
tasks. To bridge the gap between industrial inspection and scientific
research, we propose evaluation metrics for measuring the structured light
method/device’s performance. Our evaluation metric was learning from a lot of
inspection cases from the assembly line. We can judge whether the structured
light method/device can be applied to specified inspection task by our
evaluation metric.

The organization of this paper is as follows. Previous work related to
structured light and industrial inspection is briefly reviewed in Sec 2.
Preliminary knowledge of structured light is presented in Sec 3. A detailed
introduction to our benchmarks for industrial inspection based on structured
light is proposed in Sec 4. Experimental results are provided and discussed in
Sec 5.

##  2 Related Work

According to the codeword of patterns adopted by structured light, we can
categorize projector structure light into three classes: direct encoding,
spatial-multiplexing, and time-multiplexing. Direct encoding structured light
methods directly decode each pixel by its intensity/colour, so it needs few
patterns. Spatial-multiplexing structured light methods only require one
pattern image, and so the codeword of a specific pixel is decoded from several
pixels. Unlike spatial-multiplexing groups, time-multiplexing methods need to
project several patterns sequentially. As a consequence, the codeword for a
given pixel is formed by a sequence of patterns. The advantage of time-
multiplexing methods is obvious; it can achieve dense and accurate results and
is robust against the surface texture. Caused the industrial inspection’s high
requirement on accuracy and robustness, most of the inspection methods are
based on the time-multiplexing. So we will focus on surveying the time-
multiplexing projector structured light in this section.

Due to strutured light’s long research history, there have been a lot of
researchers [1, 2, 3, 4, 5, 6, 7] provided a review of advances in structured
light scanning at different times. Most efforts on the structured light have
been devoted to the following topics: laser center extracting[8, 9], accuracy
[10, 11], reflective object [12, 13], calibration [14, 15, 16], and code
strategy [17, 18, 19] etc. For the sake of the laser structured light’s
simplicity, it has been well studied. In [20], a colour 3D scanner based on
the laser structured-light imaging method that can simultaneously acquire 3D
shape data, and colour of a target object is presented. Sun [8] put forward a
closed-form solution for extracting the laser stripe centre. In [21], Methods
and systems for laser-based real-time structured light depth extraction is
desecribed. Numerous work has contributed to the time-multiplexing structured
light. One of the most widely used time-multiplexing coding strategies is the
Gray code method. In Gray code methods, each codeword only has one different
bit different between random alternate code values, and all bits are with the
same intensity, so it is robust and straightforward. Because the Gray code can
remove periodic ambiguities whereas line or phase-shifting code achieves sub-
pixel correspondence mapping, many authors have combined the Gray code with
line-shifting [22, 23, 24], or phase-shifting [25, 26, 27] stripes. In [22],
the author encoded with Gray code stripe and decoded with stripe edge acquired
by sub-pixel technology instead of pixel centre, and also corrected the error
caused by dividing projecting angle irregularly by even-width encoding stripe.
Wu et al. [25] put forward a method which combines the Gray code with phase-
shifting stripes to overcome the phase unwrapping errors in the high-speed
measurement. A structure light method [28] without phase unwrapping is
presented to adapt to the measurement environment changes. Due to the
structured light’s simplicity and accurate, many off-the-shelf products [29,
30, 31] based on structured light have been widely used.

Compared with structured light, the research of industrial inspection based on
structured light is relatively rare. In [32], the authors put forward an
inspection method based on laser structured light for checking the coplanarity
of the BGA chips. Recently, BGA chips’ coplanarity checking is often based on
projector structured light methods. A laser structured light sensor [33] is
developed to achieve on-line weld bead measurement and weld quality
inspection. In [34], the authors show serveral inpsection cases based on
structured light. A systematic algorithm [35] of generating optimal inspection
planning based on CAD-directed model is proposed for laser-guided measuring
robot (LGMR) to inspect free-form surfaces. The authors [36] surveys various
inspection systems based on 2D optical images/videos in the semiconductor
industry. With the fast development of deep learning technology in recent
years, researchers also try to introduce deep learning for inspection. A deep
residual network With adaptively parametric rectified linear units is proposed
by Zhao et al. [37] for fault diagnosis. Although methods based on deep
learning may gain plausible results, its’ general ability for varies
inspection tasks is weak, and the training process is time-consuming.

The linkage between the state-of-art structured light methods and the
inspection applications is weak. So we proposed benchmarks for engineers to
evaluate state-of-art structured light methods for industrial inspection
applications.

##  3 Structure Light

As the above mentioned, the most commonly used structured light in industrial
inspection is laser and time-multiplexing projector structured light. In this
section, we will illustrate the mathmatics behind these methods.

###  3.1 Camera model

Fig. 2: Camera’s geometric model.

Camera is an indispensable device in computer vision. The geometric model of a
typical camera is represented in Fig. 2. Based on the pinhole projection
model, one 3D point with respect to the camera coordinate system
$M^{C}={\begin{bmatrix}X^{C}&Y^{C}&Z^{C}\end{bmatrix}}$ will be project onto
the camera plane at $m^{C}={\begin{bmatrix}u^{C}&v^{C}\end{bmatrix}}$. And
this can be expressed by the following equation:

$s{\begin{bmatrix}m^{C}\\\
1\end{bmatrix}}=\underbrace{\begin{bmatrix}f_{u}^{C}&\gamma_{u}^{C}&u_{0}^{C}\\\
0&f_{v}^{C}&v_{0}^{C}\\\
0&0&1\end{bmatrix}}_{K^{C}}{\begin{bmatrix}I_{3}&\vec{0}\end{bmatrix}}{\begin{bmatrix}M^{C}\\\
1\end{bmatrix}}$ (1)

where the subscript $C$ denoete it’s camera parameters, $K^{C}$ known as the
intrinsic matrix is composed of focal length $f_{u}^{C}$ ,$f_{v}^{C}$ ,
principal point $u_{0}^{C}$ ,$v_{0}^{C}$ , and skew coefficient
$\gamma_{u}^{C}$. Let $\bar{M}^{C}$ be the normalized image projection point:

$\bar{M}^{C}=\begin{bmatrix}X^{C}/Z^{C}\\\
Y^{C}/Z^{C}\end{bmatrix}=\begin{bmatrix}x^{C}\\\ y^{C}\end{bmatrix}$ (2)

So the Equation (1) can be rewritten as follow:

${\begin{bmatrix}m^{C}\\\
1\par\end{bmatrix}}=K^{C}{\begin{bmatrix}\bar{M}^{C}\\\ 1\par\end{bmatrix}}$
(3)

We can transform the point $M^{C}$ in the camera coordinate system into the
world coordinate system $M^{W}$ . So we can correlate $M^{C}$ and $M^{W}$ as
the following equation:

${\begin{bmatrix}M^{C}\\\ 1\par\end{bmatrix}}={\begin{bmatrix}R^{C}&T^{C}\\\
0&1\par\end{bmatrix}}{\begin{bmatrix}\bar{M}^{W}\\\ 1\par\end{bmatrix}}$ (4)

where $R^{C}$ and $T^{C}$ indicate the rotation matrix and transition vector
between the camera and the world coordinate system. Due to the manufacturing
and assembly’s limitation, the simple pinhole model failed in many cases. So
we have to take the radial distortions $k_{1}^{C}$,$k_{2}^{C}$,$k_{3}^{C}$ and
tangential distortions $p_{1}^{C}$ ,$p_{2}^{C}$ into consideration. Due to the
existence of the distortions, the real normalized image projection point
$\hat{M}^{C}$ may have deviation with :

$\hat{M}^{C}=\Upsilon(\bar{M}^{C})+\Delta(\bar{M}^{C})$ (5)

where $\Upsilon(\bar{M}^{C})$ and $\Delta(\bar{M}^{C})$ refers to radial and
tangential distortion vector respectively. Let
$(r^{C})^{2}=(x^{C})^{2}+(y^{C})^{2}$ , and the radial and tangential
distortion vector can be expressed as:

$\begin{split}\Upsilon(\bar{M}^{C})=\bar{M}^{C}(1+k_{1}^{C}(r^{C})^{2}&+k_{2}^{C}(r^{C})^{4}\\\
&+k_{3}^{C}(r^{C})^{6})\end{split}$ (6)

$\Delta(\bar{M}^{C})=\begin{bmatrix}2p_{1}^{C}x^{C}y^{C}+p2^{C}((r^{C})^{2}+2(x^{C})^{2})\\\
2p_{2}^{C}x^{C}y^{C}+p1^{C}((r^{C})^{2}+2(y^{C})^{2})\end{bmatrix}$ (7)

${\begin{bmatrix}m^{C}\\\ 1\end{bmatrix}}=K^{C}{\begin{bmatrix}\hat{M}^{C}\\\
1\par\end{bmatrix}}$ (8)

And the most famous camera calibration method is proposed by Zhang [38].

###  3.2 Laser Structured Light

Laser structured light as the most simple structured light which with high
precision and easy to deploy. Laser structured light is composed of a camera
and a laser emitter. However, because the line laser can only obtain one point
coordinate on the plane at a time, so in reality, the equipment using the line
laser often needs to be matched with a displacement platform. This may reduce
the line laser’s accuracy and make it challenging to integrate into some
detection equipment.

Fig. 3: Geometric model of a typical laser structured light system.

The laser emitter emits a laser plane onto the target. Then, the laser centre
of the laser line can be extracted from the captured the illuminated image,
final the point on the laser line can be calculated with simple triangulation
method. As the Fig. 3 shows, a target surface 3D point $\hat{M}^{C}$ on laser
plane

$LP:A^{\prime}X+B^{\prime}Y+C^{\prime}Z+D^{\prime}=0$ (9)

can be captured by camera . The laser plane’s cofficients
$A^{\prime},B^{\prime},C^{\prime},D^{\prime}$ canbe calibrate according to
Equation (4) . Finally, we can get the Z coordinate as following:

$Z=\frac{-D^{\prime}}{A^{\prime}(u-u_{0}^{C})/f_{u}^{C}+B^{\prime}(v-v_{0}^{C})/f_{v}^{C}+C^{\prime}}$
(10)

###  3.3 Projector Structured Light

Line lasers can only reconstruct one laser line and need the cooperation of
the displacement platform. Researchers have introduced the projector to
replace the laser with a projector to rebuild the target surface at once.

Fig. 4: Geometric model of a typical projector structured light system

As the Fig. 4, The projector in the SLS system is often be treated as an
inverse-light-path camera. So the projector also satisfied the Equation (1-8).
Unlike the binocular stereo vision, the SLS methods can easily find the
correspondence by searching the features in the captured images. The workflow
of the SLS system can be summarized into three steps. Firstly, patterns are
projected onto the target and imaged by the camera synchronously. A decoding
method is then adopted to extract the pattern feature from the captured images
and establish the correspondences between the camera and the projector.
Finally, the target’s coordinates can be calculated via the triangulation
formulation with the SLS system’s calibration parameter.

We can transform the point $M^{C/P}$ in the camera or projector coordinate
system into the world coordinate system. So we can correlate $M^{C}$ and
$M^{P}$ as the following Equation:

${\begin{bmatrix}M^{P}\\\ 1\par\end{bmatrix}}={\begin{bmatrix}R&T\\\
0&1\par\end{bmatrix}}{\begin{bmatrix}\bar{M}^{C}\\\ 1\par\end{bmatrix}}$ (11)

where $R$ and $T$ indicate the rotation matrix and transition vector between
the projector and the camera coordinate system.

Suppose that we have the extracted point $m^{C}$ and $m^{P}$ associated with
the same spatial 3D point. According to Equation (8) , we can apply the
intrinsic parameters and distortion parameters to normalize the $m^{C/P}$ ,
and get $\hat{m}^{C/P}$ .As the Fig. 4 shows, $O_{C}O_{P}$ is epipolar line,
$E_{1}$ and $E_{2}$ are epipolar points, using the epipolar constraint:

$(\hat{m}^{C})^{T}((K^{P})^{-T}T(K^{C})^{-1})\hat{m}^{P}=0$ (12)

Finally, we can obtain the depth value $Z^{C}$ by the triangulation algorithm
as:

$Z^{C}=\frac{(R\hat{m}^{C}\hat{m}^{P})(\hat{m}^{C}T)-\|\hat{m}^{C}\|^{2}(R\hat{m}^{C}T)}{\|R\hat{m}^{C}\|^{2}\|\hat{m}^{P}\|^{2}-(R\hat{m}^{C}\hat{m}^{P})^{2}}$
(13)

##  4 Evaluation Metric

As the above mentioned, many 3D reconstruction methods based on the structured
light is proposed, and the results presented by the authors seems plausible.
When it comes to the real situation that engineers try to employ these methods
for industrial inspection, it often cost engineers a lot of time to measure
whether the structured light approach meets the inspection task’s standards.
In this section, we proposed an evaluation metric which can evaluate the
structured light methods’ performance on inspection task accurately. The well-
designed metric can significantly shorten engineers’ time to choose a suitable
structured light method for the specified inspection task. Evaluation metrics
we proposed can be an energy function for calibration or reconstruction like
[39]. Also, we hope this metric can help researchers evaluating their methods
for industrial inspection in future.

The evaluation metric we proposed is tried out from numerous inspection tasks.
Inspection method based on structured light that passes our metric can be
stable and effective in the assembly line. The metric we proposed consists of
four detailed criteria: flatness, length, height and sphericity.

In statistics, range, mean and standard variance are generally used to
describe data distribution. Range is equal to the maximum error minus the
minimum error, standard variance measures how distant or spread the numbers in
a dataset are from the mean.

$\displaystyle R(\\{X_{i}\\})$ $\displaystyle=X_{max}-X_{min}$ (14)
$\displaystyle\mu(\\{X_{i}\\})$
$\displaystyle=\frac{1}{N}\sum_{i=1}^{N}{X_{i}}$
$\displaystyle\sigma(\\{X_{i}\\})$
$\displaystyle=\left(\frac{1}{N}\sum_{i=1}^{N}{(X_{i}-\mu(X_{i}))^{2}}\right)^{\frac{1}{2}}$

In this paper, we also use the mean and standard variance to describe data
distribution. Therefore, unlike most papers which evaluate the error once, we
need to conduct several tests on different height in the structured light
device’s depth of field. The repeated test number we suggested is 50 or more.

Fig. 5: A precise flat with serveral hollow cricle markers is adopted for
evluating the flatness and length.

Measuring the workpieces’ flatness and length is the most common industrial
inspection task. To evaluate the structured light method’s flatness, length, a
precise flat is adopted, as shown in Fig. 5. To eliminate the error caused by
the thickness of printing ink, several hollow circle markers are uniformly
printed on the flat with precise distance $l_{c}$ and angularity as shown in
Fig. 5. For each hollow circle marker, we utilize the Hough circle detection
algorithm [40] to detect the marker’s centre $P_{i}$. For each of the two
adjacent points $P_{i},P_{j}$ in the horizontal or vertical direction, the
length can be calculated and denoted as $d_{j}$. So the length error can
caclulate as

$E_{d}=d_{j}-l_{c}$ (15)

and the length metric can be represente as :

$\displaystyle\mu(R(\\{E_{d}\\})),\,\,\sigma(R(\\{E_{d}\\})),\,\,\mu(\mu(\\{E_{d}\\})),\,\,\sigma(\mu(\\{E_{d}\\})),$
(16)
$\displaystyle\mu(\sigma(\\{E_{d}\\}))(Optional),\,\,\sigma(\sigma(\\{E_{d}\\}))(Optional)$

where $\mu(R(\\{E_{d}\\})),\,\,\sigma(R(\\{E_{d}\\}))$ denote the length
error’s range distribution,
$\mu(\mu(\\{E_{d}\\})),\,\,\sigma(\mu(\\{E_{d}\\}))$ for mean and
$\mu(\sigma(\\{E_{d}\\})),\,\,\sigma(\sigma(\\{E_{d}\\}))$ for standard
variance. In a real inspection task, we often have little concern for the
error’s standard variance distribution.

To evaluate the flatness, we first use a circle mask with a slightly larger
radius than the actual radius for each hollow marker and remove these points
according to Equation 1. The reference plane used for reconstruction can be
viewed as a perfect plane. Without considering the calibration errors and
reconstruction errors, the planarity should be zero. The least-square fitting
approach is applied to fitting these points, and the plane function can be
calculated as $AX+By+CZ+D=0$. For each point’s $P_{i}=(x_{i},y_{i},z_{i})$
distance to the ideal plane canbe represented as:

$E_{p}=\frac{\|ax_{i}+By_{i}+Cz_{i}+D\|}{\left(A^{2}+B^{2}+c^{2}\right)^{\frac{1}{2}}}$
(17)

and the length metric can be represented as :

$\displaystyle\mu(R(\\{E_{p}\\})),\,\,\sigma(R(\\{E_{p}\\})),\,\,\mu(\mu(\\{E_{p}\\})),\,\,\sigma(\mu(\\{E_{p}\\}))$
(18)

Fig. 6: A gage block place on the flat to measure the height error.

Different from length error $E_{d}$, which always lager than the sensors’
actual size of a pixel, the Z value of the structured light can have higher
accuracy. The most common inspection task is to calculate the height among
workpiece’s different parts. As Fig.6 show, we place a gauge block with height
$h_{c}$ onto the plane. Like measuring the planarity did, we first use the
flat’s points without the gage block and hollow circles to get the plane
function $AX+BY+CZ+D=0$; and then caculate each point
$P_{j}=(x_{j},y_{j},z_{i})$ distance to the fitting plane; final the height
error can be caculated as:

$E_{h}=\frac{\|ax_{j}+By_{j}+Cz_{j}+D\|}{\left(A^{2}+B^{2}+c^{2}\right)^{\frac{1}{2}}}-h_{c}$
(19)

So the height metric can be represented as :

$\displaystyle\mu(R(\\{E_{h}\\})),\,\,\sigma(R(\\{E_{h}\\})),\,\,\mu(\mu(\\{E_{h}\\})),\,\,\sigma(\mu(\\{E_{h}\\}))$
(20)

Fig. 7:  Twelve tungsten steel balls are placed onto the flat to measure the
sphericity.

Many algorithms, especially the off-the-shelf products, adopted many filter
method to achieve better results, which can significantly promote the criteria
such as planarity. But overuse of the filter algorithms may lead to failure to
inspect a workpiece with significant curvature. To address this issue, we
introduce the sphericity error. As we know, the sphere is very sensitive to
smooth filter methods. As Fig. 7, we place twelve tungsten steel balls with
radius $r_{c}$ onto the plane, then capture its geometric information. We can
easily separate the balls’ points from the plane via the height different, and
apply the sphere fitting algorithm to get the ball centre and the radius
$r_{i}$ of each ball. The sphericity error can be caculated as:

$E_{s}=r_{i}-r_{c}$ (21)

and the sphericity metric can be represente as :

$\displaystyle\mu(R(\\{E_{s}\\})),\,\,\sigma(R(\\{E_{s}\\})),\,\,\mu(\mu(\\{E_{s}\\})),\,\,\sigma(\mu(\\{E_{s}\\}))$
(22)

As the above stated, our metric for inspection is composed with Equation 16,
18, 20 and 22.

##  5 Experimental Results

Traditionally, inspection tasks are always raised firstly; then engineers seek
to find a suitable solution according to the task’s characteristic. The
structured light method is fit to obtain the Lambert object and monochrome
object’s geometric information. When a structured light method is selected for
inspection, engineers should choose a suitable off-the-shelf product or build
their scanning device. For example, the device’s field/depth of view should
adapt to workpiece’s size/height. It’s difficult for engineers to measure
structured light method’s accuracy for inspection. In this section, we
evaluate a self-built structured light device’s performance via our metric;
according to this metric, then we show how to check the coplanarity of the
TypeC’s pin needles.

Fig. 8: Our self-built structured light system which composed of one camera
and blue projector. Fig. 9: The coding strategy employs Gray code combined
with strip shifting pattern. Fig. 10: Evaluation metrics on our self-built
structured light device.

To measure our metrics on structured light, we first built a structured light
device with high accuracy. The established structured light system is consists
of one camera (FLIR BFS-U3-51S5M-BD, with the resolution of 2448$\times$2048,
and maximum framerate up to 75Hz, pixel size is 3.45 µm, USB3.1 interface) and
one DLP projector (TI-DLP3010, a blue light source with a wavelength of 450nm,
resolution 1280$\times$720, and maximum frame rates up to 4225Hz ), as shown
in Fig. 8. As the above stated, the structured light method the industrial
inspection adopted is often the time-multiplexing. The coding strategy we
adopted based on [41] which employed the Gray code combined with line shifting
pattern as shown in Fig. 9. To further improve our device’s accuracy, we
adopted a practical calibration framework of [42] to calibrate our system
accurately.

To evaluate our self-built device, a precise flat with several hollow circles
markers, a gauge block and twelve tungsten steel balls is used as presented in
Sec. 4. As Fig. 10 shows, Left column shows the captured point clouds of the
evaluation tools, the right column shows the detailed data distribution of
four metric’s range and mean. And Table 1 shows the four evaluation metrics’
mean and standard variance on our self-built structured light device.

And here comes an inspection sample of checking the coplanarity of the twenty-
four pin needles of TypeC. A quality TypeC product must satisfy that the
maximum and minimum error from the fitting flat must be smaller than ten µm.
By checking Table 1, we can find the planarity and height metrics’ range’s
mean far smaller than ten µm and its standard variance is relatively small. So
our self-built structured light can be integrated into the inspection machine
for checking pin needles’ coplanarity of TypeC. To further prove that our
metrics can evaluate the method/device accuracy for industrial inspection, a
non-qualified product with known range of 25 µm was reconstructed by our self-
built device; we can found that the range calculated is around the true value
within 0.3 µm shown in Fig.11 .

Table 1: Four evaluation metrics’s mean and standard variance on our self-built structured light device.  | $\mu(R(\\{E\\_x\\}))$ | $\mu(R(\\{E\\_x\\}))$ | $\mu(\mu(\\{E\\_x\\}))$ | $\sigma(\mu(\\{E\\_x\\}))$
---|---|---|---|---
E_d | 18.91 µm | 0.041 | 4.7 µm | 0.007
E_p | 1.75 µm | 0.029 | -0.22 µm | 0.003
E_h | 1.95 µm | 0.025 | 0.36 µm | 0.004
E_r | 5.78 µm | 0.031 | -1.79 µm | 0.006
Fig. 11: Top:TypeC’s texture images, Middle:TypeC’s point cloud images
reconstructed by our self-built device,Bottom:the pin needles’ coplanarity
error range caculated via our device.

##  6 Conclusion

In this paper, we put forward a metric for evaluating the structured light
method/device for industrial inspection. First, we describe the mathematical
model of laser and projector structured light. We then proposed an evaluation
metric that can evaluate the structured light methods’ performance on
inspection tasks accurately. The metric we proposed consists of four detailed
criteria such as flatness, length, height and sphericity. Each criterion is
measured by the error’s range/mean’s data distribution. In the experimental
section, we evaluate a self-built device’s performance via our metrics and
present a real inspection case. In the future, we are planning to measure the
notable structured light method via our metric for helping the engineers to
choose the suitable method/device quickly. How to incorporate our metric to
improve calibration/reconstruction’s accuracy is also another interesting
work.

## References

  * Xu et al. [2020] Xu, X, Fei, Z, Yang, J, Tan, Z, Luo, M.  Line structured light calibration method and centerline extraction: A review.  Results in Physics 2020;:103637.
  * Salvi et al. [2010] Salvi, J, Fernandez, S, Pribanic, T, Llado, X.  A state of the art in structured light patterns for surface profilometry.  Pattern recognition 2010;43(8):2666–2680.
  * Batlle et al. [1998] Batlle, J, Mouaddib, E, Salvi, J.  Recent progress in coded structured light as a technique to solve the correspondence problem: a survey.  Pattern recognition 1998;31(7):963–982.
  * Giancola et al. [2018] Giancola, S, Valenti, M, Sala, R.  A Survey on 3D Cameras: Metrological Comparison of Time-of-Flight, Structured-Light and Active Stereoscopy Technologies.  Springer; 2018.  ISBN 3319917617.
  * Geng [2011] Geng, J.  Structured-light 3d surface imaging: a tutorial.  Advances in Optics Photonics 2011;3(2):128–160.
  * Ribo and Brandner [????] Ribo, M, Brandner, M.  State of the art on vision-based structured light systems for 3d measurements.  In: International Workshop on Robotic Sensors: Robotic and Sensor Environments, 2005. IEEE.  ISBN 0780393783; ????, p. 2–6.
  * Angelsky et al. [2020] Angelsky, OV, Bekshaev, AY, Hanson, SG, Zenkova, CY, Mokhun, II, Jun, Z.  Structured light: ideas and concepts.  Front Phys 2020;8.
  * Sun et al. [2015] Sun, Q, Chen, J, Li, C.  A robust method to extract a laser stripe centre based on grey level moment.  Optics and Lasers in Engineering 2015;67:122–127.
  * Al-Temeemy and Al-Saqal [2021] Al-Temeemy, AA, Al-Saqal, SA.  Laser-based structured light technique for 3d reconstruction using extreme laser stripes extraction method with global information extraction.  Optics & Laser Technology 2021;138:106897.  URL: https://www.sciencedirect.com/science/article/pii/S0030399220315309. doi:https://doi.org/10.1016/j.optlastec.2020.106897.
  * Le et al. [2017] Le, MT, Chen, LC, Lin, CJ.  Reconstruction of accurate 3-d surfaces with sharp edges using digital structured light projection and multi-dimensional image fusion.  Optics Lasers in Engineering 2017;96:17–34.
  * Ha et al. [????] Ha, H, Oh, TH, Kweon, IS.  A multi-view structured-light system for highly accurate 3d modeling.  In: 2015 International Conference on 3D Vision. IEEE.  ISBN 1467383325; ????, p. 118–126.
  * Song et al. [2017] Song, Z, Jiang, H, Lin, H, Tang, S.  A high dynamic range structured light means for the 3d measurement of specular surface.  Optics Lasers in Engineering 2017;95:8–16.
  * Gupta et al. [????] Gupta, M, Yin, Q, Nayar, SK.  Structured light in sunlight.  In: Proceedings of the IEEE International Conference on Computer Vision. ????, p. 545–552.
  * Zhang and Huang [2006] Zhang, S, Huang, PS.  Novel method for structured light system calibration.  Optical Engineering 2006;45(8):083601.
  * Deetjen and Lentink [2018] Deetjen, ME, Lentink, D.  Automated calibration of multi-camera-projector structured light systems for volumetric high-speed 3d surface reconstructions.  Optics express 2018;26(25):33278–33304.
  * Klemm et al. [2017] Klemm, M, Seebacher, F, Hoppe, H.  High accuracy pixel-wise spatial calibration of optical see-through glasses.  Computers & Graphics 2017;64:51–61.
  * Je et al. [????] Je, C, Lee, SW, Park, RH.  High-contrast color-stripe pattern for rapid structured-light range imaging.  In: European Conference on Computer Vision. Springer; ????, p. 95–107.
  * Caspi et al. [1998] Caspi, D, Kiryati, N, Shamir, J.  Range imaging with adaptive color structured light.  IEEE Transactions on Pattern analysis machine intelligence 1998;20(5):470–480.
  * Young et al. [????] Young, M, Beeson, E, Davis, J, Rusinkiewicz, S, Ramamoorthi, R.  coded structured light.  In: 2007 IEEE Conference on Computer Vision and Pattern Recognition. IEEE.  ISBN 1424411793; ????, p. 1–8.
  * Ko and Yi [2018] Ko, Y, Yi, S.  Development of color 3d scanner using laser structured-light imaging method.  Current Optics and Photonics 2018;2(6):554–562.
  * Ackerman and Keller [2008] Ackerman, JD, Keller, KP.  Methods and systems for laser based real-time structured light depth extraction.  2008\.  US Patent 7,385,708.
  * Wu et al. [2006] Wu, HB, Chen, Y, Wu, MY, Guan, CR, Yu, XY.  3d measurement technology by structured light using stripe-edge-based gray code.  Journal of Physics: Conference Series 2006;48:537–541.  URL: http://dx.doi.org/10.1088/1742-6596/48/1/101. doi:10.1088/1742-6596/48/1/101.
  * Zheng et al. [2010] Zheng, SY, Hu, HL, Xu, X, Le, K.  Structured-light-based 3d reconstruction using gray code and line-shift stripes.  In: Advanced Materials Research; vol. 108\. Trans Tech Publ; 2010, p. 799–804.
  * Yang et al. [2008] Yang, R, Cheng, S, Yang, W, Chen, Y.  Robust and accurate surface measurement using structured light.  IEEE Transactions on Instrumentation and Measurement 2008;57(6):1275–1280.
  * Wu et al. [2019] Wu, Z, Guo, W, Zhang, Q.  High-speed three-dimensional shape measurement based on shifting gray-code light.  Optics express 2019;27(16):22631–22644.
  * Sansoni et al. [1999] Sansoni, G, Carocci, M, Rodella, R.  Three-dimensional vision based on a combination of gray-code and phase-shift light projection: analysis and compensation of the systematic errors.  Applied optics 1999;38(31):6565–6573.
  * Yu et al. [2016] Yu, S, Zhang, J, Yu, X, Sun, X, Wu, H.  Unequal-period combination approach of gray code and phase-shifting for 3-d visual measurement.  Optics Communications 2016;374:97–106.
  * Cai et al. [2020] Cai, Z, Liu, X, Pedrini, G, Osten, W, Peng, X.  Structured-light-field 3d imaging without phase unwrapping.  Optics and Lasers in Engineering 2020;129:106047.
  * Li et al. [2020] Li, X, Konaka, T, Nakamura, N.  Inspection device.  2020\.  US Patent App. 16/727,957.
  * Cofer and Hamza [2007] Cofer, D, Hamza, R.  Method and apparatus for detecting objects using structured light patterns.  2007\.  US Patent App. 11/675,117.
  * Chang et al. [2020] Chang, M, Lee, S, Chang, J, Im, S, Euijeong, S, Hansol, K, et al.  Three-dimensional scanning device using structured light.  2020\.  US Patent 10,739,131.
  * Wei et al. [2011] Wei, Z, Xiao, Z, Zhang, X, Zhou, H.  Coplanarity inspection of bga solder balls based on laser interference structure light.  In: 2011 International Conference on Optical Instruments and Technology: Optoelectronic Measurement Technology and Systems; vol. 8201. International Society for Optics and Photonics; 2011, p. 82012F.
  * Han et al. [2020] Han, Y, Fan, J, Yang, X.  A structured light vision sensor for on-line weld bead measurement and weld quality inspection.  The International Journal of Advanced Manufacturing Technology 2020;106(5):2065–2078.
  * Drouin and Beraldin [2020] Drouin, MA, Beraldin, JA.  Active triangulation 3d imaging systems for industrial inspection.  In: 3D Imaging, Analysis and Applications. Springer; 2020, p. 109–165.
  * Liu et al. [2008] Liu, W, Qu, X, Ouyang, J, Wang, Z.  Design and cad-directed inspection planning of laser-guided measuring robot.  Computers & Graphics 2008;32(6):617–623.  URL: https://www.sciencedirect.com/science/article/pii/S0097849308001209. doi:https://doi.org/10.1016/j.cag.2008.09.006.
  * Huang and Pan [2015] Huang, SH, Pan, YC.  Automated visual inspection in the semiconductor industry: A survey.  Computers in Industry 2015;66:1–10.  URL: https://www.sciencedirect.com/science/article/pii/S0166361514001845. doi:https://doi.org/10.1016/j.compind.2014.10.006.
  * Zhao et al. [2020] Zhao, M, Zhong, S, Fu, X, Tang, B, Dong, S, Pecht, M.  Deep residual networks with adaptively parametric rectifier linear units for fault diagnosis.  IEEE Transactions on Industrial Electronics 2020;68(3):2587–2597.
  * Zhang [2000] Zhang, Z.  A flexible new technique for camera calibration.  IEEE Transactions on pattern analysis and machine intelligence 2000;22(11):1330–1334.
  * Han et al. [2019] Han, H, Wu, S, Song, Z.  An accurate calibration means for the phase measuring deflectometry system.  Sensors 2019;19(24):5377.
  * Kerbyson and Atherton [1995] Kerbyson, D, Atherton, T.  Circle detection using hough transform filters 1995;.
  * Song et al. [2012] Song, Z, Chung, R, Zhang, XT.  An accurate and robust strip-edge-based structured light means for shiny surface micromeasurement in 3-d.  IEEE Transactions on Industrial Electronics 2012;60(3):1023–1032.
  * Ye and Song [????] Ye, Y, Song, Z.  A practical means for the optimization of structured light system calibration parameters.  In: 2016 IEEE International Conference on Image Processing (ICIP). IEEE.  ISBN 1467399612; ????, p. 1190–1194.