# Online Adaptive Disparity Estimation for Dynamic Scenes in Structured Light
Systems

Rukun Qiao1, Hiroshi Kawasaki2 and Hongbin Zha1 1Rukun Qiao and Hongbin Zha
are with the Key Lab of Machine Perception (MOE), School of Artificial
Intelligence, Peking University, Beijing, China<EMAIL_ADDRESS><EMAIL_ADDRESS>Kawasaki is with the Graduate School and Faculty of
Information Science and Electrical Engineering, Kyushu University, Fukuoka,
Japan<EMAIL_ADDRESS>paper has been accepted for
publications by IEEE, in 36th IEEE/RSJ International Conference on Intelligent
Robots and Systems, 2023. Copyright: © 2023 IEEE.

###### Abstract

In recent years, deep neural networks have shown remarkable progress in dense
disparity estimation from dynamic scenes in monocular structured light
systems. However, their performance significantly drops when applied in unseen
environments. To address this issue, self-supervised _online_ adaptation has
been proposed as a solution to bridge this performance gap. Unlike traditional
fine-tuning processes, online adaptation performs test-time optimization to
adapt networks to new domains. Therefore, achieving fast convergence during
the adaptation process is critical for attaining satisfactory accuracy. In
this paper, we propose an unsupervised loss function based on long sequential
inputs. It ensures better gradient directions and faster convergence. Our loss
function is designed using a multi-frame pattern flow, which comprises a set
of sparse trajectories of the projected pattern along the sequence. We
estimate the sparse pseudo ground truth with a confidence mask using a filter-
based method, which guides the online adaptation process. Our proposed
framework significantly improves the online adaptation speed and achieves
superior performance on unseen data.

##  I Introduction

Dense disparity estimation in dynamic scenes using a mono-camera with a single
projector is a task that has garnered significant attention in various fields,
including augmented reality and medical applications. Compared to object
scanning in static scenes, disparity estimation in dynamic scenes requires
disparity maps for every incoming frame, necessitating faster computation,
particularly at high frame rates. Moreover, the utilization of temporal
information is challenging owing to the higher uncertainty and complexity
caused by object motion, emphasizing the need for faster spatio-temporal
matching.

Many structured light systems with learning methods have been proposed to
tackle this matching problem [1, 2, 3]. Moreover, several researchers have
utilized implicit temporal information by introducing hidden layers between
frames to enhance the robustness and performance of disparity estimation [4,
5]. In a recent study [5], we presented TIDE-Net, which primarily focused on
the estimation of temporal residual parts and demonstrated superior
generalization performance without any online adaptation processing.

However, similar to other data-driven learning methods, structure light
systems based on machine learning suffer from performance degradation when
evaluated on unseen data, due to the domain shift issue. This problem becomes
more severe in dynamic scenes, where the domain gap between training and
testing data is amplified by the motion. Fine-tuning the trained model is a
popular approach to alleviate this problem [6, 7]. However, acquiring ground
truth of dense disparity maps in dynamic scenes is a laborious task.

Thus, we address this issue through online adaptation. Specifically, we first
pretrain the model using a synthetic dataset, where massive data with ground
truth can be generated. We then perform test-time optimization with real data
using an unsupervised loss function to adapt the model. However, this task is
challenging due to the risk of overfitting to the current data, leading to
slow convergence and catastrophic forgetting. Thus, controlled gradient
directions are necessary.

In this paper, we propose a novel loss function for online adaptation that
utilizes long-term temporal consistency. Specifically, we use the multi-frame
pattern flow, a sparse flow over multiple frames, as the basis for our loss
function. We observe that the pattern flow inherently contains long sequential
temporal consistency, which we model using a lightweight Linear Kalman Filter
process. Using the estimated pattern flow, we generate sparse pseudo ground
truth disparity to guide the network adaptation process. Additionally, we
estimate a confidence mask based on temporal continuity and spatial
consistency to model the reliability of the sparse pseudo ground truth.
Compared to the classic photometric loss commonly used for unsupervised
learning, our sparse pseudo ground truth provides better guidance for online
adaptation. Our experimental results demonstrate that our approach
significantly improves model performance on unseen data and achieves state-of-
the-art accuracy.

In summary, our contributions are:

  * •

A test-time optimization method that adapts learned structure light network to
incoming new data with a self-supervised loss.

  * •

A multi-frame pattern flow that generates pseudo ground truth and exploits
long-term temporal information to avoid overfitting.

  * •

A spatial-temporal consistency-driven confidence mask to improve the accuracy
and robustness.

Figure 1: The algorithm framework for our proposed online adaptation method.
The network, represented by the function $f$, is updated every $T$ frames.
During this temporal window, we input the historical multi-frame pattern flow
$F_{n}^{t-T}$ and incrementally compute the new pattern flow $F_{n}^{t}$.
Based on the updated pattern flow, we calculate the loss function
$\mathcal{L}$, which is then used for the online updating process.

##  II Related Work

Active Depth Estimation: By projecting a pre-designed pattern into the scene,
the dense 3D correspondence between the camera and the projector can be
extracted even if there is no texture or high-frequency feature on the object
surface, resulting in the stable and robust dense depth map [8, 9]. For
dynamic scene reconstruction, many methods choose spatial encoding approaches
to embed the positional information into the projected pattern, which is
usually called ’one-shot scan’ [10, 11, 12, 13], for they only require one
frame to calculate the depth map. The one-shot scan can avoid struggling with
the object motions between frames, but the lack of information and global
decoding limit the performance. Although this problem can be alleviated by
applying deep neural networks with a large dataset for training, the upper
bound performance is still limited from the source input.

One straightforward idea to tackle the above challenge is including the
temporal information for pattern decoding [14, 15]. Recently many researchers
focus on including temporal coherence between frames to improve the
performance of dynamic scenes [3, 16]. In our previous research [5], we also
proposed the TIDE-Net which focused on the estimation of incremental non-
linear disparity portion instead of the absolute disparity map. A hidden layer
based on the ConvGRU module was applied to inherit information from previous
frames. This method can achieve SOTA accuracy and generalization ability in
dynamic scenes while guaranteeing computational efficiency.

Domain Adaptation and Continual Learning: Although deep neural networks can
learn to estimate correspondences robustly, they often suffer from a loss of
accuracy when evaluated on unseen data. To overcome this domain gap, there has
been a significant amount of research focused on cross-domain adaptation.
While active sensor applications have seldom considered this problem, cross-
domain adaptation has drawn attention in areas such as stereo matching [17],
visual odometery [18, 19], object detection [20], and semantic segmentation
[21]. Techniques for addressing this problem can be classified into two types:
1. Reducing the domain gaps between data, such as using additional
normalization layers [7] or adding an image enhancement module based on
Generative Adversarial Networks [22]; and 2. Updating the network during
evaluation by a loss function, which is called online learning or continual
learning [23, 18, 17]. Different from the fine-tuning process, such online
learning can only go through the data once, therefore the convergence speed
needs to be considered. In this paper, we propose an unsupervised loss with
pseudo ground truth disparity and confidence mask to guide the network to
converge faster.

Figure 2: Example of multi-frame pattern flow. As objects move over time, the
intersection points of the same light ray at $u$ move forward, creating a
trajectory in the camera space. The example shows a 4-frame pattern flow.

Figure 3: The spatial neighbor checking process involves examining the
distance consistency in both projector and camera space to confirm the
connection between detected dots once the correspondence is given.

##  III Methods

###  III-A Problem Definition and Notations

We use a single camera and a projector to capture the image frames from the
scenes with non-rigid motions. Here we denote $\mathrm{\mathbf{I}}^{t}$ as the
captured images where $t$ represents the frame number, and
$\mathrm{\mathbf{P}}$ as the projected pattern. The pixel in camera and
projector space is denoted as $\mathrm{\mathbf{I}}^{t}(\mathbf{x})$ and
$\mathrm{\mathbf{P}}(\mathbf{u})$, where $\mathbf{x}=(x,y)$ and
$\mathbf{u}=(u,v)$ are the coordinates in the camera and the projector
respectively. For convenience, we denote $\rho_{x}$ and $\rho_{y}$ as the
horizontal and vertical component of the coordinates, where
$\rho_{x}(\mathbf{x})=x$ and $\rho_{y}(\mathbf{x})=y$.

The projector can be modeled as the second camera under this rectified
structured light system, therefore we can use the similar definitions like
’epipolar line’ or ’disparity’ in passive stereo systems. The 3D position can
be calculated once we find the correspondence between pixels in the camera
space and the projector space. We use $\mathbf{x}\leftrightarrow\mathbf{u}$ to
mark the correspondence relation between two pixels from the camera and the
projector. Similar to stereo systems, we can define disparity maps in the
camera space as $\mathrm{\mathbf{D}}^{t}$, where

$\mathrm{\mathbf{D}}^{t}(\mathbf{x})=\rho_{x}(\mathbf{x})-\rho_{x}(\mathbf{u}),\mathbf{x}\leftrightarrow\mathbf{u}.$
(1)

The disparity estimation problem aims to estimate the disparity map
$\mathrm{\mathbf{D}}^{t}$ for every input image $\mathrm{\mathbf{I}}^{t}$ with
the help of the projected pattern $\mathrm{\mathbf{P}}$. In our previous
research [5], the disparity map can be estimated by the TIDE-Net in an
incremental way given the camera image $\mathrm{\mathbf{I}}^{t}$ and pre-
designed pattern $\mathrm{\mathbf{P}}$. Here we just denote this network as
the function $f$ for simplicity:

$\mathrm{\mathbf{D}}^{t}=f\left(\mathrm{\mathbf{I}}^{t},\mathrm{\mathbf{P}},\mathcal{H}^{t-1}\right),$
(2)

where $\mathcal{H}^{t-1}$ denotes the history information before time $t$.
This contains the disparity map of the previous frame
$\mathrm{\mathbf{D}}^{t-1}$ and the hidden layer passed into the TIDE-Net.

In this paper, we online update this network through backpropagation (BP) with
the help of a loss function $\mathcal{L}$. Considering the ConvGRU module in
TIDE-Net needs a temporal sequence for training, we update the model with a
sliding window of length $T$, which means our model is updated every $T$
frames. We add a superscript for function $f$ to mark the updated model, and
denote $f^{0}$ as the base model pretrained offline. The upper part of Fig. 1
depicts this updating process with the help of loss function $\mathcal{L}$.
For brevity, we use the abbreviation $t-T+1:t$ for a small slice of
$t-T+1,t-T+2,...t$.

###  III-B Loss Function for Online Learning

The loss function $\mathcal{L}$ for online adapting is defined as below:

$\mathcal{L}=\mathrm{\mathbf{W}}^{t}\mathcal{L}_{D}+\alpha\mathcal{L}_{P},$
(3)

where $\mathcal{L}_{D}$ is the direct disparity loss, and $\mathcal{L}_{P}$ is
the traditional unsupervised photometric loss. The $\alpha$ is the scale
factor, which we take $0.1$ during experiments. The confidence mask
$\mathrm{\mathbf{W}}^{t}$ stands for the reliability of the first term
$\mathcal{L}_{D}$. For those failure areas that $\mathcal{L}_{D}$ is not
reliable enough, we refer to $\mathcal{L}_{P}$ for help.

The traditional photometric loss $\mathcal{L}_{P}$ is defined as below:

$\mathcal{L}_{P}=\sum_{t_{k}=t-T-1}^{t}|\mathrm{\mathbf{I}}^{t_{k}}-\pi(\mathrm{\mathbf{P}},\mathrm{\mathbf{D}}^{t_{k}})|_{1},$
(4)

where $\pi$ is the warping function given the disparity map and the projected
pattern. The direct disparity loss contains the sparse pseudo ground truth
$\mathrm{\mathbf{D}}^{t_{k}}_{PGT}$:

$\mathcal{L}_{D}=\sum_{t_{k}=t-T-1}^{t}|\mathrm{\mathbf{D}}^{t_{k}}-\mathrm{\mathbf{D}}^{t_{k}}_{PGT}|_{1}.$
(5)

Both the sparse pseudo ground truth $\mathrm{\mathbf{D}}^{t}_{PGT}$ and the
confidence mask $\mathrm{\mathbf{W}}^{t}$ can be computed from the multi-frame
pattern flow, which will be introduced in III-C. Then, the computation process
will be illustrated in III-D.

###  III-C Multi-Frame Pattern Flow in Structured Light Systems

In a structured light system, the projected pattern from a projector is
deformed due to the shape of the object’s surface and is captured by the
camera. When the objects are moving, the deformed pattern also changes. This
light flow, caused by the active projected pattern, can be observed in the
captured images. In our previous research [5], we defined this light flow as
Pattern Flow in the camera space between two adjacent frames. We now extend
this concept to the multi-frame case.

Let us define the projector pixel mentioned above as $\mathbf{u}=(u,v)$. A
light ray projected from $\mathbf{u}$ to the object’s surface is captured by
the camera. With the object moving from time $t_{0}$ to $t$, we can observe a
series of reflected light dots at the position of $\mathbf{x}^{t_{0}:t}$. We
define this series of camera positions as a ’multi-frame pattern flow’,
denoted as $F^{t}_{\mathbf{u}}=\mathbf{x}^{t_{0}:t}$, as illustrated in the
upper right of Fig. 2. In other words, the multi-frame pattern flow is a
trajectory of the same light ray from the projector in the camera space. Based
on this definition, we can re-write the disparity definition in Eq. 1 into

$\rho_{x}(\mathbf{u})=\rho_{x}(\mathbf{x})-\mathrm{\mathbf{D}}^{t}(\mathbf{x}),\mathbf{x}\in
F^{t}_{\mathbf{u}}.$ (6)

Therefore, once we compute the multi-frame pattern flow $F^{t}_{\mathbf{u}}$
of projected light ray $\mathbf{u}$, the disparity value on this trajectory
can be revealed according to Eq. 6.

Compared with the multi-frame optical flow estimation from passive image
sequences, the multi-frame pattern flow has some differences: 1. We use the
projected pattern $\mathrm{\mathbf{P}}$ as the canonical space naturally, thus
we do not need to set a reference frame during computation; 2. We can pre-
design the pattern structure to help us compute the multi-frame pattern flow;
and 3. Although we use the term ’trajectory’ to describe the estimated pattern
flow, it should be noted that the points in the trajectory do not belong to
the same physical point, but rather to the same projected light ray.
Therefore, the trajectory has to follow the epipolar constraint and is fixed
in the horizontal direction in a rectified system. Considering these
advantages we have in our structured light system, the computing of multi-
frame pattern flow can be more light and efficient compared with the optical
flow. In the next section, we will discuss how to compute the sparse multi-
frame pattern flow frame by frame with the help of pattern structure and
temporal continuity.

###  III-D Pseudo Ground Truth and Confidence for Training

In this section, we calculate the pseudo ground truth and the confidence mask
used in the loss function $\mathcal{L}$ with the help of multi-frame pattern
flow. As depicted in the lower half of Fig. 1, we first detect the light dots
projected from our projector, then we proposed a nearest-search tracking
method for multi-frame pattern flow computation. Finally, we give out the
sparse pseudo ground truth and the confidence mask for loss function based on
the computed pattern flow.

Dots detection. We use the pseudo-random light dot pattern in our structured
light system, as shown in Fig. 2. We denote
$\mathcal{B}=\\{\mathbf{b}_{m}\\}_{m=1:M}$ as the coordinates of dots center
in pre-desigend pattern, and
$\mathcal{C}^{t}=\\{\mathbf{c}^{t}_{n}\\}_{n=1:N^{t}}$ as the captured dots
center in captured image at time $t$. From the captured image of infrared
camera, the center of these dots can be detected through blob detection
algorithm from OpenCV library given a set of proper parameters.

Multi-frame pattern flow calculation. Considering the dot-based pattern, the
multi-frame pattern flow $F^{t}_{\mathbf{b}_{m}}$ can be restricted to find a
index set $i^{t_{0}:t}$, where

$F^{t}_{\mathbf{b}_{m}}=\\{\mathbf{c}^{t_{0}}_{i^{t_{0}}},...\mathbf{c}^{t}_{i^{t}}\\}.$
(7)

In this paper, we first track the movement of these dots in the camera space,
then match the trajectory with pattern dots by a filter-based method. We
denote the function $\psi$ as the nearest matching from a set of coordinates
along the horizontal direction:

$\psi_{\mathcal{C}}(\mathbf{x})=\mathop{\rm
argmin}\limits_{\mathbf{c}\in\mathcal{C}}\left|\rho_{x}(\mathbf{x})-\rho_{x}(\mathbf{c})\right|,\rho_{y}(\mathbf{x})=\rho_{y}(\mathbf{c}).$
(8)

Considering we already have the
$F^{t-1}_{\mathbf{b}_{m}}=\\{\mathbf{c}^{t_{0}:t-1}_{i^{t_{0}:t-1}}\\}$, and
we need to find $\mathbf{c}^{t}_{i^{t}}$ in the latest frame $t$. The
$\mathbf{c}^{t}_{i^{t}}$ can be selected by the nearest matching along the
horizontal direction:

$\mathbf{c}^{t}_{i^{t}}=\psi_{\mathcal{C}^{t}}(\mathbf{c}^{t-1}_{i^{t-1}}).$
(9)

We restrict the candidates to have the same y-axis value, for our system has
been rectified. We also apply the forward-backward check for this nearest
searching to avoid conflict matching. After this nearest matching, we discard
those failure flow, and then build up a new trajectory for every
$\mathbf{c}^{t}_{i}$ that does not have the flow matching. By this incremental
matching, the movement of these dots can be tracked. We denote the estimated
multi-frame pattern flow as $\\{F^{t}_{n}\\}_{n=1:N^{t}}$.

To match the computed trajectory with the pattern dots, we apply the Kalman
Filter for the matching. Since the network can give out dense disparity maps,
we can warp all the $\mathbf{c}^{t}_{i^{t}}$ into the projector space, in
which the warped coordinates should be around one exact dot center
$\mathbf{u}$, as illustrated by Eq. 6. We update the expectation $\mu$ and
variation $\sigma$ for every multi-frame pattern flow, denoted as
$\\{\mu^{t}_{n},\sigma^{t}_{n}\\}_{n=1:N^{t}}$.

Computation of pseudo groundtruth with confidence. The correspondence dot
center $\mathbf{b}_{n^{\prime}}$ in the projector can be found by the updated
$\mu^{t}_{n}$:

$\mathbf{b}_{n^{\prime}}=\psi_{\mathcal{B}}(\mu^{t}_{n}).$ (10)

Then the sparse pseudo ground truth can be computed based on Eq. 6:

$\mathrm{\mathbf{D}}^{t_{k}}(\mathbf{c}^{t_{k}}_{n})_{PGT}=\rho_{x}(\mathbf{c}^{t_{k}}_{n})-\rho_{x}(\mathbf{b}_{n^{\prime}}),\mathbf{c}^{t_{k}}_{n}\in
F^{t}_{n}.$ (11)

The confidence $\mathrm{\mathbf{W}}^{t}_{n}$ of the pseudo ground truth is
computed based on the variation $\sigma^{t}_{n}$:

$\mathrm{\mathbf{W}}^{t}_{n}(\mathbf{c}^{t_{k}}_{n})=S_{n}\exp{-\frac{(\sigma^{t}_{n})^{2}}{\beta}},$
(12)

where we set $\beta$ to 1.0 in our experiments. $S_{n}$ is a weighting
parameter based on the spatial consistency in the camera space and the
projector space. Since we have the correspondence relation
$\mathbf{c}_{n}\leftrightarrow\mathbf{b}_{n^{\prime}},\mathbf{c}_{n}\in
F^{t}_{n}$, we can check every neighbor of dot $\mathbf{b}_{n^{\prime}}$,
denoted as $\mathbf{b}_{m^{\prime}}\in\mathbb{N}(\mathbf{b}_{n^{\prime}})$,
and their correspondence pattern flow in the camera space, denoted as
$F^{t}_{m}={\mathbf{c}^{t_{0}:t}_{m}}$. The $S_{n}$ is defined as:

$S_{n}=\frac{1}{Z(\mathbf{b}_{n^{\prime}})}\sum_{\mathbf{b}_{m^{\prime}}\in\mathbb{N}(\mathbf{b}_{n^{\prime}})}\delta(\mathbf{b}_{m^{\prime}},\mathbf{b}_{n^{\prime}}).$
(13)

$Z(\mathbf{b}_{n^{\prime}})$ denotes the total number of neighbors of
$\mathbf{b}_{n^{\prime}}$. The $\delta$ denotes the valid connection:

$\delta(\mathbf{b}_{m^{\prime}},\mathbf{b}_{n^{\prime}})=\left\\{\begin{split}&1,|~{}|\mathbf{c}^{t}_{n}-\mathbf{c}^{t}_{m}|-|\mathbf{b}_{n^{\prime}}-\mathbf{b}_{m^{\prime}}|~{}|<\epsilon\\\
&0,~{}otherwise\end{split}\right.,$ (14)

where the $\epsilon$ is a constant threshold. Once we establish the
correspondence between the trajectories in the camera space and dots in the
projector space, we check the spatial neighbors based on the pre-designed
pattern. The degree of valid neighbors will influence the final confidence
weight. This spatial smoothness operation can tune down the influence of
sparse outliers and improve the final performance. An example of this spatial
neighbor checking is shown in Fig.3 for better understanding.

##  IV Experiments

In this section, we conduct a comprehensive comparison of our proposed method
with several state-of-the-art techniques, both quantitatively and
qualitatively. Additionally, we present the findings of our ablation studies,
which were performed to evaluate the contribution of each module in our
framework. Through these studies, we can gain a better understanding of the
effectiveness of our approach and how it performs compared to existing
methods.

###  IV-A Experimental Setting

Our network is pretrained on a generated synthetic dataset and then evaluated
on the unseen data. For pretraining, we first generate a virtual dataset
consisting of 2048 sequences, each containing 32 frames. We randomly sample 4
objects from the ShapeNet [24] and apply a random rigid motion sequence to
each object to create a dynamic scene. Infrared images with the projected
pattern are generated using the renderer provided in [3]. The depth range of
the dataset is limited to 0.4m - 0.8m. We train the TIDE-Net model [5] on this
synthetic dataset with the learning rate at $10^{-3}$ with a supervised L1
distance loss. The temporal window size is set to 8, which is the maximum
capacity of our 2080Ti GPU memory.

During the online adapting, we maintain the same training parameters as those
used in the pretraining. We initialize the model parameters to the pretrained
ones at the start of each scene. To minimize the impact of random factors
during online learning, we run the experiments 8 times with different seeds
and report the average accuracy.

Figure 4: Visualization of the ground truth given in [5]. We find that the dataset with ground truth provided in [5] is not precise enough for our evaluation purposes. Upon warping the pattern using the disparity ground truth provided (as indicated by the red dots in the image), we observe displacements on specific parts of the object, leading to inaccurate results. Therefore, we select the non-rigid dataset and proceeded to manually label the sparse disparity ground truth for our experiments. TABLE I: Evaluation results on the three datasets. Original Synthetic | o(1.0) (%) | o(2.0) (%) | o(5.0) (%) | Avg.L1
---|---|---|---|---
CTD [3] | 6.63 | 4.91 | 3.92 | 1.206
ASN [25] | 3.85 | 2.40 | 1.55 | 0.530
TIDE [5] | 2.03 | 1.11 | 0.55 | 0.280
MADNet (Off) | 3.97 | 3.14 | 2.38 | 0.816
MADNet [26] | 3.87 | 3.10 | 2.38 | 0.796
Ours | 2.49 | 1.16 | 0.56 | 0.412
Non-rigid Synthetic | o(1.0) (%) | o(2.0) (%) | o(5.0) (%) | Avg.L1
CTD [3] | 99.46 | 98.84 | 97.08 | 87.799
ASN [25] | 56.57 | 49.30 | 39.58 | 25.883
TIDE [5] | 51.96 | 40.67 | 33.62 | 44.597
MADNet (Off) | 96.16 | 94.78 | 93.00 | 82.905
MADNet [26] | 95.47 | 93.48 | 91.63 | 78.330
Ours | 33.67 | 24.18 | 19.55 | 22.040
Non-rigid Real | o(1.0) (%) | o(2.0) (%) | o(5.0) (%) | Avg.L1
CTD [3] | 90.80 | 83.84 | 73.88 | 23.662
ASN [25] | 56.03 | 45.54 | 37.82 | 16.616
TIDE [5] | 58.13 | 42.05 | 28.86 | 14.157
MADNet (Off) | 92.59 | 88.10 | 81.44 | 61.626
MADNet [26] | 90.56 | 85.47 | 78.71 | 59.412
Ours | 21.48 | 12.80 | 7.09 | 3.405

Figure 5: Evaluated results on the three datasets. We calculate the mesh from
the estimated disparity map for better visualization.

###  IV-B Main Results

We employ multiple datasets for evaluation of our proposed method. Firstly, we
generated a synthetic dataset using the same configuration as that used for
the pretraining. This dataset is akin to the Original Dataset in [5], but with
longer image sequences. Specifically, it comprises 8 sequences with 256 frames
for each. In this paper, We refer to this dataset as Original Synthetic.

Next, we generate another synthetic dataset, referred to as the Non-rigid
Synthetic, to evaluate the model’s adaptation ability to more complex shapes
and motions. To achieve this, we use the depth map and RGB images from four
sequences in the SceneFlow Dataset [27]. We scale the depth maps to match our
experimental settings and calculate the corresponding disparity maps. We
convert all the RGB images into grayscale and use them as the base for
infrared images, then we create the virtual projector to render the final
images.

Finally, we also employ a dataset with non-rigid motion captured in the real
world. This dataset, which is released in [5], contains 4 sequences with 256
images for each. This dataset is captured by two RealSense D435i sensors and
no ground truth is provided. To evaluate our approach, we manually label the
sparse ground truth of the last frame of every sequence. The dot center in the
captured image is detected using the approach presented in Sec. III-D, and the
correspondence to the pattern space is labeled manually. It should be noted
that although a dataset with ground truth is provided in [5], we find that it
is not accurate enough for our evaluation, particularly in terms of details,
as shown in Fig. 4. Therefore, we use the non-rigid dataset and provide the
disparity value manually. We refer to this dataset as Non-rigid Real in this
paper.

In our comparative experiments, we start by comparing our method with several
state-of-the-art techniques in structured light systems that do not involve
any adaptation processing: The connecting-the-dots(CTD) [3],
ActiveStereoNet(ASN) [25] and TIDE network [5]. This is to highlight the
importance and the necessity of online adaptation. We then compare our method
with MADNet [26], an online learning framework for stereo matching, and also
include the results of MADNet without its online adaptation process for
comparison, marked as ”MADNet (Off)”.

We evaluate two types of metrics on every estimated disparity map: 1. The
percentage of pixels with a disparity error larger than
$t\in\\{1.0,2.0,5.0\\}$, denoted as $o(t)$. 2. The average L1 distance of the
disparity for every estimated pixel. We evaluate the aforementioned metrics on
every estimated image in the synthetic datasets. However, for Non-rigid Real,
we only assess these metrics on the final frame of each sequence, where the
sparse disparity ground truth is provided. We calculate the average value of
four metrics, and the results are shown in Tab. I. The reconstructed disparity
maps and the error maps are also shown in Fig. 5. For better visualization, we
convert the disparity map into mesh to illustrate the details of reconstructed
results.

Tab. I demonstrates that all methods can produce promising disparity maps on
the Oringial Synthetic dataset. Since those methods have already converged on
the pretraining dataset, the online adaptation process does not significantly
improve accuracy and, in some cases, results in a slight decrease. However,
all methods experience a varying degree of loss of accuracy when the domain
changes. MADNet and CTD use a U-Net structure with a single-scale correlation
layer, which is highly vulnerable to domain shifting. ASN and TIDE have a 3D
cost volume module and a similar multi-scale correlation structure,
respectively, which makes them more robust to domain shifting. Under such
conditions, the online adaptation process in MADNet cannot improve the
accuracy much on the unseen data due to the drastic drop in accuracy caused by
the network structure. In contrast, our method outperforms all other methods
on the Non-rigid Synthetic and Non-rigid Real datasets, achieving a promising
boost of accuracy compared to the offline version (TIDE).

###  IV-C Ablation Study

To further analyze the effectiveness of our framework, we conduct several
ablation studies based on the Non-rigid Real dataset.

Specifically, we first evaluated the influence of our loss function modules by
ablating several of them and presenting the results in Tab. II, while Fig. 6
visualizes the reconstructed mesh. We found that both the photometric loss and
the direct disparity loss improve the network performance compared to the
total offline version. However, the disparity loss had a greater impact on
improving performance than the classic photometric loss. Additionally, the
confidence mask based on the multi-frame pattern flow was found to be crucial
for the direct disparity loss. Simply applying the direct disparity loss
without any weight mask may lead to a significant decrease in accuracy.

TABLE II: Ablation studies for loss function modules.  Loss function | o(1.0) (%) | o(2.0) (%) | o(5.0) (%) | Avg.L1
---|---|---|---|---
No adaptation | 58.13 | 42.05 | 28.86 | 14.157
$\mathcal{L}_{P}$ | 36.70 | 23.50 | 16.72 | 9.279
$\mathcal{L}_{D}$ | 59.68 | 48.26 | 42.74 | 54.859
$\mathbf{W}\mathcal{L}_{D}$ | 23.96 | 16.65 | 11.33 | 5.373
Ours | 21.48 | 12.80 | 7.09 | 3.405

Figure 6: Estimated disparity with different loss functions.

In addition to the loss functions, we also evaluated the performance of our
online adaptation process with different lengths of image sequences, presented
in Tab.7 and Fig.7. Our results show that the model performance stabilizes
after approximately 20 iterations (160 frames) with our experimental settings.

TABLE III: Ablation studies with different sequence length. Sequence Len. | o(1.0) (%) | o(2.0) (%) | o(5.0) (%) | Avg.L1
---|---|---|---|---
0 (TIDE) | 58.13 | 42.05 | 28.86 | 14.157
32 | 48.70 | 34.12 | 24.79 | 10.511
64 | 37.05 | 24.64 | 16.38 | 6.835
96 | 32.42 | 20.81 | 13.14 | 6.015
128 | 27.73 | 16.93 | 10.15 | 5.170
160 | 24.78 | 14.26 | 7.85 | 4.075
192 | 24.76 | 14.85 | 8.25 | 4.228
224 | 24.48 | 15.05 | 8.52 | 4.626
256 (Ours) | 21.48 | 12.80 | 7.09 | 3.405

Figure 7: Performance of adaptation for different sequence lengths. We
investigate the impact of sequence length on the adaptation performance by
adjusting the start point and comparing the estimated disparity on the same
last frame. The sequence length is varied by changing the number of frames in
the input sequence.

##  V Conclusion

In this paper, we proposed an online learning framework based on the TIDE-Net
architecture to adapt to new incoming data. The main contribution of this
framework is a customized loss function that utilizes the pattern structure in
monocular structured light systems. Specifically, we compute the pseudo-
ground-truth disparity with the help of multi-frame pattern flow, which can be
computed with a filter-based method incrementally. The loss function used for
the online BP process leverages the direct disparity loss and unsupervised
photometric loss through the computed confidence mask. Our method achieves
SOTA accuracy on the unseen data. The code will be available on our project
page.

There are still several limitations to our methods. The multi-frame pattern
flow computation method is quite straightforward for now. Although the nearest
matching assumption can hold in our experiment setting, the matching could be
incorrect when the disparity change is quite large, which is always caused by
fast motion or object boundaries. Besides, the baseline of the device is also
another problem. The pattern movements between frames can be very trivial when
the baseline is too small. In our future work, we may improve the multi-frame
pattern flow estimation method with semi-global optimization to handle the
harder case mentioned above.

## References

  * [1] S. Ryan Fanello, C. Rhemann, V. Tankovich, A. Kowdle, S. Orts Escolano, D. Kim, and S. Izadi, “HyperDepth: Learning depth from structured light without matching,” in _IEEE Conf. Comput. Vision and Pattern Recognit._ , 2016, pp. 5441–5450.
  * [2] S. Ryan Fanello, J. Valentin, C. Rhemann, A. Kowdle, V. Tankovich, D. Philip, and S. Izadi, “UltraStereo: Efficient learning-based matching for active stereo systems,” in _IEEE Conf. Comput. Vision and Pattern Recognit._ , 2017, pp. 6535–6544.
  * [3] G. Riegler, Y. Liao, S. Donné, V. Koltun, and A. Geiger, “Connecting the Dots: Learning representations for active monocular depth estimation,” in _IEEE Conf. Comput. Vision and Pattern Recognit._ , 2019, pp. 7616–7625.
  * [4] L. Tian, J. Liu, H. Ling, and W. Guo, “Disparity estimation in stereo video sequence with adaptive spatiotemporally consistent constraints,” _The Visual Comput._ , vol. 35, no. 10, pp. 1427–1446, 2019.
  * [5] R. Qiao, H. Kawasaki, and H. Zha, “TIDE: Temporally incremental disparity estimation via pattern flow in structured light system,” _IEEE Robotic. and Automat. Lett._ , vol. 7, pp. 5111–5118, 2022.
  * [6] F. Zhang, V. Prisacariu, R. Yang, and P. H. Torr, “GA-Net: Guided aggregation net for end-to-end stereo matching,” in _IEEE Conf. Comput. Vision and Pattern Recognit._ , 2019, pp. 185–194.
  * [7] F. Zhang, X. Qi, R. Yang, V. Prisacariu, B. Wah, and P. Torr, “Domain-invariant stereo matching networks,” in _Eur. Conf. Comput. Vision_ , 2020, pp. 420–439.
  * [8] Intel. (2018) Intel realsense D435i. [Online]. Available: https://www.intelrealsense.com/depth-camera-d435i
  * [9] Microsoft. (2010) Microsoft kinect v1. [Online]. Available: https://www.xbox.com/en-US/kinect
  * [10] L. Zhang, B. Curless, and S. M. Seitz, “Rapid shape acquisition using color structured light and multi-pass dynamic programming,” in _Int. Symposium 3D Data Process. Visualization and Transmiss._ , 2002, pp. 24–36.
  * [11] H. Kawasaki, S. Ono, Y. Horita, Y. Shiba, R. Furukawa, and S. Hiura, “Active one-shot scan for wide depth range using a light field projector based on coded aperture,” in _Int. Conf. Comput. Vision_ , 2015, pp. 3568–3576.
  * [12] R. Furukawa, H. Morinaga, Y. Sanomura, S. Tanaka, S. Yoshida, and H. Kawasaki, “Shape acquisition and registration for 3D endoscope based on grid pattern projection,” in _Eur. Conf. Comput. Vision_ , 2016, pp. 399–415.
  * [13] R. Furukawa, R. Sagawa, and H. Kawasaki, “Depth estimation using structured light flow - analysis of projected pattern flow on an object’s surface,” in _Int. Conf. Comput. Vision_ , 2017, pp. 4650–4658.
  * [14] Y. Taguchi, A. K. Agrawal, and O. Tuzel, “Motion-aware structured light using spatio-temporal decodable patterns,” in _Eur. Conf. Comput. Vision_ , 2012, pp. 832–845.
  * [15] Y. Zhang, Z. Xiong, Z. Yang, and F. Wu, “Real-time scalable depth sensing with hybrid structured light illumination,” _IEEE Trans. Image Process._ , vol. 23, no. 1, pp. 97–109, 2014.
  * [16] M. M. Johari, C. Carta, and F. Fleuret, “DepthInSpace: Exploitation and fusion of multiple video frames for structured-light depth estimation,” in _Int. Conf. Comput. Vision_ , 2021, pp. 6019–6028.
  * [17] A. Tonioni, F. Tosi, M. Poggi, S. Mattoccia, and L. D. Stefano, “Real-time self-adaptive deep stereo,” in _IEEE Conf. Comput. Vision and Pattern Recognit._ , 2019, pp. 195–204.
  * [18] S. Li, X. Wang, Y. Cao, F. Xue, Z. Yan, and H. Zha, “Self-supervised deep visual odometry with online adaptation,” in _IEEE Conf. Comput. Vision and Pattern Recognit._ , 2020, pp. 6338–6347.
  * [19] S. Li, X. Wu, Y. Cao, and H. Zha, “Generalizing to the open world: Deep visual odometry with online adaptation,” in _IEEE Conf. Comput. Vision and Pattern Recognit._ , 2021, pp. 13 179–13 188.
  * [20] W. Li, X. Liu, and Y. Yuan, “SIGMA: Semantic-complete graph matching for domain adaptive object detection,” in _IEEE Conf. Comput. Vision and Pattern Recognit._ , 2022.
  * [21] B. Xie, L. Yuan, S. Li, C. H. Liu, and X. Cheng, “Towards fewer annotations: Active learning via region impurity and prediction uncertainty for domain adaptive semantic segmentation,” in _IEEE Conf. Comput. Vision and Pattern Recognit._ , 2022.
  * [22] R. Liu, C. Yang, W. Sun, X. Wang, and H. Li, “StereoGAN: Bridging synthetic-to-real domain gap by joint optimization of domain translation and stereo matching,” in _IEEE Conf. Comput. Vision and Pattern Recognit._ , 2020, pp. 12 754–12 763.
  * [23] A. Tonioni, O. Rahnama, T. Joy, L. Di Stefano, A. Thalaiyasingam, and P. Torr, “Learning to adapt for stereo,” in _IEEE Conf. Comput. Vision and Pattern Recognit._ , 2019, pp. 9653–9662.
  * [24] A. X. Chang, T. Funkhouser, L. Guibas, P. Hanrahan, Q. Huang, Z. Li, S. Savarese, M. Savva, S. Song, H. Su, J. Xiao, L. Yi, and F. Yu, “Shapenet: An information-rich 3d model repository, Tech. Rep. arXiv:1512.03012 [cs.GR], 2015\.
  * [25] Y. Zhang, S. Khamis, C. Rhemann, J. Valentin, A. Kowdle, V. Tankovich, M. Schoenberg, S. Izadi, T. Funkhouser, and S. Fanello, “ActiveStereoNet: End-to-end self-supervised learning for active stereo systems,” in _Eur. Conf. Comput. Vision_ , 2018, pp. 802–819.
  * [26] A. Tonioni, F. Tosi, M. Poggi, S. Mattoccia, and L. D. Stefano, “Real-time self-adaptive deep stereo,” in _IEEE Conf. Comput. Vision and Pattern Recognit._ , 2019, pp. 195–204.
  * [27] N. Mayer, E. Ilg, P. Hausser, P. Fischer, D. Cremers, A. Dosovitskiy, and T. Brox, “A large dataset to train convolutional networks for disparity, optical flow, and scene flow estimation,” in _IEEE Conf. Comput. Vision and Pattern Recognit._ , 2016, pp. 4040–4048.


# Practical Doppler broadening thermometry

Nicola Agnew<EMAIL_ADDRESS>Department of Physics, SUPA,
University of Strathclyde, 107 Rottenrow E, Glasgow, G4 0NG, Scotland, UK .
Graham Machin  National Physical Laboratory, Hampton Rd, Teddington, TW11 0LW,
UK  Erling Riis  Aidan S. Arnold<EMAIL_ADDRESS>Department of
Physics, SUPA, University of Strathclyde, 107 Rottenrow E, Glasgow, G4 0NG,
Scotland, UK .

###### Abstract

We report initial research to develop a compact and practical primary
thermometer based on Doppler broadening thermometry (DBT). The DBT sensor uses
an intrinsic property of thermalized atoms, namely, the Doppler width of a
spectral line characteristic of the atoms being probed. The DBT sensor, being
founded on a primary thermometry approach, requires no calibration or
reference, and so in principle could achieve reliable long-term _in-situ_
thermodynamic temperature measurement. Here we describe our approach and
report on initial proof-of-concept investigations with alkali metal vapour
cells. Our focus is to develop long-term stable thermometers based on DBT that
can be used to reliably measure temperatures for long periods and in
environments where sensor retrieval for re-calibration is impractical such as
in nuclear waste storage facilities.

##  I  Introduction

The _mise en pratique_ for the definition of the kelvin (MeP-K-19 [1]) has
opened new possibilities for practical primary thermometry as a means of
providing in-situ traceability for temperature – a long-term objective for the
international thermometry community. Currently the overwhelming majority of
thermometry approaches (resistance based thermometers and thermocouples)
require periodic traceable calibration against the International Temperature
Scale of 1990 (ITS-90) [2] to maintain reliable operation – which is often
costly and time consuming. As a result there has been an increasing
international effort in developing practical primary thermometry standards to
be utilised in industry. One approach to providing in-situ temperature
traceability, Johnson Noise Thermometry, is already well advanced [3], whilst
a number of photonic based approaches are under active investigation [4].

Doppler broadening thermometry (DBT) is a primary photonic-based temperature
measurement scheme that capitalizes on the Doppler effect. It was first
demonstrated as a means of determining the Boltzmann constant in 2005 using
NH3 molecules [5]. Many of the subsequent DBT demonstrations [6, 7, 8, 9] have
largely focused on the study of molecular transitions. Although results have
been promising, with uncertainties as low as 24 parts per million achieved [6,
9], this approach encounters significant collisional perturbations and the low
optical cross-sections of the molecules, requiring large path lengths to
obtain a sufficient signal-to-noise ratio. This can hinder scalability and
limit the practical applications of Doppler broadening thermometry.

An alternative approach to DBT, using alkali metal vapours [10, 11, 12],
offers several advantages over molecular species. Alkali atoms possess a
relatively simple atomic structure, such that more complex effects, such as
rotational state-changing collisions, do not need to be accounted for.
Additionally, these atoms exhibit a high optical cross-section, enabling
operation at low pressures with minimal collisional broadening. This attribute
allows for the use of smaller vapour cells, which is desirable for practical
implementation and scalability of DBT techniques. However, DBT using atoms
presents its own set of challenges [10], for example the large optical cross
section leads to optical pumping effects so care must be take to perform
measurements within the weak probe regime. Additionally, the atomic structure
is highly susceptible to external magnetic fields, requiring the use of
magnetic shielding to minimize their impact on measurements.

In this paper, we present our approach and preliminary findings for using
alkali metal vapor cells for DBT. Specifically, we conducted spectroscopic
investigations on the Rubidium (Rb) D2-line at room temperature and examined
the impact of amplified spontaneous emission (ASE) from a laser diode on the
measured absorption spectrum. This initial work is aimed at addressing the
challenge of accurately measuring absorption spectra in the presence of ASE, a
common issue in spectroscopy, via a combination of experimental techniques and
post-measurement processing.

With our proposed methodology, we successfully mitigate, through the use of
appropriate corrections, the adverse effects of ASE on our spectroscopic data.
It should be noted that this work primarily focuses in the absorption of the
spectra rather than the determination of the Doppler width itself. This idea
of absorption thermometry, which underpins our current methodology, relies on
deriving temperature from the measured absorption using the Beer-Lambert law.
Although this method of temperature derivation is dependent on the vapour cell
length and thus not inherently primary in nature, we posit that this technique
can serve as a valuable check on our data when we begin extracting Doppler
broadening thermometry derived temperatures.

Furthermore, we build upon our findings by theoretically investigating the Rb
D2-line and evaluate its potential suitability for temperature measurement. To
achieve this we employ the computational theoretical program ElecSus [13, 14,
15] which models the absorption spectrum of ensembles of atoms. We study the
variation of the optical absorption as a function of temperature and cell
length. Our analysis offers insight into finding the most favourable potential
locking point for obtaining the optimal temperature reading using the
characteristic absorption of the vapour at a given frequency. Our study shows
the effects of temperature and cell length on the absorption spectrum,
providing beneficial information for optimizing the design and operation of
alkali vapor cell-based temperature sensors which give a temperature readout
via a combination of measurement of the absorption co-efficient and the
Doppler width.

##  II  Theory

It is important to model the underlying physics governing the interaction
between the Rb vapour and the laser system. This model takes a given
temperature and vapour cell length as input parameters and generates the
predicted spectroscopy signal as an output. Modeling the underlying physics of
the Rb vapour-laser interaction allows us to obtain a basis for comparison for
our experimentally obtain results. Here the Rb vapour cell is modelled as a
dielectric material with a refractive index $n$, which is probed by a laser
with an angular frequency $\omega$. The atoms’ optical response is determined
by the electric susceptibility $\chi$, which is a function of the laser’s
detuning $\delta$, from the atomic resonance
$\omega_{0},(\delta=\omega-\omega_{0}$). The electric susceptibility comprises
both real and imaginary parts, where the real part characterizes the atoms’
dispersion profile, and the imaginary part characterizes the absorption
profile - related via the Kramers-Kronig relations [16]. The imaginary part of
the susceptibility of the $j^{\textrm{th}}$ transition is calculated such
that:

$\chi(\delta_{j})=\frac{C_{j}^{2}d^{2}N_{a}}{\epsilon_{0}\hbar}f(\delta_{j}),$
(1)

where $d$ denotes the dipole matrix element and $C_{j}$ the relative strength
factor, a quantity which is dependent on the excited and ground state of the
relative hyperfine transition - see [13, 15] for further information on the
calculation of this term. The atomic number density is denoted as
$N_{\textrm{a}}$ and is calculated as follows:

$N_{\textrm{a}}=\frac{F_{\textrm{a}}\,N}{2(2I+1)},$ (2)

where $F_{\textrm{a}}$ is the isotopic abundance ($F_{\textrm{a}}\approx$ 72%
for 85Rb and $F_{\textrm{a}}\approx$ 28% for 87Rb) [17, 18], $I$ denotes the
nuclear spin quantum number ($I=5/2$ for 85Rb and $I=3/2$ for 87Rb) and $N$
the elemental number density.

The relation above is valid under the assumption that the temperature of the
atoms is low and the probe beam is weak such that the atoms are evenly
distributed among the ground state manifold. The elemental number density can
calculated as $P/kT$, wherein $P$ denotes the pressure, $T$ the temperature
and $k$ the Boltzmann constant. The pressure is derived from the vapour-
pressure model for Rb [17, 18] which is known to give an accuracy of better
than 5% [17]. With adequate temperature calibration, our results may lead to
an improved vapor-pressure model.

The line-shape $f(\delta_{j})$ of the imaginary susceptibility shown in Eq.
(1) is Lorentzian. The atoms experience a spontaneous decay at a rate
characterized by $\Gamma$ from excited state to ground state as a consequence
of the Heisenberg uncertainty principle. This leads to a natural broadening of
the transition, given by the function:

$f(\delta_{j})=\frac{i}{\Gamma/2-i\delta_{j}}.$ (3)

In addition to the homogeneous natural broadening in Eq. (3), the thermal
motion of atoms in the vapor induces a frequency shift of the interacting
laser light proportional to their velocity, leading to inhomogeneous
broadening. This phenomenon, known as Doppler broadening, gives rise to a
Gaussian line-shape in the frequency domain. The absorption profile of atoms
in the Doppler regime is commonly modeled using a Voigt profile, which is a
convolution of a Gaussian and a Lorentzian line-shape. This can be calculated
rigorously by integrating over $f(\delta_{j}-kv)$ convoluted with a Maxwell-
Boltzmann distribution such that we obtain:

$V(\delta_{j})=\frac{C_{j}^{2}d^{2}N_{a}}{\epsilon_{0}\hbar\sqrt{\pi}u}\int_{-\infty}^{\infty}f(\delta_{j}-kv)e^{-v^{2}/u^{2}}dv,$
(4)

where the speed $u$ is $\sqrt{2kT/m}$ for an atom of mass $m$ with temperature
$T$. Rather than employing integration, the Voigt profile can be approximated
using the complex error function [19]. This method decreases the computational
time required for modeling the absorption spectrum, but at the cost of less
reliable lineshape fitting.

##  III  Data Acquisition and processing

The experimental setup utilized to acquire the spectroscopy data is
illustrated in Fig. 1 (a). The setup consists of an external cavity diode
laser (ECDL) [20] that is frequency scanned across the Rb D2-line at
$780\,$nm. The laser output is directed through an iris of diameter $1.8\,$mm
to reduce the ASE. A halfwave plate and polarizing beam splitter are employed
to control the laser output power. The beam is then divided using a non-
polarizing beam splitter, with one output serving as the reference signal with
the other sent into a $75\,$mm Rb vapour cell, where it is detected using a
silicon-based photodiode (PD). The laser power into the vapour cell is set
below $1\,$W to minimise power broadening and optical pumping effects, which
have not been considered in the theoretical model [13, 14]. In this experiment
the Rb cell temperature is assumed to be that of the room, whose temperature
is recorded. However, thermal gradients within the laboratory contribute to
higher uncertainty in this measurement. In future work direct measurement of
the cell’s temperature will be performed, and in addition temperature control
of the cell will be implemented to test the limits of the spectroscopy data
ability to derive temperature in both warmer and colder environments.

Figure 1:  (a) Experimental setup for DBT. ECDL (external cavity diode laser),
OI (optical isolator), PBS (polarizing beam splitter), NPBS (non-polarizing
beam splitter) PD (photodiode). (b) Raw data collected from the reference
photodiode (red) and signal PD (black).

After acquiring spectroscopy data at room temperature, a follow-up experiment
was conducted to measure the transmission of the Rb cell when heated with a
heat gun to a high optical density. The heating was carried out until the
spectroscopy absorption signal became saturated, as evidenced by a flat
transmission floor, as shown in Fig. 2. In this case the vapour cell is heated
to an arbitrary temperature to illustrate proof of principle of the technique.
The objective was to quantify the level to which the ASE from the laser diode,
as well as stray light and current noise from the PD, contributed to an offset
in transmission signal [21]. In their absence, the floor of the signal lies at
0% transmission, indicating that all resonant laser light is absorbed by the
atoms. The ASE produced by the laser, which is off resonance, passes through
the cell to the PD, leading to an apparent increase in the transmission floor.
The ASE is quantified by taking the mean value of a linear fit across the
signal floor. Quantifying the contribution of ASE in the data is critical as
it affects the signal’s amplitude and should be suppressed as much as
possible. To reduce ASE, we incorporated a frequency-dependent optical filter
with a high degree of out-of-band blocking into the setup.

The spectral data is acquired and then processed in MATLAB. Initially, the
mean of ten successive points in the spectral data is calculated to minimize
the presence of noise in the data. Following this, the power of the laser is
computed by determining the mean voltage of the reference signals and scaling
it by a factor of 0.7 to align it with the spectroscopy PD signal - as
illustrated in Fig. 1 (b). The average current flowing through the PD was
subsequently determined using the PD’s resistance, and the power was
calculated using the frequency-dependent responsivity value specified in the
PD data sheet.

Subsequently, the transmission axis of the data was defined in the range of 0
to 1 through normalization. However, as demonstrated in Fig. 1 (b), the
maximum transmission is nonlinear [21]. Although this non-linearity can be
partially corrected by dividing the spectroscopy signal by the reference
signal before normalization, the overlap of these signals is not perfect, and
some nonlinearity persists, this is suspected to be a result polarisation
dependent optics within the setup. A secondary correction method for power
fluctuations in the observed data involves fitting a polynomial to points on
the reference scan transmission and normalising. This works best for data with
a long scan range.

To calibrate the time of the scan into frequency units, the time difference of
two of the maximum absorption points in the data is quantified then scaled
with the theoretically obtained counterpart. To aid this, a parabolic fit is
applied to the bottom 2% of the absorption peaks and its minimum calculated to
avoid data noise giving a false minimum.

Upon completion of normalization, it is necessary to eliminate the ASE from
the data. This is achieved by subtracting the transmission values at the floor
of the warm cell data from those of the room temperature data.

Discrepancies in the frequency domain arise between the experimental data and
theory because the frequency sweep exhibits non-linear behaviour whereas the
theoretical framework assumes a linear scanning process. To address this, we
apply a non-linear polynomial mapping technique to correct the theory.
Specifically we utilize the four Doppler absorption peaks observed in the
experimental data as known values, and determine the unknown parameters by
solving a system of four simultaneous equations expressed in the form:

$f(\delta_{\textrm{\,t}})=A+B\delta_{\textrm{e}}+C{\delta_{\textrm{e}}}^{2}+D{\delta_{\textrm{e}}}^{3},$
(5)

where $A$, $B$, $C$ and $D$ are our unknown values and the subscript ‘e’
refers to experimental data, and ‘t’ denotes theoretical values.

##  IV Experimental Results

Figure 2:  Warm cell measurements of the Rb D2 line, the black curve shows the
ASE for when no filter is used, the red curve shows the ASE levels when
employing a 10nm BW filter and the blue curve the ASE with a 1.2nm BW filter.

The outcomes of the warm cell measurements of the Rb D2 line are presented in
Fig. 2. The spectrum was captured under three different conditions: without
any filter; utilizing a $10\,$nm bandwidth (BW) band-pass filter at $780\,$nm;
and using a $1.2\,$nm bandwidth filter at $780\,$nm. In the absence of a
filter the average ASE level of the ECDL was observed to be $(4.0\pm 0.3)\,$%.
When a $10\,$nm BW filter was utilized, the ASE level decreased to $(1.1\pm
0.1)\,$%, and with the implementation of a $1.2\,$nm BW filter, the ASE level
further reduces to $(0.08\pm 0.02)\,$% – a 50-fold improvement. Suppression of
the ASE is of utmost importance for accurate temperature measurement. The
presence of significant ASE levels can introduce a deceptive reduction in the
transmission of the spectra. Consequently, when employing ASE sensitive
techniques for temperature measurement – such as absorption thermometry – high
ASE levels can yield erroneous readings that underestimate the actual
temperature. Thus it is crucial to minimise ASE to maintain the integrity and
reliability of the temperature measurement and minimise the the respective
uncertainty budget.

The resulting experimentally obtained spectra and residuals for the Rb D2 line
at room temperature ($297.2\,$K) are show in Fig. 3. We observe a
significantly improved agreement between the experimental results and the
theoretical prediction when a narrowband filter with a BW of $1.2\,$nm is
employed to mitigate the effects of ASE. By applying the post processing
techniques outlined previously to remove what remains of the ASE in the data
obtained with the narrowband filter, a scan root-mean-square (RMS) error of
$0.33\,$% was calculated. For the data obtained without the filter, and ASE of
$\approx 4\,$% that was neglected in the subsequent data analysis, we obtained
an RMS error of $1.29\,$%. This fourfold improvement in the agreement with
theoretical predictions underscores the efficiency of both characterising the
ASE and in employing the narrowband filter to reduce the detrimental impact of
ASE on the measurement.

Figure 3:  (a) The absorption spectrum of the Rb D2 line at room temperature
of approximately $297\,$K for a cell length of $75\,$mm. The theoretically
calculated spectrum for these values is shown in black. The spectrum measured
experimentally using a $1.2\,$nm BW filter at $790\,$nm is shown in blue and
without the filter in red, The residuals of both data sets are shown in (b)

The RMS error of the experimentally obtained data was then calculated against
a range of theoretically obtained data sets at varying temperatures to find at
which temperature the RMS error is lowest. We define the difference between
the theoretically calculated temperature at which the RMS error is lowest and
the measured temperature of the room as the temperature error. It is found
that for the data taken without the filter the RMS error is the lowest
($0.24\,$%) at a temperature $296.2\,$K, corresponding to a temperature error
of $1\,$K. Experimental data taken with the $1.2\,$nm BW filter has the best
agreement with theoretical data at $297.0\,$K, corresponding to a RMS error of
$0.27\,$% and a temperature error of $0.2\,$K. Overall these results show that
the ASE from the laser system being implemented can significantly affect the
accuracy of the observed spectral features. Furthermore, characterising the
ASE and using the appropriate filtering techniques can minimise the
detrimental impact of ASE.

Figure 4:  The theoretically calculated absorption spectrum of the Rb D2 line
is shown in (a) for a range of temperatures at a cell length of $75\,$mm. The
change in percentage transmission at the detuning values marked by vertical
lines in (a) per kelvin of temperature change for a range of temperatures and
cell lengths in displayed in (b). The detuning value that exhibits that
largest change in transmission at a given temperature and cell length of the
detuning values marked in (a) is shown in (c).

##  V Modelling Results

The absorption spectra of the Rb and Cs D1 and D2 lines were theoretically
modeled to investigate the percentage change in absorption per kelvin of
temperature variation. The temperature range examined was from $273\,$K to
$473\,$K, and the cell lengths ranged from 1 mm to 100 mm. The analysis
focused on determining the percentage change in the absorption at the maximum
of the absorption peaks as well as at the points of minimum absorption between
each peak for each cell length, these points have been marked in Fig. 4 (a)
for the Rb D2 spectrum for clarity. The absorption peaks saturate at elevated
temperatures, which can be attributed to the increase in optical depth due to
increased vapour pressure. At these saturation points, determination of the
temperature becomes unfeasible. Nevertheless, in principle, the laser can be
locked to any desired frequency, and points of minimum absorption could be
utilized as an alternative temperature indicator at higher temperature
regimes.

The percentage change in absorption per kelvin temperature change is shown in
Fig. 4 (b) and the corresponding potential locking point (out of those marked
on Fig. 4 (a)) demonstrating the highest sensitivity to temperature variation,
is presented in Fig. 4 (c). Our analysis indicates that in higher temperature
regimes beyond $375\,$K, locking to an absorption minima yields a higher
sensitivity to temperature fluctuation. In fact, the utilization of absorption
minima can produce absorption changes exceeding $2\,\%/$K within this
temperature range where the absorption maxima are saturated. However, when
examining Fig. 4 (b), a region of reduced sensitivity to temperature change is
apparent at approximately $400\,$K for both the absorption maxima and minima.
Note that this analysis is based on only six detuning values, and further
investigation into alternative areas of the spectrum may yield more fruitful
sensitivities within this range.

In future work, we will conduct a more comprehensive analysis, encompassing a
wider range of transitions, including the Rb D1 line, as well as the D1 and D2
lines of other alkali metal species. This extended investigation aims to
provide valuable insights into the most suitable atomic species for employing
the proposed thermometry technique. Such findings will facilitate informed
decision-making for future iterations of experimental testing. Furthermore, we
intend to extend our analysis to include the investigation of the change in
Doppler width per kelvin of temperature fluctuation for the selected D-lines.
This analysis will serve as an important indicator for determining the most
suitable atomic species for practical Doppler broadening thermometry
applications. At present, our model does not encompass the effects of magnetic
fields as well as collisional broadening and Dicke narrowing, which are
expected to become increasingly significant at higher temperatures. Therefore,
it becomes imperative to experimentally characterize these effects on the
chosen atomic species. By incorporating these additional factors into our
investigations, we can obtain a more comprehensive understanding of the
behaviour and the limitations of the chosen species in our application.

##  VI Conclusion

In conclusion, this study presents our initial findings in the development of
practical Doppler broadening thermometry techniques. We have demonstrated the
feasibility of deriving temperature readings from measurement of atomic
absorption using Rb. Furthermore, we have identified the significant challenge
to the fidelity of such measurements posed by the ASE originating from the
laser diode during the spectroscopic analysis.

Through the employment of narrow BW filters in the experimental setup and the
characterization of the ASE using an optically dense vapour cell, as well as
post processing techniques, we have effectively mitigated the adverse impact
of the ASE. Our methodology has resulted in a more than 50-fold reduction of
the ASE from $4.0\,$% to $0.08\,$%. Moreover, we have achieved an improvement
in the theoretical agreement of the absorption spectra at room temperature,
reducing the RMS error from $1.29\,$% to an enhanced value of $0.33\,$%.

For the purpose of open access, the author(s) has applied a Creative Commons
Attribution (CC BY) licence to any Author Accepted Manuscript (AAM) version
arising from this submission.

## References

  * Fellmuth  _et al._ [2016] B. Fellmuth, J. Fischer, G. Machin, S. Picard, P. P. M. Steur, O. Tamura, D. R. White, and H. Yoon, “The kelvin redefinition and its _mise en pratique_ ,” Philos. Trans. Royal Soc. A 374, 20150037 (2016).
  * Preston-Thomas [1990] H. Preston-Thomas, “The international temperature scale of 1990 (ITS-90),” Metrologia 27, 3–10 (1990).
  * Bramley, Cruickshank, and Aubrey [2020] P. Bramley, D. Cruickshank, and J. Aubrey, “Developments towards an industrial johnson noise thermometer,” Meas. Sci. Technol 31, 054003 (2020).
  * Dedyulin, Ahmed, and Machin [2022] S. Dedyulin, Z. Ahmed, and G. Machin, “Emerging technologies in the field of thermometry,” Meas. Sci. Technol 33, 092001 (2022).
  * Daussy  _et al._ [2005] C. Daussy, S. Briaudeau, M. Guinet, A. Amy-Klein, Y. Hermier, C. J. Bordé, and C. Chardonnet, “Spectroscopic determination of the Boltzmann constant:. first results,” in  _Laser Spectroscopy_ (2005) pp. 104–111.
  * Moretti  _et al._ [2013] L. Moretti, A. Castrillo, E. Fasci, M. D. De Vizia, G. Casa, G. Galzerano, A. Merlone, P. Laporta, and L. Gianfrani, “Determination of the boltzmann constant by means of precision measurements of ${{\mathrm{H}}_{2}}^{18}\mathrm{O}$ Line Shapes at $1.39\text{ }\text{ }\mu\textrm{m}$,” Phys. Rev. Lett. 111, 060803 (2013).
  * Gianfrani [2012] L. Gianfrani, “Highly-accurate line shape studies in the near-IR spectrum of $\textrm{H}_{2}\,^{18}\textrm{O}$: Implications for the spectroscopic determination of the Boltzmann constant,” J. Phys. Conf. Ser. 397, 012029 (2012).
  * Gianfrani [2016] L. Gianfrani, “Linking the thermodynamic temperature to an optical frequency: recent advances in Doppler broadening thermometry,” Philos. Trans. Royal Soc. A 374, 20150047 (2016).
  * Gotti  _et al._ [2018] R. Gotti, L. Moretti, D. Gatti, A. Castrillo, G. Galzerano, P. Laporta, L. Gianfrani, and M. Marangoni, “Cavity-ring-down Doppler-broadening primary thermometry,” Phys. Rev. A 97, 012512 (2018).
  * Truong  _et al._ [2011] G.-W. Truong, E. F. May, T. M. Stace, and A. N. Luiten, “Quantitative atomic spectroscopy for primary thermometry,” Phys. Rev. A 83, 033805 (2011).
  * Truong  _et al._ [2015] G.-W. Truong, D. Stuart, J. D. Anstie, E. F. May, T. M. Stace, and A. N. Luiten, “Atomic spectroscopy for primary thermometry,” Metrologia 52, S324–S342 (2015).
  * Pan  _et al._ [2019] Y. Pan, W. Liao, H. Wang, Y. Yao, J. Cai, and J. Qu, “Cesium atomic Doppler broadening thermometry for room temperature measurement,” Chin. Opt. Lett. 17, 060201 (2019).
  * Zentile  _et al._ [2015] M. A. Zentile, J. Keaveney, L. Weller, D. J. Whiting, C. S. Adams, and I. G. Hughes, “ElecSus: A program to calculate the electric susceptibility of an atomic ensemble,” Comput. Phys. Commun. 189, 162–174 (2015).
  * Keaveney, Adams, and Hughes [2018] J. Keaveney, C. S. Adams, and I. G. Hughes, “ElecSus: Extension to arbitrary geometry magneto-optics,” Comput. Phys. Commun. 224, 311–324 (2018).
  * Siddons  _et al._ [2008] P. Siddons, C. S. Adams, C. Ge, and I. G. Hughes, “Absolute absorption on rubidium D lines: comparison between theory and experiment,” J. Phys. B 41, 155004 (2008).
  * de L. Kronig [1926] R. de L. Kronig, “On the theory of dispersion of x-rays,” J. Opt. Soc. Am. A 12, 547 (1926).
  * Steck [2021a] D. A. Steck, “Rubidium 85 D line data,”  (2021a).
  * Steck [2021b] D. A. Steck, “Rubidium 87 D line data,”  (2021b).
  * Schreier [1992] F. Schreier, “The Voigt and complex error function: A comparison of computational methods,” J. Quant. Spectrosc. Radiat. Transf. 48, 743–762 (1992).
  * Daffurn, Offer, and Arnold [2021] A. Daffurn, R. F. Offer, and A. S. Arnold, “A simple, powerful diode laser system for atomic physics,” Appl. Opt. 60, 5832 (2021).
  * Pizzey  _et al._ [2022] D. Pizzey, J. D. Briscoe, F. D. Logue, F. S. Ponciano-Ojeda, S. A. Wrathmall, and I. G. Hughes, “Laser spectroscopy of hot atomic vapours: from ’scope to theoretical fit,” New J. Phys. 24, 125001 (2022).