# MoD2T:Model-Data-Driven Motion-Static Object Tracking Method

Yang Feng, Liao Pan, Wu Di, Liu Bo, Zhang Xingle  Yang Feng, Liao Pan , Wu Di,
Liu Bo, and Zhang Xingle are with Key Laboratory of Information Fusion, School
of Automation, Northwestern Polytechnical University,Xi’an,China. (e-mile:
yangfeng@nwpu.edu.cn,liaopan@mail.nwpu.edu.cn,wu_di821@mail. nwpu.edu.cn,
<EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

The domain of Multi-Object Tracking (MOT) is of paramount significance within
the realm of video analysis. However, both traditional methodologies and deep
learning-based approaches within this domain exhibit inherent limitations.
Deep learning methods driven exclusively by data exhibit challenges in
accurately discerning the motion states of objects, while traditional methods
relying on comprehensive mathematical models may suffer from suboptimal
tracking precision. To address these challenges, we introduce the Model-Data-
Driven Motion-Static Object Tracking Method (MoD2T). We propose a novel
architecture that adeptly amalgamates traditional mathematical modeling with
deep learning-based MOT frameworks, thereby effectively mitigating the
limitations associated with sole reliance on established methodologies or
advanced deep learning techniques. MoD2T’s fusion of mathematical modeling and
deep learning augments the precision of object motion determination,
consequently enhancing tracking accuracy. Our empirical experiments robustly
substantiate MoD2T’s efficacy across a diverse array of scenarios, including
UAV aerial surveillance and street-level tracking. To assess MoD2T’s
proficiency in discerning object motion states, we introduce MVF1 metric. This
novel performance metric is designed to measure the accuracy of motion state
classification, providing a comprehensive evaluation of MoD2T’s performance.
Meticulous experiments substantiate the rationale behind MVF1’s formulation.
To provide a comprehensive assessment of MoD2T’s performance, we meticulously
annotate diverse datasets and subject MoD2T to rigorous testing. The achieved
MVF1 scores, which measure the accuracy of motion state classification, are
particularly noteworthy in scenarios marked by minimal or mild camera motion,
with values of 0.774 on the KITTI dataset, 0.521 on MOT17, and 0.827 on UAVDT.

###### Index Terms:

Multi-object tracking, Motion detection, Fusion

##  I Introduction

MULTI-OBJECT tracking (MOT) stands as a pivotal cornerstone in the realm of
video analysis, capturing the dedicated attention of scholars and researchers
alike. The inception of MOT was bound by the limitations inherent in detection
techniques, prompting initial pursuits that encompassed methodologies like
image differencing, Kalman filtering, and the Hungarian algorithm, laying the
foundational bedrock of tracking frameworks[1]. Progressing forward, the
evolution witnessed the emergence of motion detection techniques, inclusive of
the application of frame differencing. These methods, underpinned by
statistical models exploiting pixel or feature-based statistical properties,
provided the groundwork for models detecting anomalies or changes.
Furthermore, statistical model-centric strategies materialized, exemplified by
Gaussian mixture models [2] and adaptive background modeling [3]. It remains
conspicuous that traditional non-deep learning methods, fixated solely on
tracking moving objects, often yield suboptimal outcomes across diverse
scenarios. This shortcoming, attributed to the intricate nature of image
complexities, inevitably curtails the precision of these conventional
methodologies.

In the ever-evolving landscape of computer vision, a profound transformation
has unfolded, primarily steered by the pervasive embrace of deep learning.
This transformative wave finds its embodiment in two prominent methodologies:
[4, 5, 6] and joint-detection-association[7, 8, 9, 10]. Tracking-by-detection
intricately harnesses the power of deep learning models for precise object
detection, seamlessly integrating them with sophisticated techniques like
Kalman filtering to pave the way for seamless object tracking. On the flip
side, joint-detection-association unfolds as an end-to-end integrated tracking
network. The deep learning paradigm brings a tapestry of advantages to the
forefront in the realm of MOT. It endows the system with unparalleled
capabilities for feature learning and representation, allowing for a nuanced
understanding of the semantic intricacies of tracked objects.

Deep learning’s forte lies in its adeptness at modeling context effectively,
enabling a holistic comprehension of the interplay between tracked objects and
their surroundings[11]. The scalability of deep learning models in handling
vast amounts of data adds another layer of strength, a crucial facet given the
large-scale nature of MOT tasks. Furthermore, the principle of transfer
learning accelerates the adaptation of models from one task to another, a
vital asset in the ever-evolving MOT landscape.

Adaptability emerges as a hallmark of deep learning, particularly in
navigating the complexities of dynamic scenes marked by occlusions and
intricate motion patterns. This innovative era has not only witnessed the
ascendancy of tracking-by-detection but has also translated into remarkable
successes across a diverse spectrum of applications[12, 13, 14].

However, these neural network-based methods heavily rely on object appearance
features during implementation. Consequently, they often track all objects
within an image, deviating from traditional methods that specifically target
moving objects. Environments such as vehicle monitoring, battlefield
operations, and autonomous driving typically require a focus on moving
objects. In these scenarios, there is a need for information regarding the
motion of objects relative to the ground, a challenge current methods struggle
to address. Furthermore, as many scenarios utilize single RGB cameras rather
than stereo or RGB-D cameras, discerning object motion becomes even more
challenging. Addressing this issue, this paper introduces a straightforward
yet practical approach for determining object motion, adaptable across various
environments.

Traditional non-deep learning methods adeptly track moving targets, harnessing
the physical and mathematical facets of images and relying on comprehensive
mathematical models. While they might exhibit reduced accuracy in select
cases, their efficacy extends across almost any environment. In contrast, deep
learning-driven MOT frameworks pivot on data-driven principles. Though their
precision is notable, they may falter when confronted with limited or
inadequate data, leading to inaccuracies in tracking object motion estimation.

To transcend the limitations inherent in both deep learning algorithms and
traditional methods, we present a tracking approach that melds conventional
mathematical modeling with data-driven deep learning methodologies. In
essence, this paper unfolds with three core contributions: 1) proposing
metric, MVF1, for evaluating the model’s ability to determine the relative
motion status of objects with respect to the ground. Experimental validation
of MVF1 is conducted, providing evidence for its rationality; 2) introducing a
plug-and-play method for discerning the motion attributes of tracked objects;
and 3) proposing a model-data-driven tracking framework(MoD2T) that harmonizes
traditional methods with MOT tracking frameworks, enabling the delineation of
object motion states.

##  II RELATED WORK

###  II-A Motion Objects Detection Methods

Traditional MOT methods primarily focus on identifying moving targets, and the
subsequent tracking process is similar to the SORT[4]. In the early stages,
image change detection methods were used for this purpose.

The development of image change detection methods can be traced back several
decades. The earliest methods were based on pixel value comparison to detect
changes [15, 1, 16]. These methods were simple and intuitive but sensitive to
complex scenes and noise.

Subsequently, methods based on pixel-level differences received extensive
research attention. These methods use pixel-level statistics or thresholds to
detect changes, such as pixel ratio [17] and statistical parameters [18].
While these methods are robust to noise, they still struggle with complex
textures and lighting variations [19].

Before the surge of deep learning, various Gaussian background model were the
state-of-the-art in this field. Specifically, for a background image, the
grayscale values of specific pixel points in the temporal domain follow a
Gaussian distribution. In other words, for a background image $B$, the
grayscale value $x$ of each point $(u,v)$ satisfies $I(x)\sim
N\left(\mu,\sigma^{2}\right)$ as shown in (1):

$I(x)=\frac{1}{\sqrt{2\pi}\sigma}e^{-\frac{(x-\mu)^{2}}{2\sigma^{2}}}$ (1)

In the process of extracting targets, the mean $\mu$ and variance $\sigma$ of
each point in the image sequence over a period of time are calculated to
create the background model. Then, for any image that contains foreground
objects, each point is evaluated based on (2):

$\frac{1}{\sqrt{2\pi}\sigma}e^{-\frac{(IG(x,y)-IB(x,y))^{2}}{2\sigma^{2}}}>T$
(2)

If the condition is met, the point is considered a background point;
otherwise, it is considered a foreground point. Here, $T$ is a constant
threshold representing the grayscale value at a given point in image $IG$, and
$IB$ is similar in nature.

In the field of image processing, where deep learning methods dominate, motion
detection methods primarily adopt sparse feature tracking[20, 21, 22] or dense
optical flow [23, 24, 25]. The former is widely used in SLAM, while the latter
is commonly applied for overall motion estimation in images. Although each
method has been proven effective for its respective applications, neither can
fully model the motion in videos: optical flow cannot capture long-term motion
trajectories, and sparse tracking cannot model the motion of all pixels.
Recently, someone proposed a method combining the Optical flow with some other
modes [26], and achieved good results.

However, these methods still have certain limitations. Sparse feature tracking
involves random selection of feature points, which may result in tracking
objects with too few or no features, or unnecessary computational overhead if
too many features are selected. The limitations of optical flow methods based
on CNN [27, 28] or generative adversarial networks (GAN)[29, 30] include
current challenges regarding their accuracy and universality. Traditional
optical flow methods, on the other hand, exhibit high tracking accuracy in
current MOT tracking frameworks.

Of course, in recent years, several models based on deep learning for
detecting changes in images have been proposed[31, 32, 33]. These methods have
been widely used in fields such as remote sensing and change detection.
However, due to the fact that their working principle relies on static image
comparison, these methods are no longer suitable for motion object detection.

###  II-B MOT Methods

The developmental trajectory of MOT methodologies has been expounded upon in
the preceding text and will not be extensively reiterated here.

Currently, the available MOT approaches capable of distinguishing object
motion exhibit limited proliferation. A significant portion of these
approaches do not solely rely on RGB image information, but rather necessitate
supplementary sensing modalities such as RGB-D cameras[34] or various
alternative sensors like LIDAR[20].

The conventional MOT approach employed within this paoer shall be elucidated
in the appendix. As for the deep learning-driven MOT method, this paper
intends to embrace the tracking-by-detection paradigm.

For the object detection stage, YOLOv8[35] will be used. YOLOv8 is a fast and
accurate object detection method that divides the image into grids and
predicts bounding boxes and their corresponding classes for object detection.

For the subsequent tracking stage, StrongSORT [36] will be employed.
StrongSORT is a deep learning-based object tracking method that combines
appearance features and motion information to achieve accurate multiple object
tracking.

The tracking framework, YOLOv8+StrongSORT, combines the YOLOv8 object detector
with the StrongSORT tracker, resulting in powerful tracking performance.
YOLOv8 can rapidly and accurately detect targets in video frames and provide
bounding box locations and class information for each detection. StrongSORT
utilizes these detection results and combines them with the appearance
features and motion information of the targets. It performs data association
and trajectory prediction for target tracking and re-identification.

Additionally, the paper has also experimented with other tracking methods,
such as YOLOv8+DeepOCSORT[6], to explore multiple tracking approaches. Our
method can be integrated with different MOT methods, and this article only
selected two.

Figure 1: Flow chart of motion judgment. The background feature extraction
process is shown in Fig 2. And, in this paper, The frame rate difference $n$
for calculating distance and background similarity is set to 3. The specific
meaning of the formula and the specific meanings of each parameter can be
found in section III. The main purpose of this process is to calculate two
indicators, one is the distance of the object’s movement in the image, and the
other is the similarity of the object’s background in the image. Finally,
these two indicators are combined to determine whether the object is moving
relative to the ground.

###  II-C Fusion Method

The fusion of tracking methods involves combining multiple individual tracking
methods or models to improve tracking performance or address various
challenges. This approach can leverage the strengths of different methods,
mitigate the limitations of individual methods, and provide more robust and
accurate tracking results.

One common method of tracking method fusion is decision-level fusion [37, 38,
39]. This method determines the final tracking result by weighting or voting
the outputs of multiple trackers. The weights of each tracker can be
dynamically adjusted based on its tracking performance in previous frames,
giving greater weight to trackers with better performance.

Another common fusion method is feature-level fusion[40, 41]. In this
approach, multiple trackers extract feature representations of the target and
fuse these features. Fusion can be as simple as averaging or weighted
averaging, or it can involve more complex model fusion methods such as
convolutional neural networks or support vector machines.

Additionally, there are information fusion-based methods[42, 43] that involve
fusing motion information, contextual information, or external prior
knowledge. These methods improve tracking performance by combining information
from multiple sources. For example, the results of multiple trackers can be
fused by incorporating the target’s motion model, scene context, or appearance
model.

Overall, the fusion of tracking methods offers a way to enhance tracking
performance by integrating multiple methods or models, leveraging their
strengths, and taking advantage of different sources of information.

##  III Motion Judgment

Figure 2: Background feature extraction process. In the context of the feature
extraction network, the background images are typically resized to dimensions
of 256x192 pixels. Backgrounds from all four orientations are resized to the
same dimensions. However, if any side of the background image is within 15
pixels of the image boundary, the portion of the background image close to
that boundary is discarded.

In this section, we present a straightforward yet highly effective approach
for ascertaining the motion status of tracked objects. Our method is designed
to precisely determine whether an object is in motion or stationary throughout
the tracking procedure.

###  III-A Moving Distance

MOT methods are employed to discern the motion trajectories of objects within
video sequences. In scenarios involving camera motion, the application of the
enhanced correlation coefficient maximization(ECC) [44] model enables robust
camera motion correction, allowing us to reliably ascertain the relative
inter-frame displacement of objects. This computed displacement serves as a
foundational metric for characterizing object motion.

We employ the Mahalanobis distance to calculate the motion distance $D$, as
shown in (1):

$D=\sqrt{(\mathbf{c}_{1}-\mathbf{c}_{2})^{T}\mathbf{C}^{-1}(\mathbf{c}_{1}-\mathbf{c}_{2})}$
(3)

where

$\displaystyle\mathbf{c}_{1}$
$\displaystyle=\left[\frac{x_{1}+\frac{w_{1}}{2}}{2},\frac{y_{1}+\frac{h_{1}}{2}}{2}\right]$
(4) $\displaystyle\mathbf{c}_{2}$
$\displaystyle=\left[\frac{x_{2}+\frac{w_{2}}{2}}{2},\frac{y_{2}+\frac{h_{2}}{2}}{2}\right]$
(5) $\mathbf{C}=\begin{bmatrix}\frac{w_{1}^{2}}{4}&0\\\
0&\frac{h_{1}^{2}}{4}\end{bmatrix}$ (6)

Here, $\mathbf{c}_{1}$ and $\mathbf{c}_{2}$ represent the center coordinates
of the two bounding boxes, and $(x_{1},y_{1},w_{1},h_{1})$ and
$(x_{2},y_{2},w_{2},h_{2})$ denote the coordinates and dimensions of the two
bounding boxes.

The rationale behind selecting Mahalanobis distance lies in its ability to
account for the covariance structure within the data. This becomes
particularly crucial when dealing with multidimensional data, as it considers
both the variances and covariances among the different dimensions. Notably, we
conducted experiments with alternative distance metrics, and Mahalanobis
distance demonstrated superior performance in capturing the underlying data
structure, especially in handling the intricacies of motion patterns
represented by the bounding boxes. This choice significantly contributes to
the accuracy and reliability of our motion distance calculations, ensuring
robustness in the face of varying data distributions and dimensions.

###  III-B Background Feature

We performed feature extraction on the background of the tracked object. The
specific approach involved segmenting the surrounding background and then
combining it. We utilized a pre-trained OSNet [45] to extract appearance
features from this background. The procedure is illustrated in the Fig.2.

Empirical findings have substantiated the enhancement in accuracy resulting
from the incorporation of this procedural step. The discernible rise in MVF1
scores, as illustrated in Tables IV, V, and VI, under the condition
$\lambda=best$, underscores its efficacy in contrast to cases where $\lambda$
is set to 0. The subsequent section will provide a comprehensive explication
of the variable $\lambda$, elucidating its impact on MVF1 scores. The optimal
value for $\lambda$, denoted as $best$, is determined through systematic
parameter tuning, with specific values tailored for each dataset. For brevity,
explicit enumeration of these dataset-specific $best$ values is omitted in
this context.

###  III-C Motion Judgment

After obtaining the background features and motion distance, we utilize the
value $\varTheta$, as shown in (7), to determine whether the object is in
motion, when $\varTheta$ is bigger than a threshold, the object is considered
stationary.

$\varTheta=\lambda A_{a}+(1-\lambda)A_{m}$ (7)

Here, $A_{a}$ represents the similarity of the background appearance, which is
calculated using the structural similarity (SSIM) index. The specific
calculation formula is given by (8). $A_{m}$ represents the normalized value
of the motion distance traveled by the tracked object. $\lambda$ is the
proportional coefficient.

$A_{a}(I_{1},I_{2})=\frac{(2\mu_{I_{1}}\mu_{I_{2}}+r_{1})(2\sigma_{I_{1}I_{2}}+r_{2})}{(\mu_{I_{1}}^{2}+\mu_{I_{2}}^{2}+r_{1})(\sigma_{I_{1}}^{2}+\sigma_{I_{2}}^{2}+r_{2})}$
(8)
$A_{m}=\left.\left\\{\begin{array}[]{c}1-\frac{D}{\beta_{d}},D\leq\beta_{d}\\\
0,\mathrm{otherwise}\\\ \end{array}\right.\right.$ (9)

Among them, $I_{1},I_{2}$ represent two background feature tensors,
$\mu_{I_{1}},\mu_{I_{2}}$ represent the mean of the two tensors,
$\sigma_{I_{1}}^{2},\sigma_{I_{2}}^{2}$ represent the variance of the two
tensors, $\sigma_{I_{1}I_{2}}$ represents the covariance of the two tensors,
$r_{1},r_{2}$ is a constant, used to avoid zero denominator, and $\beta_{d}$
is the maximum distance that the preset stationary object may move in the
image. In this paper, $\beta_{d}$ is usually set based on experience to be
0.01 times the smaller one in the length and width of the image.

##  IV MoD2T

In the previous sections, we discussed the advantages and limitations of both
traditional methods and deep learning methods. The objective of this paper is
to parallelize and integrate these two approaches to achieve improved results.
We propose a model-data-driven motion-static object tracking method (MoD2T)
that combines the outputs of traditional method and deep learning method to
enhance tracking accuracy and robustness.

The traditional method used in this paper proposes an improvement to the
Gaussian model proposed in [2], and through experiments, it demonstrates that
this method, when combined with the OC-SORT[46], achieves satisfactory results
in many environments. For the specific implementation details, please refer to
the appendix. Inspired by the work in [38], which demonstrated the benefits of
weighted fusion for multiple tracking methods, we design a fusion scheme for
the methods employed in this paper.

Let’s denote the two tracking methods as
$M=\left\\{M^{tra},M^{deep}\right\\}$. For each frame $I_{t},t=1,2,...,T$,
where $T$ represents the total number of frames in the video, these methods
generate a series of the bounding box (bbox) ($\widehat{B}_{t}^{i}$) and
motion assessment values ($\widehat{\varTheta}_{t}^{i}$) , i.e. $i=1,...,n$,
where $n$ represents the number of detections in the $t$ frame. The fusion
method employed in this paper combines their outputs
$\widehat{B}_{t}=(\widehat{B}_{t}^{tra},\widehat{B}_{t}^{deep})$ and
$\widehat{\varTheta}_{t}=(\widehat{\varTheta}_{t}^{tra},\widehat{\varTheta}_{t}^{deep})$
to obtain the fused tracking outputs
$\widehat{B}_{t}^{fusion},\widehat{\varTheta}_{t}^{fusion}$.

Considering the performance issues associated with traditional methods, our
fusion strategy may undergo certain adjustments based on the varying
performance of the traditional methods.

In a comprehensive overview, the procedural workflow of MoD2T is illustrated
in Fig.4. Subsequent to the independent processing of image data using two
distinct methodologies, we apply the motion classification technique outlined
in section III to the outcomes derived from the deep learning method. The
precise fusion strategy will be contingent upon the efficacy of the
traditional approach.

###  IV-A Traditional Methods Evaluation

Before proceeding with the evaluation, we establish two key premises. Firstly,
if it is known in advance that the traditional method yields a result with an
Intersection over MOTA[47] score greater than 0.25 on a similar dataset, then
the following evaluation process will be used. Secondly, in the absence of the
aforementioned MOTA condition, we check whether the proportion of foreground
pixels detected by the traditional method exceeds 0.5 of the entire image
size. Only if this condition is met and the MOTA criterion is not satisfied,
will we proceed with the evaluation using the following approach. It is
imperative to note that if neither of these conditions is met, both criteria
imply the inadequacy of traditional methods. The threshold of 0.5 for the
proportion of foreground pixels is chosen based on the understanding that
values exceeding 0.5 indicate rapid camera motion, rendering traditional
methods ineffective. The selection of MOTA as a primary judgment criterion is
justified by the potential limitations of traditional methods, which may lead
to misunderstandings or overlook the detection of objects in scenarios where
motion is not accurately captured.

Our evaluation is not based on the overall results of the entire image, but
rather on evaluating individual tracking results. This is because traditional
methods can produce erroneous results when performing morphological operations
such as erosion and dilation, leading to two objects being mistakenly
identified as one. Therefore, we focus on the connected components within the
bounding boxes of each moving object obtained by the traditional method for
evaluation.

For each motion pixel $p$, we examine its surrounding neighboring pixels
within a small range, typically set to three pixels, inspired by the concept
of a 3x3 convolution kernel commonly used in Convolutional Neural Networks
(CNN). This approach is akin to the local receptive field employed by CNN,
designed to capture local information in images. If additional motion pixels
are detected within this specified range, the current pixel is designated as a
new connected component and is assigned a unique label. Subsequently, we
analyze the resulting labeled image and quantify the number of connected
components. This choice of a small range, specifically three pixels, draws
inspiration from the effectiveness of 3x3 convolutional kernels in CNN,
ensuring a balance between computational efficiency and the model’s ability to
identify and label connected components of motion pixels.

We sum the pixels in these connected components, and if the pixel value
exceeds $S_{min}$, we consider it as a valid connected component. The specific
expression of this calculation can be represented by the (10):

$\displaystyle S$ $\displaystyle=\sum_{j=1}^{n}{p^{i}_{j}}$ (10)
$\displaystyle n$ $\displaystyle=\sum_{i=1}^{N}{[S_{i}>S_{min}]}$

When $n\geq 2,$ we consider that the traditional method has not worked
properly. By calculating and observing the number of connected components
obtained by the traditional method, we can assess its performance in handling
situations involving close or overlapping objects. This evaluation method
allows for a more detailed understanding of the method’s performance on
individual tracking results and provides an assessment of its effectiveness.
One more thing, in this paper, $S_{min}$ generally set to one-fifth of the
bbox area. A typical example of the effectiveness of a traditional tracking
method is illustrated in Fig. 3.

Figure 3: Schematic diagram of whether traditional methods are effective. From
the figure, it can be observed that when two objects are close in distance,
traditional methods become ineffective. At this point, we should change our
approach to fusion. Figure 4: The comprehensive architecture of MoD2T is
presented, allowing for flexibility in the integration of various deep
learning-based methods. Specifically, this study employs YOLOv8+StrongSORT and
YOLOv8+DeepOCSORT as instances of such methods. Fig.7 illustrates the
traditional method, while Fig.1 focuses on the motion judgment aspect. Within
the algorithmic effectiveness assessment module depicted in the figure, the
region enclosed by purple dashed lines signifies multiple potential outputs of
this module. The notation $p_{t}$ within the diagram represents the pixels
that traditional algorithms may identify as moving objects.

###  IV-B Fusion Strategies in the Presence of Effective Traditional Methods

First, we use Intersection of Union(IoU) to associate and match the output
results of the two methods. Then we judge whether the traditional method is
effective. When the traditional methods work well, we can attempt to fuse both
bbox and motion assessment values $\varTheta$. For the tracking results
$B^{fusion}_{i},i\,\,\in\,\,\left[1,n\right]$ of a single frame, which
includes $n$ tracked objects, the fusion is calculated by merging the outputs
of both trackers. In this paper, we compute the bbox for each individual
object using a weighted average, as shown in (11):

$B_{i}^{fusion}=w_{deep}^{b}*B_{i}^{deep}\,\,+\,\,w_{tra}^{b}*B_{i}^{tra}$
(11)

Here, the weights denoted as $w_{k}^{b}$, where the subscript $k$ signifies
distinct methods, are considered as tunable parameters. It is important to
emphasize that the weights presented in this study are computed with respect
to the performance exhibited by each method on a common dataset. The precise
computation of these weight coefficients is delineated in (12):

$\begin{gathered}w_{deep}^{b}=\alpha_{k}^{b}\frac{MOTA_{deep}}{MOTA_{tra}+MOTA_{deep}}\\\
w_{tra}^{b}=\left(1-\alpha_{k}^{b}\right)\frac{MOTA_{tra}}{MOTA_{tra}+MOTA_{deep}}\end{gathered}$
(12)

where, $\alpha_{k}^{b}$ is the confidence level, given by humans, and MOTA is
a common evaluation indicator for multi-objective tracking methods[47]. The
reason for choosing MOTA is that it mainly reflects whether the object has
been tracked, and traditional methods often miss detection of the object.
Therefore, using this indicator as the standard for calculating fusion
parameters is reasonable.

Since deep learning methods track both moving and stationary objects, the
tracking results from deep learning often contain more detections compared to
traditional methods. In cases where there are no corresponding detections from
the traditional method for the deep learning results, we directly adopt the
results from the deep learning method. In other words, we set
$w_{deep}^{b}=1,w_{tra}^{b}=0$ in this cases. This ensures that the deep
learning results are considered as the primary source of tracking information
when there is no corresponding result from the traditional methods.

The value $\varTheta$, which indicates whether an object is in motion or not,
is also calculated using a weighted average in this paper, as shown in (13).
The calculation of $\varTheta^{deep}$ is defined in (7). For
$\varTheta_{i}^{tra}$, when there is no corresponding detection from the
traditional method for the deep learning results, it is set to 1; otherwise,
it is set to 0. Meanwhile, if the traditional method does not work properly,
its value will be shown in equation (16)

$\varTheta_{i}^{fusion}=\left.w_{deep}^{f}*\varTheta_{i}^{deep}\,\,+\,\,w_{tra}^{f}*\varTheta_{i}^{tra}\,\,\right.$
(13)

Subscript $i$ means the ID of different tracking objects.

Similar to the weights for bbox, the weights for $\varTheta$ are also
determined based on the performance of each method on the same dataset. The
weight coefficients can be calculated as shown in (14),

$\begin{gathered}w_{deep}^{f}=\frac{MVF1_{deep}}{MVF1_{tra}+MVF1_{deep}}\\\
w_{tra}^{f}=\frac{MVF1_{tra}}{MVF1_{tra}+MVF1_{deep}}\end{gathered}$ (14)

where the MVF1 is an indicator used to measure whether the judgment of a
moving object is accurate, and the specific calculation method is shown in
(15). The $\text{MVF1}_{tra}$ to be 1.If traditional methods can still track
performance ($\text{MOTA}>=0.25$), then $\text{MVF1}_{tra}$ is 1, otherwise it
is 0.2. The reason for setting these two values is based on experimental
results.

###  IV-C Object Motion Assessment: MVF1 and its Elements

Recognizing the absence of a specialized metric for evaluating the accuracy of
models in predicting object motion states, we drew inspiration from the F1
score to introduce the $MVF1$ metric as an innovative evaluation criterion.

$\begin{gathered}MVF1=2\cdot\frac{MVP\cdot MVR}{MVP+MVR}\\\
MVP=\frac{MVTP}{MVTP+MVFP}\\\ MVR=\frac{MVTP}{MVTP+BF\cdot
MVFN}\end{gathered}$ (15)

Here, $MVTP$ signifies the count of samples accurately classified as either
moving or stationary objects, $MVFP$ denotes the count of samples inaccurately
labeled as moving objects when they are stationary or as stationary objects
when they are in motion, and $MVFN$ represents the count of samples absent
from the ground truth. $BF$ stands for the balance factor, a parameter
adjustable to enhance adaptability.

The introduction of $BF$ is motivated by the metric’s primary purpose of
measuring the accuracy in determining the motion state of objects. Ideally,
the impact of tracking or detection performance should be disregarded.
Nevertheless, given that motion-state determination relies on tracking, this
metric retains a certain influence from the detection process.

$BF=\frac{1}{2}\cdot(\text{precision}+\text{recall})$ (16)

where precision and recall represent the precision and recall of the object
detection. Consequently, when precision and recall are balanced, $BF$
converges to 0.5, mitigating the impact of MVFN. As precision or recall
deviates from equilibrium, $BF$ adjusts dynamically, taking into account
undetected objects based on the overall performance of object detection. As
for why detection metrics are used as balance parameters, it is mainly because
the tracking performance and detection performance of almost all current
tracking algorithms are closely related.

###  IV-D Fusion Strategy in the Presence of Malfunctioning Traditional
Methods

When the performance of traditional method is inadequate, meaning it performs
poorly on pre-prepared test sets or when the judgment criteria proposed in
this paper deem their performance to be subpar, only the value $\varTheta$ is
fused as (13). The calculation for $\varTheta_{i}^{fusion}$ and $C_{i}^{deep}$
remains unchanged. However, the calculation for $\varTheta_{i}^{tra}$ will be
calculated as shown in (16). The bbox will directly adopt the results from the
deep learning-based MOT method.

$\varTheta_{i}^{tra}=\begin{cases}\frac{p_{i}}{w*h*R},&\frac{p_{i}}{w*h}<R\\\
1,&\mathrm{otherwise}\\\ \end{cases}$ (17)

Where $w$ and $h$ represent the width and height of the bounding box,
respectively, and $p_{i}$ denotes the number of pixels in the bounding box
that correspond to the motion detection results from the traditional method.
$R$ is a correction value, which is experimentally determined to be 0.2. This
value is consistent with the determination of the S value in section IV-A.

##  V Experiment

In this section, we conduct extensive experiments to evaluate the performance
of MoD2T on several widely used datasets and compare it with several state-of-
the-art MOT methods. We also analyze the impact of various components in MoD2T
and provide insights into its effectiveness and limitations. Since we have not
found any work that can fully compare with the work in this paper, this
article did not set a baseline for comparison. We found that the work closest
to the core work of this paper is various tasks for image change detection
such as CDnet-2014[48], which include[31, 49] , but none of these works are
suitable for comparison with this paper.

###  V-A Setting

Datasets. Currently, a void exists in the availability of multi-object
tracking datasets that incorporate annotations about the dynamic state of
objects. Nevertheless, a recourse is available through repurposing datasets
originally crafted for autonomous driving, such as the KITTI [50]. This
dataset includes precise 3D coordinates of objects, thereby facilitating the
determination of object motion status within the tracking context.

The KITTI dataset [50] has achieved wide adoption within the autonomous
driving and computer vision research communities. It encompasses data
collected from diverse sensors like cameras and LiDAR mounted on vehicles. The
dataset presents an array of scenes annotated with object bounding boxes and
motion trajectories for various entities, including vehicles and pedestrians,
thereby rendering it a fitting choice for the evaluation of multi-object
tracking methodologies.

Moreover, our investigation extends to selected scenarios from the UAVDT [51]
and MOT17 [13] datasets. The UAVDT dataset is meticulously tailored for video
analysis of unmanned aerial vehicles, featuring captured video sequences
across a spectrum of scenarios. On the other hand, the MOT17 dataset
emphasizes multi-object tracking pertaining to pedestrians, offering video
footage captured from different vantage points along with annotations
detailing the motion trajectories of multiple objects.

To underscore the versatility and efficacy of our method, we subject these
UAVDT and MOT17 scenes to supplementary evaluation, bolstered by augmented
annotations. These curated scenes introduce heightened complexity and
intricate tracking scenarios, thereby affording a comprehensive assessment of
the method’s resilience and adaptability. By subjecting our method to scrutiny
across diverse datasets, we attain a holistic comprehension of its performance
in multi-object tracking tasks and corroborate its efficacy across a plethora
of scenarios.

Our assessment strategy was systematic, entailing an exhaustive performance
analysis conducted half of KITTI’s training set for training and estimating
some parameters, and use the other half for testing. Given the need for
supplemental annotations concerning object motion attributes within the MOT17
and UAVDT datasets, the entirety of these datasets was not embraced in this
inquiry. Instead, our evaluation was concentrated on representative scenes,
specifically MOT17 sequences (03, 07) and UAVDT sequences (M101, M202, M204,
M402, M601, M703, M802).

The outcomes of these exacting evaluations have been scrupulously collated in
the forthcoming table. In a bid to furnish a comprehensive insight into the
effectiveness and prowess of our proposed approach, we partitioned the KITTI
dataset into two discrete segments for the purpose of testing. One segment
encompasses sequences (0001,0017-0024), characterized by static or slowly
moving cameras, while the other segment encapsulates scenarios involving
swifter camera motions and accelerated object movements.

One more thing, although using the test datasets for calculating the fusion
weights, it is anticipated that this has minimal impact on the results.
Nevertheless, in order to ensure the fairness of our experiments, this paper
also employed additional datasets for a rough estimation of these weights.
Specifically, for KITTI, the additional data encompassed the entire training
set, which was further categorized based on the speed of motion. For MOT17,
the selected datasets were 04 and 10, and for UAVDT, it included M203, M403,
and M701. It is important to emphasize that although these datasets underwent
additional annotation and can be utilized for computing tracking performance
metrics and MVF1 scores, they were not used in the method’s performance
testing in this study.

Metrics. We use the metrics MOTA, IDF1,AssA and HOTA to evaluate tracking
performance[47, 52, 53]. MOTA primarily measures the completeness and
robustness of tracking, IDF1 focuses on the accuracy and precision of targets,
while HOTA comprehensively considers multiple aspects of performance,
including both target and trajectory-level accuracy. When comparing methods
and evaluating performance, these metrics can assist researchers in gaining a
comprehensive understanding of the strengths, weaknesses, and applicability of
the methods.

Additionally, we use the MVF1 to measure the accuracy of object motion state
predictions. The calculation method for the MVF1 is referenced in (15).

Implementation Details. We utilize OSNet[45] as the network for extracting
background appearance features. For testing on the UAVDT and KITTI , we employ
osnet pre-trained on VeRi[54], while for evaluating the MOT17 , we use weights
pre-trained on Market1501[55]. Additionally, to conserve computational
resources, the same osnet network is used in the deep learning-based MOT
method adopted in this study. The criterion used to determine correctness is
the Intersection over Union (IoU). A prediction is considered correct when the
IoU between the method’s prediction and the ground truth is greater than 0.5.
The training method of YOLOv8 in MOT17 is similar to [56]. The tracker used in
traditional methods is OC-SORT[46].

###  V-B Reasonability analysis of MVF1 design

As mentioned earlier, we propose an indicator for evaluating the accuracy of
the model’s prediction of object motion states. The definition of FN
determines that this indicator is actually not very related to the main
content we want to evaluate. Here, to demonstrate the rationality of the FN
definition and elucidate the utility of BF parameter, we selected two
sequences from the UAVDT dataset, M203 and M403, for testing, totaling
approximately 2000 frames.. We examined the impact of varying detection
performance on the MVF1 metric. Different detection accuracies were
approximated by manually removing results of the same identity throughout the
entire sequence.

Figure 5:  The fluctuations observed in MVF1 are attributed to variations in
tracking performance trends. The blue curve signifies alterations introduced
by adjusting the parameters related to the BF, while the green curve
corresponds to the scenario where the BF parameter is set to 1, and the red
curve signifies its configuration with a BF parameter value of 0.

As depicted in the Fig.5, the dynamic changes observed in MVF1, driven by
varying levels of MOTA, offer valuable insights into the nuanced impact of BF
parameter configurations on the model’s ability to assess the relative motion
states of tracked objects with respect to the ground. Furthermore, it is
noteworthy that distinct MOTA values were generated by randomly removing
entire sequences with identical ID numbers.

Examining the blue curve, representing MVF1 values under varying BF
parameters, a clear positive correlation emerges with increasing MOTA levels.
This trend emphasizes the utility of strategic BF parameter adjustments in
enhancing the model’s capacity to accurately discern the motion states of
tracked objects relative to the ground. Notably, the performance improvement
is most pronounced at higher MOTA values, underscoring the synergistic
relationship between MOTA and BF parameter optimization.

Conversely, the green curve, denoting the scenario where the BF parameter is
set to 1, suggests a nuanced trade-off. While initial MVF1 values exhibit
promise, a subsequent decline in performance is observed as MOTA increases.
This nuanced relationship underscores the importance of balancing BF parameter
settings, as extreme values may compromise tracking accuracy.

The orange curve, corresponding to a BF parameter value of 0, reveals
intriguing stability in MVF1 values across varying MOTA levels. This finding
indicates that, in specific contexts, the deliberate exclusion of background
filtering may be a viable choice, particularly when minimizing false negatives
is a critical consideration.

In essence, the result demonstrates that judicious BF parameter settings
significantly contribute to the enhanced evaluative capacity of MVF1, offering
a more nuanced assessment of the model’s proficiency in discerning the motion
states of tracked objects in relation to the ground. The observed trends
highlight the need for a context-aware approach to BF parameter tuning,
emphasizing the delicate balance required for optimal tracking performance.

###  V-C Motion State Judgment Results

TABLE I: MoD2T Motion State Judgment Performance on KITTI | Camera stationary or moving slowly | Camera moving fast
---|---|---
Method | MVF1 | $\text{MVF1}_{0}$ | $\text{MVF1}_{1}$ | MVF1 | $\text{MVF1}_{0}$ | $\text{MVF1}_{1}$
StrongSORT[36]+MJ($\lambda=0$) | 0.392 | 0.427 | 0.385 | 0.221 | 0.258 | 0.209
StrongSORT+MJ($\lambda=best$) | 0.474 | 0.503 | 0.422 | 0.219 | 0.261 | 0.210
StrongSORT+MJ($\lambda=1$) | 0.426 | 0.468 | 0.420 | 0.111 | 0.158 | 0.097
DeepOCSORT[6]+MJ($\lambda=0$) | 0.394 | 0.425 | 0.388 | 0.213 | 0.252 | 0.206
DeepOCSORT+MJ($\lambda=best$) | 0.472 | 0.501 | 0.425 | 0.220 | 0.262 | 0.213
DeepOCSORT+MJ($\lambda=1$) | 0.425 | 0.469 | 0.410 | 0.113 | 0.157 | 0.099
MoD2T(StrongSORT) | 0.775 | 0.821 | 0.763 | 0.230 | 0.239 | 0.228
MoD2T(DeepOCSORT) | 0.774 | 0.822 | 0.764 | 0.231 | 0.237 | 0.227
TABLE II: MoD2T Motion State Judgment Performance on MOT17 Method | MVF1 | $\text{MVF1}_{0}$ | $\text{MVF1}_{1}$
---|---|---|---
StrongSORT+MJ($\lambda=0$) | 0.397 | 0.436 | 0.392
StrongSORT+MJ($\lambda=best$) | 0.418 | 0.445 | 0.387
StrongSORT+MJ($\lambda=1$) | 0.352 | 0.382 | 0.331
DeepOCSORT+MJ($\lambda=0$) | 0.396 | 0.431 | 0.389
DeepOCSORT+MJ($\lambda=best$) | 0.428 | 0.457 | 0.398
DeepOCSORT+MJ($\lambda=1$) | 0.354 | 0.386 | 0.337
MoD2T(StrongSORT) | 0.518 | 0.583 | 0.483
MoD2T(DeepOCSORT) | 0.521 | 0.583 | 0.481
TABLE III: MoD2T Motion State Judgment Performance on UAVDT Method | MVF1 | $\text{MVF1}_{0}$ | $\text{MVF1}_{1}$
---|---|---|---
StrongSORT+MJ($\lambda=0$) | 0.414 | 0.452 | 0.408
StrongSORT+MJ($\lambda=best$) | 0.527 | 0.563 | 0.522
StrongSORT+MJ($\lambda=1$) | 0.385 | 0.401 | 0.381
DeepOCSORT+MJ($\lambda=0$) | 0.438 | 0.461 | 0.401
DeepOCSORT+MJ($\lambda=best$) | 0.525 | 0.561 | 0.520
DeepOCSORT+MJ($\lambda=1$) | 0.390 | 0.413 | 0.383
MoD2T(StrongSORT) | 0.825 | 0.862 | 0.820
MoD2T(DeepOCSORT) | 0.827 | 0.861 | 0.824

The specific outcomes are delineated in Tables I, II, and III. In these
tables, the term ”MJ” denotes motion judgment, $\text{MVF1}_{0}$ signifies a
background filtering (BF) coefficient set to 0, and $\text{MVF1}_{1}$ denotes
a BF coefficient set to 1.

Directing attention to the KITTI dataset, encompassing scenes characterized by
diverse camera motions, MoD2T demonstrates noteworthy enhancements in the MVF1
score, particularly in scenarios involving slow camera motion or a stationary
camera. The method excels in accurately discerning static from moving objects.
However, despite these advancements, the overall tracking performance across
all methods remains consistent, indicating that the improvements predominantly
contribute to the discrimination of static or slowly moving objects, rather
than constituting a substantial enhancement to tracking capabilities. Notably,
MoD2T’s capacity to discern object motion diminishes in scenarios
characterized by rapid camera motion.

In the UAVDT dataset, MoD2T exhibits a significant elevation in the MVF1 score
compared to deep learning-based multiple object tracking (MOT) methods, such
as StrongSORT and DeepOCSORT. Analogous to the KITTI dataset, the tracking
performance remains relatively unaffected, underscoring MoD2T’s proficiency in
detecting static or slowly moving objects without substantial gains in overall
tracking capabilities.

Similarly, in the MOT17 dataset, analogous trends emerge. MoD2T’s efficacy in
determining object motion is notably superior in scenarios with slow camera
motion or a stationary camera (e.g., MOT17-03), in contrast to situations
featuring rapid camera motion (e.g., MOT17-07). Moreover, when cross-
referencing datasets, such as UAVDT and KITTI, wherein camera motion is
minimal, it becomes apparent that discerning human motion is considerably more
challenging than detecting vehicle movement.

The similarities in results between DeepOCSORT and StrongSORT highlight their
comparable tracking performance in various motion state scenarios, assessed
through the MVF1 metric and its derivatives ($\text{MVF1}_{0}$ and
$\text{MVF1}_{1}$) in dynamic camera environments.

Across different datasets (KITTI, MOT17, and UAVDT), both methods consistently
exhibit similar trends in MVF1, $\text{MVF1}_{0}$ and $\text{MVF1}_{1}$,
indicating a sustained level of agreement in motion state judgment
performance.

This alignment in outcomes is likely attributable to shared foundational
principles or algorithmic components, resulting in similar responses to
diverse motion state scenarios. Both methods may employ comparable strategies
to address challenges posed by stationary or slowly moving objects versus
swiftly moving entities.

Minor disparities observed in specific instances may stem from dataset-
specific attributes or variations in parameters, such as the tuning parameter
$\lambda$ in the Multi-Judgment (MJ) module. Nevertheless, the overall trend
of closely aligned performance suggests a robust and consistent behavior
exhibited by both tracking approaches across diverse scenarios.

In summary, the consistent alignment of results across various datasets
emphasizes the comparable effectiveness of DeepOCSORT and StrongSORT in motion
state judgment. Despite potential differences in their underlying
architectures, both methods consistently deliver similar tracking outcomes,
showcasing their proficiency in addressing diverse motion scenarios
encountered in real-world tracking applications.

TABLE IV: MoD2T Tracking Performance on KITTI | Pedestrian | Car
---|---|---
Method | HOTA | IDF1 | MOTA | AssA | HOTA | IDF1 | MOTA | AssA
StrongSORT[36] | 0.812 | 0.876 | 0.877 | 0.838 | 0.852 | 0.915 | 0.915 | 0.858
DeepOCSORT[6] | 0.822 | 0.874 | 0.886 | 0.828 | 0.859 | 0.921 | 0.922 | 0.861
MoD2T(StrongSORT) | 0.811 | 0.876 | 0.877 | 0.837 | 0.852 | 0.915 | 0.915 | 0.858
MoD2T(DeepOCSORT) | 0.822 | 0.874 | 0.886 | 0.828 | 0.859 | 0.921 | 0.922 | 0.861
TABLE V: MoD2T Tracking Performance on MOT17 Method | HOTA | IDF1 | MOTA | AssA
---|---|---|---|---
StrongSORT | 0.682 | 0.821 | 0.825 | 0.682
DeepOCSORT | 0.714 | 0.833 | 0.825 | 0.726
MoD2T(StrongSORT) | 0.681 | 0.821 | 0.823 | 0.682
MoD2T(DeepOCSORT) | 0.714 | 0.833 | 0.825 | 0.726
TABLE VI: MoD2T Tracking Performance on UAVDT Method | HOTA | IDF1 | MOTA | AssA
---|---|---|---|---
StrongSORT | 0.812 | 0.931 | 0.932 | 0.815
DeepOCSORT | 0.828 | 0.943 | 0.945 | 0.820
MoD2T(StrongSORT) | 0.812 | 0.931 | 0.932 | 0.896
MoD2T(DeepOCSORT) | 0.828 | 0.943 | 0.945 | 0.820

###  V-D Tracking Results

MoD2T’s tracking performance is meticulously evaluated across KITTI, MOT17,
and UAVDT datasets, as detailed in Tables IV, V, and VI. The assessment
includes key metrics such as HOTA, IDF1, MOTA, and AssA.

In the KITTI dataset, MoD2T, synergized with both StrongSORT and DeepOCSORT,
exhibits tracking metrics on par with individual methods. Crucially, HOTA,
IDF1, MOTA, and AssA scores remain consistent, affirming that MoD2T seamlessly
integrates without compromising the intrinsic tracking performance of the base
methods. This emphasizes MoD2T’s compatibility, preserving the effectiveness
of existing models.

The MOT17 dataset reveals a parallel trend, where MoD2T sustains the
performance levels of StrongSORT and DeepOCSORT. HOTA, IDF1, MOTA, and AssA
scores for MoD2T align closely with standalone methods, corroborating that
MoD2T augmentation does not impede the underlying models’ tracking
proficiency.

UAVDT dataset evaluation further emphasizes MoD2T’s consistent trend in
preserving and enhancing tracking performance. Particularly, HOTA, IDF1, MOTA,
and AssA scores for MoD2T surpass standalone methods in the AssA metric,
suggesting MoD2T’s potential to improve tracking accuracy in specific
scenarios.

In essence, the results highlight that MoD2T, in collaboration with StrongSORT
and DeepOCSORT, maintains the robustness of individual tracking models while
substantially enhancing tracking performance. The introduced framework ensures
that MoD2T incorporation does not degrade the intrinsic tracking capabilities
of the base models. This reinforces MoD2T’s compatibility and efficacy in
real-world tracking applications, positioning it as a valuable enhancement for
multi-object tracking tasks.

Figure 6: Visualize the sample results of MoD2T on the KITTI test set and some
scenarios of MOT17 and UAVDT. The red box represents moving objects, and Teal
represents stationary objects. From here, it can be clearly seen that the
function of MoD2T to judge the motion status of objects performs very well
when the camera is not moving vigorously.

###  V-E Traditional Method Results

TABLE VII: Traditional Method Performance on Different Datasets.The correct IoU threshold is 0.5 Dataset | IDF1 | HOTA | MOTA | AssA
---|---|---|---|---
KITTI[50](Camera Fast Moving) | 0.013 | 0.010 | 0.012 | 0.009
KITTI(Other) | 0.063 | 0.019 | 0.063 | 0.012
MOT17[13] | 0.043 | 0.024 | 0.042 | 0.025
UAVDT[51] | 0.068 | 0.043 | 0.069 | 0.044
TABLE VIII: Traditional Method Performance on Different Datasets.The correct IoU threshold is 0.25 Dataset | IDF1 | HOTA | MOTA | AssA
---|---|---|---|---
KITTI[50](Camera Fast Moving) | 0.015 | 0.011 | 0.015 | 0.013
KITTI(Other) | 0.543 | 0.439 | 0.542 | 0.432
MOT17[13] | 0.361 | 0.262 | 0.361 | 0.365
UAVDT[51] | 0.454 | 0.362 | 0.464 | 0.361

Table VII presents the performance of traditional methods on different
datasets, with an IoU threshold of 0.5, specifically focusing on correctly
tracking moving objects. The results reveal that, at an IoU threshold of 0.5,
the performance of traditional methods is generally unsatisfactory across all
datasets. Particularly challenging is the KITTI , which involves intense
camera motion.

When the IoU threshold is relaxed to 0.25, as shown in Table VIII, the
proposed method achieves marginally better results in most scenarios. This
improvement suggests that, for traditional methods, they can often confirm the
presence of a moving object in many cases, but their accuracy in precisely
determining the object’s boundaries is limited.

However, it becomes evident that when dealing with fast camera motion in the
KITTI , the performance of traditional algorithms still falls short. This is
also one of the reasons why MoD2T underperforms in such scenarios. Therefore,
further research and refinement are necessary to enhance the method’s
robustness in fast-paced environments, or alternatively, consider adopting
more robust approaches.

In summary, traditional methods exhibit limited accuracy when evaluating
moving objects at an IoU threshold of 0.5. The proposed method shows some
improvement when the IoU threshold is relaxed, but it still faces challenges,
particularly in scenarios with fast camera motion, where the performance of
traditional algorithms also struggles. To address this issue, continuous
efforts should be made to enhance the method’s robustness in fast-motion
environments, or explore alternative, more resilient methodologies.

###  V-F Analysis of Some Parameters

This paper has meticulously expounded on the pivotal fusion parameter
configuration, elucidating its intricacies in a preceding section. However,
certain parameter settings, such as the parameter denoted as $S_{min}$ in
section IV-A, are derived from experiential considerations. To substantiate
the rationale behind these parameter choices, supplementary experimental
analyses have been conducted. The objective is to furnish a rational
explanation for the specified parameter values, shedding light on the nuanced
decisions made in their selection within the context of this research
endeavor.

Regarding $S_{min}$, its configuration in Section IV-D mirrors that of $R$
since both parameters signify the ratio of motion pixels to the entire box
pixels. Notably, the ratio $S_{min}/R$ precisely corresponds to the size of
bbox. So these two parameters are analyzed together.

To illustrate the rationality of our values. We tested the effectiveness of
different R values on the sequence of analyzing the rationality of MVF1
design, and the results are shown in the table below.

TABLE IX: R sensitivity analysis $R$ | MVF1 | $\text{MVF1}_{0}$ | $\text{MVF1}_{1}$
---|---|---|---
0.1 | 0.828 | 0.852 | 0.819
0.15 | 0.844 | 0.863 | 0.839
0.2 | 0.865 | 0.901 | 0.860
0.3 | 0.805 | 0.874 | 0.796
0.5 | 0.687 | 0.727 | 0.674
0.7 | 0.565 | 0.601 | 0.560

The choice of R = 0.2 is justified by a careful sensitivity analysis, as
depicted in Table IX. A smaller value of R, such as 0.1 or 0.15, makes the
MoD2T overly sensitive, leading to the risk of misclassifying static objects
as moving. Conversely, a larger R, for instance, 0.3, 0.5, or 0.7, tends to
result in reduced sensitivity, potentially causing dynamic objects to be
erroneously identified as stationary. Therefore, the selection of R = 0.2
strikes a balance, ensuring optimal sensitivity for motion detection while
minimizing the likelihood of false positives or negatives in distinguishing
object motion states.

In Fig 4, a notable branch exhibits a MOTA score below 0.25. This specific
threshold is selected based on the empirical observation that when the MOTA of
the traditional method falls below 0.25, it strongly indicates suboptimal
performance on sequences of this nature. In such instances, the traditional
method proves ineffective. It is essential to clarify that the value of 0.25
serves as a pragmatic threshold, lacking inherent significance beyond being a
chosen reference point for assessing traditional method performance on
specific sequences.

###  V-G Limitations

MoD2T primarily excels when dealing with minor camera motion, particularly in
scenarios involving rotary-wing unmanned aerial vehicles (UAVs). It
effectively distinguishes between stationary and slowly moving objects,
resulting in higher MVF1 scores and improved detection capabilities. However,
its performance is constrained when the camera experiences significant motion,
especially in accurately assessing the motion of mobile objects. This
limitation, sensitivity to camera motion, significantly impacts its
effectiveness in settings with frequent camera motion, like dynamic outdoor
environments or rapid camera movements. Additionally, MoD2T’s performance
varies across different object categories, particularly in assessing motion
for vehicles and pedestrians, showing significant disparities. This suggests
potential biases in specific scenarios.

Furthermore, MoD2T’s accuracy in motion assessment is contingent on the
quality of the tracking methodology employed. Generally, better tracking
methods lead to higher MVF1. MoD2T shines in scenarios with a stationary
camera or minor camera motion, proficiently distinguishing stationary from
slowly moving objects. However, in contexts with more intense camera motion,
especially in assessing mobile object motion, its performance is limited.
There are also performance differences across object categories, with better
motion assessment for vehicles compared to pedestrians. Future research should
focus on addressing these limitations by enhancing MoD2T’s ability to handle
pronounced camera motion and improving motion assessment accuracy across all
object categories.

##  VI Conclusion

This paper introduces MoD2T, a novel approach to MOT in video analysis. MoD2T
seamlessly integrates traditional mathematical modeling with deep learning-
based MOT frameworks, leveraging the strengths of both paradigms. A key
innovation is the incorporation of a novel motion-static object tracking
algorithm, rooted in motion distance and background features, enhancing
accurate determination of object motion status.

MoD2T excels in improving tracking accuracy by dynamically fusing results from
traditional methods and deep learning-based MOT frameworks. It intelligently
weights and combines tracking outcomes based on their effectiveness,
demonstrating superior performance across diverse datasets and scenarios
compared to traditional and deep learning methods.

In addition to presenting a novel tracking methodology, we propose MVF1 metric
to assess a model’s ability to discern object motion states. We provide a
concise analysis of MVF1, emphasizing its significance in comprehensively
evaluating tracking performance.

This research contributes fresh perspectives to MOT by showcasing the
potential benefits of integrating traditional and deep learning methods. MoD2T
not only addresses the limitations of individual methods but also sets the
stage for advancements in MOT within video analysis. Future work aims to
enhance MoD2T’s performance during high-speed camera movements and explore the
feasibility of treating object motion attributes as learnable parameters. This
involves leveraging neural networks for prediction and achieving end-to-end
motion-based static object discrimination, pushing the boundaries of MOT
capabilities in video analysis.

## References

  * [1] A. Lipton, H. Fujiyoshi, and R. Patil, “Moving target classification and tracking from real-time video,” in _Proceedings Fourth IEEE Workshop on Applications of Computer Vision. WACV’98 (Cat. No.98EX201)_ , 1998-10, pp. 8–14.
  * [2] K. M. Yi, K. Yun, S. W. Kim, H. J. Chang, and J. Y. Choi, “Detection of Moving Objects with Non-stationary Cameras in 5.8ms: Bringing Motion Detection to Your Mobile Device,” in _2013 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)_. IEEE, 2013-06, pp. 27–34. [Online]. Available: http://ieeexplore.ieee.org/document/6595847/
  * [3] O. Barnich and M. Van Droogenbroeck, “ViBe: A Universal Background Subtraction Algorithm for Video Sequences,” _IEEE Transactions on Image Processing_ , vol. 20, no. 6, pp. 1709–1724, 2011-06.
  * [4] A. Bewley, Z. Ge, L. Ott, F. Ramos, and B. Upcroft, “Simple online and realtime tracking,” in _2016 IEEE International Conference on Image Processing (ICIP)_ , 2016-09, pp. 3464–3468.
  * [5] N. Aharon, R. Orfaig, and B.-Z. Bobrovsky. (2022-07-07) BoT-SORT: Robust Associations Multi-Pedestrian Tracking. [Online]. Available: http://arxiv.org/abs/2206.14651
  * [6] G. Maggiolino, A. Ahmad, J. Cao, and K. Kitani. (2023-02-23) Deep OC-SORT: Multi-Pedestrian Tracking by Adaptive Re-Identification. [Online]. Available: http://arxiv.org/abs/2302.11813
  * [7] O. Cetintas, G. Brasó, and L. Leal-Taixé, “Unifying short and long-term tracking with graph hierarchies,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , June 2023, pp. 22 877–22 887.
  * [8] C. Liang, Z. Zhang, X. Zhou, B. Li, S. Zhu, and W. Hu, “Rethinking the Competition Between Detection and ReID in Multiobject Tracking,” _IEEE Transactions on Image Processing_ , vol. 31, pp. 3182–3196, 2022.
  * [9] J. Wu, J. Cao, L. Song, Y. Wang, M. Yang, and J. Yuan, “Track to detect and segment: An online multi-object tracker,” in _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , 2021, pp. 12 352–12 361.
  * [10] Y. Zhang, T. Wang, and X. Zhang, “Motrv2: Bootstrapping end-to-end multi-object tracking by pretrained object detectors,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2023, pp. 22 056–22 065.
  * [11] H. Zhou, F. Jiang, and H. Lu, “Body-part joint detection and association via extended object representation,” in _2023 IEEE International Conference on Multimedia and Expo (ICME)_. IEEE, 2023, pp. 168–173.
  * [12] P. Sun, J. Cao, Y. Jiang, Z. Yuan, S. Bai, K. Kitani, and P. Luo, “DanceTrack: Multi-Object Tracking in Uniform Appearance and Diverse Motion,” in _2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_. IEEE, 2022-06, pp. 20 961–20 970. [Online]. Available: https://ieeexplore.ieee.org/document/9879192/
  * [13] A. Milan, L. Leal-Taixe, I. Reid, S. Roth, and K. Schindler. (2016-05-03) MOT16: A Benchmark for Multi-Object Tracking. [Online]. Available: http://arxiv.org/abs/1603.00831
  * [14] P. Dendorfer, H. Rezatofighi, A. Milan, J. Shi, D. Cremers, I. Reid, S. Roth, K. Schindler, and L. Leal-Taixé. (2020-03-19) MOT20: A benchmark for multi object tracking in crowded scenes. [Online]. Available: http://arxiv.org/abs/2003.09003
  * [15] G. Tesauro _et al._ , “Temporal difference learning and td-gammon,” _Communications of the ACM_ , vol. 38, no. 3, pp. 58–68, 1995.
  * [16] M. Piccardi, “Background subtraction techniques: A review,” in _2004 IEEE International Conference on Systems, Man and Cybernetics (IEEE Cat. No.04CH37583)_ , vol. 4, 2004-10, pp. 3099–3104 vol.4.
  * [17] M. Gong, Z. Zhou, and J. Ma, “Change detection in synthetic aperture radar images based on image fusion and fuzzy clustering,” _IEEE Transactions on Image Processing_ , vol. 21, no. 4, pp. 2141–2151, 2011.
  * [18] T. Bouwmans, F. El Baf, and B. Vachon, “Background modeling using mixture of gaussians for foreground detection-a survey,” _Recent patents on computer science_ , vol. 1, no. 3, pp. 219–237, 2008.
  * [19] Z. Pan, S. Liu, and W. Fu, “A review of visual moving target tracking,” _Multimedia Tools and Applications_ , vol. 76, no. 16, pp. 16 989–17 018, 2017-08-01. [Online]. Available: https://doi.org/10.1007/s11042-016-3647-0
  * [20] F. Zhong, S. Wang, Z. Zhang, C. Chen, and Y. Wang, “Detect-SLAM: Making Object Detection and SLAM Mutually Beneficial,” in _2018 IEEE Winter Conference on Applications of Computer Vision (WACV)_ , 2018-03, pp. 1001–1010.
  * [21] J. Zhang, M. Henein, R. Mahony, and V. Ila, “Vdo-slam: a visual dynamic object-aware slam system,” _arXiv preprint arXiv:2005.11052_ , 2020.
  * [22] J. Liu, Q. Gu, D. Chen, and D. Yan, “Vslam method based on object detection in dynamic environments,” _Frontiers in Neurorobotics_ , vol. 16, p. 990453, 2022.
  * [23] P. Sand and S. Teller, “Particle video: Long-range motion estimation using point trajectories,” _International journal of computer vision_ , vol. 80, pp. 72–91, 2008.
  * [24] M. Gehrig, M. Millhäusler, D. Gehrig, and D. Scaramuzza, “E-raft: Dense optical flow from event cameras,” in _2021 International Conference on 3D Vision (3DV)_. IEEE, 2021, pp. 197–206.
  * [25] R. J. Bouwmeester, F. Paredes-Vallés, and G. C. de Croon, “Nanoflownet: Real-time dense optical flow on a nano quadcopter,” in _2023 IEEE International Conference on Robotics and Automation (ICRA)_. IEEE, 2023, pp. 1996–2003.
  * [26] Q. Wang, Y.-Y. Chang, R. Cai, Z. Li, B. Hariharan, A. Holynski, and N. Snavely, “Tracking everything everywhere all at once,” _arXiv preprint arXiv:2306.05422_ , 2023.
  * [27] T. Liu and J. Regier. An Empirical Comparison of GANs and Normalizing Flows for Density Estimation. [Online]. Available: http://arxiv.org/abs/2006.10175
  * [28] E. Ilg, N. Mayer, T. Saikia, M. Keuper, A. Dosovitskiy, and T. Brox. FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks. [Online]. Available: http://arxiv.org/abs/1612.01925
  * [29] Z. Teed and J. Deng, “RAFT: Recurrent all-pairs field transforms for optical flow,” in _Computer Vision – ECCV 2020_ , A. Vedaldi, H. Bischof, T. Brox, and J.-M. Frahm, Eds. Springer International Publishing, 2020, pp. 402–419.
  * [30] C. Doersch, A. Gupta, L. Markeeva, A. Recasens, L. Smaira, Y. Aytar, J. Carreira, A. Zisserman, and Y. Yang, “TAP-Vid: A benchmark for tracking any point in a video,” in _Advances in Neural Information Processing Systems_ , S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, Eds., vol. 35. Curran Associates, Inc., 2022, pp. 13 610–13 626. [Online]. Available: https://proceedings.neurips.cc/paper_files/paper/2022/file/58168e8a92994655d6da3939e7cc0918-Paper-Datasets_and_Benchmarks.pdf
  * [31] G. Rahmon, F. Bunyak, G. Seetharaman, and K. Palaniappan, “Motion U-Net: Multi-cue Encoder-Decoder Network for Motion Segmentation,” in _2020 25th International Conference on Pattern Recognition (ICPR)_. IEEE, 2021-01-10, pp. 8125–8132. [Online]. Available: https://ieeexplore.ieee.org/document/9413211/
  * [32] W. Shi, M. Zhang, R. Zhang, S. Chen, and Z. Zhan, “Change detection based on artificial intelligence: State-of-the-art and challenges,” _Remote Sensing_ , vol. 12, no. 10, p. 1688, 2020.
  * [33] A. Asokan and J. Anitha, “Change detection techniques for remote sensing applications: A survey,” _Earth Science Informatics_ , vol. 12, pp. 143–160, 2019.
  * [34] J. Sanchez-Riera, K. Srinivasan, K.-L. Hua, W.-H. Cheng, M. A. Hossain, and M. F. Alhamid, “Robust rgb-d hand tracking using deep learning priors,” _IEEE Transactions on Circuits and Systems for Video Technology_ , vol. 28, no. 9, pp. 2289–2301, 2017.
  * [35] G. Jocher, A. Chaurasia, and J. Qiu, “Ultralytics yolov8,” 2023. [Online]. Available: https://github.com/ultralytics/ultralytics
  * [36] Y. Du, Z. Zhao, Y. Song, Y. Zhao, F. Su, T. Gong, and H. Meng, “StrongSORT: Make DeepSORT Great Again,” _IEEE Transactions on Multimedia_ , pp. 1–14, 2023.
  * [37] S. Prabhakar and A. K. Jain, “Decision-level fusion in fingerprint verification,” _Pattern Recognition_ , vol. 35, no. 4, pp. 861–874, 2002\.
  * [38] I. Leang, S. Herbin, B. Girard, and J. Droulez, “On-line fusion of trackers for single-object tracking,” _Pattern Recognition_ , vol. 74, pp. 459–473, 2018-02. [Online]. Available: https://linkinghub.elsevier.com/retrieve/pii/S0031320317303783
  * [39] Z. Tang, T. Xu, H. Li, X.-J. Wu, X. Zhu, and J. Kittler, “Exploring fusion strategies for accurate rgbt visual object tracking,” _Information Fusion_ , p. 101881, 2023.
  * [40] X. Lan, A. J. Ma, and P. C. Yuen, “Multi-cue visual tracking using robust feature-level fusion based on joint sparse representation,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2014, pp. 1194–1201.
  * [41] Y. Pang, F. Li, X. Qiao, and A. Gilman, “Real-time tracking based on deep feature fusion,” _Multimedia Tools and Applications_ , vol. 79, pp. 27 229–27 255, 2020.
  * [42] Z. Ding, T. Hui, S. Huang, S. Liu, X. Luo, J. Huang, and X. Wei, “Progressive multimodal interaction network for referring video object segmentation,” _The 3rd Large-scale Video Object Segmentation Challenge_ , vol. 8, 2021.
  * [43] Z. Yang, P. Huang, D. He, Z. Cai, and Z. Yin, “Siammmf: multi-modal multi-level fusion object tracking based on siamese networks,” _Machine Vision and Applications_ , vol. 34, no. 1, p. 7, 2023.
  * [44] G. D. Evangelidis and E. Z. Psarakis, “Parametric image alignment using enhanced correlation coefficient maximization,” _IEEE transactions on pattern analysis and machine intelligence_ , vol. 30, no. 10, pp. 1858–1865, 2008\.
  * [45] K. Zhou, Y. Yang, A. Cavallaro, and T. Xiang, “Omni-Scale Feature Learning for Person Re-Identification,” in _2019 IEEE/CVF International Conference on Computer Vision (ICCV)_. IEEE, 2019-10, pp. 3701–3711. [Online]. Available: https://ieeexplore.ieee.org/document/9011001/
  * [46] J. Cao, J. Pang, X. Weng, R. Khirodkar, and K. Kitani, “Observation-centric sort: Rethinking sort for robust multi-object tracking,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2023, pp. 9686–9696.
  * [47] K. Bernardin and R. Stiefelhagen, “Evaluating multiple object tracking performance: the clear mot metrics,” _EURASIP Journal on Image and Video Processing_ , vol. 2008, pp. 1–10, 2008.
  * [48] Y. Wang, P.-M. Jodoin, F. Porikli, J. Konrad, Y. Benezeth, and P. Ishwar, “CDnet 2014: An Expanded Change Detection Benchmark Dataset,” in _2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)_. IEEE, 2014-06, pp. 393–400. [Online]. Available: https://ieeexplore.ieee.org/document/6910011
  * [49] L. A. Lim and H. Y. Keles, “Learning Multi-scale Features for Foreground Segmentation,” _Pattern Analysis and Applications_ , vol. 23, no. 3, pp. 1369–1380, 2020-08. [Online]. Available: http://arxiv.org/abs/1808.01477
  * [50] A. Geiger, P. Lenz, and R. Urtasun, “Are we ready for autonomous driving? the kitti vision benchmark suite,” in _Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2012.
  * [51] D. Du, Y. Qi, H. Yu, Y. Yang, K. Duan, G. Li, W. Zhang, Q. Huang, and Q. Tian, “The unmanned aerial vehicle benchmark: Object detection and tracking,” in _Proceedings of the European conference on computer vision (ECCV)_ , 2018, pp. 370–386.
  * [52] E. Ristani, F. Solera, R. Zou, R. Cucchiara, and C. Tomasi, “Performance Measures and a Data Set for Multi-target, Multi-camera Tracking,” in _Computer Vision – ECCV 2016 Workshops_ , G. Hua and H. Jégou, Eds. Springer International Publishing, 2016, vol. 9914, pp. 17–35. [Online]. Available: http://link.springer.com/10.1007/978-3-319-48881-3_2
  * [53] J. Luiten, A. Osep, P. Dendorfer, P. Torr, A. Geiger, L. Leal-Taixe, and B. Leibe, “Hota: A higher order metric for evaluating multi-object tracking,” _International Journal of Computer Vision (IJCV)_ , 2020.
  * [54] X. Liu, W. Liu, T. Mei, and H. Ma, “A deep learning-based approach to progressive vehicle re-identification for urban surveillance,” in _Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part II 14_. Springer, 2016, pp. 869–884.
  * [55] L. Zheng, L. Shen, L. Tian, S. Wang, J. Bu, and Q. Tian, “Person re-identification meets image search,” _arXiv preprint arXiv:1502.02171_ , 2015.
  * [56] Y. Zhang, P. Sun, Y. Jiang, D. Yu, F. Weng, Z. Yuan, P. Luo, W. Liu, and X. Wang, “ByteTrack: Multi-object Tracking by Associating Every Detection Box,” in _Computer Vision – ECCV 2022_ , ser. Lecture Notes in Computer Science, S. Avidan, G. Brostow, M. Cissé, G. M. Farinella, and T. Hassner, Eds. Springer Nature Switzerland, 2022, pp. 1–21.
  * [57] M. C. Heinrich, C. D. Blanke, B. J. Druker, and C. L. Corless, “Inhibition of kit tyrosine kinase activity: a novel molecular approach to the treatment of kit-positive malignancies,” _Journal of clinical oncology_ , vol. 20, no. 6, pp. 1692–1703, 2002.

## Appendices

## Traditional Method Double Gaussian Model

The process of traditional methods in this paper is very similar to [2],which
is a mixture of Gaussians, second version method(MOG2).

Firstly, we use KIT[57] to correct the camera’s motion.

Then, we adopt a dual-mode single Gaussian model with three parameters: age,
mean, and variance. The input image is divided into an $N*N$ grid, and each
grid employs a single Gaussian model (SGM). This approach effectively reduces
computational complexity while maintaining good detection performance.

Mathematically, let $G_{i}^{(t)}$ represent the pixel group of grid $i$ at
time $t$, $\left|G_{i}^{(t)}\right|$ denotes the number of pixels in this
group, and It represents the grayscale value of pixel $j$ at time $t$. The
update functions for mean, variance, and age in the single Gaussian model are
given by (18):

$\begin{gathered}\mu_{i}^{(t)}=\frac{\tilde{\alpha}_{i}^{(t-1)}}{\tilde{\alpha}_{i}^{(t-1)}+1}\tilde{\mu}_{i}^{(t-1)}+\frac{1}{\tilde{\alpha}_{i}^{(t-1)}+1}M_{i}^{(t)}\\\
\sigma_{i}^{(t)}=\frac{\tilde{\alpha}_{i}^{(t-1)}}{\tilde{\alpha}_{i}^{(t-1)}+1}\tilde{\sigma}_{i}^{(t-1)}+\frac{1}{\tilde{\alpha}_{i}^{(t-1)}+1}V_{i}^{(t)}\\\
\alpha_{i}^{(t)}=\tilde{\alpha}_{i}^{(t-1)}+1\end{gathered}$ (18)

Here, $M_{i}^{(t)}$ denotes the average grayscale value within a grid at time
$t$, and $V_{i}^{(t)}$ represents the squared difference between the moving
average grayscale and the maximum grayscale value within a grid, as shown in
(19) and (20):

$M_{i}^{(t)}=\frac{1}{\left|G_{i}\right|}\sum_{j\in G_{i}}{I_{j}^{(t)}}$ (19)
$V_{i}^{(t)}=\max_{j\in G_{i}}\left(\mu_{i}^{(t)}-I_{j}^{(t)}\right)^{2}$ (20)

If the variance exceeds a critical value
(i.e.$,\tilde{\sigma}_{i}^{(t-1)}\,\,>\,\,\theta_{v}$), the model’s age may
need to be adjusted as shown in follow:

$\tilde{\alpha}_{i}^{(t-1)}\leftarrow\tilde{\alpha}_{i}^{(t-1)}\exp\left\\{-\lambda\left(\tilde{\sigma}_{i}^{(t-1)}-\theta_{v}\right)\right\\}$
(21)

We represent the parameters of the candidate model and the current model for
grid $i$ at time $t$ as $\mu_{B,i}^{(t)},\sigma_{B,i}^{(t)}$and
$\alpha_{B,i}^{(t)}$, respectively.If the observed difference between the
average value $M_{i}^{(t)}$and the squared value $\mu_{B,i}^{(t)}$, is less
than the critical value related to $\alpha_{B,i}^{(t)}$, as shown in (22):

$\left(M_{i}^{(t)}-\mu_{A,i}^{(t)}\right)^{2}<\theta_{s}\sigma_{A,i}^{(t)}$
(22)

where $\theta_{s}$ is a critical parameter. If the above condition is not
satisfied but the observed average value matches the candidate background
model, that is (23):

$\left(M_{i}^{(t)}-\mu_{B,i}^{(t)}\right)^{2}<\theta_{s}\sigma_{B,i}^{(t)}$
(23)

If neither of the two conditions mentioned above is met, the candidate
background model is updated with the current observed value.

At the same time, if (23) is satisfied:

$\alpha_{B,i}^{(t)}>\alpha_{A,i}^{(t)}$ (24)

i.e., when the age of the candidate background model is greater than the
current background model’s age, the two models will exchange their roles.

Finally, after determining the models, for the j-th pixel within the i-th
pixel set, if (25) is satisfied, the pixel is recognized as a foreground
pixel, where d is a threshold value. Note that only the current background
model is used to decide the foreground.

$\left(I_{j}^{(t)}-\mu_{A,i}^{(t)}\right)^{2}>\theta_{d}\sigma_{A,i}^{(t)}$
(25)

Once the foreground pixels are determined, applying some morphological
operations yields bbox of the moving objects, as shown in the follow figure.

Figure 7: The framework of traditional method.