\parbox{0.49\textwidth}{
\caption{Evaluation on the effectiveness of formulating the demanders $D$ involving the idea of sub-pixel resolution.}
\label{Tab:ablation_1}
\resizebox{0.49\textwidth}{!}{
\begin{tabular}{l|cccccc}
\hline
Method & AP & $\text{AP}^{50}$ & $\text{AP}^{75}$ & $\text{AP}^{\text{M}}$ &$\text{AP}^{\text{L}}$ &AR  \\
\hline
Baseline(HRNet-W48) &77.1 &91.8 &83.8 &73.5 &83.5 &81.8\\
\hline
\textbf{Naive demanders formulation} & 77.9 & 92.5 & 84.8 & 74.5 & 83.9 & 82.4 \\
\textbf{Sub-pixel demanders formulation} & 78.8 & 92.5 & 85.1 & 75.0 & 85.3 & 83.1\\
\hline
\end{tabular}}}
The Earth Mover's Distance is a 
a technique used for measuring the difference between two probability distributions, which can be understood as the optimal cost needed to transport the mass from one distribution to another. Specifically, for calculating the Earth Mover's Distance, we regard the source distribution as a set of ($N$) suppliers $S = (s_1, ..., s_N)^\top$, where $s_n$ represents the total units of mass that the $n$-th supplier has, and we regard the target distribution as a set of ($M$) demanders $D = (d_1, ..., d_M)^\top$, where $d_m$ represents the total units of mass that the $m$-th demander requires. Besides, we also denote $C \in  R^{N \times M}_{\geq 0}$ as the cost function between the source and target distributions, where $C_{n,m}$ represents the cost for transporting a unit of mass from the $n$-th supplier to the $m$-th demander. Then we aim to find a least-cost transportation plan from the set of possible plans $P = \{p \in R^{N \times M}_{\geq 0}:~p\textbf{1} = S~\&~p^\top\textbf{1} = D\}$ to transport all mass from the $N$ suppliers to the $M$ demanders, where \textbf{1} represents a vector of ones. The Earth Mover's Distance $E_C(S, D)$ denotes the cost of the least-cost transportation plan, which can be formulated as:

\noindent\textbf{The demanders $D$.} As for the demanders, we aim to construct $D$ w.r.t. each body joint to represent its corresponding GT dot annotation.
To achieve this goal, for each body joint, a naive formulation of $D$ is to identify the pixel containing the GT dot annotation (i.e., the upper left pixel in Fig.~\ref{fig:demander}(a)) and construct a single demander (i.e., the yellow dot in Fig.~\ref{fig:demander}(a)) at the center of this pixel.
However, this naive formulation can result in a suboptimal model performance, as the demander formulated in this way can be noticeably different from what the GT dot annotation might suggest.
Generally, the predicted heatmaps outputted by most of the existing \textit{heatmap-based methods} \cite{tompson2014joint,newell2016stacked,xiao2018simple,sun2019deep,li2019rethinking,cheng2020higherhrnet,zhang2020distribution,yang2020transpose,li2021tokenpose,luo2021rethinking,YuanFHLZCW21} have a lower resolution compared to the input image. For example, for the method HRNet \cite{sun2019deep}, when the size of the input image is $384\times288$, the size of the predicted heatmap is $96\times72$ only. Due to such a resolution gap, as shown in  Fig.~\ref{fig:demander}(a), there can exist a non-negligible distance between the location of the demander and the location of the dot annotation, which can affect the performance of the pose estimation model.

To address this problem, we aim to make the formulated demanders $D$ a more accurate representation of the dot annotation by overcoming the resolution gap. To achieve this, inspired by the fact that the more accurate location information of an object can be deduced using sub-pixel resolutions \cite{khademi2012sub}, we propose to formulate the demanders $D$ with the following four steps. (1) As illustrated in the upper left of Fig.~\ref{fig:demander}(b), we first split the pixel containing the dot annotation into four "sub-pixels" (four squares separated from each other by dashed lines). (2) After that, we identify the "sub-pixel" that the dot annotation lies in (i.e., the green "sub-pixel" shown in Fig.~\ref{fig:demander}(b)). (3) Then, as shown by the four yellow dots in Fig.~\ref{fig:demander}(b), we localize a set of four demanders at the centers of both the pixel containing the identified "sub-pixel" (i.e., the upper left pixel in Fig.~\ref{fig:demander}(b)), and the three pixels adjacent to the identified "sub-pixel" (i.e., the three blue pixels in Fig.~\ref{fig:demander}(b)). (4) Finally, 
we determine the units of mass each demander requires in the following way so that the dot annotation can be precisely represented using the set of 
the four demanders.
Specifically, 
denote the 2D Euclidean distance between the centers of two adjacent pixels as $g$ (which is also the pixel size), the coordinates of the GT dot annotation as $(x_{dot}, y_{dot})$, and the coordinates of the $i$-th demander as $(x_i, y_i)$, where $i \in \{1, 2, 3, 4\}$. Then we construct the demanders $D$ as:

\noindent\textbf{The cost function $C$.} 
To measure the distribution difference between the $H \times W$ suppliers ($S$) constructed from the predicted heatmap and the $4$ demanders ($D$) constructed from the GT dot annotation via calculating their Earth Mover's Distance, we also need to formulate a cost function $C \in  R^{HW \times 4}_{\geq 0}$. 
Denote the coordinates of the $n$-th supplier as $(x_{s_n}, y_{s_n})$, where $n \in \{1, ..., H \times W\}$, and the coordinates of the $m$-th demander as $(x_{d_m}, y_{d_m})$, where $m \in \{1, 2, 3, 4\}$. We here simply formulate $C_{n,m}$, the cost per unit transported from the $n$-th supplier to the $m$-th demander, as the L2 distance between the $n$-th supplier and the $m$-th demander, i.e., $C_{n,m} = \sqrt{(x_{d_m} - x_{s_n})^2 + (y_{d_m} - y_{s_n})^2}$. Using such a cost function, our method can optimize the model directly towards accurately localizing the dot annotation of the body joint via minimizing the distribution difference between the suppliers $S$ and the demanders $D$.

The proof of Theorem~\ref{thm:theorem_1} is provided in the supplementary.
As shown in Theorem~\ref{thm:theorem_1}, when we minimize the pixel-wise loss between $\mathbf{H_{pred}}$ and $\mathbf{H_{Gau}}$, while $R(\mathcal D_{Gau}, \phi, l_{MSE})$ decreases, the second term $2 \times \mathbb{E}_{(I, \mathbf{H_{Gau}}) \sim \mathcal D_{Gau}}[\langle\mathbf{H_{pred}},\mathbf{H_{Gau}}\rangle]$ can increase and the third term $2 \times \mathbb{E}_{(I, \mathbf{H_{dot}}) \sim \mathcal D_{dot}} [\langle\mathbf{H_{pred}},\mathbf{H_{dot}}\rangle]$ cannot be guaranteed to increase or decrease. Because of this, $R(\mathcal D_{dot}, \phi, l_{MSE})$ cannot be guaranteed to decrease, and the model performance of body joint localization may not be consistently improved during such optimization of heatmap prediction. A more intuitive analysis is as follows. In the optimization process of most of the \textit{heatmap-based methods}, since all the $H \times W$ pixels from the predicted heatmap contribute to the overall pixel-wise loss, learning to fit the other pixels better instead of the pixel representing the dot annotation can also lead to a smaller overall loss, as shown in Fig.~\ref{fig:example}. Hence, when minimizing the overall pixel-wise loss, the model performance of body joint localization cannot be guaranteed to improve consistently. Besides, during training, the GT Gaussian-smoothed heatmap is constructed via using the Gaussian blob. Therefore, during testing, the predicted heatmap often has a relatively large area of pixels with large values around the dot annotation, as shown in Fig.~\ref{fig:visual}, which can make it difficult to accurately locate the body joint.

\subsection{COCO Keypoint Detection}
\noindent\textbf{Dataset \& evaluation metric.} The COCO dataset \cite{lin2014microsoft} contains more than 200k images and 250k person instances, which are annotated with 17 body joints. This dataset has three subsets including COCO training set, COCO validation set, and COCO test-dev set, which have 57k, 5k and 20k images, respectively. We conduct experiments on this dataset via first training the model on the train2017 set, and then evaluating the model on the val2017 set and test-dev2017 set. Following \cite{xiao2018simple,sun2019deep,YuanFHLZCW21}, we use standard average precision (AP) calculated based on Object Keypoint Similarity (OKS) to evaluate model performance.

\subsection{MPII Human Pose Estimation}
\noindent\textbf{Dataset \& evaluation metric.} The MPII dataset \cite{andriluka20142d} contains around 25K images and more than 40k person instances, which are annotated with 16 body joints. We adopt the standard train/val split in \cite{andriluka20142d} to build the MPII training set and validation set, and conduct all the experiments on this dataset via first training the model on the MPII training set, and then evaluating it on the MPII validation set. Following \cite{sun2019deep}, we use the head-normalized probability of correct keypoint (PCKh) \cite{andriluka20142d} score as the evaluation metric on this dataset and report the PCKh@0.5 score.

\noindent\textbf{Impact of involving the idea of sub-pixels in formulating $D$.} In our proposed method, we formulate the demanders $D$ by involving the idea of sub-pixel resolution. To investigate the impact of formulating the demanders $D$ in such a way, we compare our proposed method (\textbf{sub-pixel demanders formulation}) with a variant (\textbf{naive demanders formulation}). This variant still formulates the suppliers $S$ and the cost function $C$ in the same way, but formulates the demanders $D$ naively as a single demander at the center of the pixel containing the dot annotation, as shown in Fig.~\ref{fig:demander}(a). As shown in Tab.~\ref{Tab:ablation_1}, our proposed method consistently outperforms this variant, which shows effectiveness of our \textbf{sub-pixel demanders formulation}.
{"value": "77.1", "type": "Result", "training data/set": "COCO", "test data/set": "COCO validation", "task": ["human pose estimation", "2D human pose estimation"], "metric": "AP", "experimental settings": {"xx": "yy"}, "model": "Baseline(HRNet-W48)", "model settings": {"xx": "yy"}} {"value": "91.8", "type": "Result", "training data/set": "COCO", "test data/set": "COCO validation", "task": ["human pose estimation", "2D human pose estimation"], "metric": "\\text{AP}^{50}", "experimental settings": {"xx": "yy"}, "model": "Baseline(HRNet-W48)", "model settings": {"xx": "yy"}} {"value": "83.8", "type": "Result", "training data/set": "COCO", "test data/set": "COCO validation", "task": ["human pose estimation", "2D human pose estimation"], "metric": "\\text{AP}^{75}", "experimental settings": {"xx": "yy"}, "model": "Baseline(HRNet-W48)", "model settings": {"xx": "yy"}} {"value": "73.5", "type": "Result", "training data/set": "COCO", "test data/set": "COCO validation", "task": ["human pose estimation", "2D human pose estimation"], "metric": "\\text{AP}^{\\text{M}", "experimental settings": {"xx": "yy"}, "model": "Baseline(HRNet-W48)", "model settings": {"xx": "yy"}} {"value": "83.5", "type": "Result", "training data/set": "COCO", "test data/set": "COCO validation", "task": ["human pose estimation", "2D human pose estimation"], "metric": "\\text{AP}^{\\text{L}}", "experimental settings": {"xx": "yy"}, "model": "Baseline(HRNet-W48)", "model settings": {"xx": "yy"}} {"value": "81.8", "type": "Result", "training data/set": "COCO", "test data/set": "COCO validation", "task": ["human pose estimation", "2D human pose estimation"], "metric": "AR", "experimental settings": {"xx": "yy"}, "model": "Baseline(HRNet-W48)", "model settings": {"xx": "yy"}} {"value": "77.9", "type": "Result", "training data/set": "COCO", "test data/set": "COCO validation", "task": ["human pose estimation", "2D human pose estimation"], "metric": "AP", "experimental settings": {"xx": "yy"}, "model": "Naive demanders formulation", "model settings": {"xx": "yy"}} {"value": "92.5", "type": "Result", "training data/set": "COCO", "test data/set": "COCO validation", "task": ["human pose estimation", "2D human pose estimation"], "metric": "\\text{AP}^{50}", "experimental settings": {"xx": "yy"}, "model": "Naive demanders formulation", "model settings": {"xx": "yy"}} {"value": "84.8", "type": "Result", "training data/set": "COCO", "test data/set": "COCO validation", "task": ["human pose estimation", "2D human pose estimation"], "metric": "\\text{AP}^{75}", "experimental settings": {"xx": "yy"}, "model": "Naive demanders formulation", "model settings": {"xx": "yy"}} {"value": "74.5", "type": "Result", "training data/set": "COCO", "test data/set": "COCO validation", "task": ["human pose estimation", "2D human pose estimation"], "metric": "\\text{AP}^{\\text{M}", "experimental settings": {"xx": "yy"}, "model": "Naive demanders formulation", "model settings": {"xx": "yy"}} {"value": "83.9", "type": "Result", "training data/set": "COCO", "test data/set": "COCO validation", "task": ["human pose estimation", "2D human pose estimation"], "metric": "\\text{AP}^{\\text{L}}", "experimental settings": {"xx": "yy"}, "model": "Naive demanders formulation", "model settings": {"xx": "yy"}} {"value": "82.4", "type": "Result", "training data/set": "COCO", "test data/set": "COCO validation", "task": ["human pose estimation", "2D human pose estimation"], "metric": "AR", "experimental settings": {"xx": "yy"}, "model": "Naive demanders formulation", "model settings": {"xx": "yy"}} {"value": "78.8", "type": "Result", "training data/set": "COCO", "test data/set": "COCO validation", "task": ["human pose estimation", "2D human pose estimation"], "metric": "AP", "experimental settings": {"xx": "yy"}, "model": "Sub-pixel demanders formulation", "model settings": {"xx": "yy"}} {"value": "92.5", "type": "Result", "training data/set": "COCO", "test data/set": "COCO validation", "task": ["human pose estimation", "2D human pose estimation"], "metric": "\\text{AP}^{50}", "experimental settings": {"xx": "yy"}, "model": "Sub-pixel demanders formulation", "model settings": {"xx": "yy"}} {"value": "85.1", "type": "Result", "training data/set": "COCO", "test data/set": "COCO validation", "task": ["human pose estimation", "2D human pose estimation"], "metric": "\\text{AP}^{75}", "experimental settings": {"xx": "yy"}, "model": "Sub-pixel demanders formulation", "model settings": {"xx": "yy"}} {"value": "75.0", "type": "Result", "training data/set": "COCO", "test data/set": "COCO validation", "task": ["human pose estimation", "2D human pose estimation"], "metric": "\\text{AP}^{\\text{M}", "experimental settings": {"xx": "yy"}, "model": "Sub-pixel demanders formulation", "model settings": {"xx": "yy"}} {"value": "85.3", "type": "Result", "training data/set": "COCO", "test data/set": "COCO validation", "task": ["human pose estimation", "2D human pose estimation"], "metric": "\\text{AP}^{\\text{L}}", "experimental settings": {"xx": "yy"}, "model": "Sub-pixel demanders formulation", "model settings": {"xx": "yy"}} {"value": "83.1", "type": "Result", "training data/set": "COCO", "test data/set": "COCO validation", "task": ["human pose estimation", "2D human pose estimation"], "metric": "AR", "experimental settings": {"xx": "yy"}, "model": "Sub-pixel demanders formulation", "model settings": {"xx": "yy"}}