Muhammad Saif Ullah Khan1,2 Dhavalkumar Limbachiya2 Didier Stricker1,2
Muhammad Zeshan Afzal1,2 German Research Center for Artificial Intelligence
(DFKI)
Kaiserslautern, Germany University of Kaiserslautern-Landau (RPTU)
Kaiserslautern, Germany Estimating Human Poses Across Datasets

# Estimating Human Poses Across Datasets: A Unified Skeleton and Multi-Teacher
Distillation Approach

###### Abstract

Human pose estimation is a key task in computer vision with various
applications such as activity recognition and interactive systems. However,
the lack of consistency in the annotated skeletons across different datasets
poses challenges in developing universally applicable models. To address this
challenge, we propose a novel approach integrating multi-teacher knowledge
distillation with a unified skeleton representation. Our networks are jointly
trained on the COCO and MPII datasets, containing 17 and 16 keypoints,
respectively. We demonstrate enhanced adaptability by predicting an extended
set of 21 keypoints, 4 (COCO) and 5 (MPII) more than original annotations,
improving cross-dataset generalization. Our joint models achieved an average
accuracy of 70.89 and 76.40, compared to 53.79 and 55.78 when trained on a
single dataset and evaluated on both. Moreover, we also evaluate all 21
predicted points by our two models by reporting an AP of 66.84 and 72.75 on
the Halpe dataset. This highlights the potential of our technique to address
one of the most pressing challenges in pose estimation research and
application - the inconsistency in skeletal annotations.

##  1 Introduction

2D human pose estimation using deep learning [Dang et al.(2019)Dang, Yin,
Wang, and Zheng, Zhao et al.(2021)Zhao, Gao, Ge, and Tian, Thai et
al.(2023)Thai, Nham, Tran, Bui, Ninh, and Tran, Jiang et al.(2023)Jiang, Lu,
Zhang, Ma, Han, Lyu, Li, and Chen, Zheng et al.(2023)Zheng, Wu, Chen, Yang,
Zhu, Shen, Kehtarnavaz, and Shah] is crucial for various computer vision
applications, such as activity recognition [Song et al.(2021)Song, Yu, Yuan,
and Liu], augmented reality [Marchand et al.(2015)Marchand, Uchiyama, and
Spindler], and interactive systems [Liu et al.(2022)Liu, Liu, Zhang, Sangaiah,
Yang, and Li]. However, many datasets are available [Andriluka et
al.(2014)Andriluka, Pishchulin, Gehler, and Schiele, Lin et al.(2014)Lin,
Maire, Belongie, Hays, Perona, Ramanan, Dollár, and Zitnick, Li et
al.(2020a)Li, Xu, Liu, Huang, Xu, Wang, Fang, Ma, Chen, and Lu], each
typically featuring a unique set of annotated body joints and skeleton
topology [Dang et al.(2019)Dang, Yin, Wang, and Zheng]. This diversity
necessitates the training of separate models for each dataset, which not only
restricts the applicability of these models to their respective skeletons but
also limits their performance on new, unseen datasets with different
skeletons.

To address this critical issue, we introduce a novel framework that unifies
human pose estimation across datasets. Our approach employs multi-teacher
knowledge distillation [Hinton et al.(2015)Hinton, Vinyals, and Dean, Thai et
al.(2023)Thai, Nham, Tran, Bui, Ninh, and Tran] combined with pose union
learning to train a single, adaptable student network. This network learns a
comprehensive skeleton topology that represents the union of multiple
datasets. This significantly improves its generalization and adaptability
across various pose estimation tasks. Furthermore, by leveraging a set of pre-
trained teacher networks, our method efficiently distills the knowledge from
multiple datasets and teacher networks into a single student model. This
allows the student model to predict a broader set of keypoints than any
individual teacher.

We demonstrate the effectiveness of our proposed approach using multiple
variants of the RTMPose model [Jiang et al.(2023)Jiang, Lu, Zhang, Ma, Han,
Lyu, Li, and Chen]. We train these models on a unified dataset combining the
MPII [Andriluka et al.(2014)Andriluka, Pishchulin, Gehler, and Schiele] and
COCO [Lin et al.(2014)Lin, Maire, Belongie, Hays, Perona, Ramanan, Dollár, and
Zitnick] datasets, containing 16 and 17 annotated body keypoints,
respectively. These datasets share 12 common keypoints, with 5 unique points
in MPII and 4 unique points in COCO. We train our model to predict a superset
of the two skeletons, comprising 21 keypoints in total. We then show the
performance of our unified skeleton predictors on the validation sets of both
datasets using their respective keypoints. Moreover, we also evaluate the
precision of all 21 predicted keypoints using the groundtruth annotations from
the Halpe dataset [Li et al.(2020a)Li, Xu, Liu, Huang, Xu, Wang, Fang, Ma,
Chen, and Lu]. This dataset provides 26 body points for images from the COCO
validation set. These include all keypoints predicted by our model. Our
results indicate that our approach enables the model to learn a unified
skeleton that can be generalized to images in both datasets by accurately
predicting joints in their images that were not present in their groundtruth
annotations. This represents a step forward in addressing the limitations
imposed by dataset-specific skeletons and can enable applications that require
a more comprehensive understanding of body joints.

To summarize, the contributions of this paper are as follows:

  1. 1.

Introduction of a novel framework that unifies pose estimation across diverse
datasets by integrating multi-teacher knowledge distillation and pose union
learning, demonstrating competitive performance.

  2. 2.

Utilization of the unified model to predict previously unknown keypoints in
the COCO and MPII dataset images with high accuracy.

  3. 3.

Extensive experimental results to support our approach and validate its
effectiveness for pose estimation across the COCO and MPII datasets.

Moreover, our unified model can potentially be used to enhance the ground-
truth annotations for these datasets themselves using active learning
strategies. While we leave this as future work, our research opens multiple
avenues in cross-dataset pose estimation and keypoint dataset enhancement.

The remainder of this paper is organized as follows: Section 2 reviews related
work in human pose estimation, knowledge distillation, and cross-dataset pose
estimation, providing context for our contributions. Section 3 details our
methodology, explaining the architecture and training process of our model.
Sections 4 and 5 present our experimental setup, results, and ablation
studies, showcasing the empirical evidence supporting our claims. Finally,
Section 6 concludes the paper with a discussion of our findings and their
implications for future research in human pose estimation.

##  2 Related Work

The development of robust and accurate human pose estimation models and the
use of knowledge distillation to enhance training efficiency represent two
pivotal research areas in deep learning. This section reviews the current
state of the art in these domains and highlights the gaps our work aims to
bridge.

###  2.1 Human Pose Estimation

2D human pose estimation using deep learning is a fundamental task in computer
vision to locate human joints (keypoints) in images or video frames. The two-
stage techniques predominantly fall into either top-down or bottom-up
categories. Top-down methods [Li et al.(2021a)Li, Zhang, Wang, Yang, Yang,
Xia, and Zhou, Xu et al.(2022)Xu, Zhang, Zhang, and Tao, Khirodkar et
al.(2021)Khirodkar, Chari, Agrawal, and Tyagi, Jiang et al.(2023)Jiang, Lu,
Zhang, Ma, Han, Lyu, Li, and Chen] first detect individual people in the image
and then predict keypoints for each detected person, often leading to higher
accuracy but at the cost of computational efficiency. In contrast, bottom-up
methods [Cheng et al.(2020)Cheng, Xiao, Wang, Shi, Huang, and Zhang,
Osokin(2018), Wang et al.(2022)Wang, Li, Cai, Chen, and Han, Papandreou et
al.(2018)Papandreou, Zhu, Chen, Gidaris, Tompson, and Murphy] detect all
keypoints in the image first and then cluster them into individual poses,
which can be more scalable but might struggle with accuracy in dense human
environments. Alternatively, some recent works [Nie et al.(2019)Nie, Feng,
Zhang, and Yan, Shi et al.(2021)Shi, Wei, Yu, Tan, Ren, and Pu, Jin et
al.(2022)Jin, Xu, Wang, Xiao, Guo, Nie, and Zhao, Shi et al.(2022)Shi, Wei,
Li, Ren, and Tan, Lu et al.(2023)Lu, Jiang, Li, Li, Chen, and Yang] use a one-
stage approach using a single model for the complete pose estimation pipeline.
Keypoint predictions are typically represented directly as coordinates or,
more commonly, through heatmaps, where the latter involves predicting a
confidence map for each keypoint and has shown considerable success due to its
robustness to pixel-level noise and variability in human appearances. These
methods leverage a variety of network architectures, with more recent works
using transformers [Xu et al.(2022)Xu, Zhang, Zhang, and Tao, Li et
al.(2021a)Li, Zhang, Wang, Yang, Yang, Xia, and Zhou, Sun et al.(2024)Sun, Gu,
Wang, Yang, and Yao, Shi et al.(2022)Shi, Wei, Li, Ren, and Tan, Zheng et
al.(2021)Zheng, Zhu, Mendieta, Yang, Chen, and Ding], which have been adapted
better to capture the spatial hierarchies inherent in human poses. Despite
significant advancements, the challenge remains to enhance the precision and
efficiency of these models, especially in unconstrained real-world scenarios
where occlusions and complex human interactions are common.

###  2.2 Knowledge Distillation

Knowledge distillation [Hinton et al.(2015)Hinton, Vinyals, and Dean] is a
technique originally developed to compress the knowledge of a large, complex
model (teacher) into a smaller, more efficient model (student) with minimal
performance loss. Logit-based knowledge distillation [Hinton et
al.(2015)Hinton, Vinyals, and Dean, Chen et al.(2020)Chen, Mei, Wang, Feng,
and Chen, Jin et al.(2023)Jin, Wang, and Lin, Li et al.(2020b)Li, Huang, Chen,
Luo, Cai, and Pan, Mirzadeh et al.(2019)Mirzadeh, Farajtabar, Li, Levine,
Matsukawa, and Ghasemzadeh, Zhang et al.(2023)Zhang, Shen, Liu, Liu,
Bendersky, Najork, and Zhang, Zhao et al.(2022)Zhao, Cui, Song, Qiu, and
Liang, Zhao et al.(2022)Zhao, Cui, Song, Qiu, and Liang] utilizes the soft
output probabilities from the teacher model to guide the training of the
student model. By employing a loss function that combines temperature-scaled
cross-entropy with Kullback-Leibler divergence [Kullback and Leibler(1951)],
the student model is encouraged not only to mimic the teacher’s output
distribution but also to achieve correct label predictions on its own. This
method effectively transfers the intricate decision boundaries learned by the
teacher to the student, enhancing its performance beyond what could be
achieved through direct training alone. Feature-based distillation [Ahn et
al.(2019)Ahn, Hu, Damianou, Lawrence, and Dai, Chen et al.(2022)Chen, Mei,
Zhang, Wang, Feng, and Chen, Chen et al.(2021)Chen, Liu, Zhao, and Jia, Guo et
al.(2023)Guo, Yan, Li, and Lin, Heo et al.(2019)Heo, Kim, Yun, Park, Kwak, and
Choi, Li et al.(2021b)Li, Ye, Song, Huang, and Pan, Lin et al.(2022)Lin, Xie,
Wang, Yu, Chang, Liang, and Wang, Liu et al.(2019)Liu, Chen, Liu, Qin, Luo,
and Wang, Romero et al.(2015)Romero, Ballas, Kahou, Chassang, Gatta, and
Bengio, Tian et al.(2020)Tian, Krishnan, and Isola, Yim et al.(2017)Yim, Joo,
Bae, and Kim] and relation-based distillaion [Huang et al.(2022)Huang, You,
Wang, Qian, and Xu, Li et al.(2022)Li, Li, Wang, Zhang, Wu, and Liang, Park et
al.(2019)Park, Kim, Lu, and Cho, Peng et al.(2019)Peng, Jin, Liu, Li, Wu, Liu,
Zhou, and Zhang, Tung and Mori(2019), Yang et al.(2021)Yang, Martínez, Bulat,
and Tzimiropoulos] have also been proposed to improve the knowledge transfer
between teacher and student. Recent advancements, such as the introduction of
a teacher assistant model [Mirzadeh et al.(2020)Mirzadeh, Farajtabar, Li,
Levine, Matsukawa, and Ghasemzadeh], have further refined this process by
staging the knowledge transfer through an intermediate-sized model, which
bridges the gap between the high-capacity teacher and the smaller student.
Knowledge distillation has also been used to enhance the performance of human
pose estimation methods [Zhao et al.(2021)Zhao, Gao, Ge, and Tian, Thai et
al.(2023)Thai, Nham, Tran, Bui, Ninh, and Tran, Yang et al.(2023)Yang, Zeng,
Yuan, and Li, Li et al.(2021c)Li, Ye, Song, Huang, and Pan], including some
works [Thai et al.(2023)Thai, Nham, Tran, Bui, Ninh, and Tran, Zhao et
al.(2021)Zhao, Gao, Ge, and Tian] that use multiple teacher networks.

###  2.3 Cross-Dataset Human Pose Estimation

Cross-dataset adaption has recently gained attention for 3D pose estimation
[Kundu et al.(2020)Kundu, Patravali, RADHAKRISHNAN, et al., Gholami et
al.(2022)Gholami, Wandt, Rhodin, Ward, and Wang, Guan et al.(2021)Guan, Xu,
Wang, Ni, and Yang, Zhang et al.(2020)Zhang, Nie, and Feng]. To achieve this
adaption, some of these methods use generative networks [Nie et al.(2023)Nie,
Fang, Cheng, Lin, and Li], mesh reconstruction [Guan et al.(2021)Guan, Xu,
Wang, Ni, and Yang], or unsupervised learning techniques [Kundu et
al.(2020)Kundu, Patravali, RADHAKRISHNAN, et al.]. In 2D human pose
estimation, cross-dataset keypoint prediction remains scarce. Some methods
available in MMPose [Contributors(2020)], including RTMPose [Jiang et
al.(2023)Jiang, Lu, Zhang, Ma, Han, Lyu, Li, and Chen] and RTMO [Lu et
al.(2023)Lu, Jiang, Li, Li, Chen, and Yang] are trained on a "cocktail" of
multiple datasets called Body7 and Body8. However, they are often only trained
to predict a subset of these points instead of predicting a union of all
available annotations across the datasets.

##  3 Methodology

Figure 1: Illustration of our proposed learning approach, showing how
information from multiple datasets is integrated to create the unified
skeleton. Our student learns from both datasets and one pretrained teacher for
each dataset using a combination of distillation losses and conditional
keypoints loss.

###  3.1 Problem Formulation

Given a collection of keypoint datasets
$\mathcal{D}=\left({D_{i}}\right)_{1\leq i\leq N}$, each dataset ${D}_{i}$ is
associated with a labeled skeleton $\mathcal{K}_{i}$ that defines a set of
keypoints. We aim to construct a superset skeleton $\mathbb{K}$, containing
all unique keypoints from the individual skeletons $\mathcal{K}_{i}$.
Formally, this is represented as:

$\mathbb{K}=\bigcup_{i=1}^{N}\mathcal{K}_{i}$ (1)

In essence, $\mathbb{K}$ includes every keypoint defined across all $N$
datasets, ensuring comprehensive pose representation.

Let $\mathcal{T}=\left({T_{j}}\right)_{1\leq j\leq M}$ denote the set of
teacher networks, where each teacher $T_{j}$ is an expert on a subset of
datasets and hence a subset of the superset skeleton $\mathbb{K}$. The
knowledge of each teacher is encapsulated by its ability to predict keypoints
for its expert datasets. Specifically, for teachers focusing on COCO and MPII
datasets, the overlap and unique keypoints can be expressed as:

$\displaystyle\mathcal{K}_{\mathrm{overlap}}$
$\displaystyle=\mathcal{K}_{\mathrm{COCO}}\cap\mathcal{K}_{\mathrm{MPII}}$ (2)
$\displaystyle\mathcal{K}_{\mathrm{unique}}^{\mathrm{COCO}}$
$\displaystyle=\mathcal{K}_{\mathrm{COCO}}\backslash\mathcal{K}_{\mathrm{overlap}}$
(3) $\displaystyle\mathcal{K}_{\mathrm{unique}}^{\mathrm{MPII}}$
$\displaystyle=\mathcal{K}_{\mathrm{MPII}}\backslash\mathcal{K}_{\mathrm{overlap}}$
(4)

The student network, denoted as $\mathcal{S}$, aims to learn the superset
skeleton $\mathbb{K}$, effectively acquiring the union of knowledge from all
teachers. For the COCO+MPII scenario, this learning goal translates to:

$\mathcal{K}_{\mathrm{student}}=\mathcal{K}_{\mathrm{overlap}}\cup\mathcal{K}_{\mathrm{unique}}^{\mathrm{COCO}}\cup\mathcal{K}_{\mathrm{unique}}^{\mathrm{MPII}}$
(5)

where $\mathcal{K}_{\mathrm{student}}$ is the set of keypoints the student
learns to predict with $\left|\mathcal{K}_{\mathrm{student}}\right|=21$.

###  3.2 Loss Function

We define our loss function as a weighted sum of the task loss, which is a
conditional keypoint loss, and distillation losses for each teacher.

For an image from dataset ${D}_{i}$, the conditional keypoint loss,
$\mathcal{L}_{CK}$, between the student’s prediction $p$ and the ground truth
$g$ is defined as:

$\mathcal{L}_{CK}(p,g;{D}_{i})=\Sigma_{k\in\mathcal{K}_{i}}\left\|p_{k}-g_{k}\right\|_{2}^{2}$
(6)

where $p_{k}$ and $g_{k}$ are the predicted and true positions of keypoint
$k$, respectively.

The distillation loss for an image, irrespective of its originating dataset,
against the prediction of teacher $T_{j}$, is defined as:

$\mathcal{L}_{D}(p,T_{j})=\Sigma_{k\in\mathcal{K}_{T_{j}}}\mathrm{KL}(p_{k},t_{j,k})$
(7)

where $t_{j,k}$ is the prediction of keypoint $k$ by teacher $T_{j}$,
$\mathcal{K}_{T_{j}}$ represents the keypoints that teacher $T_{j}$ can
predict, and $\mathrm{KL}$ is the Kullback-Leibler divergence loss.

The total loss $\mathcal{L}$ for the student network is the weighted sum of
the conditional keypoint loss and the distillation losses, respectively.

$\mathcal{L}(p,g,\\{\left({T_{j}}\right)_{1\leq j\leq
M}\\};{D}_{i})=\alpha\mathcal{L}_{CK}(p,g;{D}_{i})+(1-\alpha)\sum_{j=1}^{M}\beta_{j}\mathcal{L}_{D}(p,T_{j})$
(8)

Here, $\alpha$ is a weighting parameter that balances the importance of
conditional keypoint loss and distillation losses, respectively, and
$\beta_{j}$ assigns a weight for distillation loss for teacher $T_{j}$.

##  4 Experiments

This section details the experiments we conducted to validate the
effectiveness of our proposed approach, including a description of the
baselines, experimental setup, and the results. We also show examples of
extended annotations we created for the COCO and MPII images.

###  4.1 Baselines

In our experiments, we use various RTMPose models [Jiang et al.(2023)Jiang,
Lu, Zhang, Ma, Han, Lyu, Li, and Chen] trained on the COCO Keypoints dataset
and the MPII Human Pose dataset. We compare the networks trained with our
proposed approach with their counterparts trained on individual datasets.
Specifically, we report results with RTMPose-t and RTMPose-s trained on each
dataset separately and then compare the same models when trained on a combined
dataset using our conditional loss and multi-teacher distillation. To distill
the student networks, we use pre-trained RTMPose-m models as teachers.

###  4.2 Experimental Setup

We used the pre-trained weights of RTMPose-m teachers from MMPose for both the
COCO and MPII datasets. We performed various experiments where the
students–RTMPose-s and RTMPose-t–were trained from scratch for 420 epochs. The
training used the AdamW [Loshchilov and Hutter(2017)] optimizer with an
initial learning rate of $4\mathrm{e}^{-3}$. We also used a linear learning
rate scheduler with a start rate of $1\mathrm{e}^{-5}$. Additionally, we
employed a cosine annealing learning rate schedule for the last 210 training
epochs. The training was performed with a batch size 256, and it took 2-3 days
on two V100-32GB GPUs. We followed the same data preprocessing pipeline as
used by the original RTMPose models.

###  4.3 Evaluation Metrics

In our experimental analysis, we utilize two commonly accepted metrics for
evaluating human pose estimation models: Percentage of Correct Keypoints (PCK)
and Average Precision (AP). PCK measures the accuracy of keypoint predictions
by determining the percentage of predicted keypoints that fall within a
specified pixel distance from the true keypoints, effectively capturing the
precision of spatial localization. AP, on the other hand, provides a
comprehensive measure of precision and recall across various thresholds,
assessing the model’s ability to identify keypoints while penalizing false
positives correctly. These metrics are instrumental in benchmarking the
performance of our pose estimation models across different datasets,
highlighting their precision in keypoint detection and overall detection
reliability.

Figure 2: Qualitative Results: Predicted skeletons on different images show
the accuracy of our pose estimation model across different scenarios. Notably,
our model adds five new points (nose, eyes, ears) to the MPII images (top) and
four new points (pelvis, thorax, neck, head top) to the COCO images (bottom).
These points are highlighted in red (${\color[rgb]{1,0,0}\bullet}$) in the
figure.

###  4.4 Results

The results from our experiments, summarized in Table 1, demonstrate the
performance impacts of our unified skeleton learning approach. Although the
model trained on the combined dataset using our unified learning strategy does
not always surpass the performance of models trained on individual datasets in
dataset-specific metrics, it demonstrates significant adaptability and broader
utility.

|  | MPII | COCO |  |
---|---|---|---|---|---
Model | Training | PCK |  PCK0.1 | AP |  AP0.5 |  AP0.75 | AR |  AR0.5 |  AR0.75 | Kpts | Avg
Tiny | MPII | 87.75 | 29.66 | 19.82 | 57.56 | 3.93 | 25.75 | 62.37 | 14.37 | 16 | 53.79
MPII* | 87.75 | 29.66 | 53.23 | 76.61 | 56.43 | 59.12 | 81.40 | 62.46 | 16 | 70.49
COCO* | 77.62 | 17.79 | 67.99 | 88.07 | 75.59 | 73.50 | 92.03 | 80.38 | 17 | 72.81
Unified | 79.75 | 19.58 | 62.03 | 85.66 | 69.39 | 68.12 | 90.05 | 75.13 | 21 | 70.89
Small | MPII | 89.69 | 32.28 | 21.86 | 60.85 | 53.51 | 27.96 | 65.52 | 16.17 | 16 | 55.78
MPII* | 89.69 | 32.28 | 57.48 | 78.88 | 61.57 | 63.33 | 83.61 | 67.50 | 16 | 73.59
COCO* | 81.93 | 22.82 | 72.17 | 89.19 | 79.38 | 77.23 | 92.85 | 84.74 | 17 | 77.05
Unified | 84.16 | 24.66 | 68.63 | 88.30 | 75.73 | 74.14 | 92.10 | 80.71 | 21 | 76.40
Table 1: Comparative performance of the small and tiny variants of the RTMPose
[Jiang et al.(2023)Jiang, Lu, Zhang, Ma, Han, Lyu, Li, and Chen] model on the
MPII and COCO datasets under various training schemes. Models trained on a
single dataset (MPII or COCO) show significant performance drops when
evaluated on all keypoints of the alternate dataset, compared to evaluations
on only the shared 12 keypoints (italicized values in rows marked with *). The
unified training approach (highlighted in gray) evaluates all available
keypoints for the respective dataset and consistently demonstrates enhanced
average performance across both datasets.

Dataset-Specific Performance: Models trained solely on the MPII dataset
exhibit limited generalization when applied to the COCO dataset, showing
suboptimal performance metrics whether evaluated on all 17 COCO keypoints
(rows 1 and 5) or only on the shared set of 12 keypoints (rows 2 and 6).
Conversely, when these models are trained exclusively on the COCO dataset and
assessed on MPII images for the shared keypoints (rows 3 and 7), they perform
significantly better. It is important to note, however, that these evaluations
exclude several critical MPII keypoints such as the pelvis, thorax, neck, and
head, which are not predicted by the models trained only on COCO. In contrast,
both RTMPose-s and RTMPose-t trained using our unified approach show enhanced
cross-dataset generalization. This improvement underscores the robustness and
adaptability of our method, effectively accommodating a more comprehensive
range of keypoints across datasets.

Extended Keypoint Prediction: Our training strategy enables the models to
predict a total of 21 keypoints, surpassing the number available in either the
MPII or COCO datasets individually. This expansion is useful for applications
requiring detailed pose analysis. By enabling the model to identify and
process a broader array of keypoints, we significantly extend its
applicability and performance in complex pose estimation scenarios.

##  5 Ablation Studies

Our ablation studies evaluate the specific contributions of distillation
within our framework. Moreover, we discuss the role of the conditional
keypoint loss in learning a unified skeleton.

Model | Distill | MPII (PCKh) | COCO (AP) | Average
---|---|---|---|---
Tiny | No | 80.21 | 60.69 | 70.45
Yes | 79.75 | 62.03 | 70.89
Small | No | 85.46 | 67.14 | 76.30
Yes | 84.16 | 68.63 | 76.40
Table 2: Performance summary of RTMPose-t and RTMPose-s models with and
without the application of distillation strategies on the MPII and COCO
datasets. The table highlights the effects of distillation on overall
performance metrics (PCKh for MPII and AP for COCO), showcasing the averaged
results across datasets.

Role of Distillation: The inclusion of distillation shows varied effects on
model performance, which is evident in the results summarized in Table 2. For
the "Tiny" model variant, distillation slightly reduces performance on MPII
(PCKh) but improves COCO (AP) performance, resulting in an overall higher
average score. Similarly, the "Small" model variant sees a minor dip in MPII
performance but benefits on COCO, suggesting that distillation generally
favors the COCO dataset’s characteristics. These observations underscore the
balancing act that distillation plays in aligning different dataset biases
within a single model framework.

Model | Distill | AP |  AP0.5 |  AP0.75 |  APM |  APL | AR |  AR0.5 |  AR0.75 |  ARM |  ARL
---|---|---|---|---|---|---|---|---|---|---|---
Tiny | No | 60.69 | 85.21 | 67.71 | 58.06 | 66.13 | 67.00 | 89.77 | 73.82 | 62.67 | 73.18
Yes | 62.03 | 85.66 | 69.39 | 59.14 | 67.73 | 68.12 | 90.05 | 75.13 | 63.69 | 74.37
Small | No | 67.14 | 87.49 | 74.67 | 64.17 | 73.13 | 72.95 | 91.78 | 79.77 | 68.62 | 79.17
Yes | 68.63 | 88.30 | 75.73 | 65.52 | 74.79 | 74.14 | 92.10 | 80.71 | 69.91 | 80.25
Table 3: Detailed COCO dataset performance metrics for RTMPose-t and RTMPose-s
models with and without distillation. This table displays a comprehensive
breakdown of Average Precision (AP) and Average Recall (AR) scores across
various thresholds and object sizes, illustrating the enhancement provided by
distillation.

COCO Dataset Specific Ablations: In Table 3, we look more closely at the
performance metrics for the COCO dataset across various thresholds and object
sizes. Distillation enhances all aspects of performance, particularly in
Average Precision (AP) and Average Recall (AR) at stricter thresholds and
larger object sizes. This detailed breakdown confirms that distillation not
only improves general performance metrics but also enhances the model’s
accuracy and reliability in more challenging scenarios, such as precise
localization and larger-scale object interactions.

Model | Distill | Head | Shoulder | Elbow | Wrist | Hip | Knee | Ankle | Mean |<EMAIL_ADDRESS>
---|---|---|---|---|---|---|---|---|---|---
Tiny | No | 94.51 | 91.24 | 78.61 | 69.38 | 81.34 | 72.98 | 67.69 | 80.21 | 19.51
Yes | 93.83 | 90.86 | 78.71 | 70.14 | 79.82 | 71.77 | 66.82 | 79.75 | 19.58
Small | No | 96.01 | 94.53 | 84.51 | 77.26 | 86.33 | 80.15 | 74.87 | 85.46 | 25.33
Yes | 95.74 | 93.38 | 83.30 | 76.80 | 83.59 | 78.02 | 73.78 | 84.16 | 24.66
Table 4: Ablation study results for RTMPose-t and RTMPose-s on the MPII
dataset, detailing performance across individual body keypoints. This table
compares models with and without distillation, emphasizing the effects of
distillation on precision.

MPII Dataset Specific Ablations: The impact of distillation on the MPII
dataset in Table 4 presents a nuanced picture. While the raw mean scores
slightly decrease with distillation, the detailed keypoint analysis shows
improvements in some keypoints (Wrist, Elbow), suggesting enhanced model
sensitivity to more challenging keypoints. However, primary keypoints like
Head, Shoulder, and Hip slightly decline, indicating a potential trade-off
between generalizing well across datasets and maintaining top performance on
dataset-specific primary keypoints.

Role of Conditional Loss: Our training setup uses images from two distinct
datasets. This leads to situations where certain keypoints are absent in
images from one dataset but present in another. The conditional loss is
designed to adaptively manage this by only computing losses for available
keypoints per image. This approach enables the model to learn effectively from
the available data without penalizing it for missing points. It is crucial for
building a robust model that can predict a comprehensive set of keypoints
across datasets.

Integration of Distillation and Conditional Loss: The combination of
distillation with our conditional loss formulation predominantly enhances
overall model performance. However, it is observed that there is a slight
decline in performance on the MPII dataset, where the number of images is
lower compared to COCO. This could be partly attributed to the imbalance in
dataset sizes and the specific setting of hyperparameters
($\beta_{MPII}=0.25,\beta_{COCO}=0.45,\alpha=0.30$), which influence the
degree of knowledge transfer and prioritization between datasets. Adjusting
these parameters could potentially mitigate performance disparities and
further optimize cross-dataset learning efficiency.

Effectiveness of Extended Predictions In our main experiments, we evaluated
the unified model on the COCO and MPII datasets using only the subsets of
keypoints specific to each dataset, excluding the newly added keypoints from
evaluation. To evaluate the full capabilities of our models, including all
predicted keypoints, we use the Halpe dataset [Li et al.(2020a)Li, Xu, Liu,
Huang, Xu, Wang, Fang, Ma, Chen, and Lu], which provides ground-truth
annotations for 26 body points on COCO validation images, 20 of which align
with our model’s predictions. The missing thorax annotation in the Halpe
dataset can be approximated using the midpoint between the shoulder points.
This expanded evaluation, summarized in Table 5, validates our claim that our
unified model can effectively predict points learned from one dataset on
images in the other dataset.

Model | AP |  AP0.5 |  AP0.75 |  APM |  APL | AR |  AR0.5 |  AR0.75 |  ARM |  ARL
---|---|---|---|---|---|---|---|---|---|---
Tiny | 66.84 | 90.63 | 74.66 | 64.69 | 70.03 | 69.74 | 91.44 | 76.98 | 66.65 | 73.96
Small | 72.75 | 92.69 | 80.81 | 70.35 | 76.21 | 75.20 | 93.30 | 82.17 | 72.11 | 79.48
Table 5: Performance of RTMPose models on Halpe [Li et al.(2020a)Li, Xu, Liu,
Huang, Xu, Wang, Fang, Ma, Chen, and Lu] using GT boxes, demonstrating the
effectiveness of our unified approach in accurately predicting an extended
keypoint set.

##  6 Conclusion

This paper introduced a comprehensive framework for enhancing human pose
estimation by unifying keypoint annotations from multiple datasets using a
multi-teacher distillation approach. Our model’s ability to generalize across
datasets without compromising accuracy represents a significant advancement in
pose estimation technologies. Specifically, the model’s capacity to predict an
extended set of keypoints demonstrates its potential in applications requiring
detailed human pose analysis.

Key contributions of this work include the development of a unified model that
outperforms traditional single-dataset models in cross-dataset settings and
the detailed experimental analysis that validates the effectiveness of our
approach. Future work will explore refining these techniques, potentially
using this method to extend ground-truth annotations for training sets of the
two datasets, and examining active learning strategies to make the annotation
process more precise.

## References

  * [Ahn et al.(2019)Ahn, Hu, Damianou, Lawrence, and Dai] Sungsoo Ahn, Shell Xu Hu, Andreas Damianou, Neil D Lawrence, and Zhenwen Dai.  Variational information distillation for knowledge transfer.  In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , pages 9163–9171, 2019.
  * [Andriluka et al.(2014)Andriluka, Pishchulin, Gehler, and Schiele] Mykhaylo Andriluka, Leonid Pishchulin, Peter Gehler, and Bernt Schiele.  2d human pose estimation: New benchmark and state of the art analysis.  In _IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , June 2014.
  * [Chen et al.(2020)Chen, Mei, Wang, Feng, and Chen] Defang Chen, Jian-Ping Mei, Can Wang, Yan Feng, and Chun Chen.  Online knowledge distillation with diverse peers.  In _AAAI_ , pages 3430–3437, 2020.
  * [Chen et al.(2022)Chen, Mei, Zhang, Wang, Feng, and Chen] Defang Chen, Jian-Ping Mei, Hailin Zhang, Can Wang, Yan Feng, and Chun Chen.  Knowledge distillation with the reused teacher classifier.  In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , pages 11933–11942, 2022.
  * [Chen et al.(2021)Chen, Liu, Zhao, and Jia] P. Chen, S. Liu, H. Zhao, and J. Jia.  Distilling knowledge via knowledge review.  In _2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , pages 5006–5015, Los Alamitos, CA, USA, jun 2021. IEEE Computer Society.  10.1109/CVPR46437.2021.00497.  URL https://doi.ieeecomputersociety.org/10.1109/CVPR46437.2021.00497.
  * [Cheng et al.(2020)Cheng, Xiao, Wang, Shi, Huang, and Zhang] Bowen Cheng, Bin Xiao, Jingdong Wang, Honghui Shi, Thomas S Huang, and Lei Zhang.  Higherhrnet: Scale-aware representation learning for bottom-up human pose estimation.  In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , pages 5386–5395, 2020.
  * [Contributors(2020)] MMPose Contributors.  Openmmlab pose estimation toolbox and benchmark.  https://github.com/open-mmlab/mmpose, 2020.
  * [Dang et al.(2019)Dang, Yin, Wang, and Zheng] Qi Dang, Jianqin Yin, Bin Wang, and Wenqing Zheng.  Deep learning based 2d human pose estimation: A survey.  _Tsinghua Science and Technology_ , 24(6):663–676, 2019.
  * [Gholami et al.(2022)Gholami, Wandt, Rhodin, Ward, and Wang] Mohsen Gholami, Bastian Wandt, Helge Rhodin, Rabab Ward, and Z Jane Wang.  Adaptpose: Cross-dataset adaptation for 3d human pose estimation by learnable motion generation.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 13075–13085, 2022.
  * [Guan et al.(2021)Guan, Xu, Wang, Ni, and Yang] Shanyan Guan, Jingwei Xu, Yunbo Wang, Bingbing Ni, and Xiaokang Yang.  Bilevel online adaptation for out-of-domain human mesh reconstruction.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 10472–10481, 2021.
  * [Guo et al.(2023)Guo, Yan, Li, and Lin] Ziyao Guo, Haonan Yan, Hui Li, and Xiaodong Lin.  Class attention transfer based knowledge distillation.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 11868–11877, 2023.
  * [Heo et al.(2019)Heo, Kim, Yun, Park, Kwak, and Choi] Byeongho Heo, Jeesoo Kim, Sangdoo Yun, Hyojin Park, Nojun Kwak, and Jin Young Choi.  A comprehensive overhaul of feature distillation.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , pages 1921–1930, 2019.
  * [Hinton et al.(2015)Hinton, Vinyals, and Dean] Geoffrey Hinton, Oriol Vinyals, and Jeff Dean.  Distilling the knowledge in a neural network.  _arXiv preprint arXiv:1503.02531_ , 2015.
  * [Huang et al.(2022)Huang, You, Wang, Qian, and Xu] Tao Huang, Shan You, Fei Wang, Chen Qian, and Chang Xu.  Knowledge distillation from a stronger teacher.  _Advances in Neural Information Processing Systems_ , 35:33716–33727, 2022.
  * [Jiang et al.(2023)Jiang, Lu, Zhang, Ma, Han, Lyu, Li, and Chen] Tao Jiang, Peng Lu, Li Zhang, Ningsheng Ma, Rui Han, Chengqi Lyu, Yining Li, and Kai Chen.  Rtmpose: Real-time multi-person pose estimation based on mmpose.  _arXiv preprint arXiv:2303.07399_ , 2023.
  * [Jin et al.(2022)Jin, Xu, Wang, Xiao, Guo, Nie, and Zhao] Lei Jin, Chenyang Xu, Xiaojuan Wang, Yabo Xiao, Yandong Guo, Xuecheng Nie, and Jian Zhao.  Single-stage is enough: Multi-person absolute 3d pose estimation.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 13086–13095, 2022.
  * [Jin et al.(2023)Jin, Wang, and Lin] Ying Jin, Jiaqi Wang, and Dahua Lin.  Multi-level logit distillation.  _2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , pages 24276–24285, 2023.  URL https://api.semanticscholar.org/CorpusID:260933721.
  * [Khirodkar et al.(2021)Khirodkar, Chari, Agrawal, and Tyagi] Rawal Khirodkar, Visesh Chari, Amit Agrawal, and Ambrish Tyagi.  Multi-instance pose networks: Rethinking top-down pose estimation.  In _Proceedings of the IEEE/CVF International conference on computer vision_ , pages 3122–3131, 2021.
  * [Kullback and Leibler(1951)] Solomon Kullback and Richard A Leibler.  On information and sufficiency.  _The annals of mathematical statistics_ , 22(1):79–86, 1951.
  * [Kundu et al.(2020)Kundu, Patravali, RADHAKRISHNAN, et al.] Jogendra Nath Kundu, Jay Patravali, Venkatesh Babu RADHAKRISHNAN, et al.  Unsupervised cross-dataset adaptation via probabilistic amodal 3d human pose completion.  In _Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision_ , pages 469–478, 2020.
  * [Li et al.(2022)Li, Li, Wang, Zhang, Wu, and Liang] Gang Li, Xiang Li, Yujie Wang, Shanshan Zhang, Yichao Wu, and Ding Liang.  Knowledge distillation for object detection via rank mimicking and prediction-guided feature imitation.  In _Proceedings of the AAAI conference on artificial intelligence_ , volume 36, pages 1306–1313, 2022.
  * [Li et al.(2021a)Li, Zhang, Wang, Yang, Yang, Xia, and Zhou] Yanjie Li, Shoukui Zhang, Zhicheng Wang, Sen Yang, Wankou Yang, Shu-Tao Xia, and Erjin Zhou.  Tokenpose: Learning keypoint tokens for human pose estimation.  In _Proceedings of the IEEE/CVF International conference on computer vision_ , pages 11313–11322, 2021a.
  * [Li et al.(2020a)Li, Xu, Liu, Huang, Xu, Wang, Fang, Ma, Chen, and Lu] Yong-Lu Li, Liang Xu, Xinpeng Liu, Xijie Huang, Yue Xu, Shiyi Wang, Hao-Shu Fang, Ze Ma, Mingyang Chen, and Cewu Lu.  Pastanet: Toward human activity knowledge engine.  In _CVPR_ , 2020a.
  * [Li et al.(2020b)Li, Huang, Chen, Luo, Cai, and Pan] Zheng Li, Ying Huang, Defang Chen, Tianren Luo, Ning Cai, and Zhigeng Pan.  Online knowledge distillation via multi-branch diversity enhancement.  In _Proceedings of the Asian Conference on Computer Vision (ACCV)_ , November 2020b.
  * [Li et al.(2021b)Li, Ye, Song, Huang, and Pan] Zheng Li, Jingwen Ye, Mingli Song, Ying Huang, and Zhigeng Pan.  Online knowledge distillation for efficient pose estimation.  In _Proceedings of the IEEE/CVF international conference on computer vision_ , pages 11740–11750, 2021b.
  * [Li et al.(2021c)Li, Ye, Song, Huang, and Pan] Zheng Li, Jingwen Ye, Mingli Song, Ying Huang, and Zhigeng Pan.  Online knowledge distillation for efficient pose estimation.  In _Proceedings of the IEEE/CVF international conference on computer vision_ , pages 11740–11750, 2021c.
  * [Lin et al.(2022)Lin, Xie, Wang, Yu, Chang, Liang, and Wang] Sihao Lin, Hongwei Xie, Bing Wang, Kaicheng Yu, Xiaojun Chang, Xiaodan Liang, and Gang Wang.  Knowledge distillation via the target-aware transformer.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 10915–10924, 2022.
  * [Lin et al.(2014)Lin, Maire, Belongie, Hays, Perona, Ramanan, Dollár, and Zitnick] Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick.  Microsoft coco: Common objects in context.  In _Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13_ , pages 740–755. Springer, 2014.
  * [Liu et al.(2022)Liu, Liu, Zhang, Sangaiah, Yang, and Li] Hai Liu, Tingting Liu, Zhaoli Zhang, Arun Kumar Sangaiah, Bing Yang, and Youfu Li.  Arhpe: Asymmetric relation-aware representation learning for head pose estimation in industrial human–computer interaction.  _IEEE Transactions on Industrial Informatics_ , 18(10):7107–7117, 2022.
  * [Liu et al.(2019)Liu, Chen, Liu, Qin, Luo, and Wang] Yifan Liu, Ke Chen, Chris Liu, Zengchang Qin, Zhenbo Luo, and Jingdong Wang.  Structured knowledge distillation for semantic segmentation.  In _2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , pages 2599–2608, 2019.  10.1109/CVPR.2019.00271.
  * [Loshchilov and Hutter(2017)] Ilya Loshchilov and Frank Hutter.  Decoupled weight decay regularization.  _arXiv preprint arXiv:1711.05101_ , 2017.
  * [Lu et al.(2023)Lu, Jiang, Li, Li, Chen, and Yang] Peng Lu, Tao Jiang, Yining Li, Xiangtai Li, Kai Chen, and Wenming Yang.  Rtmo: Towards high-performance one-stage real-time multi-person pose estimation.  _arXiv preprint arXiv:2312.07526_ , 2023.
  * [Marchand et al.(2015)Marchand, Uchiyama, and Spindler] Eric Marchand, Hideaki Uchiyama, and Fabien Spindler.  Pose estimation for augmented reality: a hands-on survey.  _IEEE transactions on visualization and computer graphics_ , 22(12):2633–2651, 2015.
  * [Mirzadeh et al.(2019)Mirzadeh, Farajtabar, Li, Levine, Matsukawa, and Ghasemzadeh] Seyed Iman Mirzadeh, Mehrdad Farajtabar, Ang Li, Nir Levine, Akihiro Matsukawa, and Hassan Ghasemzadeh.  Improved knowledge distillation via teacher assistant.  In _AAAI Conference on Artificial Intelligence_ , 2019.  URL https://api.semanticscholar.org/CorpusID:212908749.
  * [Mirzadeh et al.(2020)Mirzadeh, Farajtabar, Li, Levine, Matsukawa, and Ghasemzadeh] Seyed Iman Mirzadeh, Mehrdad Farajtabar, Ang Li, Nir Levine, Akihiro Matsukawa, and Hassan Ghasemzadeh.  Improved knowledge distillation via teacher assistant.  In _Proceedings of the AAAI conference on artificial intelligence_ , volume 34, pages 5191–5198, 2020.
  * [Nie et al.(2019)Nie, Feng, Zhang, and Yan] Xuecheng Nie, Jiashi Feng, Jianfeng Zhang, and Shuicheng Yan.  Single-stage multi-person pose machines.  In _Proceedings of the IEEE/CVF international conference on computer vision_ , pages 6951–6960, 2019.
  * [Nie et al.(2023)Nie, Fang, Cheng, Lin, and Li] Yuxiang Nie, Chaowei Fang, Lechao Cheng, Liang Lin, and Guanbin Li.  Adapting object size variance and class imbalance for semi-supervised object detection.  In _AAAI Conference on Artificial Intelligence_ , 2023.  URL https://api.semanticscholar.org/CorpusID:259722394.
  * [Osokin(2018)] Daniil Osokin.  Real-time 2d multi-person pose estimation on cpu: Lightweight openpose.  _arXiv preprint arXiv:1811.12004_ , 2018.
  * [Papandreou et al.(2018)Papandreou, Zhu, Chen, Gidaris, Tompson, and Murphy] George Papandreou, Tyler Zhu, Liang-Chieh Chen, Spyros Gidaris, Jonathan Tompson, and Kevin Murphy.  Personlab: Person pose estimation and instance segmentation with a bottom-up, part-based, geometric embedding model.  In _Proceedings of the European conference on computer vision (ECCV)_ , pages 269–286, 2018.
  * [Park et al.(2019)Park, Kim, Lu, and Cho] Wonpyo Park, Dongju Kim, Yan Lu, and Minsu Cho.  Relational knowledge distillation.  In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , pages 3967–3976, 2019.
  * [Peng et al.(2019)Peng, Jin, Liu, Li, Wu, Liu, Zhou, and Zhang] Baoyun Peng, Xiao Jin, Jiaheng Liu, Dongsheng Li, Yichao Wu, Yu Liu, Shunfeng Zhou, and Zhaoning Zhang.  Correlation congruence for knowledge distillation.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , pages 5007–5016, 2019.
  * [Romero et al.(2015)Romero, Ballas, Kahou, Chassang, Gatta, and Bengio] Adriana Romero, Nicolas Ballas, Samira Ebrahimi Kahou, Antoine Chassang, Carlo Gatta, and Yoshua Bengio.  Fitnets: Hints for thin deep nets, 2015.
  * [Shi et al.(2021)Shi, Wei, Yu, Tan, Ren, and Pu] Dahu Shi, Xing Wei, Xiaodong Yu, Wenming Tan, Ye Ren, and Shiliang Pu.  Inspose: instance-aware networks for single-stage multi-person pose estimation.  In _Proceedings of the 29th ACM International Conference on Multimedia_ , pages 3079–3087, 2021.
  * [Shi et al.(2022)Shi, Wei, Li, Ren, and Tan] Dahu Shi, Xing Wei, Liangqi Li, Ye Ren, and Wenming Tan.  End-to-end multi-person pose estimation with transformers.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 11069–11078, 2022.
  * [Song et al.(2021)Song, Yu, Yuan, and Liu] Liangchen Song, Gang Yu, Junsong Yuan, and Zicheng Liu.  Human pose estimation and its application to action recognition: A survey.  _Journal of Visual Communication and Image Representation_ , 76:103055, 2021.
  * [Sun et al.(2024)Sun, Gu, Wang, Yang, and Yao] Pengzhan Sun, Kerui Gu, Yunsong Wang, Linlin Yang, and Angela Yao.  Rethinking visibility in human pose estimation: Occluded pose reasoning via transformers.  In _Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision_ , pages 5903–5912, 2024.
  * [Thai et al.(2023)Thai, Nham, Tran, Bui, Ninh, and Tran] Chien Thai, Ninh Nham, Viet Tran, Minh Bui, Huong Ninh, and Hai Tran.  Multiple teacher knowledge distillation for head pose estimation without keypoints.  _SN Computer Science_ , 4(6):758, 2023.
  * [Tian et al.(2020)Tian, Krishnan, and Isola] Yonglong Tian, Dilip Krishnan, and Phillip Isola.  Contrastive representation distillation.  In _International Conference on Learning Representations_ , 2020.
  * [Tung and Mori(2019)] Frederick Tung and Greg Mori.  Similarity-preserving knowledge distillation.  In _Proceedings of the IEEE/CVF international conference on computer vision_ , pages 1365–1374, 2019.
  * [Wang et al.(2022)Wang, Li, Cai, Chen, and Han] Yihan Wang, Muyang Li, Han Cai, Wei-Ming Chen, and Song Han.  Lite pose: Efficient architecture design for 2d human pose estimation.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 13126–13136, 2022.
  * [Xu et al.(2022)Xu, Zhang, Zhang, and Tao] Yufei Xu, Jing Zhang, Qiming Zhang, and Dacheng Tao.  Vitpose: Simple vision transformer baselines for human pose estimation.  _Advances in Neural Information Processing Systems_ , 35:38571–38584, 2022.
  * [Yang et al.(2021)Yang, Martínez, Bulat, and Tzimiropoulos] Jing Yang, Brais Martínez, Adrian Bulat, and Georgios Tzimiropoulos.  Knowledge distillation via softmax regression representation learning.  In _International Conference on Learning Representations_ , 2021.  URL https://api.semanticscholar.org/CorpusID:235613564.
  * [Yang et al.(2023)Yang, Zeng, Yuan, and Li] Zhendong Yang, Ailing Zeng, Chun Yuan, and Yu Li.  Effective whole-body pose estimation with two-stages distillation.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , pages 4210–4220, 2023.
  * [Yim et al.(2017)Yim, Joo, Bae, and Kim] Junho Yim, Donggyu Joo, Ji-Hoon Bae, and Junmo Kim.  A gift from knowledge distillation: Fast optimization, network minimization and transfer learning.  _2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , pages 7130–7138, 2017.  URL https://api.semanticscholar.org/CorpusID:206596723.
  * [Zhang et al.(2020)Zhang, Nie, and Feng] Jianfeng Zhang, Xuecheng Nie, and Jiashi Feng.  Inference stage optimization for cross-scenario 3d human pose estimation.  _Advances in neural information processing systems_ , 33:2408–2419, 2020.
  * [Zhang et al.(2023)Zhang, Shen, Liu, Liu, Bendersky, Najork, and Zhang] Rongzhi Zhang, Jiaming Shen, Tianqi Liu, Jialu Liu, Michael Bendersky, Marc Najork, and Chao Zhang.  Do not blindly imitate the teacher: Using perturbed loss for knowledge distillation, 2023.
  * [Zhao et al.(2022)Zhao, Cui, Song, Qiu, and Liang] Borui Zhao, Quan Cui, Renjie Song, Yiyu Qiu, and Jiajun Liang.  Decoupled knowledge distillation.  In _Proceedings of the IEEE/CVF Conference on computer vision and pattern recognition_ , pages 11953–11962, 2022.
  * [Zhao et al.(2021)Zhao, Gao, Ge, and Tian] Zhong-Qiu Zhao, Yao Gao, Yuchen Ge, and Weidong Tian.  Orderly dual-teacher knowledge distillation for lightweight human pose estimation.  _arXiv preprint arXiv:2104.10414_ , 2021.
  * [Zheng et al.(2021)Zheng, Zhu, Mendieta, Yang, Chen, and Ding] Ce Zheng, Sijie Zhu, Matias Mendieta, Taojiannan Yang, Chen Chen, and Zhengming Ding.  3d human pose estimation with spatial and temporal transformers.  In _Proceedings of the IEEE/CVF international conference on computer vision_ , pages 11656–11665, 2021.
  * [Zheng et al.(2023)Zheng, Wu, Chen, Yang, Zhu, Shen, Kehtarnavaz, and Shah] Ce Zheng, Wenhan Wu, Chen Chen, Taojiannan Yang, Sijie Zhu, Ju Shen, Nasser Kehtarnavaz, and Mubarak Shah.  Deep learning-based human pose estimation: A survey.  _ACM Computing Surveys_ , 56(1):1–37, 2023.