Title: CENTROID-BASED JOINT REPRESENTATION FOR HUMAN POSE ESTIMATION AND INSTANCE SEGMENTATION. Abstract: Joint pose estimation and instance segmentation combines keypoint heatmaps
with segmentation masks for multi-person pose and instance-level segmenta-
tion. Unlike easy cases with explicit heatmap activation, hard cases with im-
plicit heatmap due to multi-person entanglement, overlap, and occlusions requires
joint representation with a segmentation mask in end-to-end training. This pa-
per presents a new centroid-based joint representation method called CENTER-
FOCUS. It follows a bottom-up paradigm to generate Strong Keypoint Feature
Maps for both soft and hard keypoints and improve keypoints detection accuracy
as well as the confidence score by introducing KeyCentroids and a Body Heat
Map. CENTERFOCUS then uses the high-resolution representation of keypoint as
a center of attraction for the pixels in the embedding space to generate MaskCen-
troid to cluster the pixels to a particular human instance to whom it belongs, even
if 70% of the body is occluded. Finally, we propose a new PoseSeg algorithm
that collects the feature representation of a 2D human pose and segmentation for
the joint structure of the pose and instance segmentation. We then experimentally
demonstrate the effectiveness and generalization ability of our system on chal-
lenging scenarios such as occlusions, entangled limbs, and overlapping people.
The experimental results show the effectiveness of CENTERFOCUS outperforms
representative models on the challenging MS COCO and OCHuman benchmarks
in terms of both accuracy and runtime performance, Ablation experiments analyze
the impact of each component of the system. The code will be released publicly.