a straightforward attention mechanism: pooling under the segmentation mask. Formally, given the pixel indicator function $\phi_s(i,j)$ for a segment $s$ (see Sec. \[sec:instance-seg\]), and the corresponding input $N$-dimensional ROI-pooled feature map $\mathrm{X}_s:\mathbb{R}^2\to\mathbb{R}^N$, mask-pooling computes a feature vector $x_s\in\mathbb{R}^N$ as: $$x_s = \frac{\sum_i\sum_j\phi_s(i, j)\mathrm{X}_s(i, j)}{\sum_i\sum_j\phi_s(i, j)}\,.$$ During training we pool under the ground truth segmentation masks, while at inference time we switch to the masks predicted by the RSH.

Training Losses {#sec:training}
---------------

[MOTSNet]{}is trained by minimizing the following loss function: $$L = L_\text{TH} + \lambda (L_\text{RPH} + L_\text{RSH})\,,$$ where $L_\text{RPH}$ and $L_\text{RSH}$ represent the Region Proposal Head and Region Segmentation Head losses as defined in [@Por+19_cvpr], and $\lambda$ is a weighting parameter.

$L_\text{TH}$ is the loss component associated with the Tracking Head, the following batch hard triplet loss [@hermans2017defense]: $$\begin{gathered}
  L_\text{TH} = \frac{1}{|{\ensuremath{\mathcal{S}}}_B|} \sum_{s\in{\ensuremath{\mathcal{S}}}_B}
    \max(\max_{\hat{s} \in \mu_B(s)} \|a_s^{y_s} - a_{\hat{s}}^{y_{\hat{s}}}\| \\
    - \min_{\hat{s} \in \bar{\mu}_B(s)} \|a_s^{y_s} - a_{\hat{s}}^{y_{\hat{s}}}\| + \beta,0)\,,\end{gathered}$$ where ${\ensuremath{\mathcal{S}}}_B$ is the set of predicted, positive segments in the current batch, $a_s^{y_s}$ is the class-specific embedding vector predicted by the TH for a certain segment $s$ and ground truth class $y_s$, and $\beta$ is a margin parameter. The definition of a “positive” segment follows the same logic as 