in the RSH (see [@Por+19_cvpr]), a segment is positive if its bounding box has high IoU with a ground truth segment’s bounding box. The functions $\mu_B(s)$ and $\bar{\mu}_B(s)$ map $s$ to the sets of its “matching” and “non-matching” segments in ${\ensuremath{\mathcal{S}}}_B$, respectively. These are defined as: $$\begin{aligned}
  \mu_B(s) &= \left\{\hat{s}\in{\ensuremath{\mathcal{S}}}_B \mid y_s = y_{\hat{s}} \land M(s, \hat{s})\right\}\,,\\
  \bar{\mu}_B(s) &= \left\{\hat{s}\in{\ensuremath{\mathcal{S}}}_B \mid y_s = y_{\hat{s}} \land \neg M(s, \hat{s})\right\}\,,\end{aligned}$$ where $M(s, \hat{s})$ is true iff $s$ and $\hat{s}$ belong to the same tracklet in the sequence. Note that we are restricting the loss to only compare embedding vectors of segments belonging to the same class, effectively letting the network learn a set of class-specific embedding spaces.

In order to overcome the issue due to occlusions mentioned in Sec. \[sec:tracklets\], when training on [KITTI Synth]{}we also apply the following heuristic. Ground truth segments are only considered in the tracking loss if they are associated with a tracklet that appears in more than half of the frames in the current batch. This ensures that, when an occlusion in the scene causes a tracklet to split, the two pieces are never mistakenly treated by the network as two different objects.

Inference {#sec:inference}
---------

During 