142 sequences (we excluded the 9 sequences overlapping with the validation set of [KITTI MOTS]{}), for a total of $\sim 44$k images, captured with a professional rig including stereo cameras, LIDAR, GPS and IMU. Next we describe our pipeline which, relying only on monocular images and GPS data is able to generate accurate MOTS annotations for the dataset.

Generation of Instance Segmentation Results {#sec:instance-seg}
-------------------------------------------

We begin with segmenting object instances in each frame per video and consider a predefined set ${\ensuremath{\mathcal{Y}}}$ of $37$ object classes that belong to the Mapillary Vistas dataset [@MVD2017]. To extract the instance segments, we run the Seamless Scene Segmentation method [@Por+19_cvpr], augmented with a ResNeXt-101-32$\times$8d [@Xie2016] backbone and trained on Mapillary Vistas. By doing so, we obtain a set of object segments ${\ensuremath{\mathcal{S}}}$ per video sequence. For each segment $s\in{\ensuremath{\mathcal{S}}}$, we denote by $t_s\geq 0$ the frame where the segment was extracted, by $y_s\in{\ensuremath{\mathcal{Y}}}$ the class it belongs to and by $\phi_s:\mathbb R^2\to\{0,1\}$ a pixel indicator function representing the segmentation mask, $\phi_s(i,j)=1$ if and only if pixel $(i,j)$ belongs to the segment. For convenience, we also introduce the notation ${\ensuremath{\mathcal{S}}}_t$ to denote the set of all segments extracted from frame $t$ of a given 