Automated MOTS training data generation – instance detection, segmentation and actual tracking – is possible at scale and with very high fidelity, approximating the quality of previous state-of-the-art results [@Voigtlaender_2019_CVPR] without even directly training for it.

-   Deep-learning based, supervised optical flow [@Yin_2019_CVPR] can be effectively trained on pixel-based correspondences obtained from Structure-from-Motion and furthermore used for tracklet extraction.

-   Direct exploitation of instance segmentation masks through our proposed mask-pooling layer yields to significant improvements of sMOTSA scores

-   The combination of our general-purpose MOTS data generation pipeline with our proposed [MOTSNet]{}improves by up to 1.2% for cars and 7.5% for pedestrians over the previously best method on [KITTI MOTS]{}in absolute terms, while using $13.7\%$ fewer parameters. Similarly we can gain $4.1\%$ over the previously best-performing work on the MOTSChallenge dataset.

We provide extensive ablation studies on the [KITTI MOTS]{}, MOTSChallenge, and Berkeley Deep Drive BDD100k [@Yu2018] datasets and obtain results consistently improving with a large margin over the prior state-of-the-art [@Voigtlaender_2019_CVPR].

Related Works
=============

Since the MOTS task was only very recently introduced in [@Voigtlaender_2019_CVPR], directly related works are scarce. Instead, the bulk of related works in a broader sense comes from the MOT (multi-object tracking) and VOS (video 