set ${\ensuremath{\mathcal{V}}}_{t-1}$ of ${\ensuremath{\mathcal{G}}}_{t-1}$ and ${\ensuremath{\mathcal{S}}}_t$, ${\ensuremath{\mathcal{V}}}_{t}={\ensuremath{\mathcal{V}}}_{t-1}\cup {\ensuremath{\mathcal{S}}}_t$. The edge set is computed by solving a linear assignment problem between ${\ensuremath{\mathcal{S}}}_{t-1}$ and ${\ensuremath{\mathcal{S}}}_t$, where the payoff function is constructed by comparing segments in ${\ensuremath{\mathcal{S}}}_t$ against segments in ${\ensuremath{\mathcal{S}}}_{t-1}$ warped to frame $t$ via the mapping $\overleftarrow f_{t}$. We design the payoff function $\pi(\hat s, s)$ between a segment $\hat s\in{\ensuremath{\mathcal{S}}}_{t-1}$ in frame $t-1$ and a segment $s\in{\ensuremath{\mathcal{S}}}_t$ in frame $t$ as follows: $$\pi(\hat s, s)=\text{IoU}(\phi_{s},\phi_{\hat s}\circ\overleftarrow f_{t})+\eta(\hat s, s)\,,$$ where $\text{IoU}(\cdot,\cdot)$ computes the Intersection-over-Union of the two masks given as input, and $\eta(\hat s, s)\in\{-\infty,0\}$ is a characteristic function imposing constraints regarding valid mappings. Specifically, $\eta(\hat s, s)$ returns $0$ if and only if the two segments belong to the same category, $y_{\hat s}=y_s$, and none of the following conditions on $s$ hold:

-   $b_1(s)-b_2(s)<\tau_0$

-   $b_1(s)<\tau_1$, and

-   $b_1(s)/r(s)<\tau_2$,

where $b_1(s)$ and $b_2(s)$ denote the largest and second-largest areas of intersection with $s$ obtained by segments warped from frame $t-1$ and having class $y_s$, and $r(s)$ denotes the area of $s$ that is not covered by segments of class $y_s$ warped from frame $t-1$. We solve the linear assignment problem by maximizing the total payoff under a relaxed assignment 