continue the track (fig. \[figure:siamese\_track\_rcnn\], [magenta]{}). When the person is not present in this search region the track branch should produce a low score $\hat{v}$ to terminate the track ([green]{}). (ii) The detection branch is responsible for localizing all people that appear at time $t+\delta$, independently of their status at time $t$ ([yellow]{}). (iii) The re-id branch is responsible for generating discriminative embedding features that can help decide whether a newly localized person belongs to a previously terminated track and should be reinstated or requires a new track to be initiated ([red]{}).

With this high level structure in mind, we design our network by starting with the Faster-RCNN architecture [@ren2015faster; @he2017mask] and enriching it with two additional branches: track and re-id. In the following sections we present the details of how we structure and train each branch.

Detection branch
----------------

Our detection branch is a common Faster-RCNN architecture [@ren2015faster; @he2017mask] that consists of a Region Proposal Network (RPN), followed by classification and regression of the generated proposals. The classification loss is a multi-class cross entropy loss and the regression loss is a smooth $\ell_1$-loss [@girshick2015fast]. Differently from the original [@ren2015faster; @he2017mask], our Siamese Track-RCNN takes pairs Faster-RCNN of frames as 