\sum_{t=1}^{T} V_t, \\
    V_t = t * ( P_{detection}(&S_t)  + P_{compatible}(S_t, S_{t+1}) ),
\end{split}\label{eqn:reward}\end{aligned}$$

where $V_t$ is the sum of the detection score $P_{detection}$ and the compatible score $P_{compatibility}$ at action $t$, and weighted by the time $t$ which encourages the searching path to be long. $P_{detection}$ and $P_{compatibility}$ ensure that the selected boxes are reliable and compatible. $P_{detection}(t)$ is the detection probability of $S_t$, and $P_{compatibility} = dist(S_t, S_{t+1})$, where $dist(\cdot)$ is the IoU between two boxes. An optimal policy $\pi$ outputs an optimal action sequence, which is defined as a path with maximum reward from the root to a leaf. We seek a path that maximizes the reward function in Eqn. (\[eqn:reward\]).

After the root was created, Monte Carlo simulations selected actions and followed the sequence $S$ to create a new node. After a number of simulations, the tree was well populated, and the optimal path was selected. Each depth of the tree is the time step, with root at $t=1$ and leaves at $t = T$.

![image](1_3_det_results_small.pdf){height="7cm"}

### Next Action 

At time $t+1$, we need to select a proposal $S_{t+1}$ close enough to $S_t$ while keeping the category same as $S_{t+1}^*$. Since there may be a few 