At a time step $t$, the agent observes the state $s_t \in \mathcal{S}$ that contains sufficient information characterizing the current bbox prediction, its history, its image patch and contextual features, and proceeds to decide on a specific action $a_t \in \mathcal{A}$. When terminated, the finite sequence of actions is collectively evaluated by reward $r:=r_t$. several such visual examples of the action sequences are shown in Fig \[fig\_67\]. Note that the sequence lengths and the first action in these examples of Fig \[fig\_67\] happen to be the same, while in general they could be very different. The box-plots in Fig. \[fig\_numOfActions\] summarize the distribution of action sequence length over three different benchmark datasets. The medium action sequence length is around 10, that is, 10 consecutive actions from an initial bbox prediction to reach the final decision. Training the localization agent involves maximizing the expected discounted sum of rewards, and in practice we consider the REINFORCE [@Wil:mlj01] method. In what follows, we will delineate the concrete details of the involved MDP.

**State:** At time $t$, our localization agent resides at an information state $s_t \in \mathcal{S}$ that encodes the current bbox prediction and the corresponding image patch features, as well as the 