is at the terminal. The agent therefore has to remember information acquired at terminals in order to properly choose, at the root, to which terminal it will go.

![Cumulative reward per episode obtained on our object gathering task, with [OOIs]{}, without [OOIs]{}, and using an LSTM over options. [OOIs]{}learns an expert-level policy much quicker than an LSTM over options. The LSTM curve flattens-out (with high variance) after about 30K episodes.[]{data-label="fig:kresults"}](terminals-results/plot)

The agent has access to 12 memoryless options that go to red ($\omega_{R1..R4}$), green ($\omega_{G1..G4}$) or blue objects ($\omega_{B1..B4}$), and terminate when the agent is close enough to them to read a QR-code displayed on them. The initiation set of $\omega_{R1,R2}$ is $\omega_{G1..G4}$, of $\omega_{R3,R4}$ is $\omega_{B1..B4}$, and of $\omega_{G_i,B_i}$ is $\omega_{R_i} ~ \forall i = 1..4$. This description of the options and their [OOIs]{}is purposefully uninformative, and illustrates how little information the agent has about the task. The option set used in this experiment is also richer than the simple example of Section \[sec:theory\], so that the solution of the problem, not going back to an empty terminal, is not encoded in [OOIs]{}but must be learned by the agent.

Agents with and without [OOIs]{}learn top-level policies over these options. 