good memory-based policy for our motivating example, where the agent has to bring objects from two terminals to the root, can be described as “go to the green terminal, then go to the root, then go back to the green terminal if it was full, to the blue terminal otherwise”, and symmetrically so for the blue terminal. This sequence of sub-tasks, that contains a condition, is easily translated to a set of options. Two options, $\omega_{GF}$ and $\omega_{GE}$, sharing a single policy, go from the green terminal to the root (using low-level motor actions). $\omega_{GF}$ is executed when the terminal is full, $\omega_{GE}$ when it is empty. At the root, the option that goes back to the green terminal can only follow $\omega_{GF}$, not $\omega_{GE}$. When the green terminal is empty, going back to it is therefore forbidden, which forces the agent to switch to the blue terminal when the green one is empty.

We now formally define our main contribution, Option-Observation Initiation Sets ([OOIs]{}), that allow to describe which options may follow which ones. We define the initiation set $I_\omega$ of option $\omega$ so that the set [$\mathcal{O}_t$]{} of options available at time $t$ depends on the observation $x_t$ 