We compare them to a *fixed* agent, using an expert top-level policy that interprets the options as follows: $\omega_{R1..R4}$ go to the root from a full/empty green/blue terminal (and are selected accordingly at the terminals depending on the QR-code displayed on them), while $\omega_{G1..G4,B1..B4}$ go to the green/blue terminal from the root when the previous terminal was full/empty and green/blue. At the root, [OOIs]{}ensure that only one option amongst *go to green after a full green*, *go to green after an empty blue*, *go to blue after a full blue* and *go to blue after an empty green* is selected by the top-level policy: the one that corresponds to what color the last terminal was and whether it was full or empty. The agent goes to a terminal until it is empty, then switches to the other terminal, leading to an average reward of 10.

When the top-level policy is learned, [OOIs]{}allow the task to be solved, as shown in Figure \[fig:kresults\], while standard initiation sets do not allow the task to be learned. Because experiments on a robot are slow, we developed a small simulator for this task, and used it to produce Figure \[fig:kresults\] after having successfully asserted 