objects, alternating between terminals several times. The repetitive nature of this task is incompatible with HQ-Learning [@Wiering1997]. Options with standard initiation sets are not able to solve this task, as the top-level policy is memoryless [@Sutton1999] and cannot remember from which terminal the robot arrives at the root, and whether that terminal was full or empty. Because the terminals are a dozen feet away from the root, almost a hundred primitive actions have to be executed to complete any root/terminal journey. Without options, this represents a time horizon much larger than usually handled by recurrent neural networks [@Bakker2001] or finite history windows [@Lin1993].

[OOIs]{}allow each option to be selected conditionally on the previously executed one (see Section \[sec:theory\]), which is much simpler than combining options and recurrent neural networks [@Sridharan2010]. The ability of [OOIs]{}to solve complex POMDPs builds on the time abstraction capabilities and expressiveness of options. Section \[sec:khepera\] shows that [OOIs]{}allow a policy for our robotic task to be learned to expert level. Additional experiments demonstrate that both the top-level and option policies can be learned by the agent (see Section \[sec:duplicatedinput\]), and that [OOIs]{}lead to substantial gains over standard initiation sets even if the option set is reduced 