in Figure \[fig:fsc\] cannot be mapped to a set of options and a policy over options (without [OOIs]{}). This proves that options without [OOIs]{}are strictly less expressive than FSCs.

Options without [OOIs]{}are not as expressive as Finite State Controllers.

Figure \[fig:fsc\] shows a Finite State Controller that emits a sequence of alternating A’s and B’s, based on a constant uninformative observation $x_{\emptyset}$. This task requires memory because the observation does not provide any information about what was the last letter to be emitted, or which one must now be emitted. Options having memoryless policies, options executing for multiple time-steps are unable to represent the FSC exactly. A combination of options that execute for a single time-step cannot represent the FSC either, as the options framework is unable to represent memory-based policies with single-time-step options [@Sutton1999].

Experiments {#sec:experiments}
===========

The experiments in this section illustrate how [OOIs]{}allow agents to perform optimally in environments where options without [OOIs]{}fail. Section \[sec:khepera\] shows that [OOIs]{}allow the agent to learn an expert-level policy for our motivating example (Section \[sec:motivation\]). Section \[sec:duplicatedinput\] shows that the top-level and option policies required by a repetitive task can be learned, and that learning option policies allow the agent to 