agent only observes a single input character at a time, and can write at most one character to the output tape per time-step.

The input tape is a sequence of $N$ symbols $x \in \Omega$, with $\Omega = \{A, B, C, D, E\}$ and $N$ a random number between 20 and 30. The agent observes a single symbol $x_t \in \Omega$, read from the $i$-th position in the input sequence, and does not observe $i$. When $t = 1$, $i = 0$. There are 20 actions ($5 \times 2 \times 2$), each of them representing a symbol (5), whether it must be pushed onto the output tape (2), and whether $i$ should be incremented or decremented (2). A reward of 1 is given for each correct symbol written to the output tape. The episode finishes with a reward of -0.5 when an incorrect symbol is written.

![Cumulative reward per episode obtained on modified DuplicatedInput, with random or designed [OOIs]{}, without [OOIs]{}and using an LSTM over options. Despite our efforts, an LSTM over options repeatedly learns then forgets optimal policies, as shown by the high variance of its line.[]{data-label="fig:diresults"}](modifiedduplicatedinput-results/plot)

The agent has access to two options, $\omega_1$ and $\omega_2$. [OOIs]{}are designed 