that”). We group these two signals into a single binary signal which we then provide to our learning algorithm.\
Before we started experimenting in the “Open Zork“ domain, i.e., playing in Zork without any modifications to the domain, we evaluated the performance on two subdomains of Zork. These subdomains are inspired by the Zork plot and are referred to as the Egg Quest and the Troll Quest (Figure \[ZRK\_WORLD\], right, and Appendix B). For these subdomains, we introduced an additional reward signal (in addition to the reward provided by the environment) to guide the agent towards solving specific tasks and make the results more visible. In addition, a reward of $-1$ is applied at every time step to encourage the agent to favor short paths. When solving “Open Zork“ we only use the environment reward. The optimal time that it takes to solve each quest is $6$ in-game timesteps for the Egg quest, $11$ for the Troll quest and $350$ for “Open Zork”. The agent’s goal in each subdomain is to maximize its cumulative reward. Each trajectory terminates upon completing the quest or after $T$ steps are taken. We set the discounted factor during training to $\gamma=0.8$ but use $\gamma=1$ 