the opening excerpt, an action might be ‘open the mailbox’ (Figure \[ZRK\_ENV\]). Once the player describes his/her action, it is processed by a sophisticated natural language parser. Based on the parser’s results, the game presents the outcome of the action. The ultimate goal of Zork is to collect the Twenty Treasures of Zork and install them in the trophy case. Finding the treasures require solving a variety of puzzles such as navigation in complex mazes and intricate action sequences. During the game, the player is awarded points for performing deeds that bring him closer to the game’s goal (e.g., solving puzzles). Placing all of the treasures into the trophy case generates a total score of $350$ points for the player. Points that are generated from the game’s scoring system are given to the agent as a reward. Zork presents multiple challenges to the player, like building plans to achieve long-term goals; dealing with random events like troll attacks; remembering implicit clues as well as learning the interactions between objects in the game and specific actions. The **elimination signal** in Zork is given by the Zork environment in two forms, a “wrong parse” flag, and text feedback (e.g. “you cannot take 