- Goal Misspecification: Failures on the ALFRED benchmark often occurred due to goal misspecification, where the LLM did not accurately recover the formal goal predicate, especially when faced with ambiguities in human language.
- Policy Inaccuracy: The learned policies sometimes failed to account for low-level, often geometric details of the environment.
- Operator Overspecification: Some learned operators were too specific, e.g., the learned SliceObject operator specified a particular type of knife, leading to planning failures if that knife type was unavailable.
- Limitations in Hierarchical Planning: The paper acknowledges that it doesn't address some core problems in general hierarchical planning. For instance, it assumes access to symbolic predicates representing the environment state and doesn't tackle finer-grained motor planning. The paper also only considers one representative pre-trained LLM and not others like GPT-4.
- Goal Misspecification: Failures on the ALFRED benchmark often occurred due to goal misspecification, where the LLM did not accurately recover the formal goal predicate, especially when faced with ambiguities in human language.