Scalability: The proposed encoding method is templated-based (Line 155-156). Although the input encoding scheme (Section 7.1) may be a trivial problem, the encoding scheme may still affect the performance. Searching for the optimal encoding scheme is an expensive process, which may bring a high cost of hand-crafted engineering. Besides, the data gathering method also relies on hand-designed templates (Line 220).
Presentation: The related work of PLM is adequately cited. But the authors should also introduce the background of policy learning so that the significance of this work can be highlighted.
Performance: Compared to the work that uses traditional networks like DQN, the integration of PLM may affect the inference speed.
Clarity: Most parts of this paper are well written. However, there are some typos in the paper:
Line 53: pretrained LMs -> pre-trained LMs
Line 104: language -> language. (missing full stop mark)
Some papers should be cited in a proper way: Line 108: [23], Line 109: [36], Line 285:[15], Line 287 [15]. For example, in Line 108, "[23] show that" needs to be rewritten as "Frozen Pretrained Transformer (FPT) [23] show that".
[Rebuttal Updates] The authors provided the additional experiments for addressing my concern of scalability. The authors also revised the typos and added the related works.
Societal Impact: No potential negative societal impact. The authors provide a new perspective to aid policy learning with a pre-trained language model.
Limitation: 1) Building text descriptions for each task still requires human labor. We do not know what textual format is optimal for policy learning. It varies from task to task, model to model. On the other hand, as I stated in Question 1, the long-text input could restrict the scalability of this framework. 2) The proposed methods also need humans to design some templates/rules, as the authors mentioned in the conclusion part.
1) Building text descriptions for each task still requires human labor. We do not know what textual format is optimal for policy learning. It varies from task to task, model to model. On the other hand, as I stated in Question 1, the long-text input could restrict the scalability of this framework.