- The paper's presentation is difficult to follow, with numerous typos and inconsistencies in notation. For example:
- Line 84, "In summery" -> "In summary".
- In Figure 1, "LLaVA as dicision model" -> "LLaVA as decision model."
- Line 215, "donate" should be "denote"; additionally, $\pi_{ref}$ is duplicated.
- The definitions of subscripts and superscripts for action (i.e., $a_t^1$ and $a_t^2$) in line 245 and in Equations (4), (6), (7), (8), and (9) are inconsistent.
- Line 213 references the tuple with $o_t$, but it is unclear where $s_{1 \sim t-1}$ originates.
- The authors should include a background section to introduce the basic RL framework, including elements of the MDP, trajectories, and policy, to clarify the RL context being considered. Without this, it is difficult to follow the subsequent sections. Additionally, a brief overview of the original DPO algorithm should be provided so that modifications proposed in the methods section are clearly distinguishable.
- In Section 3.1, the authors state that the VLM is used as a planner; however, it is unclear how the plan is generated. It appears that the VLM functions directly as a policy, outputting final actions to step into the environment, as illustrated in Figure 1. Thus, it may be misleading to frame the proposed method as a "re-planning framework" (line 197). Can author clarify this point?
- What type of action space does the paper consider? Is it continuous or discrete? If it is discrete, how is the MSE calculated in Eq. (7)?
- In line 201, what does "well-fine-tuned model" refer to? Is this the VLM fine-tuned by the proposed method?
- Throughout the paper, what does $\tau_t^{t-1}$ represent?
- The authors should include a background section to introduce the basic RL framework, including elements of the MDP, trajectories, and policy, to clarify the RL context being considered. Without this, it is difficult to follow the subsequent sections. Additionally, a brief overview of the original DPO algorithm should be provided so that modifications proposed in the methods section are clearly distinguishable.