Weakness:
It is unclear whether the gain of BooT comes from 1. Extra data 2. Different architecture (pretrained gpt2 vs not) 3. Some inherent property in the sequence model as opposed to other world models that may only predict the observation and the reward.
It is unclear from the paper whether bootstrapping is novel beyond supervised learning (e.g. in RL)
There are quite a few additional limitations not mentioned in the paper (l349-350): 1. The extra two hyperparameters introduced k and η require finetuning, which depends on availability to the environment or a good OPE method. 2. As mentioned in l37-39, for other tasks in general, it is unclear whether the dataset available is sufficient to train a BooT, unless we try it, which will incur extra training time and cost, as mentioned in l349-350.
1. The extra two hyperparameters introduced k and η require finetuning, which depends on availability to the environment or a good OPE method.