1. evaluation on a single domain
The method is evaluated only on the tasks from Meta World, a robotic manipulation domain. Hence, it is difficult to judge whether the results will generalize to other domains. I strongly recommend running experiments on a different benchmark such as Atari which is commonly used in the literature. This would also verify whether the method works with discrete action spaces and high-dimensional observations.
2. evaluation on a setting created by the authors, no well-established external benchmark
The authors seem to create their own train and test splits in Meta World. This seems strange since Meta World recommends a particular train and test split (e.g. MT10 or MT50) in order to ensure fair comparison across different papers. I strongly suggest running experiments on a pre-established setting so that your results can easily be compared with prior work (without having to re-implement or re-run them). You don't need to get SOTA results, just show how it compares with reasonable baselines like the ones you already include. Otherwise, there is a big question mark around why you created your own "benchmark" when a very similar one exists already and whether this was somehow carefully designed to make your approach look better.
3. limited number of baselines
While you do have some transformer-based baselines I believe the method could greatly benefit from additional ones like BC, transformer-BC, and other offline RL methods like CQL or IQL. Such comparisons could help shed more light into whether the transformer architecture is crucial, the hypernetwork initialization, the adaptation layers, or the training objective.
4. more analysis is needed
It isn't clear how the methods compare with the given expert demonstrations on the new tasks. Do they learn to imitate the policy or do they learn a better policy than the given demonstration? I suggest comparing with the performance of the demonstration or policy from which the demonstration was collected.
If the environment is deterministic and the agent gets to see expert demonstrations, isn't the problem of learning to imitate it quite easy? What happens if there is more stochasticity in the environments or the given demonstration isn't optimal?
When finetuning transformers, it is often the case that they forget the tasks they were trained on. It would be valuable to show the performance of your different methods on the tasks they were trained on after being finetuned on the downstream tasks. Are some of them better than the others at preserving previously learned skills?
5. missing some important details
The paper seems to be missing some important details regarding the experimental setup. For example, it wasn't clear to me how the learning from observations setting works. At some point you mention that you condition on the expert observations while collecting online data. Does this assume the ability to reset the environment in any state / observation? If so, this is a big assumption that should be more clearly emphasized and discussed. how exactly are you using the expert observations in combination with online learning?
There are also some missing details regarding the expertise of the demonstrations at test time. Are these demonstrations coming from an an expert or how good are they? Minor
sometimes you refer to generalization to new tasks. however, you finetune your models, so i believe a better term would be transfer or adaptation to new tasks.
1. evaluation on a single domain The method is evaluated only on the tasks from Meta World, a robotic manipulation domain. Hence, it is difficult to judge whether the results will generalize to other domains. I strongly recommend running experiments on a different benchmark such as Atari which is commonly used in the literature. This would also verify whether the method works with discrete action spaces and high-dimensional observations.