Weakness: 1. Literature review is not adequate. Even with the content in the appendix, this is no discussion of off-policy evaluation for reinforcement learning or non-stationary multi-armed bandits. 2. The claim and the discussion of “active non-stationarity” is somewhat confusing (more on this later in Questions). 3. The authors seem to overstate their contribution.
4. Baseline methods are weak and not presenting state-of-the-art.
There is no discussion of limitation. Along with my questions regarding the difference between this work and reinforcement learning, one possible direction in the conclusion is to talk about the similarity and difference, and to what extent the results are generalizable to RL settings.
4. Baseline methods are weak and not presenting state-of-the-art. There is no discussion of limitation. Along with my questions regarding the difference between this work and reinforcement learning, one possible direction in the conclusion is to talk about the similarity and difference, and to what extent the results are generalizable to RL settings.