5 Limitations
a. The authors briefly talk about the limitations of the approach in section 5. The main limitation they draw attention to is the challenge of moving closer to the local maxima of the reward function in the latter stages of optimization. To resolve this they discuss combining their method with local optimization techniques; however, I wonder whether the temperature approach they discuss in the earlier part of their paper (combined with some annealing scheme) could also be used here?
b. One limitation the authors do not mention is how the method scales in terms of the size of the state and action space. The loss function requires for every current state the sum over all previous states and actions that may have led to the current state (see term 1 of Eq.9). I assume this may become intractable for very large state-action spaces (and the flows one is trying to model become very small). Can one approximate the sum using a subset? Also what about continuous state/action spaces?
6 Societal impact
The authors state that they foresee no negative social impacts of their work (line 379). While I do not believe this work has the potential for significant negative social impact (and I'm not quite sure if/how I'm meant to review this aspect of their work), the authors could always mention the social impact of increased automation, or the risks from the dual use of their method, etc.
6 Societal impact The authors state that they foresee no negative social impacts of their work (line 379). While I do not believe this work has the potential for significant negative social impact (and I'm not quite sure if/how I'm meant to review this aspect of their work), the authors could always mention the social impact of increased automation, or the risks from the dual use of their method, etc.