* A big concern for me is that this paper was hard to read. Since it is very applications specific, I am not familiar with a lot of the theory or the inverse problem(s) considered here. As a result, I am unable to appreciate the key aspects of the paper. For example, the introduction directly gets into the details of wave based imaging without sufficient detail or context with more commonly considered inverse problems. This makes it unapproachable for someone not familiar with this exact application. There is quite a bit of detail left in the supplement, but I believe this should be in the main paper for the contribution to be fully appreciated. * A second point about it being so applications specific is that the paper lacks context to existing methods, for e.g. how can FIONets be useful for someone outside of wave-based imaging? * Another issue is that there are no quantitative comparisons in the main paper (but in the supplement), leaving only qualitative comparisons. * There are no comparisons to any other method other than a U-Net (which essentially serves as an ablation of whether or not including the physics based network helps). Considering this is a linear inverse problem, what are other existing solutions to this problem? It is imperative to compare the proposed FIONet to iterative or classical solutions to the problem to place them in context. * Regarding the OOD experiments, this is indeed interesting because the trained network is able to give strong OOD generalization. However, particularly in imaging in the recent few years several papers have shown that untrained NNs (like deep image prior Ulyanov et al., CVPR 2018) can be used to solve inverse problems across a very wide class of images. It maybe good to mention this in the paper and place the current method in context and Ideally, also compare with those class of methods. * I am not very sure how to read or interpret figure 7 describing the diffeomorphisms. * A minor comment, there is already a model called “routing networks” (Rosenbaum et al, ICLR 2018) which are different from those described in the paper. In the interest of mitigating confusion for the reader it maybe better to clarify or re-name the model.
* Regarding the OOD experiments, this is indeed interesting because the trained network is able to give strong OOD generalization. However, particularly in imaging in the recent few years several papers have shown that untrained NNs (like deep image prior Ulyanov et al., CVPR 2018) can be used to solve inverse problems across a very wide class of images. It maybe good to mention this in the paper and place the current method in context and Ideally, also compare with those class of methods.