Though the training procedure is novel, a part of the algorithm is not well-justified to follow the physics and optics nature of this problem. A few key challenges in depth from defocus are missing, and the results lack a full analysis. See details below:
- the authors leverage multiple datasets, including building their own to train the model. However, different dataset is captured by different cameras, and thus the focusing distance, aperture settings, and native image resolution all affect the circle of confusion, how are those ambiguities taken into consideration during training?
- related to the point above, the paper doesn't describe the pre-processing stage, neither did it mention how the image is passed into the network. Is the native resolution preserved, or is it downsampled?
- According to Held et al "Using Blur to Affect Perceived Distance and Size", disparity and defocus can be approximated by a scalar that is related to the aperture and the focus plane distance. In the focal stack synthesis stage, how is the estimated depth map converted to a defocus map to synthesize the blur?
- the paper doesn't describe how is the focal stack synthesized, what's the forward model of using a defocus map and an image to synthesize defocused image? how do you handle the edges where depth discontinuities happen?
- in 3.4, what does “Make the original in-focus region to be more clear” mean? in-focus is defined to be sharpest region an optical system can resolve, how can it be more clear?
- the paper doesn't address handling textureless regions, which is a challenging scenario in depth from defocus. Related to this point, how are the ArUco markers placed? is it random?
- fig 8 shows images with different focusing distance, but it only shows 1m and 5m, which both exist in the training data. How about focusing distance other than those appeared in training? does it generalize well?
- what is the limit of the amount of blur presented in the input that the proposed models would fail? Are there any efforts in testing on smartphone images where the defocus is *just* noticeable by human eyes? how do the model performances differ for different defocus levels?
Minor suggestions
- figure text should be rasterized, and figures should maintain its aspect ratio.
- figure 3 is confusing as if the two nets are drawn to be independent from each other -- CNN layers are represented differently, one has output labeled while the other doesn't. It's not labeled as the notation written in the text so it's hard to reference the figure from the text, or vice versa.
- the results shown in the paper are low-resolution, it'd be helpful to have zoomed in regions of the rendered focal stack or all-in-focus images to inspect the quality.
- the sensor plane notation 's' introduced in 3.1 should be consistent in format with the other notations.
- calling 'hyper-spectral' is confusing. Hyperspectral imaging is defined as the imaging technique that obtains the spectrum for each pixel in the image of a scene.
- the paper doesn't describe how is the focal stack synthesized, what's the forward model of using a defocus map and an image to synthesize defocused image? how do you handle the edges where depth discontinuities happen?