* The paper makes several “hand-wavy” arguments, which are suitable for supporting the claims in the paper; but it is unclear if they would generalize for analyzing / developing other algorithms. For instance: 1. Replacing `n^2/(2*s^2)` with an arbitrary parameter `lambda` (lines 119-121) 2. Taking SGD learning rate ~ 0.1 (line 164) — unlike the Adam default value, it is unclear what the justification behind this value is.
1. Replacing `n^2/(2*s^2)` with an arbitrary parameter `lambda` (lines 119-121) 2. Taking SGD learning rate ~ 0.1 (line 164) — unlike the Adam default value, it is unclear what the justification behind this value is.