1. How Fourier features accelerate NTK convergence in the high-frequency range? Did I overlook something or it's not analyzed? This is an essential theoretical support to the merits of Fourier features. 2. The theory part is limited to the behavior on NTK. I understand analyzing Fourier features on MLPs is highly difficult, but I'm a bit worried there would be a significant gap between NTK and the actual behavior of MLPs (although they are asymptotically equivalent). 3. Examples in Section 5 are limited to 1D functions, which are a bit toyish.
1. How Fourier features accelerate NTK convergence in the high-frequency range? Did I overlook something or it's not analyzed? This is an essential theoretical support to the merits of Fourier features.