that are meaningful. More significantly, the most successful algorithms for phase retrieval are not based on gradient descent. In this paper we apply these same techniques to the training of neural networks. As with gradient descent there are many options and we present only a particular approach that is both flexible and empirically successful.

Phase retrieval uses the measured magnitudes of a complex-valued signal, together with generic properties (signal support, sparsity) to reconstruct the signal’s phases. It is possible to define loss functions for phase retrieval and attempt the discovery of the phases with gradient-based methods. This approach, called Wirtinger flow [@candes2015phase], has led to a recent revival of interest in the theoretical problem. However, this line of research has not produced any practical algorithms to displace the non-gradient algorithms that are used in applications. Instead of minimizing a loss function, these algorithms try to discover a point $x_\mathrm{sol}$ that lies in the intersection of two sets: $$\label{AintB}
x_\mathrm{sol}\in A\cap B.$$ The two sets live in a high dimensional Euclidean space, not unlike the space of parameters for a network trained on images. In phase retrieval $A$ is the set of all images with given (Fourier-transform) magnitudes and $B$ is 