The phase problem in diffraction physics is one of the oldest inverse problems in all of science. The central difficulty that any approach to solving this inverse problem must overcome is that half of the information, namely the phase of the diffracted beam, is always missing. In the context of electron microscopy, the phase problem is generally non-linear and solutions provided by phase-retrieval techniques are known to be poor approximations to the physics of electrons interacting with matter. Here, we show that a semi-supervised learning approach can effectively solve the phase problem in electron microscopy/scattering. In particular, we introduce a new Deep Neural Network (DNN), Y-net, which simultaneously learns a reconstruction algorithm via supervised training in addition to learning a physics-based regularization via unsupervised training. We demonstrate that this constrained, semi-supervised approach is an order of magnitude more data-efficient and accurate than the same model trained in a purely supervised fashion. In addition, the architecture of the Y-net model provides for a straightforward evaluation of the consistency of the model's prediction during inference and is generally applicable to the phase problem in other settings. Advances in materials have shaped the course of human civilization from the bronze age to the silicon-powered information age. Future advances in materials science depend critically on our ability to determine, with atomic resolution (10 −10 m), a material's local electron density. This goal is within reach, for the first time, via a computational imaging technique commonly known as 4D-STEM (scanning transmission electron microscopy). Figure 1: 4D-STEM is a computational imaging technique, where a picometer-size beam is scanned across a material and a diffraction pattern is collected at each spatial location. The resultant dataset is 4-D dimensional, where each "pixel" is indexed by (x, y, k x , k y ), where k = α/λ, λ is the wavelength and α is the diffraction angle of the electron beam. Successful inversion of the 4D-STEM data should, in principle, provide local electron density maps of materials with higher spatial resolution and sensitivities than in another existing technique. In 4D-STEM, a picometer-sized (10 −12 m) electron beam is 2-D raster scanned across the material to collect a diffraction pattern with picometer spatial resolutions from each (x, y) position (see Figure  1 ). The resultant dataset is 4-D dimensional, where each "pixel" is indexed by (x, y, k x , k y ), where (k x , k y ) are the wave-vectors associated with a diffracting beam. A 4D-STEM dataset encodes information about the material's electron density from the vantage point of a single atom. Decoding electron diffraction patterns into the local electronic density is a longstanding inverse problem for two principal reasons Zuo & Spence (2013) . First, the quantum interaction of electrons with matter is strong, which produces numerous interference processes and the resultant inverse problem in non-linear. Second, diffraction patterns provide incomplete data, since they only provide intensities as opposed to the complex-valued diffracted electron wavefunction. Here, we introduce a semi-supervised and physics-constrained Deep Neural Network (DNN) to solve the phase problem in 4D-STEM, thereby reconstructing both the local electron density and the incident electron beam wavefunction. Estimating both of these quantities allows one to a posteriori quantify the reconstruction error. We also discuss how our approach is naturally extensible via differentiable programming. In summary, we found that by combining supervised and unsupervised learning to train a new DNN architecture, we can learn both a solution to an inverse problem as well as learn a physics-based regularization, leading to vastly improved reconstruction quality and data efficiency. We are currently extending the presented framework by substituting the learnable regularization with the full forward model in Equation 1 using techniques from differentiable programming.
We introduce a semi-supervised deep neural network to approximate the solution of the phase problem in electron microscopy