The ability to synthesize realistic patterns of neural activity is crucial for studying neural information processing. Here we used the Generative Adversarial Networks (GANs) framework to simulate the concerted activity of a population of neurons.
 We adapted the Wasserstein-GAN variant to facilitate the generation of unconstrained neural population activity patterns while still benefiting from parameter sharing in the temporal domain.
 We demonstrate that our proposed GAN, which we termed Spike-GAN, generates spike trains that match accurately the first- and second-order statistics of datasets of tens of neurons and also approximates well their higher-order statistics. We applied Spike-GAN to a real dataset recorded from salamander retina and showed that it performs as well as state-of-the-art approaches based on the maximum entropy and the dichotomized Gaussian frameworks. Importantly, Spike-GAN does not require to specify a priori the statistics to be matched by the model, and so constitutes a more flexible method than these alternative approaches.
 Finally, we show how to exploit a trained Spike-GAN  to construct 'importance maps' to detect the most relevant statistical structures present in a spike train. 
 Spike-GAN provides a powerful, easy-to-use technique for generating realistic spiking neural activity and for describing the most relevant features of the large-scale neural population recordings studied in modern systems neuroscience.
 Understanding how to generate synthetic spike trains simulating the activity of a population of neurons is crucial for systems neuroscience. In computational neuroscience, important uses of faithfully generated spike trains include creating biologically consistent inputs needed for the simulation of realistic neural networks, generating large datasets to be used for the development and validation of new spike train analysis techniques, and estimating the probabilities of neural responses in order to extrapolate the information coding capacity of neurons beyond what can be computed from the neural data obtained experimentally BID14 BID29 . In experimental systems neuroscience, the ability to develop models that produce realistic neural population patterns and that identify the key sets of features in these patterns is fundamental to disentangling the encoding strategies used by neurons for sensation or behavior and to design closed-loop experiments BID16 in which synthetic patterns, representing salient features of neural information, are fed to systems of electrical micro-stimulation BID44 or patterned light optogenetics BID3 for naturalistic intervention on neural circuits.One successful way to generate realistic spike trains is that of using a bottom-up approach, focusing explicitly on replicating selected low-level aspects of spike trains statistics. Popular methods include renewal processes BID42 ; BID10 ), latent variable models BID23 BID22 and maximum entropy approaches BID43 BID40 BID39 , which typically model the spiking activity under the assumption that only first and second-order correlations play a relevant role in neural coding (but see BID4 ; BID18 ; BID32 ). Other methods model spike train responses assuming linear stimulus selectivity and generating single trial spike trains using simple models of input-output neural nonlinearities and neural noise BID15 BID35 BID19 . These methods have had a considerable success in modeling the activity of populations of neurons in response to sensory stimuli BID35 . Nevertheless, these models are not completely general and may fail to faithfully represent spike trains in many situations. This is because neural variability changes wildly across different cortical areas BID24 due to the fact that responses, especially in higher-order areas and in behaving animals, have complex non-linear tuning to many parameters and are affected by many behavioral variables (e.g. the level of attention BID9 ).An alternative approach is to apply deep-learning methods to model neural activity in response to a given set of stimuli using supervised learning techniques BID26 . The potential advantage of this type of approach is that it does not require to explicitly specify any aspect of the spike train statistics. However , applications of deep networks to generate faithful spike patterns have been rare. Here, we explore the applicability of the Generative Adversarial Networks (GANs) framework BID12 to this problem. Three aspects of GANs make this technique a good candidate to model neural activity. First , GANs are an unsupervised learning technique and therefore do not need labeled data (although they can make use of labels BID31 BID5 ). This greatly increases the amount of neural data available to train them. Second , recently proposed modifications of the original GANs make them good at fitting distributions presenting multiple modes BID13 . This is an aspect that is crucial for neural data because the presentation of even a single stimulus can elicit very different spatio-temporal patterns of population activity BID7 BID28 . We thus need a method that generates sharp realistic samples instead of producing samples that are a compromise between two modes (which is typical, for instance, of methods seeking to minimize the mean squared error between the desired output and the model's prediction BID11 BID20 ). Finally , using as their main building block deep neural networks, GANs inherit the capacity of scaling up to large amounts of data and therefore constitute a good candidate to model the ever growing datasets provided by experimental methods like chronic multi-electrode and optical recording techniques.In the present work we extend the GAN framework to synthesize realistic neural activity. We adapt the recently proposed Wasserstein-GAN (WGAN) which has been proven to stabilize training, by modifying the network architecture to model invariance in the temporal dimension while keeping dense connectivity across the modeled neurons. We show that the proposed GAN, which we called Spike-GAN, is able to produce highly realistic spike trains matching the first and second-order statistics of a population of neurons. We further demonstrate the applicability of Spike-GAN by applying it to a real dataset recorded from the salamander retina and comparing the activity patterns the model generates to those obtained with a maximum entropy model BID46 and with a dichotomized Gaussian method BID22 . Finally, we describe a new procedure to detect, in a given activity pattern, those spikes participating in a specific feature characteristic of the probability distribution underlying the training dataset. We explored the application of the Generative Adversarial Networks framework BID12 to synthesize neural responses that approximate the statistics of the activity patterns of a Figure 4 : A) An example pattern showing the different packets highlighted with different colors and sorted to help visualization. The probability of each type of packet to occur was set to 0.1. Packets of the same type do not overlap in time. B) Realistic neural population pattern (gray spikes do not participate in any packet). C) Examples of activity patterns (grayscale panels) in which only one type of packet is usually present (one or two times) during a period of time from 16 to 32 ms. Packets are highlighted as white spikes. Heatmaps: importance maps showing the change that disrupting specific spikes has on the critic's output. Note that packet spikes normally show higher values. We used a sliding window of 8 ms (with a step size of 2 ms) to selectively shuffle the activity of each neuron at different time periods. The Spike-GAN used to obtain these importance maps was trained for 50000 iterations on 8192 samples. D) Average of 200 randomly selected importance maps across the neurons dimension, yielding importance as a function of time. E) Average of the same 200 randomly selected importance maps across the time dimension, yielding importance as a function of neurons. Errorbars correspond to standard error. population of neurons. For this purpose, we put forward Spike-GAN, by adapting the WGAN variant proposed by to allow sharing weights across time while maintaining a densely connected structure across neurons. We found that our method reproduced to an excellent approximation the spatio-temporal statistics of neural activity on which it was trained. Importantly, it does so without the need for these statistics to be handcrafted in advance, which avoids making a priori assumptions about which features of the external world make neurons fire.Recently, BID33 have proposed a deep learning method, LFADS (Latent Factor Analysis via Dynamical Systems), to model the activity of a population of neurons using a variational autoencoder (in which the encoder and decoder are recurrent neural networks). LFADS allows inferring the trial-by-trial population dynamics underlying the modeled spike train patterns and thus can be seen as a complementary method to Spike-GAN, which does not explicitly provide the latent factors governing the response of the neurons. Regarding the application of the GANs framework to the field of neuroscience, BID1 proposed a GAN-based approach for fitting network models to experimental data consisting of a set of tuning curves extracted from a population of neurons. However, to the best of our knowledge our work is the first to use GANs to directly produce realistic neural patterns simulating the activity of populations of tenths of neurons.Building on the work by BID47 , we showed how to use Spike-GAN to visualize the particular features that characterize the training dataset. Specifically, Spike-GAN can be used to obtain importance maps that highlight the spikes that participate in generating activity motifs that are most salient in the spike trains. This can be useful for unsupervised identification of highly salient low-dimensional representations of neural activity, which can then be used to describe and interpret experimental results and discover the key units of neural information used for functions such as sensation and behavior.A further and promising application of importance maps is that of designing realistic patterns of stimulation that can be used to perturb populations of neurons using electrical or optical neural stimulation techniques BID44 BID8 . The ability of Spike-GAN to generate realistic neural activity including its temporal dynamics and to identify its most salient features suggests that it may become a very relevant tool to design perturbations. In FIG1 we provide a more detailed description of a potential application of Spike-GAN, in which importance maps may allow inferring the set of neurons participating in the encoding of the information about a given set of stimuli FIG1 ) and the spatio-temporal structure of the packets elicited by each stimulus FIG1 .We have compared Spike-GAN with two alternative methods based on the maximum entropy and the dichotomized Gaussian frameworks. These methods offer the possibility of computing the sample probabilities (MaxEnt model) and separately specifying the signal and noise correlations present in the generated samples (DG model). Spike-GAN does not have these features; nevertheless, it does have important advantages over the mentioned methods. First, Spike-GAN is more flexible than the MaxEnt and DG models, being able to fit any type of spatio-temporal structure present in the data. Further, it does not require making a priori assumptions about which statistical properties of a dataset are relevant and thus need to be matched. Finally, Spike-GAN is based on the deep neural network framework, and is therefore able to directly benefit from the engineering advances emerging in this rapidly-growing field. Conceivably, this will enable Spike-GAN, or methods derived from it, to make in the future better and better use of the datasets of ever increasing size that are produced by the experimental neuroscience community.
Using Wasserstein-GANs to generate realistic neural activity and to detect the most relevant features present in neural population patterns. A method for simulating spike trains from populations of neurons which match empirical data using a semi-convolutional GAN. The paper proposes to use GANs for synthesizing realistic neural activity patterns