We propose a new architecture for distributed image compression from a group of distributed data sources. The work is motivated by practical needs of data-driven codec design, low power consumption, robustness, and data privacy. The proposed architecture, which we refer to as Distributed Recurrent Autoencoder for Scalable Image Compression (DRASIC), is able to train distributed encoders and one joint decoder on correlated data sources. Its compression capability is much better than the method of training codecs separately. Meanwhile, for 10 distributed sources, our distributed system remarkably performs within 2 dB peak signal-to-noise ratio (PSNR) of that of a single codec trained with all data sources. We experiment distributed sources with different correlations and show how our methodology well matches the Slepian-Wolf Theorem in Distributed Source Coding (DSC). Our method is also shown to be robust to the lack of presence of encoded data from a number of distributed sources. Moreover, it is scalable in the sense that codes can be decoded simultaneously at more than one compression quality level. To the best of our knowledge, this is the first data-driven DSC framework for general distributed code design with deep learning. It has been shown by a variety of previous works that deep neural networks (DNN) can achieve comparable results as classical image compression techniques (Toderici et al., 2015; Ballé et al., 2016; Gregor et al., 2016; Theis et al., 2017; Liu et al., 2018; Li et al., 2018; Mentzer et al., 2018) . Most of these methods are based on autoencoder networks and quantization of bottleneck representations. These models usually rely on entropy codec to further compress codes. Moreover, to achieve different compression rates it is unavoidable to train multiple models with different regularization parameters separately, which is often computationally intensive. In this work, we are motivated to develop an architecture that has the following advantages. First, unlike classical distributed source coding (DSC) which requires customized code design for different scenarios (Xiong et al., 2004) , a data-driven distributed compression framework can handle nontrivial distribution of image sources with arbitrary correlations. Second, the computation complexity of encoders (e.g. mobile devices) can be transferred to the decoder (e.g. a remote server). Such a system of low complexity encoders can be used in a variety of application domains, such as multi-view video coding (Girod et al., 2005) , sensor networks (Xiong et al., 2004) , and under-water image processing where communication bandwidth and computational power are quite restricted (Stojanovic & Preisig, 2009; Schettini & Corchs, 2010) . Third, the distributed framework can be more robust against heterogeneous noises or malfunctions of encoders, and such robustness can be crucial in, e.g., unreliable sensor networks (Girod et al., 2005; Ishwar et al., 2005; Xiao et al., 2006) . Last but not least, the architecture is naturally scalable in the sense that codes can be decoded at more than one compression quality level, and it allows efficient coding of correlated sources which are not physically co-located. This is especially attractive in video streaming applications (Guillemot et al., 2007; Gehrig, 2008) . It is tempting to think that splitting raw data into different encoders compromises the compression quality. It is thus natural to ask this question: Can distributed encoders perform as well as a single encoder trained with all data sources together? A positive answer from a theoretical perspective was given in the context of information theory, where DSC is an important problem regarding the compression of multiple correlated data sources. The Slepian-Wolf Theorem shows that lossless coding of two or more correlated data sources with separate encoders and a joint decoder can compress data as efficiently as the optimal coding using a joint encoder and decoder (Slepian & Wolf, 1973; Cover, 1975) . The extension to lossy compression with Gaussian data sources was proposed as Wyner-Ziv Theorem (Wyner & Ziv, 1976) . Although these theorems were published in 1970s, it was after about 30 years that practical applications such as Distributed Source Coding Using Syndromes (DISCUS) emerged (Pradhan & Ramchandran, 2003) . One of the main advantages of DSC is that the computation complexity of the encoder is transferred to the decoder. A system architecture with low complexity encoders can be a significant advantage in applications such as multi-view video coding and sensor networks (Girod et al., 2005; Xiong et al., 2004) . Motivated by the theoretical development of DSC, in this work we propose a DNN architecture that consists of distributed encoders and a joint decoder (illustrated in Fig. 1 and 2 ). We show that distributed encoders can perform as well as a single encoder trained with all data sources together. Our proposed DSC framework is data-driven by nature, and it can be applied to distributed data even with unknown correlation structure. The paper is outlined below. We review previous related works in Section 2. We describe our proposed architecture for general image compression and its basic modules in Subsections 3.1-3.4. Then we elaborate the Deep Distributed Source Coding framework in Subsection 3.5. Experimental results are shown in Section 4, followed by conclusions in Section 5. We introduced a data-driven Distributed Source Coding framework based on Distributed Recurrent Autoencoder for Scalable Image Compression (DRASIC). Compared to classical code design, our method has the following advantages. First, instead of explicitly estimating the correlations among data sources in advance, we use data-driven approach to learn the dependencies with the neural network parameters. Given enough training data, our method can handle an arbitrary number of sources with arbitrary correlations. Second, we showed the robustness of our framework. Unlike classical code design which may require careful data source synchronization, each distributed encoder of our model, once trained and deployed, can be used independently of others because the dependencies are already learned by the model parameters. Third, as one of the most important applications of Distributed Source Coding, low complexity encoders were shown to be feasible based on our experimental results. Data sources trained with less data and fewer number of iterations can still approach the theoretical limit obtained by pulling all the data. Last but not least, our recurrent model can reconstruct images efficiently even at low compression quality. We point out two interesting directions of future work. First, the compression quality of the proposed architecture may be improved by introducing spatially adaptive weights over different iterations, e.g. by using context models for adaptive arithmetic coding. Second, the network architecture may be further extended to handle time-dependent data sources.
We introduce a data-driven Distributed Source Coding framework based on Distributed Recurrent Autoencoder for Scalable Image Compression (DRASIC). The paper proposed a distributed recurrent auto-encoder for image compression that uses a ConvLSTM to learn binary codes that are constructed progressively from residuals of previously encoded information The authors propose a method to train image compression models on multiple sources, with a separate encoder on each source, and a shared decoder. 