Robustness and security of machine learning (ML) systems are intertwined, wherein a non-robust ML system (classifiers, regressors, etc.) can be subject to attacks using a wide variety of exploits. With the advent of scalable deep learning methodologies, a lot of emphasis has been put on the robustness of supervised, unsupervised and reinforcement learning algorithms. Here, we study the robustness of the latent space of a deep variational autoencoder (dVAE), an unsupervised generative framework, to show that it is indeed possible to perturb the latent space, flip the class predictions and keep the classification probability approximately equal before and after an attack. This means that an agent that looks at the outputs of a decoder would remain oblivious to an attack. The ability to encode data reliably is essential for many tasks including image compression, data retrieval and communication. As data is transmitted between communication channels, error detection and correction is often employed to deduce the presence of erroneous bits BID12 . The source of such errors can be a result of imperfection in the transmitter, channel or in the receiver. Often times, such errors can be deliberate where a man-in-middle attack BID3 BID2 can result in deleterious erasure of information, yet to the receiver, it may end up as appearing untampered BID8 .In deep learning, we are able to learn an encoding process using unsupervised learning such as in autoencoders (AE) BID7 ; however, we are less able to design methods for checking whether encodings have been tampered with. Therefore , there are two facets of this problem -the first, is to come up with methodologies of tampering with the models and second, is to detect the adversarial breach. In what follows, we will concentrate only on the first problem by presenting a method for tampering autoencoders. An autoencoder has two components: the encoder maps the input to a latent space, while the decoder maps the latent space to the requisite output. A vanilla autoencoder can, therefore, be used to compress the input to a lower dimensional latent (or feature) space. Other forms of autoencoder include the denoising AE BID16 that recovers an undistorted input from a partially corrupted input; the compressive AE BID14 designed for image compression and the variational AE BID7 ) that assumes that the data is generated from a directed graphical model with the encoder operationalized to learn the posterior distribution of the latent space. Autoencoders have wide use in data analytics, computer vision, natural language processing, etc.We propose an attack that targets the latent encodings of autoencoders, such that if an attack is successful the output of an autoencoder will have a different semantic meaning to the input. Formally, we consider an autoencoder consisting of an encoder and decoder model designed to reconstruct an input data sample such that the label information associated with the input data is maintained. For example, consider a dataset of images , x with the labels, y = {0, 1}, and an encoder, E : x ! z and a decoder, D : z ! x where z is a latent encoding for x. If the encoder and decoder are operating normally, the label of the reconstructed data sample,ŷ = class(D(E(x))) should be the same as the label of the input data sample, where class(·) is the soft output of a binary classifier.In this paper, we focus on learning an attack transformation, T z, such that if z is the latent encoding for a data sample, x, with label 0, T z is the latent encoding for a data sample with label 1. The attack is designed to flip the label of the original input and change its content. Note that the same T is applied to each encoding and is not specific to either the input data sample or the encoding, it is only dependent on the label of the input data sample.The success of an attack may be measured in three ways:1. The number of elements in the latent encoding , changed by the attack process should be small. If the encoding has a particular length, changing multiple elements may make the attack more detectable.2. When a decoder is applied to tampered encodings, the decoded data samples should be indistinguishable from other decoded data samples that have not been tampered with.3. Decoded tampered-encodings should be classified with opposite label to the original (untampered) data sample.Our contribution lies in studying transforms with these properties. Experimentally, we find that optimizing for requirement (1) may implicitly encourage requirement (2). Crucially, in contrast to previous work BID6 , our approach does not require knowledge of the model (here a VAE) parameters; we need access only to the encodings and the output of a classifier, making our approach more practical . Finally, we owe the success of this attack method primarily to the near-linear structure of the VAE latent space BID7 ) -which our attack exploits. In this paper, we propose the idea of latent poisoning -an efficient methodology for an adversarial attack i.e., by structured modification of the latent space of a variational autoencoder. Both additive and multiplicative perturbation, with sparse and dense structure, show that it is indeed possible to flip the predictive class with minimum changes to the latent code.Our experiments show that additive perturbations are easier to operationalize than the multiplicative transformation of the latent space. It is likely that additive perturbations have reasonable performance because of the near-linear structure of the latent space. It has been shown that given two images and their corresponding points in latent space, it is possible to linearly interpolate between samples in latent space to synthesize intermediate images that transit smoothly between the two initial images BID7 BID13 . If the two images were drawn from each of the binary classes, and a smooth interpolation existed between them, this would mean that additive perturbation in the latent space, along this vector, would allow movement of samples from one class to the other.How can we counter such a poisoning of the latent space? It might be helpful to look into the predictive probability and its uncertainty on outputs from an autoencoder. If the uncertainty is above a threshold value, an attack may be detected. Detection via predictive probability and its uncertainty, as well as alternative methods, such as inspection of the latent encoding, become even more difficult when the attacker has altered the latent distribution minimally (under a norm).Given the prevalence of machine learning algorithms, the robustness of such algorithms is increasingly becoming important BID9 BID0 , possibly at par with reporting test error of such systems.
Adversarial attacks on the latent space of variational autoencoders to change the semantic meaning of inputs This paper concerns security and machine learning and proposes a man-in-middle attack that alters the VAE encoding of input data so that decoded output will be misclassified.