Adversarial Attacks on Machine Learning-Aided Visualizations
Takanori Fujiwara Kostiantyn Kucher Junpeng Wang Rafael M. Martins Andreas Kerren Anders Ynnerman


Adversarial Attacks on Machine Learning-Aided Visualizations]Adversarial Attacks on Machine Learning-Aided Visualizations













[1]Takanori Fujiwaratakanori.fujiwara@liu.se
1]Kostiantyn Kucher
2]Junpeng Wang
3]Rafael M. Martins
1,3]Andreas Kerren
1]Anders Ynnerman

[1]Linköping University, Sweden
[2]Visa Research, United States
[3]Linnaeus University, Sweden





Research in ML4VIS investigates how to use machine learning (ML) techniques to generate visualizations, and the field is rapidly growing with high societal impact. However, as with any computational pipeline that employs ML processes, ML4VIS approaches are susceptible to a range of ML-specific adversarial attacks. These attacks can manipulate visualization generations, causing analysts to be tricked and their judgments to be impaired. Due to a lack of synthesis from both visualization and ML perspectives, this security aspect is largely overlooked by the current ML4VIS literature. To bridge this gap, we investigate the potential vulnerabilities of ML-aided visualizations from adversarial attacks using a holistic lens of both visualization and ML perspectives. 
We first identify the attack surface (i.e., attack entry points) that is unique in ML-aided visualizations. 
We then exemplify five different adversarial attacks.
These examples highlight the range of possible attacks when considering the attack surface and multiple different adversary capabilities. Our results show that adversaries can induce various attacks, such as creating arbitrary and deceptive visualizations, by systematically identifying input attributes that are influential in ML inferences. Based on our observations of the attack surface characteristics and the attack examples, we underline the importance of comprehensive studies of security issues and defense mechanisms as a call of urgency for the ML4VIS community.







[
    [
    3 September 2024
====================



fancy



§ INTRODUCTION


Visualizations play a critical role in depicting relationships or patterns within an underlying dataset such that analysts can effectively explore, interact, and communicate the data.
Recently, researchers are actively investigating various applications of using machine learning (ML) techniques when generating visualizations optimized for certain tasks (e.g., chart recommendation <cit.>, text-chart translation <cit.>, and interaction prediction <cit.>).
This approach is often referred to as ML for visualization, or ML4VIS for short <cit.>.
Despite the wide usage, we argue that the current research focus on ML4VIS is imbalanced. The majority of ML4VIS research mainly focuses on the benefits these techniques provide <cit.>, and consequently largely overlooks the security issues these techniques introduce.

Critically understanding security issues related to ML4VIS requires a more holistic perspective that is both visualization- and ML-oriented. 
This dual perspective is necessary as the joint use of ML and visualization techniques may cause additional security problems that are not prevalent in each respective field. 
However, relevant research investigations on these matters are currently disconnected. 
For example, though some visualization research has explored the vulnerabilities of visualization approaches, the focus of these investigations targets more general visualization practices (e.g., the influence of the choice of the aspect ratio on the recognition of correlation patterns <cit.>). These insights do not consider the specific security issues that ML approaches introduce (e.g., manipulating data transformation when generating ML-aided visualizations). 
Similarly, though the ML community is intensively studying the vulnerability of ML models from adversarial attacks <cit.>, these security vulnerabilities have not been studied with respect to visualization. 
Adversaries can then take advantage of loopholes that are unattended.
For instance, while direct human intervention is vital to prevent critical harm from ML mispredictions <cit.>, this scenario is generally not applicable to ML4VIS. 
In ML4VIS, an adversary's goal could be to harm the end users by corrupting output visualizations that humans use for their judgments.
This lack of an interdisciplinary perspective can introduce security concerns that both communities are overlooking. 

This paper aims to convey the importance of systematic studies related to the security vulnerabilities of ML4VIS approaches. 
To reveal structural vulnerabilities of ML-aided visualizations, we first outline the attack surface (i.e., a set of entry points for adversaries) by referring to the ML4VIS pipeline introduced by Wang et al. <cit.>.
We then characterize the unique aspects of the attack surface of ML-aided visualizations.
For this characterization, we focus on ML-aided visualizations that incorporate neural networks (NNs), as NNs are rapidly gaining increased attention and wide usage <cit.>.
To demonstrate the unique characteristics of the attack surface with real possible threats, we design concrete examples of adversarial attacks on representative state-of-the-art ML4VIS methods.
The results from these exemplary attacks highlight critical vulnerabilities in the current ML4VIS approach that can directly impact future applications.

Based on the insights from our literature review, characterization of the attack surface, and attack examples, we discuss the identified research gaps on the lack of studies on vulnerabilities specific to ML4VIS and propose a research agenda outlining future research directions from a security perspective.
Our research reflects the urgency of this matter as several government and official institutions (e.g., EU <cit.>, UK <cit.>, and US <cit.>) have already introduced and developed ML security regulations. 
We similarly argue that it is important that visualization researchers also take action to minimize the risk of any harm that can stem from using ML in conjunction with visualization. 
In summary, our primary contributions are to be:


  
  * characterization of the attack surface of ML-aided visualizations, which produces an indication of potential threats (<ref>);
  
  * five examples of adversarial attacks using two representative ML-aided visualizations that (1) describe the attack strategies and (2) analyze the attacked results (<ref>); and
  
  * a research agenda outlining the future research direction for ML4VIS from a security perspective (<ref>).





§ BACKGROUND AND RELATED WORK



Addressing security issues in ML-aided visualizations requires a dual set of ML and visualization considerations. 
We describe relevant works related to adversarial attacks and vulnerabilities from ML and visualization research. 
We also discuss a general overview of ML-aided visualizations. 




 §.§ Adversarial Attacks on Machine Learning Models and Defenses



With ML techniques actively used in real-world settings, Dalvi et al. <cit.> posits a critical research agenda that can address the issue of how “the presence of an adversary actively manipulating the data [can] defeat the data miner”.
As deep NNs are rapidly being used in various domains, such as vision and speech recognition, a significant portion of ML research is devoted to addressing security issues in NNs. 
One notable early result is an adversarial example designed for convolutional NN (CNN) models by Szegedy et al. <cit.>.
They demonstrated that an adversarial example can be easily constructed by adding a human-imperceptible perturbation into an input image. 
This perturbation can readily cause image misclassification even with state-of-the-art CNN models. 
These adversarial examples pose critical issues in real-world settings where, for example, an adversary may craft a stop traffic sign that an autonomous car will misclassify as a yield sign <cit.>. Researchers have since studied a multitude of efficient and intense adversarial examples on CNNs.

Adversarial examples on CNNs can be generally categorized as either white-box attacks or black-box attacks depending on the adversary's knowledge about the model of interest.
When an adversary knows detailed information about a target CNN model (i.e., white-box attacks), they can efficiently construct adversarial examples by referring to the gradients of the model's loss function <cit.>.
In contrast, black-box attacks are when the adversary has limited information available on a target model. 
In these cases, the adversary generally has two strategies. First, an adversary can build a substitute model that performs the same classification task as the target model. 
The substitute model can then be used to generate adversarial examples like in white-box attacks <cit.>. 
Another common black-box attack strategy is to reverse engineer a target model from the collected input-output relationships by sending queries to the model <cit.>. 
Research continues to highlight new attack methods that have grown and diversified for other NNs, such as recurrent NNs (RNNs) and graph NNs (GNNs) <cit.>.

Besides crafting adversarial examples, data poisoning <cit.> (i.e., adding malicious training data) is another effective way to corrupt the model as exemplified by the problematic tweets made by Microsoft's chatbot Tay <cit.>.
Data poisoning can also be used to perform backdoor attacks. 
Backdoor attacks can make NN models behave maliciously only when models receive inputs stamped with a trigger crafted by the adversaries <cit.>. 
Multiple surveys <cit.> highlight how backdoor attacks can be easily prepared with various approaches, such as sharing maliciously pre-trained models with others. 
The growing range of these attacks highlights the need for defense strategies.

Defense strategies against adversarial attacks vary but all present shortcomings. 
One straightforward way to protect against white-box attacks is gradient masking, which conceals the gradient information from adversaries <cit.>. However, even after gradient masking, an adversary can perform a black-box attack with a substitute model.
Another defense strategy is to generate more robust NN models through adversarial training (i.e., training using artificially created adversarial examples) <cit.>.
NN models produced with adversarial training are still vulnerable to out-of-sample adversarial examples.
A third popular strategy focuses on input validation and preprocessing, such as applying statistical methods to detect abnormal inputs (e.g., PCA-based detection <cit.>) and data compression to exclude the perturbations (e.g., JPEG compression <cit.>).
However, this strategy is highly domain-specific and is not generalizable across domains <cit.>.
We discussed only three possible defense strategies. 
We refer readers to the taxonomy by Papernot et al. <cit.> and multiple surveys and reports for more details on attack and defense strategies  <cit.>).

Visual analytics approaches have also devoted to investigating the mechanism of adversarial attacks on NNs to establish better defense strategies <cit.>. 
However, there is a research void in studying the impact of adversarial attacks on the generation of visualizations.




 §.§ Vulnerabilities of Visualizations



Visualizations can become obscure, misleading, and even deceptive as a consequence of poorly prepared data <cit.>, problematic visual designs <cit.>, viewers' cognitive bias <cit.>, or any combinations of these.
For example, a visualization with missing value imputations that is not suited for target analysis tasks can decrease viewers' performance <cit.>.
Subtle skews in visualizations, such as 3D effects on pie charts, can lead to wrong conclusions <cit.>.
A viewer's belief in the existence of correlations between two variables (e.g., the numbers of guns and crimes) can also influence the cognition of the correlation strength <cit.>.
The choice of a visual representation in itself can additionally lead to bias in estimating a user's confidence in the presented visual representations <cit.>.
McNutt et al. <cit.> provided a conceptual model that frames these visualization vulnerabilities along the visual analytics pipeline.

By exploiting these visualization vulnerabilities, adversaries can easily create malicious visualizations.
Correll and Heer <cit.> discussed a man-in-the-middle attack on visualizations: an attack from a malicious visualization designer who aims to distract communication between data and viewers.
Adversaries (i.e., designers) can intentionally break visualization conventions <cit.> to create visualizations with problematic designs. 
Our work argues that even when adversaries do not have such a strong capability of manipulating visualization designs, various practical attacks can be imposed on ML-aided visualizations (see <ref> and <ref>).

To the best of our knowledge, no existing work explicitly studied defense strategies against adversarial attacks on visualizations.
Existing  works targeted on detecting flaws in visualizations <cit.>, mitigating cognitive biases <cit.>, and testing the robustness of visualizations <cit.>.
For example, Chen et al. <cit.> developed a linting tool to detect erroneous visual specifications. 
McNutt et al. <cit.> introduced metamorphic testing for visualization, which measures how strongly a change to the input data can perturb a visualization outcome. 
This current state of literature presents security issues that are being overlooked within the visualization community. 




 §.§ Machine Learning-Aided Visualizations

Existing works <cit.> provide comprehensive surveys on ML4VIS related to visual analytics, information visualization, and scientific visualization.
Based on these surveys, the following facts emphasize the importance of needing future studies regarding adversarial attacks on ML-aided visualizations.

    
  Increase of research interest in ML4VIS.
    Research on ML4VIS is rapidly growing. 
    Until 2015, only a few ML4VIS-related papers were published annually. 
    Since then, the number of ML4VIS-related publications radically increased <cit.>.
    This trend indicates an increasing interest in, and need for, using ML for visualization. 
    
    
  Existence of broad and critical attack targets. 
    ML4VIS approach is employed for various tasks, including data processing for visualization, feature extraction, visualization generation, user behavior prediction, and interpretation support of visualized results. 
    Also, ML4VIS approaches are utilized in life-critical applications, such as 3D medical image analysis and reconstruction <cit.> as well as cancer genome analysis <cit.>.
    Because visualization is a fundamental tool to communicate and analyze data, by attacking ML-aided visualizations, adversaries could significantly and negatively impact critical applications in areas such as business, politics, and medicine.
    
    
  Potential exposure to immediate threats.
    Many existing ML-aided visualizations are based on NN-based models, using multilayer perceptrons (MLPs), CNNs, RNNs, GNNs, autoencoders, and/or generative adversarial networks.
    Adversarial attack methods have been developed for all of these NNs <cit.>. 
    Therefore, ML-aided visualizations might be inherently exposed to potential threats.





§ ATTACK SURFACE OF MACHINE LEARNING-AIDED VISUALIZATIONS



We analyze the attack surface (i.e., set of potential entry points for attacks) of ML-aided visualizations.
For this analysis, we use Wang et al.'s ML4VIS pipeline <cit.>, which is derived from an extensive survey on ML4VIS approaches, to review the potential processes, inputs, and outputs involved in ML-aided visualizations.



 §.§ ML4VIS Pipeline



As shown in <ref>, Wang et al.'s ML4VIS pipeline consists of seven processes that can be individually aided by ML: Data Processing4VIS, Data-VIS Mapping, Insight Communication, Style Imitation, VIS Interaction, User Profiling, and VIS Reading.
Note that we consider that visualizations are ML-aided if they utilize ML for any of these processes. 
ML does not have to be involved in all the processes.
For example, a user may utilize ML for data processing but still manually design the visual encodings to produce a visualization. 
Below, we briefly describe each process.

Data Processing4VIS prepares data for the subsequent visualization processes.
A typical ML method used for this process is dimensionality reduction (DR). 
This process can use raw data, existing visualizations, or both as input. 
For example, DR can utilize the previously generated visualization result to perform visually consistent updates <cit.>.
It is worth noting that NN-based DR methods are becoming more actively developed for Data Processing4VIS <cit.>.

Data-VIS Mapping encodes processed data into visualizations by assigning visual marks and channels.
For this process, ML can recommend marks and channels suitable for a given dataset or accelerate the encoding process. 
These enhancements are often achieved by training NNs using supervised learning on labeled training data (e.g., data tables and their suitable visual marks) <cit.>.

Insight Communication aims to produce visualizations that effectively convey insights found in the data.
Wang et al. <cit.> distinguish this process from Data-VIS mapping based on whether the insights are used as inputs.
Because insights are frequently represented with texts, NN-based natural language processing can be applied to this process <cit.>.



Style Imitation reflects a given visualization's style in other visualizations. 
Visual styles include applied color palettes, chart decorations, etc.
Similar to Data-VIS Mapping, this process can be aided by NNs trained with supervised learning <cit.>.

VIS Interaction takes user actions as input and then accordingly updates visualizations by using any of the aforementioned processes.
For this process, ML can aid in interpreting users' intentions as well as refine the interaction results.  
For example, NN-based classification models are used to improve the accuracy and speed of selecting visual elements (e.g., points in 2D scatterplots) <cit.>.

User Profiling is to understand users from their action histories.
ML is applied to predict user behaviors (e.g., next interactions) or user characteristics (e.g., cognitive abilities, analysis goals, and personalities) <cit.>.
With such predictions, visualizations can provide better interaction latency, suggestions for the next actions, and desirable marks and channels for users and their analysis.

VIS Reading is the process of reviewing visualizations and understanding the encoded information. 
Through this process, the goal is to understand the visualized data, extract applied styles, and obtain insights. 
ML can help users automate VIS Reading, instead of relying on manual human inspections. 
For example, NNs can reconstruct input data corresponding to visualized results <cit.>. 

As shown in <ref>, these processes are intertwined via the inputs and outputs that are produced by or used for the other processes.
In addition, as with other visualization pipelines <cit.>, there are likely iterative updates for each process based on newly added data and user actions.
We also want to note that the dataflow of this pipeline only reflects inputs and outputs at the inference phase. 
For the training phase, pre-generated outputs can be used as training data for supervised learning (e.g., Data-VIS mapping can be trained with a set of pairs of data and its visualization) for each process.




 §.§ Characterization of Attack Surfaces



We consider that all employed processes, inputs, and outputs in Wang et al.'s ML4VIS pipeline compose an attack surface given how adversaries may be able to manipulate inputs, corrupt ML/non-ML processes, and tamper with the output results.
Probable attack surfaces across different ML-aided visualizations reveal critical, unique characteristics in ML-aided visualizations:

    
  C1. ML inputs and outputs specific to visualization.
    Though existing ML-aided visualizations usually customize and utilize NNs developed from ML research <cit.>, the inputs used for the training and inference phases are often specific to the visualization (e.g., <cit.>).
    Target outputs and loss functions to produce such visualization outputs can also be unique for ML-aided visualizations.
    For example, ML-aided visualizations often adapt NNs to analyze visualizations based on their visual marks and channels rather than pixel-based images <cit.>.
    Consequently, adversaries might design attack methods for ML-aided visualizations that are significantly different from those studied in the ML field, and existing vulnerability assessment and defense methods for NNs <cit.> might not be as effective for such attacks.
    
    
  C2. Exposure of ML outputs (and inputs) to adversaries.
    Visualizations are usually intended to be reviewed by users. The processes within the ML4VIS pipeline, such as Data-VIS Mapping, Insight Communication, and Style Imitation, all produce visualizations as their outputs.
    Thus, when these processes employ ML, the visualizations are inference results that are abundant with information, which will inherently be observed by users as well as adversaries. 
    When the data generated by Data Processing4VIS is visualized without additional processes (e.g., directly visualizing DR results as 2D scatterplots <cit.>), Data Processing4VIS also suffers from the same issue.
    Moreover, interactive visualizations often support details-on-demand <cit.>, enabling adversaries to access detailed information of raw or processed input data.
    This attack surface characteristic can enable adversaries to gain further knowledge of ML models (i.e., contributing to the reconnaissance stage of cyberattacks <cit.>).
    In contrast, C1 provides new opportunities to create attacks specific to visualization (i.e., contributing to the weaponization stage <cit.>).
    
    
  C3. Long, complex pipeline with interdependent processes.
    ML-aided visualizations may involve a large number of interdependent processes. 
    This interdependency introduces additional opportunities for adversaries to amplify a cascade of attacks throughout the pipeline.
    For instance, adversaries may be able to create slight influences on Data Processing4VIS so that the subsequent processes will cause critical issues (see <ref> for a concrete example).
    This cascade of attacks can be even amplified when feedback loops are involved <cit.>.
    Furthermore, the ML4VIS approach inherits each respective attack surface that exists in ML <cit.> and visualization <cit.> pipelines, resulting in an even larger attack surface. 
    This attack surface indicates that ML-aided visualizations might be exposed to more potential threats and could be more complex to defend than cases that only use ML or visualization.
    
    
  C4. Active involvement of users in the ML processes.
    Users are actively involved with ML-aided visualizations. They interpret (i.e., VIS Reading) and interactively analyze (i.e., VIS Interaction and User Profiling) the visual content.
    This user involvement provides adversaries opportunities to manipulate users to project attacks or target users as their attack objectives.
    For example, by exploiting the information memorized in NNs, adversaries might reveal users' private information, such as their cognitive ability <cit.>.

    
  C5. Threats on human judgment.
    Human intervention is vital to avoid critical harm from actions instigated by ML outputs <cit.>. 
    For example, even if an autonomous car misclassifies a stop sign as a yield sign from an attack, a human driver could still hit the brakes <cit.>.
    However, with ML4VIS, adversaries may intentionally attack visualizations that require human judgment, making them deceptive and thereby creating situations where human intervention is not as readily feasible (<ref> demonstrates such situations).
    Although closely related to C4, we separately list this attack surface characteristic. Effective human intervention would be the final defense to protect users from potential harm.







§ CONCRETE ADVERSARIAL ATTACK EXAMPLES



To showcase possible threats while highlighting the uniqueness of the attack surface, we designed attacks on state-of-the-art ML-aided visualizations <cit.>.
A summary is presented in <ref>. 
Across the attacks, we make different assumptions that cover different levels of the adversaries' capabilities (e.g., black-box attack vs. white-box attack).
The source code used for our attacks is available online <cit.>.

Our attacks are on two representative methods that focus on different parts of the ML4VIS pipeline: (1) parametric UMAP (PUMAP) <cit.> for Data Processing4VIS and (2) MultiVision <cit.> for Data-VIS Mapping.
These methods are selected for three reasons.
First, we aim to cover multiple different processes in the ML4VIS pipeline so we can highlight all of the identified attack surface characteristics (C1–5 in <ref>).
Second, we consider the research influence of these two processes.
Data Processing4VIS is the root process of the ML4VIS pipeline.
As a result, attacks on this process can trickle a cascade of attacks on the subsequent processes.
On the other hand, Data-VIS Mapping is one of the most frequently studied ML4VIS processes <cit.>.
Lastly, after our extensive search, we found only a limited portion of the published works provide publicly available, executable source codes and datasets <cit.>.
These source codes and datasets are necessary to precisely replicate their ML models.
Consequently, we cover only a few processes in the ML4VIS pipeline—designing attacks on the other processes remains as future work. 




 §.§ Attack Target 1: Parametric DR for Data Processing4VIS


We first provide the background of the attack target.
Nonlinear DR methods, such as t-SNE and UMAP, are commonly used for visual analysis of high-dimensional data <cit.>.
Using NNs, parametric extensions of nonlinear DR methods have been developed (e.g., parametric t-SNE <cit.> and PUMAP <cit.>).
Unlike its non-parametric counterpart, the parametric method produces a parametric mapping that projects data instances onto a low-dimensional space. 

<ref> compares pipelines employed by UMAP and PUMAP.
Conventional UMAP first constructs a graph representation of the input high-dimensional data.
The subsequent step is an iterative optimization that does not involve NNs. 
Using this optimization, UMAP layouts the instances onto a low-dimensional space (often in 2D) so that instances similar in the graph representation are spatially proximate (refer to <cit.> for details).
There is no direct, numerical connection between high-dimensional data and its low-dimensional representation. 
Thus, UMAP does not provide parametric mapping.
In contrast, PUMAP feeds high-dimensional data to the MLP's input layer, learns the hidden layers' neuron weights for parametric mapping, and produces low-dimensional coordinates from the output layer.
PUMAP only uses the graph representation for a loss function during the training phase.
At the inference phase, PUMAP can directly project new inputs onto the low-dimensional space by using the trained neuron weights. 



This parametric mapping ability is useful for visual analysis, such as when analyzing streaming data.
Since the projection can be performed without rerunning DR methods, parametric DR is computationally efficient and provides visually stable results (e.g., avoiding the arbitrary rotation caused for each update <cit.>).
Parametric DR is actively studied due to this ability and its convenience of NN-based optimizations <cit.>.
Also, similar to conventional DR methods, parametric DR can be widely used for visualizations.
Therefore, investigating the vulnerabilities of parametric DR methods is critical to ensure security for ML-aided visualizations.




  §.§.§ Attack Manipulating One Attribute Value



Attack design.  For our first attack, using PUMAP as an example, we demonstrate a rudimentary attack that can be performed even with limited adversarial capability. 
This basic example provides evidence of how NN-based parametric DRs are susceptible to maliciously crafted inputs.
We assume that PUMAP is trained with the default setting as introduced by Sainburg et al. <cit.> (e.g., using an MLP of three 100-neuron hidden layers with rectified linear activation functions, or ReLUs) and that PUMAP outputs are visualized as scatterplots.
In addition, we assume the following: adversaries only have the capability to observe their inputs into the trained PUMAP and the corresponding scatterplots; their goal is to produce misleading scatterplots.

From the assumptions above, we designed a one-attribute attack: a black-box attack that manipulates one attribute value of an input instance.
The one-attribute attack is designed based on our observations that PUMAP tends to have highly influential attributes on a learned parametric mapping.
The one-attribute attack consists of two steps.
First, for any given input instance, we observe the change in its low-dimensional coordinate 
before and after adding a small perturbation for each attribute.
We add a small perturbation to hide suspicious behaviors before performing the main attacks.
Although this step is an exhaustive search, we only need to feed (d+1) instances at minimum, where d is the number of attributes.
For the second step, we create an adversarial input by adding a relatively large perturbation to the most influential attribute. 
By doing so, we can locate the instance in an arbitrary coordinate in the low-dimensional space.


Attack results and analyses.  We provide a demonstrative example of the one-attribute attack, using the Wine dataset <cit.>, which consists of 178 instances, 13 attributes, and 3 cultivar labels.
Since PUMAP is an unsupervised learning method, we use data labels only for the visual encodings to better convey our analysis results.
We first normalize the dataset and then use it to train PUMAP.
<ref>-a shows the resulting 2D scatterplot, where circles correspond to the 178 instances projected by the trained PUMAP.

Based on the one-attribute attack, flavanoids is identified as the most influential attribute. 
For presentation purposes, we select one benign input from Cultivar 3 and then craft an adversarial input by adding a value of 10 into the benign input's flavanoids.
We then project both the benign and adversarial inputs onto <ref>-a.
The adversarial input is positioned near Cultivar 1 instead of Cultivar 3. 

<ref>-b shows a histogram of flavanoids for each cultivar, where  are approximately placed from right to left in order. 
From the histograms, we expect that flavanoids is useful for PUMAP to decide the low-dimensional coordinate of each instance (e.g., the higher value of flavanoids, the more characteristics of Cultivar 1).
However, as seen in <ref>-b, although the adversarial input has an extremely high value of flavanoids, the trained PUMAP does not show it as an outlier.
In <ref>-c, we further investigate the influence of the perturbation strength by incrementing flavanoids from 0 to 15 in step 1. 
As the perturbation strength increases, the adversarial input moves from the top right and passes by all three cultivars. 
By simply changing one attribute, this result highlights how adversaries can manipulate a user's visual perception of which cluster the input instance belongs (e.g., Cultivar 2 when adding 5) while not exposing any extreme or outlying characteristics of the adversarial input.
One can consider that the one-attribute attack is analogous to the one-pixel attack designed for image datasets <cit.>.
Although we performed the one-attribute attack on the dataset with 13 attributes, this example is mainly intended to provide a concise demonstration. In reality, the attack may occur for datasets with numerous attributes (e.g., 1,000 attributes).
In such situations, similar to finding the one abnormal pixel in an image, the identification of the one-attribute attack can be non-trivial when humans solely rely on manual or visual inspection of each attribute's data distribution.



We suspect that the observed phenomenon above—the projected coordinate can be controlled even by changing only one attribute value—occurs due to the “linearity” of modern NNs. 
Modern NNs utilize close-to-linear activation functions (e.g., ReLU <cit.>) and linear multiplications of neuron inputs and weights.
Goodfellow et al. <cit.> also hypothesized this linearity characteristic can explain the success of adversarial examples on CNNs. 
As a result, for NN-based parametric DR, some of the attributes could almost be linearly mapped onto one direction of low-dimensional coordinates, as shown in .
The supplementary material <cit.> exhibits close-to-linear mappings even for different datasets and activation functions.
This inherent issue in NN-based parametric DR should be addressed for more secure use. 




  §.§.§ Attack Using a Substitute Model



Attack design.  When adversaries can observe various combinations of inputs and the corresponding outputs for parametric DR, they can construct a substitute model that can craft adversarial inputs flexibly and effectively.
Interactive visualization systems showing DR results have a high likelihood of this situation occurring. 
These systems often provide training data information on demand to allow users to examine the DR results <cit.>.
Therefore, we assume that it is possible and reasonable that an adversary's goal is to construct a substitute model to the attack target model and generate deceptive visualizations.

<ref> shows our architecture for a substitute model attack.
The substitute model learns the parameters that produce a nearly identical low-dimensional representation to the one produced by the attack target model. 
This learning is achieved by setting a loss function that minimizes the positional differences (e.g., the sum of pairwise Euclidean distances) between the instances in the low-dimensional representations that are produced by the attack target and substitute model.
To construct this substitute model, adversaries do not need to know how the attack target model's parametric mapping is learned (e.g., the use of PUMAP).
In addition, the substitute model's NN architecture and implementation do not have to be the same as the attack target model's. 
Adversaries need to only have sufficient neurons/parameters to mimic the attack target's parametric mapping. 
Since adversaries can access all information of the substitute model (e.g., gradients), they can efficiently craft adversarial inputs so that the inputs are projected at specific positions in the substitute model's low-dimensional representation. 
Then, adversaries can feed the crafted adversarial inputs to the attack target model, where the inputs would be projected closely to the aimed positions.


Attack results and analyses.  We demonstrate attacks on the same ML model as <ref> (i.e., PUMAP trained with the Wine dataset). 
We construct our substitute model with an MLP of three 50-neuron hidden layers, using PyTorch. 
Note: PUMAP employs TensorFlow and an MLP of three 100-neuron hidden layers.

We first showcase how to create adversarial inputs.
As shown in <ref>-a1, we can create an adversarial input that is projected onto a specific position (e.g., (-2.5, -2.5)) with the substitute model.
The adversarial inputs can be optimized by adjusting the input's attribute values based on their gradients. 
The gradients are related to the error between the aimed and projected positions.
When we feed the crafted adversarial input to PUMAP, as shown in <ref>-a2, the adversarial input is placed close to the aimed position.
Due to the linearity of NNs, we expect that similar placement could be achieved without manipulating many attributes. 
For example, two attributes would be enough given how this example's low-dimensional space is in 2D.
To validate this hypothesis, using one arbitrarily selected benign input (in <ref>, one instance from ), we perform this same creation procedure while only manipulating two attribute values of the benign input.
Examples of the produced adversarial inputs are shown in <ref>-b and c.
We can see that the adversarial inputs are placed closely to the aimed position in both the substitute model and PUMAP results.





This analysis exposes two critical vulnerabilities for PUMAP as well as for other NN-based parametric DR methods.
First, it can be easy for adversaries to construct a substitute model that has an almost identical parametric mapping to the attack target model's. 
As such, a substitute model can effectively create adversarial inputs without necessarily involving the attack target model.
Second, as discussed in <ref>, NN-based parametric DR methods are likely to have strong linearity.
By exploiting this linearity characteristic, adversaries can aim to place malicious inputs to desired low-dimensional coordinates by manipulating only a few attributes. 
In the supplementary material <cit.>, we exhibit that this same issue occurs even for three other cases: a PUMAP with a smaller NN, a parametric t-SNE, and different datasets (including a dataset for cancer detection as a life-critical example). 



From these analysis insights, we can induce that visualizations can be easily obscured, misleading, or deceptive from crafted adversarial inputs.
For example, as shown in <ref>-a, adversaries can overwrite the existing visual cluster corresponding to Cultivar 2.
At the VIS Interaction step, if users aim to identify and understand the clusters from the tainted scatterplot and histograms <cit.>, they would fundamentally misunderstand the characteristics of the clusters. 
In <ref>-b, a majority of the manipulated cluster (i.e., Cultivar 2) tends to have a higher value of flavanoids than Cultivar 1. 
In contrast, flavanoids values for the original cluster for Cultivar 2 are mostly in-between Cultivar 1 and Cultivar 3 (see <ref>-b).
Also, if the Data-VIS Mapping process adjusts the visualization axes based on the value range, adversaries can craft an adversarial input that is projected to be an outlier, as shown in <ref>-c.
This radical change of axis scales influences the visual space used for the main region to be much smaller, and thereby affecting the VIS Reading process to disturb observing the main region.
Adversaries may utilize this situation to hide subsequent attacks influencing the main region.




 §.§ Attack Target 2: Chart Recommendation for Data-VIS Mapping


Chart recommendation is a canonical use of ML for Data-VIS Mapping.
Given input data, chart recommendation systems offer suggestions and generate the appropriate charts.
By using large datasets of collected charts (e.g., <cit.>), researchers applied supervised learning to NNs and developed various chart recommendation systems <cit.>.




We use MultiVision <cit.> as a representative example of chart recommendation systems.
Given a data table, MultiVision is designed to rank and recommend multiple charts.
See <ref> for an overview of MultiVision's pipeline for chart recommendation.
The following procedure is applied to both the training and inference phases: 

    
  * given an input data table, generate sets of data columns for 2D chart generation (e.g., selecting two columns/attributes that would be represented as the scatterplot's x- and y-axes);
    
  * extract various features for each column, such as a word embedding corresponding to the column name, data type (e.g., quantitative, nominal, and temporal), and statistics (e.g., the ratio of negative values, standard deviation, and skewness);
    
  * score the sets of columns by feeding the corresponding features to an NN using bidirectional long short-term memory (LSTM); 
    
  * select an appropriate chart type (e.g., bar, line, or pie) for each set of columns by using another NN that jointly employs a bidirectional LSTM and an MLP; 
    
  * based on the score from the step 3 and the confidence score of the chart type selection from the step 4, rank the recommendation level of each output chart;
    
  * visualize the top-recommended charts with Vega-Lite <cit.>.



 While chart recommendation systems are useful to help reduce the burden of creating charts for analysts, adversarial attacks can make systems recommend meaningless or deceptive charts and hide important information from the analysts. 
Existing NN-based chart recommendations <cit.> share similar approaches to MultiVision by extracting data table features as ML inputs and using RNN-related models to interpret the relationships among the set of data columns.  




  §.§.§ Knowledge-Based Attack



Attack design.  We perform a simple, but critical attack on Wu et al.'s pre-trained MultiVision <cit.>.
Here, we assume that adversaries know or can guess some basic speci­fications related to the employed ML model.
This assumption fits into cases when adversaries can recognize that the recommendation system uses a similar approach to MultiVision.
Even if adversaries partially know the MultiVision procedure described above, they can strategically craft adversarial inputs. 
For example, to induce misprediction, they can influence data features by manipulating column names or values. 
They can even apply very subtle manipulations to make attacks difficult to notice without a close examination.
To guess the important features for the recommendation, we perform a trial-and-error process to find a subtle change that causes invalid recommendations.


Attack results and analyses.  We show attack results using the Gapminder dataset <cit.>, which the original authors of MultiVision used for their user study.
We use this dataset to ensure that MultiVision works in the intended manner. 
This dataset consists of 693 instances and 6 attributes with no missing values. 
In the supplementary materials <cit.>, we provide the attack results for another dataset.



<ref>-a shows the top 2 recommended charts by MultiVision before our attack. 
MultiVision suggests reviewing the relationships between life_expect (life expectancy) and other attributes (e.g., fertility) throughout the years. 
From our trials, we decided to generate an adversarial input by replacing one randomly selected value of life_expect with a blank space (similar to a missing value).
This manipulation leads to the results shown in <ref>-b.
Though life_expect is still selected as an important attribute, the resultant charts now do not adequately convey the relationships between life_expect and other attributes. 
This effect is likely because life_expect is categorized as a nominal attribute due to the blank space we added. 
Nonetheless, a nominal version of life_expect is not helpful in understanding the data, and 
MultiVision's susceptibility highlights how it should have selected other attributes for the recommendation charts.




  §.§.§ Attack Referring to Gradients



Attack design.  When the chart recommendation system is a white box, adversaries can craft adversarial inputs more efficiently.
MultiVision employs two different NNs to recommend column sets and chart types respectively.
We refer to the gradients for column features toward decreasing the scores corresponding to the highest-ranked chart. 
Then, we manipulate an input data table to change the column features that have large gradient magnitudes.






Attack results and analyses.  We craft an adversarial input for the Gapminder dataset.
As shown in <ref>-a1, MultiVision originally recommended 
life_expect (x-axis), fertility (y-axis), and year (line width) as the column set; line chart as the chart type.
By accessing the NNs used in MultiVision, we extract the aforementioned gradients to disturb this line chart recommendation. 

<ref> shows the gradients of the NNs and we list only three out of 96 features used by MultiVision.
We observe that  (column index divided by the number of columns) has significantly large gradient magnitudes for both column set and chart type recommendations.
This value indicates that switching the column order introduces a high probability of changing the overall recommendation.
From this observation, we shuffle the order of columns to attack MultiVision, resulting in the recommendation shown in <ref>.
As expected, the recommended charts are radically different from this attack, and the chart no longer seems to be able to support analyzing the dataset.

Bidirectional LSTM considers both forward and backward orders of inputs and theoretically should mitigate the influence of the order of columns. 
However, this attack results clearly highlight that MultiVision is still vulnerable to changes in the column order.
This result draws similar implications to various chart recommendation systems <cit.> as they also rely on RNN-related models. 
Therefore, the possibility of this vulnerability may be common across these systems.




  §.§.§ Attack Propagating across Multiple Processes



Attack design.  This attack focuses on creating adversarial examples for visualizations that utilize NNs for multiple ML4VIS processes.
We consider a system that first performs Data Processing4VIS using PUMAP to derive 2D features from the raw data. 
Then, the subsequent Data-VIS Mapping process employs MultiVision to select the appropriate chart to visualize the 2D features.
We assume that when attacking the target system, adversaries can add new instances to the raw data but cannot directly change a data table input used in MultiVision.
We also assume that adversaries are capable of accessing a pre-trained MultiVision model (e.g., the pre-trained model available from an online repository).
With these two capabilities, adversaries aim to induce misprediction for the chart recommendation.

Similar to <ref>, we can refer to the gradients of the pre-trained MultiVision models. 
However, when considering the first capability of adversaries, we can only indirectly influence attribute values of the input data table. 
For example, we cannot change the column order and column names.
When MultiVision extracts the column features, attribute values (in our case, 2D features from PUMAP) are converted into statistics that are interdependent from each other (e.g., the ratios of negative values and skewness).
Consequently, it is not trivial to associate these statistics' gradients with the data table's attributes. 
Instead, we generate various 2D features with multiple different magnitudes of values and select one that changes the ranks of the chart type or column set scores.
Then, to find an adversarial instance that PUMAP transforms to the selected 2D feature, we build and exploit a substitute model of PUMAP by taking the same approach as <ref>.


Attack results and analyses.  We demonstrate an attack that aims to change MultiVision's recommendation by influencing PUMAP's inference.
<ref>-a shows MultiVision's top-recommended chart for PUMAP's 2D feature outputs extracted from the Wine dataset.
The recommended chart reasonably visualizes the distribution of features. 
Using the attack design described above, we create and add one adversarial input that is processed as negative 2D feature values with large magnitudes.
Feeding this input to the attack target generates the recommendation shown in <ref>-b. 
In this case, the chart is less suitable for visualizing the distribution of 2D features when compared with <ref>-a.
This result highlights how MultiVision can drastically change the chart recommendation even with one additional outlier. 





§ DISCUSSION: TOWARD SECURE MACHINE LEARNING-AIDED VISUALIZATION



We now discuss the implication of the attacks demonstrated in <ref> as well as highlight potential threats in the real world. 
We then discuss the insights we gained by studying these concrete attacks.
Lastly, we provide a set of suggestions to help further investigate the vulnerabilities of ML-aided visualizations to move toward a more secure use.



 §.§ Discussion on the Performed Attack Examples


Possible critical scenarios in the real world.  Although we performed the attacks using demonstrative datasets and ML-aided visualizations, these attacks can be easily applied to various real-life scenarios.

Adversaries can directly influence NN models used in various domains.
For example, in the case where NN-based parametric DR is used to monitor a product in real time <cit.>, adversaries might notice that the product weight has an influence on the projection. 
This attack would result similarly to the one-attribute attack example.
Then, they might physically control the weight to hide any other abnormal product status.
They may also intentionally cause problematic x, y-axes scaling in visualizations to conceal any subsequent attacks from analysts.
Another possible scenario is how these adversarial attacks can have political influence. 
For example, parametric DR can be employed to analyze political opinions, such as sentiment analysis in social media posts <cit.>. 
Adversaries could find that the projection can be controlled by a post's length and overwrite existing visual clusters by sending many posts of different lengths to hide important opinion differences.

Adversaries can also attack chart recommendations in practical scenarios. 
When a company analyzes its web customer data using chart recommendation services, adversaries (disguising as customers) can make their own data with malicious values to cause problematic recommendations.
As another example, since diagnosing NN models themselves requires analyses from various aspects <cit.>, a company may utilize a chart recommendation service to efficiently detect adversarial attacks on their NNs (i.e., utilizing NN-based chart recommendation to analyze the other NNs).
If a chart recommendation service has vulnerabilities, adversaries can exploit them to hide their attacks on the company's NN model, using a similar approach to the attack propagating across multiple processes we performed.

Need of defense strategies for the identified vulnerabilities.  We identified clear vulnerabilities of PUMAP and MultiVision. 
These vulnerabilities are likely to exist in other NN-based parametric DR methods and chart recommendation systems.
The design and success of the performed attacks are highly related to the unique characteristics of the ML4VIS attack surface, as described in <ref>.

The attacks on PUMAP utilize ML inputs and outputs exposed by the visualization, exploit the linearity of NNs, and demonstrate cases that can potentially cause user misjudgments.
Developing defense strategies against adversaries requires more research effort. 
For example, concealing the inputs and outputs may reduce the analysis capability or the interpretability of ML-aided visualizations, requiring developers to find a good balance between preserving security and supporting analyses.
The linearity of NNs can be mitigated by employing other activation functions such as the sigmoid. 
However, even these functions have a region that is close to being linear. 
As a result, these functions alone cannot significantly resolve the issues related to the linearity of NNs (see <cit.>).  
Through these insights, we may need to design new activation functions to device secure parametric DR methods.

One of the root causes of the success of the attacks on MultiVision could be a mismatch between the employed models (i.e., bidirectional LSTMs) and visualization-specific inputs (i.e., data tables).
Though we want to capture the relationship among table columns, we do not want to place importance on the order of columns when applying ML to data tables.
Although the influence of the column order change can be avoided by shuffling the order during the training phase, a new NN architecture suitable for data tables should essentially be developed.

Also, the attack propagating across multiple ML processes exemplifies a significant security vulnerability for long, complex ML4VIS pipelines.
This attack highlights that not only do we need to investigate each of the ML4VIS processes, but also study how to defend an ML-aided visualization as a whole system.




 §.§ Suggestions for Future Research



Enhance studies on attacks specific to ML-aided visualizations.  To develop defense strategies, we first need to analyze possible attacks that exploit the vulnerabilities of ML-aided visualizations.
We expect that various attacks can be specific to ML-aided visualizations based on our observations on their attack surface (<ref>). 
As a primary step toward a better understanding of ML4VIS' vulnerability, this work demonstrates several critical attacks.
As discussed in <ref>, only a limited work makes their source code, training and testing datasets, and pre-trained ML models available for the public <cit.>.
In contrast, in the ML field, they are often publicly available given how they are vital to efficiently and accurately study limitations of ML models <cit.>. 
Thus, we encourage more visualization researchers to make efforts to provide full accounts of data and software publicly, and thereby enable further analyses of ML-aided visualizations.
Also, this immaturity of security studies in ML4VIS indicates our study's limitation. 
The current characterization of the attack surface is based on an abstract-level analysis and may not reflect a variety of possible real attacks (since they are yet unseen).

Identify and inform potential vulnerabilities.  Furthermore, we suggest that researchers routinely identify and publicly inform potential vulnerabilities in their ML-aided visualizations. 
Authors have intimate knowledge of their methods (e.g., designs, algorithms, and datasets).
For example, adding discussions on the vulnerabilities in the respective publications would largely benefit risk assessments.
If we believe the developed visualizations provide significant value (e.g., having a large number of users and deriving highly valuable knowledge) <cit.>, these discussions are crucial in light of potential threats.
Although discussing the vulnerabilities involves a risk of distributing information about attack entries, unsolved critical issues should be reported before the research results are applied to practical applications.
In addition to these individual efforts, there is a growing need to develop methods that systematically evaluate vulnerabilities <cit.>, which would reduce the need for time-consuming manual inspections.

Investigate the role of human intervention.  Lastly, we pose another two open questions: Should we utilize human intervention for detecting adversarial attacks? If so, how?
For pure ML models, when inputs and the corresponding ML predictions have a human-noticeable mismatch (e.g., when a stop sign is recognized as a yield sign), human intervention is useful to avoid harm caused by this mismatch.
However, for ML-aided visualizations, adversaries can aim to create deceptive visualized results to make human intervention unreliable or impossible.
One potential approach to still use human intervention is to provide multiple visualizations that exhibit different responses to the changes in input data. 
For example, for the adversarial inputs crafted in <ref>, if we constantly show both the scatterplot and histogram (e.g., <ref>-a and b), a user would detect the beginning of attacks from the histogram.
Similarly, visual supports designed for the detection of biases in ML model outputs <cit.> may be useful.
However, approaches that rely on additional visualizations can increase the cognitive load of users, which would be especially problematic when visualizations are used for real-time monitoring purposes.
Further research is required to answer many of these fundamental questions and provide clear and comprehensive guidelines for developers and users.



§ CONCLUSION


We systematically reviewed the vulnerabilities of ML-aided visualizations. 
We described the uniqueness of the attack surface of ML-aided visualizations and demonstrated security risks with five concrete examples of adversarial attacks.
Our results show that more research efforts are needed to address security aspects and advance defense strategies against these adversarial attacks. 

This work also suggests several future research directions, such as investigating diverse adversarial attacks, systematically testing to evaluate the robustness of ML-aided visualizations, and evaluating the human role in defense against adversarial attacks.
In addition to pursuing these directions, we suggest future research to study the interrelationships between security and other closely related topics, such as privacy preservation and trust building <cit.> in ML-aided visualizations.
This work contributes as a stepping stone toward a holistic study for both maximizing benefits and minimizing risks in the use of ML for visualizations.




Supplementary information

We provide the supplementary materials online <cit.>.
The materials include the source code for the adversarial attack examples in <ref>; additional experiments related to the attack examples; full-size figures and tables of the attack results; and a list of publications that provide publicly available source code.


Acknowledgments

The authors wish to thank S. Sandra Bae and Daniel Jönsson for their assistance in improving the clarity of the paper's content.
This work has been supported in part by the Knut and Alice Wallenberg Foundation through grant KAW 2019.0024 and the ELLIIT environment for strategic research in Sweden. 





































