Bypassing DARCY Defense: Indistinguishable Universal Adversarial Triggers
Zuquan Peng Yuanyuan He Jianbing Ni Ben Niu

DasAtom: A Divide-and-Shuttle Atom Approach to Quantum Circuit Transformation
    Yunqi Huang, Dingchao Gao, Shenggang Ying, and Sanjiang Li^*
        
Yunqi Huang and Sanjiang Li are with Centre for Quantum Software and Information (QSI), Faculty of Engineering and Information Technology, University of Technology Sydney, NSW 2007, Australia. Dingchao Gao and Shenggang Ying are with Institute of Software, Chinese Academy of Sciences, and University of Chinese Academy of Sciences, Beijing, China
Corresponding author (E-mail: Sanjiang.Li@uts.edu.au)

    September 5, 2024
============================================================================================================================================================================================================================================================================================================================================================================================================================================================================================

§ ABSTRACT 

Neural networks (NN) classification models for Natural Language Processing (NLP) are vulnerable to the Universal Adversarial Triggers (UAT) attack that triggers a model to produce a specific prediction for any input.
DARCY borrows the "honeypot" concept to bait multiple trapdoors, for effectively detecting the adversarial examples generated by UAT.
Unfortunately, we find a new UAT generation method, called IndisUAT, which produces triggers (i.e., tokens) and uses them to craft the adversarial examples whose feature distribution is indistinguishable from that of the benign examples in a randomly-chosen category at the detection layer of DARCY.
The produced adversarial examples incur the maximal loss of predicting results in the DARCY-protected models.
Meanwhile, the produced triggers are effective in black-box models for text generation, text inference, and reading comprehension.
Finally, the evaluation results under NN models for NLP tasks indicate that the IndisUAT method can effectively circumvent DARCY and penetrate other defenses. 

For example, IndisUAT can reduce the true positive rate of DARCY's detection at least 40.8% and 90.6%, and drop the accuracy at least 33.3% and 51.6% in the RNN and CNN models, respectively.
IndisUAT reduces the accuracy of the BERT's adversarial defense model by at least 34.0%, and makes the GPT-2 language model to spew racist outputs even when conditioned on non-racial context.









§ INTRODUCTION

Textual Neural Networks (NN) classification models used in Natural Language Processing (NLP) are vulnerable to be fooled and forced to output specific results for any input by attackers with adversarial examples carefully crafted by perturbing original texts <cit.>.
It is noticeable that adversarial examples have successfully cheated the NN classification models in a large number of applications, such as fake news detection <cit.>, sentiment analysis <cit.>, and spam detection <cit.>.

The early methods of adversarial example generation are instance-based search methods, which search adversarial examples for specific inputs, but they can be easily identified by spelling detection and semantic analysis.
The current methods mainly rely on learning models that learn and generate adversarial examples for various unknown discrete textual inputs, e.g.,
HotFlip <cit.>, Universal Adversarial Triggers (UAT) <cit.>, and MALCOM <cit.>.
The learning-based methods are attractive, since 1 they have high attack success rates and low computational overhead; 2 they are highly transferable from white-box models to black-box models, even if they have different tokenizations and architectures; and 3 they are usually effective to fool other models, e.g., reading comprehension and conditional text generation models.
UAT <cit.>, as one of powerful learning-based attacks, can drop the accuracy of the text inference model from 89.94% to near zero by simply adding short trigger sequences (i.e., a token or a sequence of tokens) chosen from a vocabulary into the original examples. Besides, the adversarial examples generated by UAT for a Char-based reading comprehension model are also effective in fooling an ELMO-based model.

To defend against UAT attacks, DARCY <cit.> has been firstly proposed. 
It artfully uses the "honeypot" concept and searches and injects multiple trapdoors (i.e., words) into a textual NN for minimizing the Negative Log-Likelihood (NLL) loss. A binary detector is trained for identifying UAT adversarial examples from the examples by using the binary NLL loss.


Therefore, adversarial examples can be detected when the features of the adversarial examples match the signatures of the detection layer where the trapdoors are located.




The literature <cit.> introduced two methods to attack DARCY.
The first one sorts triggers and uses the l+1-th trigger instead of top-l (l=20) triggers to construct an adversarial example, which prevents the detection of DARCY on a couple of trapdoors.
The second method uses the trapdoor information estimated by a reverse engineering approach to construct an alternative detection model, and carefully generates triggers that can circumvent the detection.
However, both methods activate the detection layer of DARCY and fail to circumvent DARCY that injects a normal number of trapdoors, e.g., more than 5 trapdoors.













In this paper, we design a novel UAT generation method, named Indistinguishable UAT (IndisUAT).
The IndisUAT attack is a black-box and un-targeted attack
that can effectively circumvent DARCY's detection.



The tokens (i.e., words, sub-words, or characters) in the trigger sequences are updated iteratively to search the trigger sequences whose signatures are mismatched with the trapdoors' signatures, so that the trigger sequences do not activate the detection layer of DACRY where the trapdoors are located. Meanwhile, the searched trigger sequences increase the probability that the prediction results stay away from the ground truth.
Fig. <ref> shows an example of IndisUAT.
IndisUAT has the following distinguished features:



  * IndisUAT effectively circumvents DARCY, since IndisUAT estimates the feature distribution of benign examples in the view of DARCY's detection layer, and produces adversarial examples to match the feature distribution estimates.





  * IndisUAT generate adversarial examples that incur the maximal loss of predicting results in the DARCY-protected models, so that the success rate of the IndisUAT attack is high.



  * Extensive experiments show that IndisUAT drops the true positive rate of DARCY's detection at least 40.8% and 90.6%, and drops the accuracy at least 33.3% and 51.6% in RNN and CNN models, respectively;
IndisUAT works for both CNN and BERT models defended by adversarial methods, as IndisUAT results in the decrease of the accuracy at least 27.5% and 34.0%, respectively;
IndisUAT can be migrated from the classification to other NLP tasks (e.g., text generation and QA question answering).
 
The IndisUAT code will be available after this paper is published.




§ BACKGROUND



 §.§ Related work






 
Adversarial Attacks in NLP.
The concept of adversarial examples was first introduced by <cit.>. Later, <cit.> found that even minor perturbations of target answers can have negative impacts on reading comprehension tasks. Thus, many generation methods of adversarial examples were proposed for different attack levels (i.e., character-level, word-level, sentence-level, and multi-level) and in different models (e.g., DNN models and pre-trained models).
For example, Textfooler <cit.> in BERT <cit.> and TextBugger <cit.> for multi-level attacks can significantly change the outputs of DNN models.

However, these methods are not universal (input-agnostic), which means that they have poor transferability.






To improve the transferability, <cit.> propose the UAT attack that is an universal attack method for many NLP tasks such as text classification, reading comprehension, text generation, and text inference.
The UAT attack is independent of the victim classification models and the position of triggers, and it only needs original data and a model that has similar effects on a victim classification model to generate word-level and character-level triggers.
Thus, the UAT attack is highly transferable and resource-efficient. Subsequently, <cit.> added a semantic information processing step during the UAT generation to make UAT more consistent with the natural English phrases.
However, the UAT attacks can effective be detected by DARCY.







Defenses Against Adversarial Attacks in NLP.
Many defense methods <cit.> have been proposed to prevent adversarial attacks by adding noisy words into inputs of models in NLP. The amount of the added noisy data determines the robustness of the trained models. However, if too much noise data is injected into the inputs, the output of the model is discovered to get worse. Subsequently, adversarial training
methods <cit.> add noises into the embedding layer of a model instead of the inputs and do not need the injection of extra adversarial examples. They maximize the disruption to the embedding layer and minimize the corresponding loss by the addition of the noises during the training process.
Thus, the adversarial training methods can avoid the over-fitting issue and improve the generalization performance of the model.
Unfortunately, they usually fail to protect the models against pervasive UAT attacks.

<cit.> recently proposed DARCY, an defense method that first traps UAT and protects text classification models against UAT attacks.
DARCY artfully introduces the honeypot concept and uses a backdoor poisoning method to generate trapdoors. The trapdoors are mixed with original data and trained together to get a detector model that can capture adversarial examples. DARCY is currently the most effective defense method against UAT attacks.



 §.§ Analysis of DARCY's detection

The detection performance of DARCY is outstanding due to the following reasons:
1 the pertinent adversarial examples drop into trapdoors and activate a trapdoor when the feature of the adversarial example matches the signature of the trapdoor, so that the adversarial examples can be captured;


2 the signature of each trapdoor is different from that of benign examples in the target category, and the signatures are also different between trapdoors to guarantee a low false-positive rate and the effectiveness of trapdoors; and
3 the detector is built from a single network, and its detection rate increases with the number of trapdoors.





In IndisUAT, the features of the trigger-crafted adversarial examples are similar to those of the benign examples. Therefore, these adversarial examples do not activate the trapdoors located on the DARCY's detection layer. At the same time, the adversarial examples for a randomly-chosen target class are far away from the original ground truth and close to the target class, so as to achieve the purpose of the attack.




§ INDISTINGUISHABLE UAT



 §.§ Detection Layer Estimation

The IndisUAT attacker can perform the following steps to estimate the distribution of outputs corresponding to benign examples on the detection layer of DARCY.







 
 
(1) Randomly select the candidate examples from the benign examples detected by DARCY to form a set, i.e., D_f^L, where L is the randomly-chosen target class. For each example-label pair (x_i, y_i) ∈ D_f^L, example x_i ∉D^L and label y_i ∉ L, where |D^L_f|=N, D^L is a dataset belonging to L.

(2) Feed the chosen data  D^L_f into ℱ_g, where ℱ_g is the binary detector trained in Sec. <ref>.


(3) Estimate the feature distribution of the outputs on the detection layer for benign examples that do not belong to the class L, i.e., ℱ_g^tgt∼ [E[ℱ_g(x_1)], ⋯, E[ℱ_g(x_N)]], where E[ℱ_g(x_i)] is the expected output of ℱ_g with an input x_i ∈ D_f^L.



 §.§ Generation of Candidate Triggers


The IndisUAT attacker can perform the following steps to generate candidate triggers.
  




(1) Set the vocabulary set 𝒱  as described in Sec. <ref>. Set the length of a trigger (a sequence of words) N, an initial token t_init∈𝒱, the number of  candidate triggers k, and the threshold of the cosine similarity τ.
A trigger T_L^* is initialized on line 1, Alg. <ref>.


(2) For each batch in D^L_f, run the HotFlip method <cit.> on line 3 of Alg. <ref> to generate the candidate tokens that are as close as possible to the class L in the feature space. The technical details are presented in Sec. <ref>.

(3) For each candidate token, replace T_L^*[0] with the candidate token on line 4 of Alg. <ref> by executing Alg. <ref>, and obtain an initial set of k candidate triggers.
For each i∈[1, N-1] and each initial candidate trigger, run Alg. <ref> to return a set of tuples and finally get a set T_cand. Each tuple contains a candidate trigger T_L^*, the loss for the target prediction ℒ, and the cosine similarity between detecting results before and after adding candidate trigger c^tgt.
The key steps in Alg. <ref> are as follows: 1 replace the id-th word of the trigger with a token to obtain a trigger T_L^* on line 4;
2 run the model F_θ with inputs T_L^* and original text examples in batch on line 5, and get ℒ= ℒ(ℱ_θ(x', L), ℱ_θ^tgt(x, L))=ℒ(ℱ_θ(x ⊕ T_L^*, L), ℱ_θ^tgt(x, L)) for each x∈ batch, where x' is an candidate adversarial example created by T_L^*; and
3 calculate the cosine similarity between detecting results before and after adding T_L^* to get c^tgt on lines 6-7.




 §.§ Triggers Selection and Update


The IndisUAT attacker can perform the following steps to use a two-objective optimization and select triggers that can bypass DARCY's defense and successfully attack the class L.


(1) Filter out the candidate triggers satisfying c^tgt≥τ in each iteration on line 11, Alg. <ref>, and obtain the set of final remaining candidate triggers  T_cand.
It indicates that the detecting results of adversarial examples generated by adding triggers in T_cand are similar to those of benign examples in D^L_f for the class L, so the adversarial examples can circumvent the DARCY's trapdoors.

(2) Build Eq. (<ref>) to select the desired triggers and adversarial examples as:

    min_x'∈ D'{cos(ℱ_g^tgt(x'),ℱ_g^tgt(x))},
       max_x'∈ D'^{ℒ(ℱ_θ(x, x'))},
        s.t., x'=x ⊕ T_L^*∈ D', x ∈ D^L_f
       T_L^*∈ T_cand.
In the first objective function, the cosine similarity is calculated as c^tgt on line 7, Alg. <ref>.
Since x' can be an adversarial example only if it is misclassified to L, the low similarity between detecting results of x outside the class L and x' indicates the higher attack success probability and detected probability.
Thus, the threshold τ strikes a balance between the likelihood of being detected by DARCY and the effectiveness of the IndisUAT attack. τ can be adaptively adjusted in each iteration.
In the second objective function, the loss of predicting results is calculated as ℒ on line 5, Alg. 2.
The maximal loss indicates that ℱ_θ misclassifies the selected x' to the class L with a high probability, thus the selected trigger T_L^* shows a strong attack.

(3) At each iteration in solving Eq. (<ref>), firstly update the embedding for every token in the trigger as shown in Eq. (<ref>), Sec. <ref>. Then, convert the updated embedding back to the corresponding tokens, and obtain a set of the tokens in triggers and a set of corresponding tuples to refresh T_cand. 
Finally, find the tuple having maximal ℒ_j in T_cand to obtain the updated trigger T_L^*=cand_j^*, where j^* =max_(cand_j, ℒ_j, c_j^tgt)∈ T_cand (ℒ_j).
An example of IndisUAT is shown in Fig. <ref>.















§ PRINCIPLE ANALYSIS


IndisUAT searches and selects the adversarial examples indistinguishable to benign examples in the feature space without sacrificing their attack effects, so that IndisUAT deviates from the convergence direction of adversarial examples in original UAT method and keeps the adversarial examples away from DARCY's trapdoors.
Thus, the detection layer of DARCY is inactive to the adversarial examples generated by IndisUAT.


Fig. <ref> compares the downscaled feature distributions of original examples and adversarial examples before and after UAT 
attack and IndisUAT attack.
The triggers generated by the UAT method result in an obvious difference between the benign examples and the adversarial examples for the detector.
Adversarial examples can be detected by DARCY due to the difference in Fig. <ref>. 
The adversarial examples generated by IndisUAT deviate from those produced by UAT in the feature space, and merge with original examples.
Since there is no obvious dividing lines between the adversarial examples and the original examples as shown in Fig. <ref> and Fig. <ref>, the original model and DARCY have difficulty in distinguishing the adversarial examples from others.
Thus, the probability of detecting IndisUAT-crafted adversarial examples for DARCY is low. 

The T-SNE <cit.> is used to generate the distribution results of the examples. More detailed analysis is analyzed in Sec. <ref>. 
 
















§ EXPERIMENTAL EVALUATION



 §.§ Settings


Datasets and Threshold setting. 
We use the same datasets as DARCY did, including Movie Reviews (MR) <cit.>, Binary Sentiment Treebank (SST) <cit.>, Subjectivity (SJ) <cit.>, and AG News (AG) <cit.>. 
Their detailed information is shown in Table <ref>, Sec. <ref>.
We split each dataset into D_train, D_attack, and D_test at the ratio of 8:1:1.
All datasets are relatively class-balanced.
We set the threshold τ=0.8.



Victim Models. We attack the most widely-used models including RNN, CNN <cit.>, ELMO <cit.>, and BERT <cit.>. 

Besides DARCY, adversarial training methods are used to defend adversarial attacks, including PGD <cit.>, FreeAt <cit.>, and FreeLb <cit.>.



We report the average results on D_test over at least 5 iterations.

Attack Methods. We compared IndisUAT's performance with three adversarial attack algorithms: (1) Textfooler <cit.> that preferentially replaces the important words for victim models;
(2) PWWS <cit.> that crafts adversarial examples using the word saliency and the corresponding classification probability;
and (3) TextBugger <cit.> that finds the important words or sentences and chooses an optimal one from the generated five kinds of perturbations to craft adversarial examples.


Baselines. For text classification tasks, we use the results from the original model and the DARCY's detector with 5 trapdoors as the benchmarks for the attacks on the original model and the detector model, respectively.
For other tasks, we use the results from the original model as benchmarks. For the original task,  benchmark is the result improved by using a pre-training model.

Evaluation Metrics. We use the same metrics as DARCY <cit.> did, including Area Under the Curve (detection AUC), True Positive Rate (TPR), False Positive Rate (FPR), and  Classification Accuracy (ACC).

The attacker expects a lower AUC, TPR, ACC, and a higher FPR.






 §.§ Effect of IndisUAT on DARCY Defense

We choose the clean model as a baseline. 
Table <ref> shows that IndisUAT circumvents the detection of DARCY with a high probability.


For the RNN and CNN models, IndisUAT has  lower ACC than other attack methods.
IndisUAT incurs the ACC of the RNN model at least 33.3% on all datasets below the baseline, and meanwhile reduces the TPR of the DARCY's detector at least 40.8% on all datasets.
For the BERT model, the ACC drops at least 27.3%, and the detecting TPR drops at least 27.4% on all datasets after the IndisUAT attack.
The IndisUAT attack performs better for the CNN model, since it reduces the ACC of the CNN model at least 51.6%  compared with the baseline, and the TPR of the detection of DARCY is reduced at least 90.6%. 
Therefore, DARCY is more vulnerable when it protects the CNN model under the IndisUAT attack.

DARCY can strengthen its detecting ability through increasing the injected trapdoors. 
However, the ACC of the models falls sharply as the number of trapdoors increases as shown in Fig. <ref>.
When 50 trapdoors are added into the CNN model, the ACC drops by 34.64%.
For the models with low ACC, the DARCY's detector is not able to distinguish the adversarial examples with a high accuracy.
Thus, it is technically unfeasible for DARCY to defend against the IndisUAT attack by adding unlimited trapdoors.
We discuss the effect of the number of injected trapdoors k on IndisUAT in Fig. <ref>.
We observe that k has an obviously milder impact on the BERT model than that on the RNN and CNN models.
Besides, the AUC, and the TPR are significantly lower than those of baseline in all cases. When k=20, the ACC of the BERT model decreases by 38.2% and 37.8% with DARCY on MR and SJ datasets, respectively.
The corresponding TPR decreases by 53.1% and 31.9%, respectively.





 §.§ Effect of IndisUAT on Adversarial Defense

Table <ref> shows that the IndisUAT attack is at work for the adversarial defenses based on PGD, FreeAt, and FreeLb.
The ACC drops by 6.8% to 68.1% after adding the triggers generated by IndisUAT in all cases. 
IndisUAT has the least impact on the result from the RNN model over the AG dataset, and its ACC only drops by 8.9% at most.
For the BERT model on the AG dataset, IndisUAT has the most impact on the ACC and incurs a drop of 44.2%-68.1% in the ACC.

The IndisUAT-crafted adversarial examples are semantically similar to the original examples compared with the trapdoors as shown in Table <ref> and Table <ref> by Universal Sentence Encoder (USE) <cit.>.
Thus, IndisUAT is difficult to be identified by semantic detection methods and has good concealment.






 §.§ Effect of IndisUAT on Other Tasks


IndisUAT can be used to attack the models for text generation, text inference, and reading comprehension in addition to the text classification task.
A custom attack dictionary is used to make the models much more risky and vulnerable to unknown attacks.
We target many pre-trained models, adversarial trained models, and trained models to illustrate that IndisUAT is still highly transferable.

Text Generation. IndisUAT is used to generate triggers for racist, malicious speech on the GPT-2 <cit.> model with 117M parameters.
Applying the triggers to the GPT-2 with 345M parameter model is able to generate malicious or racially charged text as shown in
Table <ref>. The detailed results refer to Sec. <ref>.

Reading Comprehension.
The SQuAD dataset is used for the questions about why, who, where, when.
 The F1 score of the result from BiDAF <cit.> is set as a metric, and only a complete mismatch indicates a successful attack <cit.>.
Table <ref> shows the results, where the triggers generated under BiDAF (white box) migrated to the BiDAF model with ELMO embeddings (BiDAF-ELMO, black box).



Text Inference. 
The top-5 triggers are searched and used to attack the ESIM <cit.> (white-box) model for inference tasks.
IndisUAT is highly transferable, since the triggers directly attack black-box models (DA <cit.>, DA model with ELMO <cit.> embeddings (DA-ELMO)) and incur a remarkable decrease in the ACC in Table <ref>.






  









































§ CONCLUSION

We propose a novel UAT attack that can bypass the DARCY defense called IndisUAT.
IndisUAT estimates the feature distribution of benign examples and produces adversarial examples to be similar enough to the distribution estimates at the DARCY's detection layer. 
Meanwhile, the adversarial examples with the maximal loss of predicted results of the original model are selected to attack the model with a high success rate.

Extensive experiments show that IndisUAT circumvents the DARCY defense even with decades of injected trapdoors, while reducing the accuracy of the original model, adversarial training model, and pre-training model.
Beside the text classification tasks, IndisUAT is at work for other tasks, e.g., text generation, text inference, and reading comprehension.
Therefore, IndisUAT is powerful and raises a warning to model builders and defenders.
It is challenging to propose approaches to protect the textual NN models against IndisUAT in the future.





§ LIMITATIONS

IndisUAT generally outperforms other attack methods for many reasons.
First, IndisUAT, as an universal attack method, does not require the white-box (gradient) access and the access to the target model at the inference stage.
The widespread existence of trigger sequences lowers the barrier for attackers to enter into the model.
Second, the trigger search is bath-oriented in the IndisUAT method, while other attacks rely on the results of a single example, so the overall attack effect of IndisUAT is stronger than that of others.
Third, the trigger search can be extended to find more powerful trigger sequences in an extended vocabulary. The time complexity of searching triggers increases linearly with the size of the vocabulary.
However, this increased complexity is negligible, since Top-K, beam search, and KDTree methods can be used to speed up the search process by discarding trigger sequences with low impact on the results.
If the information of the detector is fully obtained, IndisUAT is highly transferable to attack even the black-box defense models with different tokenizations and architectures.








§ BROADER IMPACT STATEMENT

IndisUAT inspired by FIA <cit.> uses the cosine similarity to build adversarial examples against honeypot-injected defense models.
Although the IndisUAT attack is specifically designed to bypass the DARCY defense, it also provides effective ideas of adversarial examples generation to circumvent similar detection and defense mechanisms. 
The vulnerability of the learning model can be found using adversarial attack methods, and its robustness can be improved using adversarial defense methods.
Meanwhile, it is necessary for researchers to design novel methods that can filter out potential adversarial examples to improve the robustness of learning models.















§ APPENDIX




 §.§ Preliminaries




  §.§.§ UAT Attack


Given a textual DNN model ℱ parameterized by θ, an attacker adds a perturbation δ to the original data x, and obtains a perturbed example x'≡ x + δ. x' is an adversarial example, if the addition of x' results in a different classification output, i.e., ℱ_θ(x') ≠ℱ_θ(x).
UAT attack <cit.> consists of two steps.

(1) Trigger Search.

The task loss ℒ for the target class L is minimized to search the best trigger S, i.e., min_Sℒ = -∑_x logℱ_θ(x⊕ S, L).
Trigger S is a fixed phrase  consisting of k tokens (original example tokens). ⊕ is token-wise concatenation.
 
(2) Trigger Update. 
UAT method updates the embedding value e'_i to minimize its influence on the average gradient of the task loss over a batch ∇_e_adv_iℒ, i.e.,

    argmin_e'_i∈𝒱 [ e'_i - e_adv_i ] ^ T∇_e_adv_iℒ,

where 𝒱 is the set of all token embeddings in the model's vocabulary, and T is the first-order Taylor approximation.
The embeddings are converted back to their associated tokens, and the tokens  that alter the corresponding classification results are selected as the updated triggers.




  §.§.§ DARCY



DARCY <cit.> consists of the following three steps.

(1) Trapdoor Search. To defend attacks on a target label L of model ℱ, DARCY performs a multiple-greedy-trapdoor search algorithm H with the inputs of (K, D_train, L) to select K trapdoors S_L^*={w_1,w_2,⋯, w_K}. H has the properties of fidelity, robustness, and class-awareness.

(2) Trapdoor Injection. DARCY injects S_L^* into ℱ by populating a set of trapdoor-embedded examples, and obtains a new dataset D_trap^L{ (S_L^*⊕x, L):(x,y) ∈D_y ≠ L}, where D_y ≠ L{D_train : y ≠ L }.
DARCY baits S_L^* into F by training ℱ to minimize the NLL loss on both original examples and trapdoor-embedded examples.


(3) Trapdoor Detection. DARCY trains a binary classifier ℱ_g using the binary NLL loss, i.e., min_θ_ℱ_gℒ_ℱ_g=∑_x∈ D_train -log(ℱ_g(x)) - log(1-ℱ_g(x')),
where θ_ℱ_g denotes the parameters of ℱ_g, and x' ≡ x ⊕ S_L^*.






  §.§.§ HotFlip


In the HotFlip method <cit.>, the attacker inputs the adversarial examples into the original model, and then uses the back-propagation learning process of the model to obtain the gradients of the trained triggers.
The attacker calculates the model product of the gradient vectors corresponding to the triggers and the trained triggers at the embedding layer.
 The trigger-involved dimension of the model product matrix can be denoted as a vector.
 All components of the vector are sorted to select the k-highest components, and the attacker gets the words in 𝒱 corresponding to these k components as the k candidate tokens.

 































 §.§ More Detailed Analysis





  §.§.§ Threshold Analysis


The threshold τ is critical to adaptively circumvent the DARCY defense with k trapdoors. 

 When k is small, e.g., k < 5,
τ can ensure that the features of the adversarial examples are as similar as possible to the target class and they are not matched with the signature of the detection layer. 

When k is large, e.g., k > 10,

the detector is extremely sensitive.
Thus, τ should be large for Eq. (<ref>) by selecting T_cand, e.g., a value close to 1.
Then, the first objective of the IndisUAT attack in Eq. (<ref>) is to find the adversarial examples whose output under DARCY is very similar to the detection output of original data under DARCY.





  §.§.§ Trigger Analysis


In the process of generating triggers, the smaller length of the trigger has higher concealment.
The default length of triggers in IndisUAT is 3.

IndisUAT uses the beam search and pruning method to accelerate searches and achieve a low time complexity O(|𝒱|), where 𝒱 is the vocabulary set.
Thus, the speed of searching triggers in the IndisUAT method is fast. 

The searched triggers are effective, because of the constraints on the similarity part of Eq. (<ref>) and the HotFlip method. 
For example, even if the length of a trigger is small, e.g., 3, it can successfully compromise the DARCY's detector with 20 trapdoors.

Thus, the IndisUAT method produces effective and imperceptible triggers.



 §.§ Further Details of Experiments




  * Table <ref> shows the detailed statistics of four datasets used in the experiments as mentioned in Sec. <ref>.


  * Table <ref> shows the details of the malicious output of the text generation model in Sec. <ref>.



 §.§ Reproducibility



  §.§.§ Source Code

We release the source code of IndisUAT at: xxxxxxsource code


  §.§.§ Computing Infrastructure

We run all experiments on the machines with Ubuntu OS (v22.04), Intel(R) Xeon(R) Silver 4210R CPU @ 2.40GHz, 93GB of RAM, and an RTX 3090. All implementations are written in Python (v3.7.5) with Pytorch (v1.11.0+cu113), Numpy (v1.19.5), Scikit-learn (v0.21.3), allennlp (v0.9.0), Textattack (v0.3.7)[https://github.com/QData/TextAttack]. We use the Transformers (v3.3.0)[https://huggingface.co/] library for training transformers-based BERT. Note that, the version of python can also be 3.6.9.




  §.§.§ Model’s Architecture and # of Parameters



The structure of the CNN model with 6M parameter consists of three 2D convolutional layers, a max-pooling layer, a dropout layer with probability 0.5, and a Fully Connected Network (FCN) with softmax activation for prediction.

The pre-trained GloVe <cit.> is used to transform the original discrete texts into continuous features and feed them into the models.
The RNN model with 6.1M parameters uses a GRU layer to replace the convolution layers of CNN, and its other layers remain the same.


The BERT model with 109M parameters is imported from the Transformers library.
The ELMO[https://allenai.org/allennlp/software/elmo] model with 13.6M has a LSTM network, and the size of the input layer and that of the hidden layer of LSTM are 128 and 1024, respectively.
We construct a vocabulary set, called 𝒱, for the trigger search in IndisUAT. 

𝒱 contains 330K words, 126k words are extracted from the datasets shown in Table. <ref>, and the other words are randomly produced. 
The features of all words in 𝒱 are taken from the GloVe pre-trained features.
In our experiments, DARCY is run with the vocabulary set 𝒱.




  §.§.§ Implementation of Other Attacks

We use the tool kit of Textattack <cit.> to generate the adversarial examples of PWWS, TextBugger, and Textfooler.

The parameters setting is shown in Table <ref>. The bert-base-uncased version of BERT model is used, and the structures of CNN and RNN are the same as those presented in Sec. <ref>. 
These adversarial attacks and the IndisUAT attack use the same test datasets, which are extracted from the four datasets shown in Table <ref>.








































