Con-ReCall: Detecting Pre-training Data in LLMs via Contrastive Decoding
Cheng Wang Yiwei Wang Bryan Hooi Yujun Cai Nanyun Peng Kai-Wei Chang


Efficient Multi-Task Large Model Training via Data Heterogeneity-aware Model Management
    Yujie Wang^1, Shenhan Zhu^1, Fangcheng Fu^1, Xupeng Miao^2, Jie Zhang^3, Juan Zhu^3, Fan Hong^3, Yong Li^3, Bin Cui^1
    September 9, 2024
=========================================================================================================================



§ ABSTRACT 

The training data in large language models is key to their success, but it also presents privacy and security risks, as it may contain sensitive information. Detecting pre-training data is crucial for mitigating these concerns. 
Existing methods typically analyze target text in isolation or solely with non-member contexts, overlooking potential insights from simultaneously considering both member and non-member contexts. 
While previous work suggested that member contexts provide little information due to the minor distributional shift they induce, our analysis reveals that these subtle shifts can be effectively leveraged when contrasted with non-member contexts.
In this paper, we propose , a novel approach that leverages the asymmetric distributional shifts induced by member and non-member contexts through contrastive decoding, amplifying subtle differences to enhance membership inference. Extensive empirical evaluations demonstrate that  achieves state-of-the-art performance on the WikiMIA benchmark and is robust against various text manipulation techniques. 




§ INTRODUCTION


Large Language Models (LLMs) <cit.> have revolutionized natural language processing by achieving remarkable performance across a wide range of language tasks.
These models owe their success to extensive training datasets, often encompassing trillions of tokens.
However, the sheer volume of these datasets makes it practically infeasible to meticulously filter out all inappropriate data points.
Consequently, LLMs may unintentionally memorize sensitive information, raising significant privacy and security concerns.
This memorization can include test data from benchmarks <cit.>, copyrighted materials <cit.>, and personally identifiable information <cit.>, leading to practical issues such as skewed evaluation results, potential legal ramifications, and severe privacy breaches.
Therefore, developing effective techniques to detect unintended memorization in LLMs is crucial.



Existing methods for detecting pre-training data <cit.> typically analyze target text either in isolation or alongside with non-member contexts, while commonly neglecting member contexts. 
This omission is based on the belief that member contexts induce only minor distributional shifts, offering limited additional value <cit.>.



However, our analysis reveals that these subtle shifts in member contexts, though often dismissed, hold valuable information that has been underexploited. The central insight of our work is that information derived from member contexts gains significant importance when contrasted with non-member contexts. This observation led to the development of , a novel approach that harnesses the contrastive power of prefixing target text with both member and non-member contexts. By exploiting the asymmetric distributional shifts induced by these different prefixes,  provides more nuanced and reliable signals for membership inference. This contrastive strategy not only uncovers previously overlooked information but also enhances the accuracy and robustness of pre-training data detection, offering a more comprehensive solution than existing methods.

To demonstrate the effectiveness of ,
we conduct extensive empirical evaluations
on the method across a variety of models of different sizes. Our experiments show that  outperforms the current state-of-the-art method by a significant margin, as shown in Figure <ref>. Notably,  only requires a gray-box access to LLMs, i.e., token probabilities, and does not necessitate a reference model, enhancing its applicability in real-world scenarios.



We summarize our contributions as follows: 1) We introduce , a novel contrastive decoding approach that effectively utilizes both member and non-member contexts, significantly enhancing the distinction between member and non-member data in LLMs. 2) Through extensive experiments, we demonstrate that  achieves substantial improvements over existing baselines, highlighting its effectiveness and resilience in detecting pre-training data. 3) We demonstrate that  is robust against text manipulation techniques, including random deletion, synonym substitution, and paraphrasing, maintaining superior performance and resilience to potential evasion strategies.



§ RELATED WORK




  
Membership inference attack.

Membership inference attack (MIA) was first proposed by <cit.>. MIA has been extensively studied, particularly in classification models within the computer vision domain <cit.>. While there is growing attention to MIA in language models, most work has focused on detecting fine-tuning data <cit.>. MIA can serve as a powerful tool for detecting copyrighted materials <cit.>, personally identifiable information <cit.> and test-set contamination <cit.>.



  
Detecting Pre-training Data in LLMs.

Although detecting pre-training data is an instance of MIA, it faces greater challenges compared to traditional MIA. Classical MIA <cit.> typically requires training a shadow model using data sampled from the training data distribution. However, for large language models, many developers are reluctant to release the full training data <cit.>, making it impractical to train shadow models. Additionally, due to the sheer volume of training data, LLMs are usually trained for a single epoch, which makes memorization inherently difficult and detection even more challenging <cit.>.

To our knowledge, <cit.> was the first to investigate this problem, contributing a baseline method and the WikiMIA benchmark. Their method, Min-K%, despite its simplicity, serves as a powerful baseline. <cit.> enhanced Min-K% by normalizing token log-probabilities. The ReCall method  introduces relative conditional log-likelihoods and achieves current state-of-the-art performance. 



  
Contrastive Decoding.

Contrastive decoding is primarily a method for text generation. Depending on the elements being contrasted, it serves different purposes. For example, DExperts <cit.> use outputs from a model exposed to toxicity to guide the target model away from undesirable outputs. Context-aware decoding <cit.> contrasts model outputs given a query with and without relevant context. <cit.> further enhance context-aware decoding by providing irrelevant context in addition to relevant context. In this paper, we adapt the idea of contrastive decoding to MIA, where the contrast occurs between target data prefixed with member and non-member contexts. 



§ 




 §.§ Problem Formulation


Consider a model ℳ trained on dataset 𝒟. The objective of a membership inference attack is to ascertain whether a data point x belongs to 𝒟 (i.e., x ∈𝒟) or not (i.e., x ∉𝒟). Formally, we aim to develop a scoring function s(x, ℳ) →ℝ, where the membership prediction is determined by a threshold τ:

    x ∈𝒟   if  s(x, ℳ) ≥τ
    
      x ∉𝒟   if  s(x, ℳ) < τ
    .








 §.§ Motivation


Our key insight is that prefixing target text with contextually similar content increases its log-likelihood, while dissimilar content decreases it. Member prefixes boost log-likelihoods for member data but reduce them for non-member data, with non-member prefixes having the opposite effect. This principle stems from language models' fundamental tendency to generate contextually consistent text. 

To quantify the impact of different prefixes, we use the Wasserstein distance to measure the distributional shifts these prefixes induce. For discrete probability distributions P and Q defined on a finite set X, the Wasserstein distance W is given by:


    W(P, Q) = ∑_x∈ X |F_P(x) - F_Q(x)|,

where F_P and F_Q are the cumulative distribution functions of P and Q respectively. To capture the directionality of the shift, we introduce a signed variant of this metric:


    W_signed(P, Q) = sign(𝔼_Q[X] - 𝔼_P[X]) · W(P, Q).


Our experiments reveal striking asymmetries in how member and non-member data respond to different prefixes. Figure <ref> illustrates these asymmetries, showing the signed Wasserstein distances between original and prefixed distributions across varying numbers of shots, where shots refer to the number of non-member data points used in the prefix.



We observe two key phenomena:


    
  * Asymmetric Shift Direction: Member data exhibits minimal shift when prefixed with other member contexts, indicating a degree of distributional stability. However, when prefixed with non-member contexts, it undergoes a significant negative shift. In contrast, non-member data displays a negative shift when prefixed with member contexts and a positive shift with non-member prefixes.
    
    
  * Asymmetric Shift Intensity: Non-member data demonstrated heightened sensitivity to contextual modifications, manifesting as larger magnitude shifts in the probability distribution, regardless of the prefix type. Member data, while generally more stable, still exhibited notable sensitivity, particularly to non-member prefixes.


These results corroborate our initial analysis and establish a robust basis for our contrastive approach. The asymmetric shifts in both direction and intensity provide crucial insights for developing a membership inference technique that leverages these distributional differences effectively.



 §.§ Contrastive Decoding with Member and Non-member Prefixes


Building on the insights from our analysis, we propose , a method that exploits the contrastive information between member and non-member prefixes to enhance membership inference through contrastive decoding. Our approach is directly motivated by the two key observations from the previous section:


    
  * The asymmetric shift direction suggests that comparing the effects of member and non-member prefixes could provide a strong signal for membership inference.
    
  * The asymmetric shift intensity indicates the need for a mechanism to control the relative importance of these effects in the decoding process.


These insights lead us to formulate the membership score s(x, M) for a target text x and model M as follows:


    LL(x|P_non-member) - γ· LL(x|P_member)/LL(x)
    ,

where LL(·) denotes the log-likelihood, P_member and P_non-member are prefixes composed of member and non-member contexts respectively, and γ is a parameter controlling the strength of the contrast.

This formulation provides a robust signal for membership inference by leveraging the distributional differences revealed in our analysis. Figure <ref> illustrates how our contrastive approach amplifies the distributional differences

Importantly,  requires only gray-box access to the model, utilizing solely token probabilities. This characteristic enhances its practical utility in real-world applications where full model access may not be available, making it a versatile tool for detecting pre-training data in large language models.



§ EXPERIMENTS


In this section, we will evaluate the effectiveness of  across various experimental settings, demonstrating its superior performance compared to existing methods.



 §.§ Setup




  
Baselines.

In our experiment, we evaluate  against seven baseline methods. Loss  directly uses the loss of the input as the membership score. Ref  requires another reference model, which is trained on a dataset with a distribution similar to 𝒟, to calibrate the loss calculated in the Loss method. Zlib  instead calibrates the loss by using the input's Zlib entropy. Neighbor  perturbs the input sequence to generate n neighbor data points, and the loss of x is compared with the average loss of the n neighbors. Min-K%  is based on the intuition that a member sequence should have few outlier words with low probability; hence, the top-k% words having the minimum probability are averaged as the membership score. Min-K%++  is a normalized version of Min-K% with some improvements.   calculates the relative conditional log-likelihood between x and x prefixed with a non-member contexts P_non-member. More details can be found in Table <ref>.



  
Datasets.

We primarily use WikiMIA <cit.> as our benchmark. WikiMIA consists of texts from Wikipedia, with members and non-members determined using the knowledge cutoff time, meaning that texts released after the knowledge cutoff time of the model are naturally non-members. WikiMIA is divided into three subsets based on text length, denoted as WikiMIA-32, WikiMIA-64, and WikiMIA-128. 

Another more challenging benchmark is MIMIR <cit.>, which is derived from the Pile <cit.> dataset. The benchmark is constructed using a train-test split, effectively minimizing the temporal shift present in WikiMIA, thereby ensuring a more similar distribution between members and non-members. More details about these two benchmarks are presented in Appendix <ref>.



  
Models.

For the WikiMIA benchmark, we use Mamba-1.4B <cit.>, Pythia-6.9B <cit.>, GPT-NeoX-20B <cit.>, and LLaMA-30B <cit.>, consistent with <cit.>. For the MIMIR benchmark, we use models from the Pythia family, specifically 2.8B, 6.9B, and 12B. Since Ref <cit.> requires a reference model, we use the smallest version of the model from that series as the reference model, for example, Pythia-70M for Pythia models, consistent with previous works <cit.>. 






  
Metrics.
Following the standard evaluation metrics <cit.>, we report the AUC (area under the ROC curve) to measure the trade-off between the True Positive Rate (TPR) and False Positive Rate (FPR). We also include TPR at low FPRs (TPR@5%FPR) as an additional metrics. 



  
Implementation Details.
For Min-K% and Min-K%++, we vary the hyperparameter k from 10 to 100 in steps of 10. For , we optimize γ from 0.1 to 1.0 in steps of 0.1. Following <cit.>, we use seven shots for both ReCall and  on WikiMIA. For MIMIR, due to its increased difficulty, we vary the number of shots from 1 to 10. In all cases, we report the best performance. For more details, see Appendix <ref>.



 §.§ Results




  
Results on WikiMIA.

Table <ref> summarizes the experimental results on WikiMIA, demonstrating 's significant improvements over baseline methods. In terms of AUC performance, our method improved upon ReCall by 7.4%, 6.6%, and 5.7% on WikiMIA-32, -64, and -128 respectively, achieving an average improvement of 6.6% and state-of-the-art performance. For TPR@5%FPR,  outperformed the runner-up by even larger margins: 30.0%, 34.8%, and 27.6% on WikiMIA-32, -64, and -128 respectively, with an average improvement of 30.8%. Notably,  achieves the best performance across models of different sizes, from Mamba-1.4B to LLaMA-30B, demonstrating its robustness and effectiveness. The consistent performance across varying sequence lengths suggests that  effectively identifies membership information in both short and long text samples, underlining its potential as a powerful tool for detecting pre-training data in large language models in diverse scenarios.



  
Results on MIMIR.

We summarize the experimental results on MIMIR in Appendix <ref>.
The performance of  on the MIMIR benchmark demonstrates its competitive edge across various datasets and model sizes. In the 7-gram setting,  consistently achieved top-tier results, often outperforming baseline methods. Notably, on several datasets, our method frequently secured the highest scores in both AUC and TPR metrics. 
In the 13-gram setting,  maintained its strong performance, particularly with larger model sizes. While overall performance decreased compared to the 7-gram setting,  still held leading positions across multiple datasets. It's worth noting that  exhibited superior performance when dealing with larger models, indicating good scalability for more complex and larger language models. Although other methods occasionally showed slight advantages in certain datasets, 's overall robust performance underscores its potential as an effective method for detecting pre-training data in large language models.







 §.§ Ablation Study


We focus on WikiMIA with the Pythia-6.9B model for ablation study.



  
Ablation on γ.

In , we introduce a hyperparameter γ, which controls the contrastive strength between member and non-member prefixes. The AUC performance across different γ values for the WikiMIA dataset is depicted in Figure <ref>. The red vertical lines mark the γ = 0 case, where  reverts to the baseline ReCall method. 


The performance of  fluctuates as γ varies, meaning that there exist an optimal value for γ for us to get the best performance. However, even without any fine-tuning on γ, our method still outperforms ReCall and other baselines.





  
Ablation on the number of shots.

The prefix is derived by concatenating a series of member or non-member strings, i.e., P = p_1⊕ p_2⊕⋯⊕ p_n, and we refer to the number of strings as shots following <cit.>'s convention. In this section, we evaluate the relationship between AUC performance and the number of shots. We vary the number of shots on the WikiMIA dataset using the Pythia-6.9B model, and summarize the results in Figure <ref>. 

The general trend shows that increasing the number of shots improves the AUC, as more shots provide more information. Both ReCall and  exhibit this trend, but  significantly enhances the AUC compared to ReCall and outperforms all baseline methods.



§ ANALYSIS



To further evaluate the effectiveness and practicality of , we conducted additional analyses focusing on its robustness and adaptability in real-world scenarios. These investigations provide deeper insights into the method's performance under various challenging conditions.



 §.§ Robustness of 


As membership inference attacks gain prominence, it is crucial to evaluate the robustness of these methods against potential evasion techniques. In real-world scenarios, data may not always be presented in its original form due to various factors such as text preprocessing, natural language variations, or intentional obfuscation. Therefore, a robust membership inference method should maintain its effectiveness even when faced with altered versions of the target data.

To assess the robustness of , we employ three text manipulation techniques. First, we use Random Deletion, where we randomly remove a certain percentage of words from the original text, using deletion rates of 10%, 15%, and 20% in our experiments. Second, we apply Synonym Substitution, replacing a portion of the words in the text with their synonyms. For this technique, we use substitution rates of 10%, 15%, and 20%, utilizing WordNet <cit.> for synonym selection. Lastly, we leverage the WikiMIA-paraphrased dataset <cit.>, which offers paraphrased versions of the original WikiMIA <cit.> texts. This dataset, created using ChatGPT[OpenAI.  <https://chat.openai.com/chat>] to rephrase the original text while preserving its meaning, provides a standardized benchmark for evaluating robustness against paraphrasing.

We evaluate the effectiveness of baselines and  after transforming texts using the above techniques. Our experiments are conducted using Pythia-6.9B <cit.> and LLaMA-30B <cit.> models on the WikiMIA-32 <cit.> dataset. Table <ref> presents the AUC performance for each method under various text manipulation scenarios. The results demonstrate that  consistently outperforms baseline methods across all text manipulation techniques, maintaining its superior performance even when faced with altered versions of the target data. Notably,  shows particular resilience to synonym substitution and paraphrasing, where it experiences minimal performance degradation compared to other methods. This robustness underscores 's effectiveness in real-world scenarios where data may undergo various transformations.



 §.§ Approximation of Members


In real-world scenarios, access to member data may be limited or even impossible. Therefore, it is crucial to develop methods that can approximate member data effectively. Our approach to approximating members is driven by two primary motivations. First, large language models (LLMs) are likely to retain information about significant events that occurred before their knowledge cutoff date. This retention suggests that LLMs have the potential to recall and replicate crucial aspects of such events when prompted. Second, when presented with incomplete information and tasked with its completion, LLMs can effectively leverage their internalized knowledge to generate contextually appropriate continuations. These two motivations underpin our method, where we first utilize an external LLM to enumerate major historical events. We then truncate these events and prompt the target LLM to complete them, hypothesizing that the generated content can serve as an effective approximation of the original data within the training set.

To test this approach, we first employed GPT-4o <cit.> to generate descriptions of seven major events that occurred before 2020 (the knowledge cutoff date for the Pythia models). We then truncated these descriptions and prompted the target model to complete them. This method allows us to simulate the generation of data resembling the original members without directly accessing the original training set. Details of the prompts and the corresponding responses can be found in Appendix <ref>.

We evaluated this method using a fixed number of seven shots for consistency with our previous experiments. The results, summarized in Table <ref>, demonstrate that even without prior knowledge of actual member data, this approximation approach yields competitive results, outperforming several baseline methods.

This finding suggests that when direct access to member data is not feasible, leveraging the model's own knowledge to generate member-like content can be an effective alternative.





§ CONCLUSION


In this paper, we introduced , a novel contrastive decoding approach for detecting pre-training data in large language models. By leveraging both member and non-member contexts, CON-RECALL significantly enhances the distinction between member and non-member data. Through extensive experiments on multiple benchmarks, we demonstrated that CON-RECALL achieves substantial improvements over existing baselines, highlighting its effectiveness in detecting pre-training data. Moreover, CON-RECALL showed robustness against various text manipulation techniques, including random deletion, synonym substitution, and paraphrasing, maintaining superior performance and resilience to potential evasion strategies. These results underscore CON-RECALL's potential as a powerful tool for addressing privacy and security concerns in large language models, while also opening new avenues for future research in this critical area.



 §.§ Limitations


The efficacy of  is predicated on gray-box access to the language model, permitting its application to open-source models and those providing token probabilities. However, this prerequisite constrains its utility in black-box scenarios, such as API calls or online chat interfaces. Furthermore, the performance of  is contingent upon the selection of member and non-member prefixes. The development of robust, automated strategies for optimal prefix selection remains an open research question. While our experiments demonstrate a degree of resilience against basic text manipulations, the method's robustness in the face of more sophisticated adversarial evasion techniques warrants further rigorous investigation.



 §.§ Ethical Considerations


The primary objective in developing  is to address privacy and security concerns by advancing detection techniques for pre-training data in large language models. However, it is imperative to acknowledge the potential for misuse by malicious actors who might exploit this technology to reveal sensitive information. Consequently, the deployment of  necessitates meticulous consideration of ethical implications and the establishment of stringent safeguards. Future work should focus on developing guidelines for the responsible use of such techniques, balancing the benefits of enhanced model transparency with the imperative of protecting individual privacy and data security.







§ DATASETS STATISTICS









§ ADDITIONAL IMPLEMENTATION DETAILS



All models are obtained from Huggingface[<https://huggingface.co/>] and deployed with 4 NVIDIA RTX 3090 GPUs.

In our evaluation process, we carefully handled the data to ensure fair comparison across all methods. The specifics of our data handling varied between the WikiMIA and MIMIR datasets:

For the WikiMIA dataset, we selected 7 samples each from the member and non-member sets to use as prefixes. The number of shots was fixed at 7 for all experiments on this dataset. 

For the MIMIR dataset, we removed 10 samples each from the member and non-member datasets to create our prefix pool. Unlike WikiMIA, we varied the number of shots from 1 to 10 and reported the best-performing configuration.

For both datasets, the samples used for prefixes were removed from the evaluation set for all methods, including baselines, ensuring a fair comparison across different methods.



§ MEMBER APPROXIMATION DETAILS



In this section, we detail our method for approximating member data when direct access to the original training set is not feasible. Our approach involves two steps: first, using GPT-4o <cit.> to generate descriptions of significant events, and then using these partially truncated descriptions to prompt our target model.

We begin by providing GPT-4o with the following prompt:

[title=Prompt for GPT-4o]
Provide a concise overview of seven pivotal global events that occurred before 2020. Each event should be described in under 32 words and presented in a narrative format without explicit headings.


GPT-4o generated the following response:

[title=Response from GPT-4o]
The world witnessed the fall of the Berlin Wall in 1989, symbolizing the end of the Cold War. The 9/11 attacks in 2001 reshaped global security dynamics. The 2008 financial crisis led to worldwide economic instability. In 1945, World War II concluded, altering geopolitical landscapes. The 1969 Moon landing marked humanity's leap into space. The 1979 Iranian Revolution transformed Middle Eastern politics. The 1918 Spanish flu pandemic highlighted vulnerabilities in global health.


We then truncated these responses to create partial prompts:

[title=Truncated Responses]

    
  * The world witnessed the fall of the Berlin Wall in 
    
  * The 9/11 attacks in 
    
  * The 2008 financial crisis led to 
    
  * In 1945, World War II concluded, 
    
  * The 1969 Moon landing marked 
    
  * The 1979 Iranian Revolution transformed Middle Eastern 
    
  * The 1918 Spanish flu pandemic highlighted 



These truncated texts were then used as prompts for our target model to complete, simulating the generation of member-like content. To ensure consistency with our experimental setup, we set the maximum number of new tokens (max_new_tokens) to match the length of the target text. For example, when working with WikiMIA-32, max_new_tokens was set to 32.



§ MIMIR RESULTS




