Infilling Score: A Pretraining Data Detection Algorithm for Large Language Models
Under review as a conference paper at ICLR 2025

INFILLING SCORE ✼ A PRETRAINING DATA DETECTION
ALGORITHM FOR LARGE LANGUAGE MODELS

Anonymous authors
Paper under double-blind review

ABSTRACT

In pretraining data detection, the goal is to detect whether a given sentence is in the
dataset used for training a Large Language Model (LLM). Recent methods (such as
Min-K% and Min-K%++) reveal that most training corpora are likely contaminated
with both sensitive content and evaluation benchmarks, leading to inflated test set
performance. These methods sometimes fail to detect samples from the pretraining
data, primarily because they depend on statistics composed of causal token likeli-
hoods. We introduce Infilling Score, a new test-statistic based on non-causal token
likelihoods. Infilling Score can be computed for autoregressive models without
re-training using Bayes rule. A naive application of Bayes rule scales linearly with
the vocabulary size. However, we propose a ratio test-statistic whose computation
is invariant to vocabulary size. Empirically, our method achieves a significant accu-
racy gain over state-of-the-art methods including Min-K%, and Min-K%++ on the
WikiMIA benchmark across seven models with different parameter sizes. Further,
we achieve higher AUC compared to reference-free methods on the challenging
MIMIR benchmark. Finally, we create a benchmark dataset consisting of recent
data sources published after the release of Llama-3; this benchmark provides a
statistical baseline to indicate potential corpora used for Llama-3 training.

1

INTRODUCTION

The significant progress in language modeling can largely be attributed to development and deploy-
ment of large-scale models that utilize extensive training corpora, often encompassing trillions of
tokens (Li et al., 2024; Dubey et al., 2024). The selection and curation of data for training such
Large Language Models (LLMs) is very complex and expensive. Further, recent developers of LLMs
withhold details regarding the sources of their pretraining datasets (Dubey et al., 2024; OpenAI
et al., 2024; Touvron et al., 2023b). This lack of transparency has raised concerns regarding the
inadvertent inclusion of copyrighted content (Chang et al., 2023; Min et al., 2023; Meeus et al., 2023)
or personally identifiable information (Mozes et al., 2023; Panda et al., 2024), potentially leading to
ethical and legal challenges (Grynbaum & Mac, 2023). Furthermore, the inclusion of benchmark
datasets within the training corpora itself can compromise the integrity of model evaluations. This
practice may inflate test performance metrics without accurately reflecting the model’s capabilities
(Oren et al., 2023; Zhou et al., 2023).

Recent work has focused on the problem of determining whether specific sequences of tokens have
been previously seen by a language model (Shi et al., 2024; Zhang et al., 2024; Duan et al., 2024).
These investigations are categorized under a growing field of attacks on LLMs known as Membership
Inference Attacks (MIA) (Shokri et al., 2017; Mattern et al., 2023b; Carlini et al., 2022). Many studies
in this area focus on fine-tuning data detection (Song & Shmatikov, 2019; Shejwalkar et al., 2021;
Mahloujifar et al., 2021). However, pretraining data detection attacks are becoming increasingly
important as they can reveal whether a model has been trained on potentially sensitive data and
prevent evaluation data contamination (Jiang et al., 2024; Yang et al., 2023).

We introduce a novel method for identifying whether a given text sequence was part of a language
model’s pretraining data. Our method uses a new test-statistic that we call the Infilling Score. Our
approach performs a non-causal test to compute the infilling probability of a token, based on the
tokens that appear before and after this token in the sentence. An autoregressive language model

1

000
001

002
003
004
005
006
007

008
009
010
011
012
013

014
015
016
017
018

019
020
021
022
023
024

025
026
027
028
029
030

031
032
033
034
035
036

037
038
039
040
041

042
043
044
045
046
047

048
049
050
051
052
053

Under review as a conference paper at ICLR 2025

054
055

056
057
058
059
060
061

062
063
064
065
066
067

068
069
070
071
072

073
074
075
076
077
078

079
080
081
082
083
084

085
086
087
088
089
090

091
092
093
094
095

096
097
098
099
100
101

102
103
104
105
106
107

generates causal likelihoods (i.e. the probability of a word appearing after some context). We find
that non-causal likelihoods lead to more accurate tests for membership inference. These likelihoods
can be computed using a causal autoregressive model.The computation involves applying Bayes’
rule and the law of total probability, and needs a marginalization over the vocabulary to compute a
partition function. Unfortunately, computing this partition function requires calling the autoregressive
LM many times, one for each vocabulary entry. This would require tens of thousands of calls to
the autoregressive LLM to compute a single non-causal probability for one token, and hence is not
practical. Our central idea is to propose an approximate test-statistic whose computation is much
faster, does not require an exact computation of this partition function and does not depend on the
vocabulary size.

Our method achieves a significant accuracy gain over state-of-the-art methods including Min-K%, and
Min-K%++ on the WikiMIA benchmark across seven models. On WikiMIA, our method outperforms
the previous state of the art in AUC. It achieves up to 10% improvement on Llama models when
testing long sequences (256 tokens). Further, we achieve higher AUC compared to reference-free
methods on the challenging MIMIR benchmark.

Our main contributions are summarized below:

(1) We introduce the Infilling Score, a new reference-free method for detecting pretraining
data using infilling likelihood of tokens within the candidate sentence (Section 3). While
SoTA methods: MIN-K% and MIN-K%++ rely on a statistic based on past tokens only, our
method computes a new test statistic considering both past and future tokens in the sentence.

(2) We develop an efficient algorithm for computing this new score. Though our method
conceptually shares similarities with a likelihood computed via Bayes rule, computationally
it is much different: whereas any natural approach for computing a Bayes rule calculation
scales with vocabulary size, our algorithm has computation invariant to vocabulary size.

(3) We conduct extensive experiments on the standard (a) WikiMIA (Shi et al., 2024) and,
(b) MIMIR (Duan et al., 2024) to verify the efficacy of our method (Section 4). On these
benchmarks, we compare our method with state-of-the-art MIA methods including MIN-
K% (Shi et al., 2024) and MIN-K%++ (Zhang et al., 2024). On WikiMIA, our method
achieves 11% improvement over MIN-K% and 5% improvement over MIN-K%++ in terms
of AUROC on average. We attribute the notable performance gain of our method to infilling
probability (Section 3).

(4) We curate a dataset of book excerpts that have not been seen by the LLMs released before
April 2024 (Section 4.1). Employing our Infilling Score, we detect a list of books which
have (likely) been used for training Llama-3-8B (Dubey et al., 2024) (4.4.3).

2 BACKGROUND

In this section, we discuss the standard definition of Membership Inference Attack (MIA) and recent
advances along this line of research.
Problem setup. Given a sentence x = {xi}N
i=0 and a Large Language Model (LLM) denoted by M,
the goal of MIA is to build a detector h (x, M) → {0, 1} that can infer the membership of x in the
training corpus D = {xj}j∈[n] of M. Existing MIA methods for LLMs (Shi et al., 2024; Zhang
et al., 2024; Carlini et al., 2021; Mattern et al., 2023a) assign a score to each sample x and use a
binary threshold to determine its membership class, with 1 indicating x ∈ D and 0 otherwise.

2.1 CHALLENGES IN PRETRAINING DATA DETECTION USING MIA METHODS

2.1.1 DETECTION DIFFICULTY

Prior works (Hardt et al., 2016; Bassily et al., 2020) have shown that the total variation (TV) distance
between the distribution of seen and unseen data is proportional to the learning rate, size of the
dataset |D| and the frequency of the test sentence x. Since TV captures the separability between
these distributions, low TV makes it difficult to infer the membership class of a given x.

2

Under review as a conference paper at ICLR 2025

2.1.2 ARCHITECTURE AND PRETRAINING DISTRIBUTION

Membership inference attacks for LLM pretraining data detection are broadly categorized into two
classes: (a) reference-based methods and (b) reference-free methods. Reference-based methods
such as Reference (Carlini et al., 2021) infer the membership of a sentence x by computing the
likelihood of x using two different LLMs. They compare the perplexity of x under the target LLM
with the perplexity of x under a smaller language model. The smaller model M shares the same
architecture as M, and is trained on a subset of samples, D, collected from the same underlying
distribution of D. The intuition is that smaller networks have less capacity to memorize sentences
from the pretraining dataset. One crucial limitation of these methods is that reference model may not
always exist. Although LLM developers often do not disclose information about the distribution of
pretraining data, reference-based MIAs (Carlini et al., 2021) assume the knowledge of the architecture
and underlying pretraining distribution, making these methods less practical.

Among reference-free methods, Min-K% (Shi et al., 2024) hypothesizes that when a sentence is
seen by the model, i.e., x ∈ D, it usually contains a number of tokens with low causal probabilities
(outliers). Formally, given a sequence of tokens x = {xi}N

i=0, Min-K% score is given by:

Min-K%(x) =

1
|min-k%|

(cid:88)

xi∈min-k%

Min-K%token(xi),

where Min-K%token(xi) = log p (xi|x<i) .

(1)

(2)

Here, Min-K%token(xi) denotes the score for each token xi. The set min-k% contains k% of the
input tokens which correspond to the bottom k% scores within the sequence. If the average score for
this set is less than τ (k), where τ (k) denotes the binary threshold for a fixed k, then Min-K% detects
the sequence x as “unseen”. Note that the classification threshold τ (k) is determined empirically
using a validation dataset.

A recently proposed method, Min-K%++ (Zhang et al., 2024), improves the detection accuracy of
Min-K% by normalizing the next-tokens log likelihood log p(xi|x<i) as follows:

Min-K%++(x) =

1
|min-k%|

(cid:88)

Min-K%++token(xi),

where Min-K%++token(xi) =

xi∈min-k%
log p (xi|x<i) − µx<i
σx<i

(3)

(4)

µx<i = Ez∼p(.|x<i)[log p(z|x<i)] and σx<i = (cid:112)Ez∼p(.|x<i)[(log p(z|x<i) − µx<i )2] are the mean
and standard deviation of the next-token likelihood.

Both Min-K% and Min-K%++ rely on the “causal” likelihood predictions of the model. However,
the causal likelihood of xi does not consider the information from the entire sentence context, as it
only depends on the preceding tokens x<i. We propose that sentences seen during training (x ∈ D)
typically have a number of tokens with low infilling probabilities. By using the non-causal token
likelihoods which depend on both preceding and succeeding tokens (x<i and x>i), we achieve a
more accurate statistic than causal likelihoods alone. This enables our Infilling Score method to
outperform previous pretraining data detection approaches on standard benchmarks.

3 METHOD

We describe our method in this section. First we describe the computation of our new ratio statistic,
and explain why it offers computational scalability compared to a straightforward application of
Bayes Rule. Next, we describe how this score is used to detect data in the pretraining set. Finally, we
explain how we employ our method to detect pretraining samples in Llama-3.

3

108
109

110
111
112
113
114
115

116
117
118
119
120
121

122
123
124
125
126

127
128
129
130
131
132

133
134
135
136
137
138

139
140
141
142
143
144

145
146
147
148
149

150
151
152
153
154
155

156
157
158
159
160
161

Under review as a conference paper at ICLR 2025

Ground truth:
Masked input:

She
She
x1

pasta
Italian
ate
ate <MASKED> pasta
m3
x2

x4

3.1 COMPUTING THE INFILLING LIKELIHOOD

In this setting, we search for the most likely token to infill m3 using other tokens in the sentence, i.e.,
{x1, x2, x4}. Using the law of total probability, we get:

p(x3|x1, x2, x4) =

p(x4|x1, x2, x3)p(x3|x1, x2)
p(x4|x1, x2)

=

(cid:80)

p(x4|x1, x2, x3)p(x3|x1, x2)
3∈V p(x4|x1, x2, x′
x′

3)p(x′

3|x1, x2)

.

(5)

Observe that the partition function in the denominator of equation 5 is expensive to compute as it
requires summation over all the tokens in the vocabulary V. In the naive case, the number of LLM
calls required to compute the infilling likelihood scales linearly both with vocabulary size and the
sequence length. This is because for each token, the denominator in equation 5 scales linearly in the
vocabulary size, and this computation needs to be repeated for each token. The vocabulary size can
be as large as 128K in recent LLMs (Dubey et al., 2024).

To address the scalability challenge, we introduce a ratio test-statistic. Our main idea is to compute the
ratio of the infilling probability of the ground-truth token and the maximum causal likelihood token.
Using this proposed statistic, we circumvent the need to compute the computationally expensive
partition function. In the above setting, we define the ratio test-statistic of token x3 as:

p(x3|x1, x2, x4)
p(x∗
3|x1, x2, x4)

, where x∗

3 = arg max
x′

3∈V

p(x′

3|x1, x2).

(6)

This ratio compares the infilling likelihood of the ground-truth token to that of the model’s causal
prediction. If x3 is an outlier the ratio is closer to 0, and when x3 is among the model’s top predictions,
this ratio is closer to 1. Since the partition function in equation 5 is the same for p(x3|x1, x2, x4) and
p(x∗
3|x1, x2, x4), it gets cancelled in the ratio test-statistic. This drastically reduces the number of
LLM calls from O(N |V|) to O(N ), making our test-statistic independent of the size of the vocabulary
(details in 4.5). Interestingly, we can exactly compute this ratio analytically using auto-regressive
models without re-training. We then compute the log of this ratio and normalize the probabilities to
capture the relative significance of each token in the vocabulary. First, we derive

log

p(x3|x1, x2, x4)
p(x∗
3|x1, x2, x4)

= log

p(x4|x1, x2, x3)p(x3|x1, x2)
p(x4|x1, x2, x∗
3|x1, x2)

3)p(x∗

= log p(x4|x1, x2, x3) + log p(x3|x1, x2) − log p(x4|x1, x2, x∗

3) − log p(x∗

3|x1, x2),

(7)

(8)

Generalizing (equation 7) to use m future tokens for calculating the infilling ratio of token i, we get:

log

p(xi|x1:i−1, xi+1:n)

p(x∗

i |x1:i−1, xi+1:i+m)

=

i+m
(cid:88)

j=i+1

i+m
(cid:88)

j=i+1

log p(xj|x1, x2, ..., xi, ...xj−1) + log p(xi|x1:i−1)−

(9)

log p(xj|x1, x2, ..., x∗

i , ...xj−1) − log p(x∗

i |x1:i−1),

where x1:i denotes the sequence x1, x2, ...xi, and x∗

i = arg maxx′

i∈V p(x′

i|x1:i−1).

4

162
163

164
165
166
167
168
169

170
171
172
173
174
175

176
177
178
179
180

181
182
183
184
185
186

187
188
189
190
191
192

193
194
195
196
197
198

199
200
201
202
203

204
205
206
207
208
209

210
211
212
213
214
215

Under review as a conference paper at ICLR 2025

216
217

218
219
220
221
222
223

224
225
226
227
228
229

230
231
232
233
234

235
236
237
238
239
240

241
242
243
244
245
246

247
248
249
250
251
252

253
254
255
256
257

258
259
260
261
262
263

264
265
266
267
268
269

As suggested in Zhang et al. (2024), we normalize the terms to compute our infilling score for a given
token x3:

InfillingScoretoken(xi) =

−

i+m
(cid:88)

j=i+1

i+m
(cid:88)

j=i+1

log p(xj|x1, x2, ..., xi, ...xj−1) − µx1:j
σx1:j

+

log p(xi|x1:i−1) − µx1:i
σx1:i

log p(xj|x1, x2, ..., x∗
σx1:j

i , ...xj−1) − µx1:j

−

log p(x∗

i |x1:i−1) − µx1:i

σx1:i

(10)
(cid:113)Ez∼p(.|x1:j )[(log p(z|x1:j) − µx1:j )2], are
where µx1:j = Ez∼p(.|x1:j )[log p(z|x1:j)], and σx1:j =
the mean and standard deviation of the next token log probability, log p(xj|x1, ..., xj−1), over the
whole vocabulary. In contrast to equation 5, there is no normalization in the denominator needed in
equation 10. Note that the non-causal terms in equation 6 are all replaced by causal terms which can
be computed through LLM logits. To implement, we need two calls to the LLM – the first with input
as the sequence x1, ..., xi, ..., xN and the second call with input as x1, ..., x∗
i , ..., xN . Note that the
means and standard deviations can be computed from these logits. Thus, equation 10 requires only
two calls to the LLM per token. Hence with N tokens, the total number of calls to the LLM scales
as 2N , in contrast to the naive approach where the scaling is N |V|. We will see in our experiments
(see Section 4.5) that this leads to a dramatic decrease in runtime, with two orders of magnitude
improvement.

3.2 PRETRAINING DATA DETECTION

To detect the membership of a given sentence x, we find the set of min-k% tokens with low Infilling
Scores in the sentence, and compute the average score over this subset. Our final test-statistic
becomes:

InfillingScore(x) =

1
|min-k%|

(cid:88)

xi∈min-k%

InfillingScoretoken(xi).

(11)

Our experiments suggest that InfillingScore(x) is higher for a given sentence x which was seen by
the model during pretraining. Thus, the infilling score enables us to build a detector h(·, M) for an
LLM M to infer the membership class of x as:

h(x, M) =

(cid:26)0
1

InfillingScore(x) < τ
otherwise

,

(12)

where τ denotes the binary threshold that is applied on the soft scores.

4 EXPERIMENTS

4.1 BENCHMARKS

We conduct comprehensive tests to evaluate the performance of our newly proposed test-statistic
against state-of-the-art reference-based and reference-free methods. We experiment with various
models and different parameter sizes. Initially, we examine the established pretraining data detection
benchmarks: WikiMIA (Shi et al., 2024) and MIMIR (Duan et al., 2024). WikiMIA is a temporal
MIA dataset commonly used for evaluating pretraining data detection methods. This benchmark
contains excerpts from Wikipedia event articles, and classifies samples based on the timestamp of
the articles. Samples from articles published before the training of an LLM are classified as “seen”,
and samples after the training are classified as “unseen”. Hence, this benchmark applies only to a
subset of LLMs, depending on their training and release time. WikiMIA has four different subsets
with sequence lengths of 32, 64, 128, and 256. Zhang et al. (2024) also published a “Paraphrased”
version of WikiMIA which uses ChatGPT to paraphrase the samples.

A more challenging benchmark, MIMIR (Duan et al., 2024), aims to evaluate pretraining data
detection methods when the distributions of “seen” and “unseen” text samples have high n-gram
overlap. MIMIR consists of samples from the Pile (Gao et al., 2020) across seven domains: English

5

Under review as a conference paper at ICLR 2025

270
271

272
273
274
275
276
277

278
279
280
281
282
283

284
285
286
287
288

289
290
291
292
293
294

295
296
297
298
299
300

301
302
303
304
305
306

307
308
309
310
311

312
313
314
315
316
317

318
319
320
321
322
323

Wikipedia, ArXiv, Github, Pile CC, PubMed Central, DM Mathematics, and HackerNews. Parts from
the train subset of the Pile are labeled as “seen” while parts of the test set are labeled as “unseen”.
These seen and unseen samples are selected to have very high n-gram overlaps, making it significantly
more challenging to infer training data membership.

Previous membership inference benchmarks such as WikiMIA, BookMIA (Shi et al., 2024), and
BookTection (Duarte et al., 2024) cannot be reliably used for Llama-3 because the model was trained
more recently. To address this, we curate a new dataset consisting of book excerpts published after
the release of Llama-3 labeled as “unseen” data. In this new dataset the “seen” data comes from
classical fiction books published before 1965. We sample a set of 100 excerpts, with each excerpt
containing 200 tokens. The “unseen” data consists of excerpts from books published after April 2024,
similarly having size of 200 tokens.

4.2 MODELS AND METRICS

We use the WikiMIA benchmark to evaluate our Infilling Score method on Llama (7B, 13B, 30B)
(Touvron et al., 2023a), Pythia (2.8B, 6.9B) (Biderman et al., 2023), GPT-NeoX-20B (Black et al.,
2022), and Mamba-1.4B (Gu & Dao, 2023) models. WikiMIA is applicable to models released
between 2017 and 2023. Samples from the Wikipedia event articles published in and after 2023 are
labeled as “unseen”, and samples from articles published before 2017 are labeled as “unseen”.

For experiments on the MIMIR benchmark, we evaluate our method using Pythia (160M and 1.4B) on
a subset of the Pile (Gao et al., 2020) dataset sampled across seven different domains. Pythia model
has been pretrained on the training set of the Pile dataset (Biderman et al., 2023). Therefore, MIMIR
benchmark has labeled samples from the train/test of the Pile as “seen”/“unseen”, respectively.

We evaluate Infilling Score for membership classification against the state-of-the-art methods using
the area under the ROC curve (AUROC) metric. As suggested in prior studies (Carlini et al.,
2022; Mireshghallah et al., 2022), we also report the True Positive rate at low False Positive rate
(TPR@5%FPR).

4.3 BASELINES

We compare our proposed method with multiple state-of-the-art methods as our baselines. Reference
method (Carlini et al., 2021) relies on the ratio of the sample perplexity (e.g. next token likelihood)
estimated by the target model to the sample perplexity estimated by a smaller reference model. Zlib
is another reference-based method which uses the Zlib compression entropy for calibrating the score
(Carlini et al., 2021). Neighbor method (Mattern et al., 2023a) replaces tokens within a sequence
using a pretrained masked language model to generate similar sentences. The method identifies if a
sample belongs to the training data by comparing the loss of the original sample with the average loss
of its neighboring sentences. The same algorithm is also used for detecting machine generated text in
(Mitchell et al., 2023). We compare our results with both Min-K%(Shi et al., 2024) and Min-K%++
methods (Zhang et al., 2024) extensively for performance evaluations because both methods are the
current state-of-the-art reference-free baselines, falling under the same category as our Infilling Score.

4.4 RESULTS

4.4.1 EVALUATION ON WIKIMIA

Table 1 presents the results comparing our Infilling Score method with state-of-the-art methods
evaluated on the WikiMIA benchmark. In addition, we evaluate the effectiveness of our method using
TPR at low FPR in Table 2. Our experimental setup is consistent with prior work such as Min-K%++
and Min-K%. For 32-token sequences we only use one future token, and for longer sequences we use
5 future tokens. We fix k = 20% across all experiments.

On average, our method shows a 5% improvement in AUC over Min-K%++ across various model
sizes and different inputs sequence lengths. As hypothesized in Section 3, Infilling Score consistently
outperforms existing reference-based and reference-free methods in detecting Llama pretraining data.
We empirically show that predicting the token-level likelihoods, using the information in both the
past and future tokens is more accurate for pretraining data detection. For longer sequences. This is
specially helpful for samples with longer sequence lengths where there are more tokens in the context

6

Under review as a conference paper at ICLR 2025

324
325

326
327
328
329
330
331

332
333
334
335
336
337

338
339
340
341
342

343
344
345
346
347
348

349
350
351
352
353
354

355
356
357
358
359
360

361
362
363
364
365

366
367
368
369
370
371

372
373
374
375
376
377

Seq.

Method

length

32

64

128

256

Infilling Score (Ours)
Min-K%++ (Zhang et al., 2024)
Min-K%(Shi et al., 2024)
Neighbor (Mattern et al., 2023a)
Zlib (Carlini et al., 2021)
Ref (Carlini et al., 2021)

Infilling Score (Ours)
Min-K%++ (Zhang et al., 2024)
Min-K%(Shi et al., 2024)
Neighbor (Mattern et al., 2023a)
Zlib (Carlini et al., 2021)
Ref (Carlini et al., 2021)

Infilling Score (Ours)
Min-K%++ (Zhang et al., 2024)
Min-K%(Shi et al., 2024)
Neighbor (Mattern et al., 2023a)
Zlib (Carlini et al., 2021)
Ref (Carlini et al., 2021)

Infilling Score (Ours)
Min-K%++ (Zhang et al., 2024)
Min-K%(Shi et al., 2024)
Zlib (Carlini et al., 2021)

Mamba-1.4B

NeoX-20B

Pythia-2.8B

Pythia-6.9B

Llama-7B

Llama-13B

Llama-30B Average

Ori.

66.6
66.8
63.2
64.1
61.9
62.2

67.3
67.2
62.2
60.6
60.4
60.6

69.6
68.8
66.8
64.8
65.6
65.2

70.1
65.5
69.8
67.6

Para.

66.1
66.1
62.9
63.6
62.3
62.3

62.9
63.3
58.0
60.6
59.1
59.6

66.6
65.6
64.5
62.6
65.3
61.1

-
-
-
-

Ori.

75.6
75.0
71.8
70.2
69.0
67.2

76.8
76.0
72.2
67.1
67.6
65.7

78.1
75.9
75.0
71.6
71.8
67.8

77.0
71.9
78.0
73.2

Para. Ori.

Para. Ori.

Para. Ori.

Para. Ori.

Para. Ori.

Para.

73.1
69.6
69.7
68.3
68.2
66.3

73.1
67.5
66.1
67.4
66.4
65.9

74.9
72.2
72.6
69.6
71.8
67.8

-
-
-
-

65.0
64.4
61.8
62.1
62.1
61.3

65.7
65.0
61.2
61.3
60.5
59.6

67.1
66.8
66.9
65.2
65.0
59.6

73.6
63.9
70.0
69.3

63.9
62.4
61.7
64.5
62.3
61.2

58.9
58.5
56.8
59.6
59.0
59.2

64.1
63.4
64.7
61.9
65.0
59.5

-
-
-
-

69.7
70.3
66.3
65.8
64.3
63.6

71.4
71.6
65.0
63.2
62.6
62.4

70.4
70.4
69.5
67.5
67.6
63.3

70.5
65.5
71.1
69.8

68.2
68.0
65.2
65.5
64.2
63.5

64.2
64.8
61.1
63.1
61.6
62.9

67.5
66.8
67.0
64.3
67.4
62.9

-
-
-
-

88.1
85.1
66.3
-
66.7
-

89.7
85.7
63.3
-
63.4
-

87.6
85.7
70.1
-
68.3
-

96.6
82.5
72.4
71.2

88.0
84.0
67.0
-
67.3
-

86.8
80.8
61.8
-
63.6
-

83.4
82.2
68.1
-
68.4
-

-
-
-
-

88.6
84.8
68.0
65.8
67.8
57.9

90.1
86.7
66.0
64.1
65.3
63.4

88.3
83.9
71.5
68.3
69.7
62.6

95.3
82.3
72.9
73.1

87.0
82.7
68.4
65.0
68.3
56.2

84.5
78.8
64.0
64.7
65.3
60.9

83.5
76.3
68.7
64.0
69.6
59.7

-
-
-
-

87.3
84.3
70.1
67.6
69.8
63.5

88.3
84.7
68.5
67.1
67.5
69.0

86.7
82.6
73.9
72.2
71.8
71.9

89.8
77.3
72.1
72.8

84.7
81.2
70.7
66.3
70.4
62.4

81.2
74.9
65.7
66.7
67.4
65.4

79.5
73.8
70.2
67.2
71.5
70.0

-
-
-
-

76.56
74.62
66.65
65.73
66.04
62.3

75.78
73.25
63.71
63.79
63.55
63.88

76.23
73.88
69.25
66.60
68.48
64.28

81.84
72.70
72.33
71.00

Table 1: AUROC results on the Original and Paraphrased subsets of the WikiMIA benchmark (Shi
et al., 2024). Note that the paraphrased version of the 256-token subset of WikiMIA is not published
on HuggingFace which is why some results are missing for 256 tokens. Bold shows the best result
and underline shows the second best results in each section. As seen, our Infilling Score method
outperforms previous work for detecting pretraining samples for EleutherAI’s Pythia (Biderman et al.,
2023) and GPT-NeoX (Black et al., 2022), Mamba (Gu & Dao, 2023), and Meta’s Llama (Touvron
et al., 2023a) models across various model sizes.

Seq.

Method

length

32

64

128

256

Infilling Score (Ours)
Min-K%++ (Zhang et al., 2024)
Min-K%(Shi et al., 2024)
Neighbor (Mattern et al., 2023a)
Zlib (Carlini et al., 2021)
Ref (Carlini et al., 2021)

Infilling Score (Ours)
Min-K%++ (Zhang et al., 2024)
Min-K%(Shi et al., 2024)
Neighbor (Mattern et al., 2023a)
Zlib (Carlini et al., 2021)
Ref (Carlini et al., 2021)

Infilling Score (Ours)
Min-K%++ (Zhang et al., 2024)
Min-K%(Shi et al., 2024)
Neighbor (Mattern et al., 2023a)
Zlib (Carlini et al., 2021)
Ref (Carlini et al., 2021)

Infilling Score (Ours)
Min-K%++ (Zhang et al., 2024)
Min-K%(Shi et al., 2024)
Zlib (Carlini et al., 2021)

Mamba-1.4B

NeoX-20B

Pythia-2.8B

Pythia-6.9B

Llama-7B

Llama-13B

Llama-30B Average

Ori.

14.0
12.9
14.7
11.9
15.5
7.8

19.4
16.6
19.4
8.8
14.1
4.6

16.6
16.6
16.6
15.8
19.4
10.1

25.5
15.7
13.7
23.5

Para.

16.5
10.6
15.2
7.2
13.2
5.9

10.2
7.0
8.4
9.5
15.1
8.1

15.8
10.1
14.4
13.7
17.3
11.5

-
-
-
-

Ori.

27.6
19.4
27.9
22.2
19.9
1.5

27.8
20.4
20.4
13.0
16.6
15.5

25.9
23.0
25.2
15.8
23.0
15.8

29.4
13.7
21.6
23.5

Para. Ori.

Para. Ori.

Para. Ori.

Para. Ori.

Para. Ori.

Para.

23.0
12.9
19.6
15.2
18.6
15.2

21.8
13.0
17.6
18.3
19.4
14.1

33.1
19.4
22.3
18.7
21.6
19.4

-
-
-
-

13.7
14.2
17.1
15.0
15.8
6.2

18.0
16.2
18.3
10.2
14.4
10.6

15.8
17.3
13.7
8.6
18.7
10.1

19.6
13.7
13.7
19.6

13.7
13.9
16.5
8.5
14.5
7.2

13.4
9.9
11.3
11.3
16.6
13.0

13.4
14.4
14.4
12.2
16.6
7.2

-
-
-
-

17.3
17.1
17.8
16.5
16.3
6.7

21.1
26.1
19.0
10.9
16.2
12.0

20.9
22.3
18.0
10.8
20.9
13.7

29.4
11.8
15.7
27.5

20.7
17.1
21.7
9.6
12.7
6.2

14.8
14.1
12.7
12.7
15.8
16.2

21.6
21.6
17.3
17.3
20.9
8.6

-
-
-
-

34.1
33.6
15.2
-
13.7
-

50.7
39.4
14.4
-
11.3
-

38.1
46.8
19.4
-
14.4
-

80.4
47.1
17.6
21.6

35.9
31.5
16.0
-
14.2
-

28.5
26.8
13.7
-
14.8
-

33.8
38.8
21.6
-
18.7
-

-
-
-
-

30.5
38.5
18.9
11.6
11.6
4.7

53.5
34.1
17.2
10.2
12.7
4.2

41.0
41.0
25.9
12.9
18.7
10.8

80.4
37.3
19.6
27.5

29.7
35.9
17.6
8.5
15.0
5.4

34.9
26.4
13.4
14.4
13.4
4.6

30.9
21.5
14.4
11.6
16.9
8.1

-
-
-
-

33.1
31.3
21.2
9.3
14.5
9.8

44.0
36.3
17.6
9.9
15.5
11.3

24.5
38.1
23.7
15.1
18.0
10.8

72.5
19.6
13.7
29.4

38.2
27.4
18.1
9.3
15.0
7.5

27.8
21.5
14.4
11.6
16.9
8.1

31.7
21.6
18.7
14.4
19.4
18.7

-
-
-
-

24.85
22.59
18.39
12.07
15.03
7.01

27.56
21.98
15.55
11.73
15.20
10.19

25.93
25.18
18.97
13.91
18.89
12.07

48.17
22.70
16.51
24.66

Table 2: True Positive rate at low False Positive rate (FPR=5%) results on the Original and Paraphrased
subsets of the WikiMIA benchmark (Shi et al., 2024). Note that the paraphrased version of the 256-
token subset of WikiMIA is not published on HuggingFace, which is why some results are missing for
256 tokens. Bold shows the best results and underline shows the second best results in each section.
As shown, our Infilling Score method on average achieves higher True Positive rate compared to
existing methods, with the best performance on 256-token long sequences.

to use for inference. Since our method offers the capability to leverage the future as well as past
tokens, it shows a significant gain over current state-of-the-art method when input sequences are long.

4.4.2 EVALUATION ON MIMIR

Table 3 shows the results comparing our Infilling Score method with SoTA methods evaluated on
the challenging MIMIR benchmark. In the MIMIR dataset, samples from the “seen” and “unseen”
classes are sampled from the same dataset to ensure 13-gram overlap of up to 0.8 between the classes.
Reference-based models show high performance on this benchmark. However, the drawback of this

7

Under review as a conference paper at ICLR 2025

Wikipedia

Github

Pile CC

PubMed Central

Method

160M 1.4B 160M 1.4B 160M 1.4B 160M 1.4B

Infilling Score (Ours)
Min-K%++ (Zhang et al., 2024)
Min-K%(Shi et al., 2024)
Zlib (Carlini et al., 2021)
Ref (Carlini et al., 2021)

49.7
49.7
50.2
51.1
51.2

53.4
53.7
51.3
52.0
55.2

65.5
64.8
65.7
67.4
63.9

70.0
69.6
69.9
71.0
67.1

ArXiv

DM Math

53.3
53.7
51.0
50.6
51.0
50.3
50.1
49.6
49.2
52.2
HackerNews

52.3
50.6
50.6
49.9
51.3

53.5
51.4
50.3
50.0
53.1
Average

Method

160M 1.4B 160M 1.4B 160M 1.4B 160M 1.4B

Infilling Score (Ours)
Min-K%++ (Zhang et al., 2024)
Min-K%(Shi et al., 2024)
Zlib (Carlini et al., 2021)
Ref (Carlini et al., 2021)

51.0
50.1
51.0
50.1
49.4

51.3
51.1
51.7
50.9
51.5

53.5
50.5
49.4
48.1
51.1

50.4
50.9
49.7
48.2
51.1

50.9
50.7
50.9
49.7
49.1

52.6
51.3
51.3
50.3
52.2

53.4
52.4
52.6
52.3
52.2

54.9
54.1
53.6
53.2
54.6

Table 3: AUROC results on MIMIR dataset (Duan et al., 2024) for Pythia models for different
sizes. Similar to Zhang et al. (2024), we experiment on a subset of MIMIR with maximum 13-gram
overlap of 0.8 between samples form “seen” and “unseen” class. Bold shows the best results and
underline shows the second best results in each section. As shown, our Infilling Score method overall
outperforms existing reference-free and reference-based methods.

Year Pub. Book Title

Contamination Rate

.

1817
2006
1812
2003
1986
2009
1991
2009
1998
1996
2009
1889
2003
2009
1982
2000
2008
2007
2007
2005
2006
2008

Persuasion
Oakleaf bearers
Grimms’ Fairy Tales
The Sacred Land
Howl’s Moving Castle
CATCHING FIRE
Red Magic
Tenth Grade Bleeds
Mad Ship
Too Good to Leave, Too Bad to Stay
Crouching Vampire, Hidden Fang
Three Men in a Boat (To Say Nothing of the Dog)
Something from the Nightside
The Silver Eagle
The Man From St. Petersburg
Ship of Destiny
The Painted Man
The Center Cannot Hold
Raintree: Sanctuary
Sister of the Dead
The Corfu Trilogy
Ascendancy of the Last

99
76
73
73
69
68
66
64
61
58
56
56
54
53
53
53
53
52
52
52
50
50

Table 4: Books detected in the pretraining data of Llama-3-8B (Dubey et al., 2024). Contamination
rate shows the percentage of excerpts sampled from the books which were classified as “seen” using
the Infilling Score method.

approach is that it requires testing multiple different LLMs to determine the best performing baseline
(Duan et al., 2024; Zhang et al., 2024). Despite the competitive nature of the benchmark, our Infilling
Score achieves the best performance compared to both reference-free and reference-based models on
average over different domains.

4.4.3 DETECTING PRETRAINING DATA OF LLAMA-3

We apply Infilling Score to detect books that were likely used in the pretraining of the Llama3-8B
model, recently released by Meta (Dubey et al., 2024). Llama3 is known to be trained using over 15T
tokens of data (7x larger training set than Llama-2) according to Dubey et al. (2024). No information
about the source and distribution of this data is disclosed by the developers, making it difficult to
construct a labeled MIA dataset of books suitable for this model.

We used our books dataset as a validation set to find the best hyperparameters (k% and # future
tokens, m, and the classification threshold τ ) for identifying samples used in pretraining Llama3.
Since Llama3 has been released in 2024, existing temporal benchmarks such as WikiMIA, BookMIA
(Shi et al., 2024), and BookTection (Duarte et al., 2024) cannot be used for pretraining data detection
on this model. We found that using the next 100 tokens when calculating the Infilling Score shows the
highest accuracy on this benchmark. Table 12 shows the performance of our method on this dataset.

8

378
379

380
381
382
383
384
385

386
387
388
389
390
391

392
393
394
395
396

397
398
399
400
401
402

403
404
405
406
407
408

409
410
411
412
413
414

415
416
417
418
419

420
421
422
423
424
425

426
427
428
429
430
431

Under review as a conference paper at ICLR 2025

432
433

434
435
436
437
438
439

440
441
442
443
444
445

446
447
448
449
450

451
452
453
454
455
456

457
458
459
460
461
462

463
464
465
466
467
468

469
470
471
472
473

474
475
476
477
478
479

480
481
482
483
484
485

Figure 1: Figure shows an example of the distribution of the Infilling Scores for “seen” and “unseen”
excerpts in our validation dataset which consists of text from fiction books. Scores are normalized in
each distribution. The unseen data comes from recent novels published after the training of Llama
3. For the classic novel Persuasion, our method detects 99% of the excerpt to be in the training set.
As seen in this histogram, the distribution for Persuasion matches other seen novels and is clearly
separated from unseen data, as one would expect.

We employ our method on 20,000 excerpts sampled from 200 books. Table 4 presenters the list of
books which we found to be in the training dataset of Llama3-8B with ≥ 50% contamination rate.
Contamination rate shows the percentage of excerpts detected as ’seen’ for each publication. Figure 1
shows that books with high contamination rate have higher sample statistic overlap with the “seen”
excerpts in our validation dataset.

4.4.4 ABLATION STUDY ON THE NUMBER OF FUTURE TOKENS TO USE

It is important to note that the number of future tokens used to calculate the Infilling Score determines
the performance gain of our method. As shown in the Figure 2 increasing the number of future tokens
does not necessarily lead to a higher AUC. However, on the WikiMIA benchmark, using about 5
future tokens leads to relatively better AUC across various context lengths on WikiMIA using Llama-
7B and Llama-13B. We conduct all experiments with different input sequence lengths (32, 64, 128,
and 256) to examine the effect of the number of future tokens across various context lengths. While
the ideal number of next tokens to use remains consistent across various model sizes, the optimal
number may differ depending on data distribution and model architecture. We investigate various
values for m within {0, 1, 3, 5, 10, 20, . . . , N }, where N represents the input sequence length. It’s
important to note that the hyperparameter search does not increase the computational complexity, as
incorporating additional future tokens does not require extra calls to the LLM. We provide additional
results in Appendix A.

Figure 2: The figures show the AUROC achieved by the Infilling Score as the number of future
tokens increases. These results are shown for input sequence lengths of 32, 64, 128, and 256. The
left figure presents the results for Llama-7B, while the right figure shows the results for Llama-13B.
Our baseline, representing existing methods, uses zero future tokens. The optimal number of future
tokens to use is 1 for sequences of 32 tokens. For longer sequences of up to 256 tokens, the optimal
number is around 5 for both models.

9

Under review as a conference paper at ICLR 2025

486
487

488
489
490
491
492
493

494
495
496
497
498
499

500
501
502
503
504

505
506
507
508
509
510

511
512
513
514
515
516

517
518
519
520
521
522

523
524
525
526
527

528
529
530
531
532
533

534
535
536
537
538
539

4.5 ALGORITHM RUNTIME

Table 5 compares the runtime of our Infilling Score algorithm with straightforward application
of Bayes, and Min-K%++ (Shi et al., 2024) using Llama-7B. Although both the naive approach
and Infilling score are slower than Min-K%, these methods yield a more accurate estimate of
token likelihoods for membership inference. Note that our proposed test-statistic, Infilling Score,
significantly reduces the computational complexity compared to the naive approach, delivering an
accurate membership inference score within a feasible runtime. WikiMIA dataset has 776 sequences
of length 32, 542 of length 64, 250 of length 128, and 82 of length 256 tokens. The compute
cost increases with sequence length. The 256-token sequences require approximately 2,460 seconds
compared to 776 seconds for 32-token sequences (30 seconds per sequence), highlighting the trade-off
between detection accuracy and computational efficiency.

Seq. length Min-K%++
0.028 sec.
0.042 sec.
0.064 sec.
0.106 sec.

32
64
128
256

Infilling Score Naive Approach

0.952 sec.
3.11 sec.
9.47 sec.
29.98 sec.

207 sec.
334 sec.
581 sec.
1141 sec.

Table 5: Algorithm runtime results comparing Infilling Score, Min-K%++, and the naive approach
discussed in Section 3, sequences of 32, 64, 128, and 256 tokens using Llama-7B on a H200 GPU.

To evaluate the impact of the number of future tokens used, m, on the runtime, we measure the
runtime using 1, 5, and 10 future tokens. As discussed in Section 3.1, the number of LLM calls
required by our Infilling Score algorithm is independent of the number of future tokens used. However,
increasing the number of future tokens also increases the number of terms in the summations in
equation 10. The additional computations have a minimal impact on the runtime as shown in Table 6.

Seq. length

# future tokens

1

32

5

10

1

64

5

10

1

128

5

10

1

256

5

10

Runtime

0.952 sec.

0.953 sec.

0.956 sec.

3.11 sec.

3.12 sec.

3.12 sec.

9.47 sec.

9.48 sec.

9.49 sec.

29.98 sec.

30.01 sec.

30.04 sec.

Table 6: Algorithm runtime as the number of future tokens used increases. As the table indicates,
increasing the number of future tokens to use has minimal impact on runtime.

5 CONCLUSIONS

Limitations One limitation is that computing the Infilling Score requires grey-box access to the LLM,
meaning access to the sample log probabilities estimated by the model. This requirement is common
among most of the existing membership inference methods. Another limitation of our approach lies
in its runtime complexity. As described in Section 3.1, the order of LLM calls required for computing
the infilling likelihood (for a sequence of length N ) with the naive Bayes method is N |V|, which
scales linearly with both sequence length N , and vocabulary size |V|. By introducing the Infilling
Score, we reduce the number of LLM calls to 2N . However, prior methods such as Min-K%and
Min-K%++ require only a single LLM call (to test a sequence of length N ), and are faster compared
to our proposed algorithm.

To conclude, we proposed a novel method that can detect if text sequences have been present in the
training set with significantly better accuracy compared to prior work. Our new test statistic allows us
to derive non-causal likelihoods (up to a multiplicative factor) from pre-trained autoregressive models
and may have other uses, beyond membership inference. Although our method is slower compared to
previous methods, it can be practically run in a few seconds for large foundation models.

Our results present evidence that numerous books and other recent sources of text have been in the
training data of modern LLMs. This test can further be used for measuring dataset contamination
rates, and also evaluating decontamination methods. An important research direction would be to
create larger evaluation datasets for membership inference, and include high n-gram overlap samples
for recent sources that remain unseen to llama3 and other recently released frontier models.

10

Under review as a conference paper at ICLR 2025

540
541

542
543
544
545
546
547

548
549
550
551
552
553

554
555
556
557
558

559
560
561
562
563
564

565
566
567
568
569
570

571
572
573
574
575
576

577
578
579
580
581

582
583
584
585
586
587

588
589
590
591
592
593

REFERENCES

Raef Bassily, Vitaly Feldman, Cristóbal Guzmán, and Kunal Talwar. Stability of stochastic gradient

descent on nonsmooth convex losses, 2020.

Stella Biderman, Hailey Schoelkopf, Quentin Anthony, Herbie Bradley, Kyle O’Brien, Eric Hallahan,
Mohammad Aflah Khan, Shivanshu Purohit, USVSN Sai Prashanth, Edward Raff, Aviya Skowron,
Lintang Sutawika, and Oskar van der Wal. Pythia: A suite for analyzing large language models
across training and scaling, 2023.

Sid Black, Stella Biderman, Eric Hallahan, Quentin Anthony, Leo Gao, Laurence Golding, Horace He,
Connor Leahy, Kyle McDonell, Jason Phang, Michael Pieler, USVSN Sai Prashanth, Shivanshu
Purohit, Laria Reynolds, Jonathan Tow, Ben Wang, and Samuel Weinbach. Gpt-neox-20b: An
open-source autoregressive language model, 2022.

Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine
Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, Alina Oprea, and Colin Raffel.
Extracting training data from large language models, 2021.

Nicholas Carlini, Steve Chien, Milad Nasr, Shuang Song, Andreas Terzis, and Florian Tramer.

Membership inference attacks from first principles, 2022.

Kent K. Chang, Mackenzie Cramer, Sandeep Soni, and David Bamman. Speak, memory: An

archaeology of books known to chatgpt/gpt-4, 2023.

Michael Duan, Anshuman Suri, Niloofar Mireshghallah, Sewon Min, Weijia Shi, Luke Zettlemoyer,
Yulia Tsvetkov, Yejin Choi, David Evans, and Hannaneh Hajishirzi. Do membership inference
attacks work on large language models? arXiv preprint arXiv:2402.07841, 2024.

André V. Duarte, Xuandong Zhao, Arlindo L. Oliveira, and Lei Li. De-cop: Detecting copyrighted

content in language models training data, 2024.

Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha
Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn,
Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, Arun Rao, Aston
Zhang, Aurelien Rodriguez, Austen Gregerson, Ava Spataru, Baptiste Roziere, Bethany Biron,
Binh Tang, Bobbie Chern, Charlotte Caucheteux, Chaya Nayak, Chloe Bi, Chris Marra, Chris
McConnell, Christian Keller, Christophe Touret, Chunyang Wu, Corinne Wong, Cristian Canton
Ferrer, Cyrus Nikolaidis, Damien Allonsius, Daniel Song, Danielle Pintz, Danny Livshits, David
Esiobu, Dhruv Choudhary, Dhruv Mahajan, Diego Garcia-Olano, Diego Perino, Dieuwke Hupkes,
Egor Lakomkin, Ehab AlBadawy, Elina Lobanova, Emily Dinan, Eric Michael Smith, Filip
Radenovic, Frank Zhang, Gabriel Synnaeve, Gabrielle Lee, Georgia Lewis Anderson, Graeme
Nail, Gregoire Mialon, Guan Pang, Guillem Cucurell, Hailey Nguyen, Hannah Korevaar, Hu Xu,
Hugo Touvron, Iliyan Zarov, Imanol Arrieta Ibarra, Isabel Kloumann, Ishan Misra, Ivan Evtimov,
Jade Copet, Jaewon Lee, Jan Geffert, Jana Vranes, Jason Park, Jay Mahadeokar, Jeet Shah,
Jelmer van der Linde, Jennifer Billock, Jenny Hong, Jenya Lee, Jeremy Fu, Jianfeng Chi, Jianyu
Huang, Jiawen Liu, Jie Wang, Jiecao Yu, Joanna Bitton, Joe Spisak, Jongsoo Park, Joseph
Rocca, Joshua Johnstun, Joshua Saxe, Junteng Jia, Kalyan Vasuden Alwala, Kartikeya Upasani,
Kate Plawiak, Ke Li, Kenneth Heafield, Kevin Stone, Khalid El-Arini, Krithika Iyer, Kshitiz
Malik, Kuenley Chiu, Kunal Bhalla, Lauren Rantala-Yeary, Laurens van der Maaten, Lawrence
Chen, Liang Tan, Liz Jenkins, Louis Martin, Lovish Madaan, Lubo Malo, Lukas Blecher, Lukas
Landzaat, Luke de Oliveira, Madeline Muzzi, Mahesh Pasupuleti, Mannat Singh, Manohar Paluri,
Marcin Kardas, Mathew Oldham, Mathieu Rita, Maya Pavlova, Melanie Kambadur, Mike Lewis,
Min Si, Mitesh Kumar Singh, Mona Hassan, Naman Goyal, Narjes Torabi, Nikolay Bashlykov,
Nikolay Bogoychev, Niladri Chatterji, Olivier Duchenne, Onur Çelebi, Patrick Alrassy, Pengchuan
Zhang, Pengwei Li, Petar Vasic, Peter Weng, Prajjwal Bhargava, Pratik Dubal, Praveen Krishnan,
Punit Singh Koura, Puxin Xu, Qing He, Qingxiao Dong, Ragavan Srinivasan, Raj Ganapathy,
Ramon Calderer, Ricardo Silveira Cabral, Robert Stojnic, Roberta Raileanu, Rohit Girdhar, Rohit
Patel, Romain Sauvestre, Ronnie Polidoro, Roshan Sumbaly, Ross Taylor, Ruan Silva, Rui Hou,
Rui Wang, Saghar Hosseini, Sahana Chennabasappa, Sanjay Singh, Sean Bell, Seohyun Sonia
Kim, Sergey Edunov, Shaoliang Nie, Sharan Narang, Sharath Raparthy, Sheng Shen, Shengye Wan,
Shruti Bhosale, Shun Zhang, Simon Vandenhende, Soumya Batra, Spencer Whitman, Sten Sootla,

11

Under review as a conference paper at ICLR 2025

594
595

596
597
598
599
600
601

602
603
604
605
606
607

608
609
610
611
612

613
614
615
616
617
618

619
620
621
622
623
624

625
626
627
628
629
630

631
632
633
634
635

636
637
638
639
640
641

642
643
644
645
646
647

Stephane Collot, Suchin Gururangan, Sydney Borodinsky, Tamar Herman, Tara Fowler, Tarek
Sheasha, Thomas Georgiou, Thomas Scialom, Tobias Speckbacher, Todor Mihaylov, Tong Xiao,
Ujjwal Karn, Vedanuj Goswami, Vibhor Gupta, Vignesh Ramanathan, Viktor Kerkez, Vincent
Gonguet, Virginie Do, Vish Vogeti, Vladan Petrovic, Weiwei Chu, Wenhan Xiong, Wenyin Fu,
Whitney Meers, Xavier Martinet, Xiaodong Wang, Xiaoqing Ellen Tan, Xinfeng Xie, Xuchao Jia,
Xuewei Wang, Yaelle Goldschlag, Yashesh Gaur, Yasmine Babaei, Yi Wen, Yiwen Song, Yuchen
Zhang, Yue Li, Yuning Mao, Zacharie Delpierre Coudert, Zheng Yan, Zhengxing Chen, Zoe
Papakipos, Aaditya Singh, Aaron Grattafiori, Abha Jain, Adam Kelsey, Adam Shajnfeld, Adithya
Gangidi, Adolfo Victoria, Ahuva Goldstand, Ajay Menon, Ajay Sharma, Alex Boesenberg, Alex
Vaughan, Alexei Baevski, Allie Feinstein, Amanda Kallet, Amit Sangani, Anam Yunus, Andrei
Lupu, Andres Alvarado, Andrew Caples, Andrew Gu, Andrew Ho, Andrew Poulton, Andrew
Ryan, Ankit Ramchandani, Annie Franco, Aparajita Saraf, Arkabandhu Chowdhury, Ashley
Gabriel, Ashwin Bharambe, Assaf Eisenman, Azadeh Yazdan, Beau James, Ben Maurer, Benjamin
Leonhardi, Bernie Huang, Beth Loyd, Beto De Paola, Bhargavi Paranjape, Bing Liu, Bo Wu,
Boyu Ni, Braden Hancock, Bram Wasti, Brandon Spence, Brani Stojkovic, Brian Gamido, Britt
Montalvo, Carl Parker, Carly Burton, Catalina Mejia, Changhan Wang, Changkyu Kim, Chao
Zhou, Chester Hu, Ching-Hsiang Chu, Chris Cai, Chris Tindal, Christoph Feichtenhofer, Damon
Civin, Dana Beaty, Daniel Kreymer, Daniel Li, Danny Wyatt, David Adkins, David Xu, Davide
Testuggine, Delia David, Devi Parikh, Diana Liskovich, Didem Foss, Dingkang Wang, Duc Le,
Dustin Holland, Edward Dowling, Eissa Jamil, Elaine Montgomery, Eleonora Presani, Emily
Hahn, Emily Wood, Erik Brinkman, Esteban Arcaute, Evan Dunbar, Evan Smothers, Fei Sun, Felix
Kreuk, Feng Tian, Firat Ozgenel, Francesco Caggioni, Francisco Guzmán, Frank Kanayet, Frank
Seide, Gabriela Medina Florez, Gabriella Schwarz, Gada Badeer, Georgia Swee, Gil Halpern,
Govind Thattai, Grant Herman, Grigory Sizov, Guangyi, Zhang, Guna Lakshminarayanan, Hamid
Shojanazeri, Han Zou, Hannah Wang, Hanwen Zha, Haroun Habeeb, Harrison Rudolph, Helen
Suk, Henry Aspegren, Hunter Goldman, Ibrahim Damlaj, Igor Molybog, Igor Tufanov, Irina-
Elena Veliche, Itai Gat, Jake Weissman, James Geboski, James Kohli, Japhet Asher, Jean-Baptiste
Gaya, Jeff Marcus, Jeff Tang, Jennifer Chan, Jenny Zhen, Jeremy Reizenstein, Jeremy Teboul,
Jessica Zhong, Jian Jin, Jingyi Yang, Joe Cummings, Jon Carvill, Jon Shepard, Jonathan McPhie,
Jonathan Torres, Josh Ginsburg, Junjie Wang, Kai Wu, Kam Hou U, Karan Saxena, Karthik
Prasad, Kartikay Khandelwal, Katayoun Zand, Kathy Matosich, Kaushik Veeraraghavan, Kelly
Michelena, Keqian Li, Kun Huang, Kunal Chawla, Kushal Lakhotia, Kyle Huang, Lailin Chen,
Lakshya Garg, Lavender A, Leandro Silva, Lee Bell, Lei Zhang, Liangpeng Guo, Licheng Yu,
Liron Moshkovich, Luca Wehrstedt, Madian Khabsa, Manav Avalani, Manish Bhatt, Maria
Tsimpoukelli, Martynas Mankus, Matan Hasson, Matthew Lennie, Matthias Reso, Maxim Groshev,
Maxim Naumov, Maya Lathi, Meghan Keneally, Michael L. Seltzer, Michal Valko, Michelle
Restrepo, Mihir Patel, Mik Vyatskov, Mikayel Samvelyan, Mike Clark, Mike Macey, Mike Wang,
Miquel Jubert Hermoso, Mo Metanat, Mohammad Rastegari, Munish Bansal, Nandhini Santhanam,
Natascha Parks, Natasha White, Navyata Bawa, Nayan Singhal, Nick Egebo, Nicolas Usunier,
Nikolay Pavlovich Laptev, Ning Dong, Ning Zhang, Norman Cheng, Oleg Chernoguz, Olivia
Hart, Omkar Salpekar, Ozlem Kalinli, Parkin Kent, Parth Parekh, Paul Saab, Pavan Balaji, Pedro
Rittner, Philip Bontrager, Pierre Roux, Piotr Dollar, Polina Zvyagina, Prashant Ratanchandani,
Pritish Yuvraj, Qian Liang, Rachad Alao, Rachel Rodriguez, Rafi Ayub, Raghotham Murthy,
Raghu Nayani, Rahul Mitra, Raymond Li, Rebekkah Hogan, Robin Battey, Rocky Wang, Rohan
Maheswari, Russ Howes, Ruty Rinott, Sai Jayesh Bondu, Samyak Datta, Sara Chugh, Sara
Hunt, Sargun Dhillon, Sasha Sidorov, Satadru Pan, Saurabh Verma, Seiji Yamamoto, Sharadh
Ramaswamy, Shaun Lindsay, Shaun Lindsay, Sheng Feng, Shenghao Lin, Shengxin Cindy Zha,
Shiva Shankar, Shuqiang Zhang, Shuqiang Zhang, Sinong Wang, Sneha Agarwal, Soji Sajuyigbe,
Soumith Chintala, Stephanie Max, Stephen Chen, Steve Kehoe, Steve Satterfield, Sudarshan
Govindaprasad, Sumit Gupta, Sungmin Cho, Sunny Virk, Suraj Subramanian, Sy Choudhury,
Sydney Goldman, Tal Remez, Tamar Glaser, Tamara Best, Thilo Kohler, Thomas Robinson, Tianhe
Li, Tianjun Zhang, Tim Matthews, Timothy Chou, Tzook Shaked, Varun Vontimitta, Victoria Ajayi,
Victoria Montanez, Vijai Mohan, Vinay Satish Kumar, Vishal Mangla, Vítor Albiero, Vlad Ionescu,
Vlad Poenaru, Vlad Tiberiu Mihailescu, Vladimir Ivanov, Wei Li, Wenchen Wang, Wenwen Jiang,
Wes Bouaziz, Will Constable, Xiaocheng Tang, Xiaofang Wang, Xiaojian Wu, Xiaolan Wang,
Xide Xia, Xilun Wu, Xinbo Gao, Yanjun Chen, Ye Hu, Ye Jia, Ye Qi, Yenda Li, Yilin Zhang,
Ying Zhang, Yossi Adi, Youngjin Nam, Yu, Wang, Yuchen Hao, Yundi Qian, Yuzi He, Zach Rait,
Zachary DeVito, Zef Rosnbrick, Zhaoduo Wen, Zhenyu Yang, and Zhiwei Zhao. The llama 3 herd
of models, 2024. URL https://arxiv.org/abs/2407.21783.

12

Under review as a conference paper at ICLR 2025

648
649

650
651
652
653
654
655

656
657
658
659
660
661

662
663
664
665
666

667
668
669
670
671
672

673
674
675
676
677
678

679
680
681
682
683
684

685
686
687
688
689

690
691
692
693
694
695

696
697
698
699
700
701

Leo Gao, Stella Biderman, Sid Black, Laurence Golding, Travis Hoppe, Charles Foster, Jason Phang,
Horace He, Anish Thite, Noa Nabeshima, Shawn Presser, and Connor Leahy. The pile: An 800gb
dataset of diverse text for language modeling, 2020.

Michael M. Grynbaum and Ryan Mac. The times sues openai and microsoft over a.i. use
of copyrighted work. https://www.nytimes.com/2023/12/27/business/media/
new-york-times-open-ai-microsoft-lawsuit.html, 2023.

Albert Gu and Tri Dao. Mamba: Linear-time sequence modeling with selective state spaces, 2023.

Moritz Hardt, Benjamin Recht, and Yoram Singer. Train faster, generalize better: Stability of

stochastic gradient descent, 2016.

Minhao Jiang, Ken Ziyu Liu, Ming Zhong, Rylan Schaeffer, Siru Ouyang, Jiawei Han, and Sanmi
Koyejo. Investigating data contamination for pre-training language models, 2024. URL https:
//arxiv.org/abs/2401.06059.

Jeffrey Li, Alex Fang, Georgios Smyrnis, Maor Ivgi, Matt Jordan, Samir Gadre, Hritik Bansal, Etash
Guha, Sedrick Keh, Kushal Arora, Saurabh Garg, Rui Xin, Niklas Muennighoff, Reinhard Heckel,
Jean Mercat, Mayee Chen, Suchin Gururangan, Mitchell Wortsman, Alon Albalak, Yonatan Bitton,
Marianna Nezhurina, Amro Abbas, Cheng-Yu Hsieh, Dhruba Ghosh, Josh Gardner, Maciej Kilian,
Hanlin Zhang, Rulin Shao, Sarah Pratt, Sunny Sanyal, Gabriel Ilharco, Giannis Daras, Kalyani
Marathe, Aaron Gokaslan, Jieyu Zhang, Khyathi Chandu, Thao Nguyen, Igor Vasiljevic, Sham
Kakade, Shuran Song, Sujay Sanghavi, Fartash Faghri, Sewoong Oh, Luke Zettlemoyer, Kyle Lo,
Alaaeldin El-Nouby, Hadi Pouransari, Alexander Toshev, Stephanie Wang, Dirk Groeneveld, Luca
Soldaini, Pang Wei Koh, Jenia Jitsev, Thomas Kollar, Alexandros G. Dimakis, Yair Carmon, Achal
Dave, Ludwig Schmidt, and Vaishaal Shankar. Datacomp-lm: In search of the next generation of
training sets for language models, 2024. URL https://arxiv.org/abs/2406.11794.

Saeed Mahloujifar, Huseyin A. Inan, Melissa Chase, Esha Ghosh, and Marcello Hasegawa. Member-

ship inference on word embedding and beyond, 2021.

Justus Mattern, Fatemehsadat Mireshghallah, Zhijing Jin, Bernhard Schoelkopf, Mrinmaya Sachan,
and Taylor Berg-Kirkpatrick. Membership inference attacks against language models via neigh-
bourhood comparison. In Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki (eds.), Findings
of the Association for Computational Linguistics: ACL 2023, pp. 11330–11343, Toronto, Canada,
July 2023a. Association for Computational Linguistics. doi: 10.18653/v1/2023.findings-acl.719.
URL https://aclanthology.org/2023.findings-acl.719.

Justus Mattern, Fatemehsadat Mireshghallah, Zhijing Jin, Bernhard Schölkopf, Mrinmaya Sachan,
and Taylor Berg-Kirkpatrick. Membership inference attacks against language models via neigh-
bourhood comparison, 2023b.

Matthieu Meeus, Shubham Jain, Marek Rei, and Yves-Alexandre de Montjoye. Did the neurons read

your book? document-level membership inference for large language models, 2023.

Sewon Min, Suchin Gururangan, Eric Wallace, Hannaneh Hajishirzi, Noah A. Smith, and Luke

Zettlemoyer. Silo language models: Isolating legal risk in a nonparametric datastore, 2023.

Fatemehsadat Mireshghallah, Kartik Goyal, Archit Uniyal, Taylor Berg-Kirkpatrick, and Reza Shokri.
Quantifying privacy risks of masked language models using membership inference attacks, 2022.

Eric Mitchell, Yoonho Lee, Alexander Khazatsky, Christopher D. Manning, and Chelsea Finn.

Detectgpt: Zero-shot machine-generated text detection using probability curvature, 2023.

Maximilian Mozes, Xuanli He, Bennett Kleinberg, and Lewis D. Griffin. Use of llms for illicit

purposes: Threats, prevention measures, and vulnerabilities, 2023.

OpenAI, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni
Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, Red Avila, Igor
Babuschkin, Suchir Balaji, Valerie Balcom, Paul Baltescu, Haiming Bao, Mohammad Bavarian,
Jeff Belgum, Irwan Bello, Jake Berdine, Gabriel Bernadett-Shapiro, Christopher Berner, Lenny
Bogdonoff, Oleg Boiko, Madelaine Boyd, Anna-Luisa Brakman, Greg Brockman, Tim Brooks,

13

Under review as a conference paper at ICLR 2025

702
703

704
705
706
707
708
709

710
711
712
713
714
715

716
717
718
719
720

721
722
723
724
725
726

727
728
729
730
731
732

733
734
735
736
737
738

739
740
741
742
743

744
745
746
747
748
749

750
751
752
753
754
755

Miles Brundage, Kevin Button, Trevor Cai, Rosie Campbell, Andrew Cann, Brittany Carey, Chelsea
Carlson, Rory Carmichael, Brooke Chan, Che Chang, Fotis Chantzis, Derek Chen, Sully Chen,
Ruby Chen, Jason Chen, Mark Chen, Ben Chess, Chester Cho, Casey Chu, Hyung Won Chung,
Dave Cummings, Jeremiah Currier, Yunxing Dai, Cory Decareaux, Thomas Degry, Noah Deutsch,
Damien Deville, Arka Dhar, David Dohan, Steve Dowling, Sheila Dunning, Adrien Ecoffet, Atty
Eleti, Tyna Eloundou, David Farhi, Liam Fedus, Niko Felix, Simón Posada Fishman, Juston Forte,
Isabella Fulford, Leo Gao, Elie Georges, Christian Gibson, Vik Goel, Tarun Gogineni, Gabriel
Goh, Rapha Gontijo-Lopes, Jonathan Gordon, Morgan Grafstein, Scott Gray, Ryan Greene, Joshua
Gross, Shixiang Shane Gu, Yufei Guo, Chris Hallacy, Jesse Han, Jeff Harris, Yuchen He, Mike
Heaton, Johannes Heidecke, Chris Hesse, Alan Hickey, Wade Hickey, Peter Hoeschele, Brandon
Houghton, Kenny Hsu, Shengli Hu, Xin Hu, Joost Huizinga, Shantanu Jain, Shawn Jain, Joanne
Jang, Angela Jiang, Roger Jiang, Haozhun Jin, Denny Jin, Shino Jomoto, Billie Jonn, Heewoo
Jun, Tomer Kaftan, Łukasz Kaiser, Ali Kamali, Ingmar Kanitscheider, Nitish Shirish Keskar,
Tabarak Khan, Logan Kilpatrick, Jong Wook Kim, Christina Kim, Yongjik Kim, Jan Hendrik
Kirchner, Jamie Kiros, Matt Knight, Daniel Kokotajlo, Łukasz Kondraciuk, Andrew Kondrich,
Aris Konstantinidis, Kyle Kosic, Gretchen Krueger, Vishal Kuo, Michael Lampe, Ikai Lan, Teddy
Lee, Jan Leike, Jade Leung, Daniel Levy, Chak Ming Li, Rachel Lim, Molly Lin, Stephanie
Lin, Mateusz Litwin, Theresa Lopez, Ryan Lowe, Patricia Lue, Anna Makanju, Kim Malfacini,
Sam Manning, Todor Markov, Yaniv Markovski, Bianca Martin, Katie Mayer, Andrew Mayne,
Bob McGrew, Scott Mayer McKinney, Christine McLeavey, Paul McMillan, Jake McNeil, David
Medina, Aalok Mehta, Jacob Menick, Luke Metz, Andrey Mishchenko, Pamela Mishkin, Vinnie
Monaco, Evan Morikawa, Daniel Mossing, Tong Mu, Mira Murati, Oleg Murk, David Mély,
Ashvin Nair, Reiichiro Nakano, Rajeev Nayak, Arvind Neelakantan, Richard Ngo, Hyeonwoo
Noh, Long Ouyang, Cullen O’Keefe, Jakub Pachocki, Alex Paino, Joe Palermo, Ashley Pantuliano,
Giambattista Parascandolo, Joel Parish, Emy Parparita, Alex Passos, Mikhail Pavlov, Andrew Peng,
Adam Perelman, Filipe de Avila Belbute Peres, Michael Petrov, Henrique Ponde de Oliveira Pinto,
Michael, Pokorny, Michelle Pokrass, Vitchyr H. Pong, Tolly Powell, Alethea Power, Boris Power,
Elizabeth Proehl, Raul Puri, Alec Radford, Jack Rae, Aditya Ramesh, Cameron Raymond, Francis
Real, Kendra Rimbach, Carl Ross, Bob Rotsted, Henri Roussez, Nick Ryder, Mario Saltarelli, Ted
Sanders, Shibani Santurkar, Girish Sastry, Heather Schmidt, David Schnurr, John Schulman, Daniel
Selsam, Kyla Sheppard, Toki Sherbakov, Jessica Shieh, Sarah Shoker, Pranav Shyam, Szymon
Sidor, Eric Sigler, Maddie Simens, Jordan Sitkin, Katarina Slama, Ian Sohl, Benjamin Sokolowsky,
Yang Song, Natalie Staudacher, Felipe Petroski Such, Natalie Summers, Ilya Sutskever, Jie Tang,
Nikolas Tezak, Madeleine B. Thompson, Phil Tillet, Amin Tootoonchian, Elizabeth Tseng, Preston
Tuggle, Nick Turley, Jerry Tworek, Juan Felipe Cerón Uribe, Andrea Vallone, Arun Vijayvergiya,
Chelsea Voss, Carroll Wainwright, Justin Jay Wang, Alvin Wang, Ben Wang, Jonathan Ward, Jason
Wei, CJ Weinmann, Akila Welihinda, Peter Welinder, Jiayi Weng, Lilian Weng, Matt Wiethoff,
Dave Willner, Clemens Winter, Samuel Wolrich, Hannah Wong, Lauren Workman, Sherwin Wu,
Jeff Wu, Michael Wu, Kai Xiao, Tao Xu, Sarah Yoo, Kevin Yu, Qiming Yuan, Wojciech Zaremba,
Rowan Zellers, Chong Zhang, Marvin Zhang, Shengjia Zhao, Tianhao Zheng, Juntang Zhuang,
William Zhuk, and Barret Zoph. Gpt-4 technical report, 2024.

Yonatan Oren, Nicole Meister, Niladri Chatterji, Faisal Ladhak, and Tatsunori B. Hashimoto. Proving

test set contamination in black box language models, 2023.

Ashwinee Panda, Christopher A. Choquette-Choo, Zhengming Zhang, Yaoqing Yang, and Prateek

Mittal. Teach llms to phish: Stealing private information from language models, 2024.

Virat Shejwalkar, Huseyin A. Inan, Amir Houmansadr, and Robert Sim. Membership inference
attacks against nlp classification models. 2021. URL https://api.semanticscholar.
org/CorpusID:245222525.

Weijia Shi, Anirudh Ajith, Mengzhou Xia, Yangsibo Huang, Daogao Liu, Terra Blevins, Danqi Chen,

and Luke Zettlemoyer. Detecting pretraining data from large language models, 2024.

Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. Membership inference attacks

against machine learning models, 2017.

Congzheng Song and Vitaly Shmatikov. Auditing data provenance in text-generation models, 2019.

14

Under review as a conference paper at ICLR 2025

Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée
Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand
Joulin, Edouard Grave, and Guillaume Lample. Llama: Open and efficient foundation language
models, 2023a.

Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay
Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cris-
tian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu,
Wenyin Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, Naman Goyal, Anthony Hartshorn,
Saghar Hosseini, Rui Hou, Hakan Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa, Isabel
Kloumann, Artem Korenev, Punit Singh Koura, Marie-Anne Lachaux, Thibaut Lavril, Jenya Lee,
Diana Liskovich, Yinghai Lu, Yuning Mao, Xavier Martinet, Todor Mihaylov, Pushkar Mishra,
Igor Molybog, Yixin Nie, Andrew Poulton, Jeremy Reizenstein, Rashi Rungta, Kalyan Saladi,
Alan Schelten, Ruan Silva, Eric Michael Smith, Ranjan Subramanian, Xiaoqing Ellen Tan, Binh
Tang, Ross Taylor, Adina Williams, Jian Xiang Kuan, Puxin Xu, Zheng Yan, Iliyan Zarov, Yuchen
Zhang, Angela Fan, Melanie Kambadur, Sharan Narang, Aurelien Rodriguez, Robert Stojnic,
Sergey Edunov, and Thomas Scialom. Llama 2: Open foundation and fine-tuned chat models,
2023b.

Shuo Yang, Wei-Lin Chiang, Lianmin Zheng, Joseph E. Gonzalez, and Ion Stoica. Rethinking
benchmark and contamination for language models with rephrased samples, 2023. URL https:
//arxiv.org/abs/2311.04850.

Jingyang Zhang, Jingwei Sun, Eric Yeats, Yang Ouyang, Martin Kuo, Jianyi Zhang, Hao Yang, and
Hai Li. Min-k%++: Improved baseline for detecting pre-training data from large language models.
arXiv preprint arXiv:2404.02936, 2024.

Kun Zhou, Yutao Zhu, Zhipeng Chen, Wentong Chen, Wayne Xin Zhao, Xu Chen, Yankai Lin,
Ji-Rong Wen, and Jiawei Han. Don’t make your llm an evaluation benchmark cheater, 2023.

15

756
757

758
759
760
761
762
763

764
765
766
767
768
769

770
771
772
773
774

775
776
777
778
779
780

781
782
783
784
785
786

787
788
789
790
791
792

793
794
795
796
797

798
799
800
801
802
803

804
805
806
807
808
809

Under review as a conference paper at ICLR 2025

A CHOICE OF HYPERPARAMETERS

Infilling Score algorithm has two hyperparameters: m which represents the number of future tokens
to use, and k which represents the k% tokens with minimum probabilities to use. We sweep over
1, 3, 5, 10 and 20 future tokens, and k = 0.1, 0.2, ...0.5. Tables 7, 8, 9, and 10 show AUROC and
TPR at low FPR results on WikiMIA subsets with sequence lengths of 32, 64, 128, and 256. Based
on the results, the optimal number of future tokens is 1 for sequences of 32 tokens and 5 for longer
sequences. We find that k = 0.1 often works best across different model sizes and sequence lengths.

# future tokens

k (Min-k%) AUROC FPR@TPR95 TPR@FPR05 AUROC FPR@TPR95 TPR@FPR05 AUROC FPR@TPR95 TPR@FPR05

Llama-7B

Llama-13B

Llama-30B

1

3

5

10

20

0.1
0.2
0.3
0.4
0.5

0.1
0.2
0.3
0.4
0.5

0.1
0.2
0.3
0.4
0.5

0.1
0.2
0.3
0.4
0.5

0.1
0.2
0.3
0.4
0.5

89.10
88.10
88.00
88.10
88.00

88.20
87.80
87.80
87.80
87.70

88.00
87.60
87.60
87.70
87.50

86.80
86.60
86.60
86.60
86.40

82.00
81.80
81.80
81.80
81.40

33.90
36.80
37.80
36.00
37.30

34.70
35.70
37.30
36.00
37.30

35.50
37.50
37.80
37.50
37.80

39.60
41.10
41.40
41.60
41.60

47.30
47.80
47.80
48.10
46.80

30.50
27.60
27.90
25.60
25.30

29.50
31.00
31.80
32.00
28.20

34.10
32.80
32.80
33.90
30.50

28.40
26.60
27.10
26.90
25.10

22.70
23.50
22.50
23.00
21.70

89.20
88.60
88.60
88.60
88.60

89.00
88.70
88.60
88.70
88.70

88.40
88.30
88.20
88.30
88.30

86.80
86.60
86.60
86.70
86.60

81.20
81.00
80.90
81.00
80.80

32.90
35.50
36.20
35.20
36.80

32.10
38.30
38.80
38.30
38.00

32.90
33.70
36.20
35.20
34.40

37.00
37.50
36.20
35.50
36.00

44.20
45.50
44.50
45.80
45.00

24.50
26.40
26.90
27.60
25.60

29.50
28.70
28.40
30.50
30.00

27.60
29.50
27.60
30.50
28.90

26.90
26.90
26.40
26.10
26.40

19.60
20.20
19.90
20.20
20.20

87.80
87.30
87.30
87.30
87.30

86.80
86.60
86.60
86.60
86.60

87.30
87.20
87.30
87.30
87.30

85.30
85.20
85.20
85.20
85.30

81.30
81.20
81.10
81.20
81.20

37.30
37.80
39.10
37.80
38.80

36.00
37.50
37.50
37.50
36.80

35.20
35.20
35.50
35.70
35.50

39.80
40.60
40.90
41.40
40.90

47.80
49.10
49.40
48.60
48.30

27.90
27.40
28.90
28.20
27.10

27.90
27.40
28.90
27.10
28.70

31.80
33.10
31.50
32.00
31.00

28.40
27.90
27.90
27.90
27.90

22.20
22.20
22.20
22.20
22.20

Table 7: Complete Infilling Score results testing Llama-7B, Llama-13B, and Llama-30B models on
the Original subset of the WikiMIA 32-token sequences (Shi et al., 2024). For this subset, using one
future token results in the best performance.

# future tokens

k (Min-k%) AUROC FPR@TPR95 TPR@FPR05 AUROC FPR@TPR95 TPR@FPR05 AUROC FPR@TPR95 TPR@FPR05

Llama-7B

Llama-13B

Llama-30B

1

3

5

10

20

0.1
0.2
0.3
0.4
0.5

0.1
0.2
0.3
0.4
0.5

0.1
0.2
0.3
0.4
0.5

0.1
0.2
0.3
0.4
0.5

0.1
0.2
0.3
0.4
0.5

89.60
89.00
89.00
89.00
89.00

89.70
89.60
89.60
89.70
89.70

89.70
89.70
89.70
89.70
89.70

88.70
88.70
88.70
88.70
88.60

83.00
83.00
82.90
83.00
82.90

33.70
36.40
35.70
36.80
36.40

37.60
38.00
37.60
37.60
37.60

38.00
37.20
38.40
37.20
38.00

40.30
41.10
41.10
40.70
41.10

51.90
52.70
52.70
52.30
52.70

44.00
45.40
45.80
46.80
45.80

41.20
38.70
37.30
39.80
39.40

45.80
45.40
45.80
46.10
46.50

52.80
52.50
52.10
53.20
53.50

33.10
32.40
32.70
32.40
33.10

89.90
89.60
89.60
89.60
89.60

90.00
90.00
90.00
90.00
90.00

90.10
90.10
90.10
90.10
90.00

88.70
88.70
88.70
88.80
88.50

82.50
82.50
82.50
82.50
82.30

35.30
38.40
39.50
37.60
39.50

36.80
37.60
38.80
38.40
38.40

37.20
39.50
39.10
39.10
38.80

40.70
41.50
41.50
41.90
42.20

51.20
50.80
50.80
50.80
50.80

46.10
47.90
47.20
47.20
47.20

52.10
53.50
52.50
53.50
53.20

48.60
49.30
50.00
49.60
46.10

47.50
47.20
47.50
47.50
46.80

26.40
27.10
27.10
27.50
27.10

87.70
87.50
87.50
87.60
87.60

88.00
88.00
88.00
88.00
88.00

88.30
88.30
88.30
88.40
88.30

87.30
87.30
87.30
87.30
87.30

82.50
82.50
82.50
82.50
82.40

36.40
35.70
35.70
35.70
35.70

37.60
35.70
35.70
35.70
35.70

35.30
34.90
34.10
35.30
35.30

38.80
38.00
38.00
38.40
37.60

49.60
49.60
49.20
49.20
48.80

39.10
38.40
39.40
38.70
39.10

36.30
37.30
37.30
37.70
37.70

43.00
43.70
44.00
44.00
43.70

37.70
38.70
38.70
39.10
38.00

36.30
35.60
35.90
35.60
35.60

Table 8: Complete Infilling Score results testing Llama-7B, Llama-13B, and Llama-30B models on
the Original subset of the WikiMIA 64-token sequences (Shi et al., 2024). For this subset, using five
future tokens results in the best performance.

16

810
811

812
813
814
815
816
817

818
819
820
821
822
823

824
825
826
827
828

829
830
831
832
833
834

835
836
837
838
839
840

841
842
843
844
845
846

847
848
849
850
851

852
853
854
855
856
857

858
859
860
861
862
863

Under review as a conference paper at ICLR 2025

864
865

866
867
868
869
870
871

872
873
874
875
876
877

878
879
880
881
882

883
884
885
886
887
888

889
890
891
892
893
894

895
896
897
898
899
900

901
902
903
904
905

906
907
908
909
910
911

912
913
914
915
916
917

# future tokens

k (Min-k%) AUROC FPR@TPR95 TPR@FPR05 AUROC FPR@TPR95 TPR@FPR05 AUROC FPR@TPR95 TPR@FPR05

Llama-7B

Llama-13B

Llama-30B

1

3

5

10

20

0.1
0.2
0.3
0.4
0.5

0.1
0.2
0.3
0.4
0.5

0.1
0.2
0.3
0.4
0.5

0.1
0.2
0.3
0.4
0.5

0.1
0.2
0.3
0.4
0.5

87.10
86.80
86.80
86.80
86.80

87.20
87.10
87.10
87.10
87.10

87.70
87.60
87.60
87.60
87.60

87.60
87.50
87.60
87.50
87.50

81.50
81.40
81.40
81.40
81.40

36.90
36.90
36.00
36.90
36.90

37.80
37.80
37.80
37.80
37.80

38.70
38.70
38.70
38.70
38.70

45.90
44.10
43.20
44.10
43.20

54.10
55.00
55.00
55.00
55.00

36.00
35.30
35.30
36.00
36.70

22.30
23.70
23.00
23.70
22.30

37.40
38.10
36.70
37.40
36.70

23.70
24.50
23.70
23.70
23.70

21.60
23.00
21.60
21.60
21.60

88.40
88.20
88.20
88.20
88.20

87.90
87.80
87.80
87.80
87.80

88.40
88.30
88.30
88.30
88.30

87.60
87.60
87.50
87.50
87.50

82.30
82.20
82.20
82.20
82.10

35.10
34.20
34.20
34.20
34.20

36.00
34.20
34.20
35.10
34.20

37.80
37.80
37.80
37.80
36.90

39.60
40.50
40.50
39.60
39.60

53.20
52.30
52.30
52.30
52.30

40.30
39.60
40.30
39.60
41.00

36.70
37.40
37.40
37.40
37.40

34.50
33.80
34.50
34.50
35.30

33.80
34.50
35.30
35.30
33.80

18.70
18.70
18.70
18.70
18.70

84.90
84.70
84.70
84.70
84.70

85.30
85.20
85.20
85.20
85.20

86.70
86.70
86.60
86.60
86.60

86.00
86.00
85.90
85.90
85.90

83.20
83.10
83.10
83.20
83.10

33.30
36.90
36.90
37.80
36.90

39.60
39.60
39.60
39.60
39.60

37.80
37.80
37.80
37.80
37.80

38.70
38.70
38.70
37.80
38.70

52.30
51.40
51.40
51.40
51.40

24.50
24.50
24.50
24.50
24.50

13.70
14.40
13.70
13.70
13.70

19.40
20.10
19.40
19.40
20.10

18.00
18.00
18.00
18.00
18.00

20.90
20.90
21.60
20.90
20.90

Table 9: Complete Infilling Score results testing Llama-7B, Llama-13B, and Llama-30B models on
the Original subset of the WikiMIA 128-token sequences (Shi et al., 2024). Again, using five future
tokens results in the best performance.

# future tokens

k (Min-k%) AUROC FPR@TPR95 TPR@FPR05 AUROC FPR@TPR95 TPR@FPR05 AUROC FPR@TPR95 TPR@FPR05

Llama-7B

Llama-13B

Llama-30B

1

3

5

10

20

0.1
0.2
0.3
0.4
0.5

0.1
0.2
0.3
0.4
0.5

0.1
0.2
0.3
0.4
0.5

0.1
0.2
0.3
0.4
0.5

0.1
0.2
0.3
0.4
0.5

93.80
93.80
93.70
93.80
93.90

96.30
96.10
96.00
96.00
96.00

96.80
96.60
96.50
96.60
96.60

95.70
95.90
95.80
95.80
95.80

93.70
93.70
93.40
93.70
93.50

51.60
51.60
51.60
51.60
51.60

29.00
32.30
35.50
35.50
35.50

22.60
22.60
25.80
22.60
25.80

29.00
25.80
29.00
29.00
29.00

22.60
22.60
22.60
22.60
22.60

80.40
80.40
80.40
80.40
80.40

78.40
74.50
74.50
74.50
72.50

74.50
74.50
74.50
74.50
74.50

78.40
78.40
78.40
78.40
78.40

66.70
66.70
68.60
68.60
66.70

92.90
92.80
92.70
92.70
92.70

95.30
95.30
95.30
95.30
95.30

95.30
95.30
95.20
95.20
95.20

93.00
93.10
93.20
93.10
93.20

90.70
90.60
90.60
90.60
90.50

25.80
29.00
29.00
29.00
29.00

19.40
19.40
19.40
19.40
19.40

22.60
22.60
22.60
22.60
22.60

29.00
29.00
29.00
29.00
29.00

35.50
35.50
35.50
35.50
35.50

66.70
68.60
66.70
66.70
66.70

72.50
72.50
72.50
72.50
72.50

80.40
80.40
80.40
80.40
80.40

54.90
54.90
54.90
54.90
54.90

51.00
51.00
51.00
51.00
51.00

85.60
85.60
85.60
85.60
85.70

90.60
90.60
90.60
90.60
90.60

89.80
89.80
89.80
89.80
89.80

87.40
87.50
87.60
87.50
87.50

85.60
85.60
85.80
85.70
85.70

32.30
35.50
32.30
35.50
32.30

41.90
41.90
41.90
41.90
41.90

35.50
35.50
35.50
35.50
35.50

45.20
45.20
45.20
45.20
45.20

48.40
48.40
48.40
48.40
48.40

21.60
21.60
19.60
21.60
19.60

72.50
72.50
72.50
72.50
72.50

47.10
47.10
47.10
47.10
47.10

49.00
49.00
49.00
49.00
49.00

35.30
33.30
35.30
35.30
35.30

Table 10: Complete Infilling Score results testing Llama-7B, Llama-13B, and Llama-30B models on
the Original subset of the WikiMIA 256-token sequences (Shi et al., 2024). Similar to the WikiMIA
64-token and 128-token sequence subsets, using 5 future tokens results in the best performance.

B ADDITIONAL RESULTS

B.1 STATISTICAL ANALYSIS: INFILLING SCORE VS. MIN-K%++

We employ a bootstrap-based statistical comparison to evaluate Infilling Score and Min-K%++.
We use 1,000 bootstrap iterations to estimate the the mean difference between AUROC metrics
from these methods, along with the standard errors to construct 95% confidence intervals for the
true performance gap. Table 11 shows that Infilling Score consistently outperforms Min-K%++
across different sequence lengths (32, 64, 128, and 256 tokens) and model sizes (7B, 13B, and 30B
parameters).

17

Under review as a conference paper at ICLR 2025

Sequence Length

Model

AUROC (%)

Std Err AUROC (%)

Std Err Difference (%)

p-value

Infilling Score

Min-K%++

Comparison

32 tokens

64 tokens

128 tokens

256 tokens

llama-7b
llama-13b
llama-30b

llama-7b
llama-13b
llama-30b

llama-7b
llama-13b
llama-30b

llama-7b
llama-13b
llama-30b

89.185
88.850
87.628

89.788
90.029
88.206

87.364
88.145
86.207

96.307
95.124
90.737

1.173
1.232
1.236

1.341
1.265
1.447

2.272
2.214
2.797

1.761
2.271
3.782

85.182
84.852
84.390

85.922
85.692
84.828

84.896
83.740
82.398

82.354
82.326
77.411

1.328
1.333
1.329

1.659
1.642
1.705

2.395
2.463
2.602

4.662
4.740
5.643

4.003 ± 1.130
3.998 ± 1.222
3.239 ± 1.157

3.866 ± 1.492
4.338 ± 1.539
3.378 ± 1.601

2.468 ± 2.654
4.405 ± 2.649
3.809 ± 1.993

0.000***
0.004**
0.006**

0.012*
0.010*
0.040*

0.348
0.080
0.064

13.952 ± 4.296
12.797 ± 3.952
13.326 ± 4.459

0.000***
0.000***
0.002**

Table 11: Comparing performance of Infilling Score versus Min-K%++ across different sequence
lengths and model sizes. Results show bootstrap estimates with 1000 iterations. The mean difference
indicates Infilling Score’s improvement over Min-K%++. Statistical significance is denoted as: * (p
< 0.05), ** (p < 0.01), *** (p < 0.001).

B.2 DETECTING PRE-TRAINING DATA FROM BOOKS

We compare the AUROC of Infilling Score with existing methods on a labeled validation subset of
book excerpts. As discussed in Section 4.1, this validation subset contains book excerpts labeled as
“seen” and “unseen”. Infilling Score significantly outperforms existing methods in detecting “seen”
examples.

Method

Infilling Score (Ours)
Min-K%++ (Zhang et al., 2024)
Min-K%(Shi et al., 2024)
Zlib (Carlini et al., 2021)

AUC

0.79
0.53
0.71
0.68

Table 12: Comparing AUROC of Infilling Score, Min-K%++, Min-K%, and Zlib methods on the
validation dataset, detecting book excerpts in Llama3-8B pretraining data.

C COMPUTE RESOURCES

We ran our experiments on A100 (40 GB) and H200 (120 GB) GPUs. Testing Infilling Score on the
WikiMIA benchmark on an A100 node takes approximately between 20 minutes (for a 3B parameter
model) and 35 minutes (for a 30B parameter model). For Llama models, we used float16 data type.
On the MIMIR benchmark, where there are 1000 long samples per class, the test approximately takes
10 hours on each subset on an A100 node.

D INFILLING SCORE ALGORITHM

918
919

920
921
922
923
924
925

926
927
928
929
930
931

932
933
934
935
936

937
938
939
940
941
942

943
944
945
946
947
948

949
950
951
952
953
954

955
956
957
958
959

960
961
962
963
964
965

966
967
968
969
970
971

18

Under review as a conference paper at ICLR 2025

Algorithm 1: Infilling Score
Input: Sequence: x : x1, x2...xN , Threshold τ

1 for i = 1 to N do
2
3

4

5

6
7

(cid:113)Ez∼p(.|x1 ...xi−1)[(log p(z|x1...xi−1) − µx<i )2]
i|x1...xi−1)

Compute log p(xi|x1...xi−1)
µx<i ← Ez∼p(.|x1...xi−1)[log p(z|x1...xi−1)]
σx<i ←
Find x∗
Compute log p(x∗
r ← (log p(xi|x1...xi−1) − µx<i )/σx<i − (log p(x∗
for j = i + 1 to i + m do

∈V p(x′
i ← arg maxx′
i
i |x1...xi−1)

8
9
10
11
12
13
14 end
15 Min-K%(x) ← k% of tokens from x with the lowest InfillingScoretoken(xi)
16 InfillingScore(x) = (cid:80)
17 return InfillingScore(x) < τ

end
InfillingScoretoken(xi) ← r

xi∈min-k% InfillingScoretoken(xi)

Compute log p(xj |x1...xj−1)
Compute log p(xj |x1...xi∗ ...xj−1)
r ← r + (log p(xj |x1...xj−1) − µx<i)/σx<i − (log p(xj |x1...xi∗ ...xj−1) − µx<i)/σx<i

i |x1...xi−1) − µx<i )/σx<i

972
973

974
975
976
977
978
979

980
981
982
983
984
985

986
987
988
989
990

991
992
993
994
995
996

997
998
999
1000
1001
1002

1003
1004
1005
1006
1007
1008

1009
1010
1011
1012
1013

1014
1015
1016
1017
1018
1019

1020
1021
1022
1023
1024
1025

19

