# STAR: SocioTechnical Approach to Red Teaming Language Models

Laura Weidinger*1 John Mellor*1 Bernat Guillén Pegueroles2 Nahema Marchal1
Ravin Kumar3 Kristian Lum1 Canfer Akbulut1 Mark Diaz2 Stevie Bergman1
Mikel Rodriguez1 Verena Rieser1 William Isaac1

1Google DeepMind 2Google 3Google Labs
<EMAIL_ADDRESS>

* denotes equal contribution

###### Abstract

This research introduces STAR, a sociotechnical framework that improves on
current best practices for red teaming safety of large language models. STAR
makes two key contributions: it enhances steerability by generating
parameterised instructions for human red teamers, leading to improved coverage
of the risk surface. Parameterised instructions also provide more detailed
insights into model failures at no increased cost. Second, STAR improves
signal quality by matching demographics to assess harms for specific groups,
resulting in more sensitive annotations. STAR further employs a novel step of
arbitration to leverage diverse viewpoints and improve label reliability,
treating disagreement not as noise but as a valuable contribution to signal
quality.

STAR: SocioTechnical Approach to Red Teaming Language Models

Laura Weidinger*1 John Mellor*1 Bernat Guillén Pegueroles2 Nahema Marchal1
Ravin Kumar3 Kristian Lum1 Canfer Akbulut1 Mark Diaz2 Stevie Bergman1 Mikel
Rodriguez1 Verena Rieser1 William Isaac1 1Google DeepMind 2Google 3Google Labs
<EMAIL_ADDRESS>* denotes equal contribution

##  1 Introduction

Red teaming has emerged as an important tool for for discovering flaws,
vulnerabilities, and risks in generative Artificial Intelligence (AI) systems,
including large language models (e.g. Ganguli et al., 2022; White House, 2023;
Thoppilan et al., 2022; Zou et al., 2023) and multimodal generative models
Parrish et al. (2023). It is used by AI developers to provide assurances
toward decision-makers and public stakeholders (Feffer et al., 2024), and is
increasingly requested or mandated by regulators and other institutions tasked
with upholding public safety (White House, 2023).

Despite the growing use of red teaming, there is a lack of consensus on best
practices, making it difficult to compare results and establish standards
Feffer et al. (2024); Anthropic (2023). This hinders the progress of safety
research in AI, and makes it challenging for the public to assess AI safety.

Figure 1:  STAR procedurally generates parametric instructions to ensure
comprehensive AI red teaming.

In this paper, we introduce STAR: a SocioTechnical Approach to Red teaming,
and propose methods for direct comparison to current state-of-the-art red
teaming methods. STAR is a customisable framework designed to improve the
effectiveness and efficiency of red teaming for AI. STAR makes several
methodological innovations that offer two key advantages: better
_steerability_ , enabling targeted risk exploration at no increased cost; and
_higher quality signal_ through expert- and demographic matching, and a new
arbitration step that leverages annotator reasoning. We present these
methodological innovations and empirical results on their strengths and
limitations, aiming to contribute to best practices in red teaming generative
AI.

##  2 Background

Red teaming is an adaptive method used to complement static AI evaluations
like benchmarking (Zhuo et al., 2023). It involves adversarial exploration of
a system’s risk surface to identify inputs that could trigger harmful outputs.
In the context of generative AI systems, attackers provide prompts, and
annotators evaluate system responses to determine if they constitute safety
failures.

Prior red teaming efforts of generative AI have varied widely, targeting
failure modes ranging from system integrity failures to social harms. Red
teaming approaches range from human attacks (Ganguli et al., 2022; White
House, 2023; Thoppilan et al., 2022; Nakamura et al., 2024; OpenAI, 2023) to
automated methods (Radharapu et al., 2023; Parrish et al., 2023; Perez et al.,
2022; Samvelyan et al., 2024) or hybrid approaches (Xu et al., 2021). Novel
results are often released alongside new models, though some stand-alone
methodological papers exist (Radharapu et al., 2023; Parrish et al., 2023;
Nakamura et al., 2024; Xu et al., 2021). This paper focuses on open challenges
in human red teaming of language models for social harms.

###  2.1 Steerability

A common challenge in AI red teaming is ensuring comprehensive and even
coverage of the risk surface. Uneven coverage can lead to redundant attack
clusters and missed vulnerabilities or blind spots.

Unintentional skews in red teaming may result from practical factors such as
attacker demographics or task design. For example, open-ended approaches are
intended to foster broad exploration, but can inadvertently lead to clustered
redundancies as red teamers may naturally gravitate towards familiar or easily
exploitable vulnerabilities. This tendency can be amplified by incentive
structures that reward quick or easily identifiable harms. Furthermore, a lack
of demographic diversity among human red teamers can exacerbate this issue, as
attacks often reflect attackers own, inherently limited, experiences and
perspectives (Ganguli et al., 2022; Feffer et al., 2024).

Prior work to addresses this challenge still has limitations. One strategy is
to simply increase the number of attacks, but this is costly and doesn’t
guarantee comprehensive coverage, as multiple attackers may still exploit the
same harm vector. Principled approaches include dynamic incentives that reward
the discovery of impactful vulnerabilities (Attenberg et al., 2015), framing
diverse prompt generation as a quality-diversity search (Samvelyan et al.,
2024) and using parametric instructions (Radharapu et al., 2023), though these
approaches have not been applied to human red teaming of generative AI.

###  2.2 Signal Quality

Another significant challenge in red teaming is ensuring high quality of
collected human data, especially when assessing harms that rely on subjective
judgments. Prior work has shown comparably high rates of disagreement between
raters when evaluating attack success (Ganguli et al., 2022; Xu et al., 2021).
While often dismissed as noise, this disagreement can be a valuable source of
information, reflecting the diverse perspectives that are essential to
consider in evaluating AI model safety Aroyo and Welty (2015); Plank (2022).
Simply taking a majority vote loses such signal, and risks overlooking
minority judgments rooted in marginalised experiences.

Reduced signal quality may also stem from to skewed demographics of red
teamers, as race, gender, and geo-cultural region have been shown to influence
judgments on objectionable or adversarially generated content (Jiang et al.,
2021; Goyal et al., 2022; Homan et al., 2023; Aroyo et al., 2023; DeVos et
al., 2022). Yet, red teaming and annotation teams often lack demographic
diversity (Feffer et al., 2024), even when efforts are made to recruit
diversely. In prior studies, the majority of red teamers identified as white,
cis-gendered, heterosexual, and without disabilities, with men often
outnumbering women (Ganguli et al., 2022; Thoppilan et al., 2022).111Only very
few red teaming reports document annotator demographics. This lack of
representation is likely to be more pronounced in red teaming efforts that did
not deliberately recruit a diverse pool of workers. Furthermore, most red
teaming focuses on English-language attacks, excluding many demographic groups
and their languages (Nakamura et al., 2024). Such demographic skew can lead to
undetected risks for these communities, potentially perpetuating
disproportionate risks of harm when AI systems are deployed (Yong et al.,
2024). To ensure broad coverage and legitimate and reliable data points, red
teaming should involve diverse groups, encompassing a wider range of
perspectives and experiences (Bockting et al., 2023). In addition, principled
approaches are needed to account for meaningful annotator disagreement.

##  3 STAR: SocioTechnical Approach to Red teaming

###  3.1 Improving Steerability

STAR addresses the steerability challenge by providing procedurally generated
instructions to ensure comprehensive and even coverage of a targeted risk
area. STAR’s instructions contain multiple parameters that delineate the
targeted risk area (see detailed instructions in Appendix C). This content-
agnostic approach is adaptable to any target area. As a proof of concept, we
focus on red teaming for ethical and social harms, as codified in different
‘rules’ in a proprietary content safety policy, see B. To demonstrate that
STAR enables steerability also in complex manifolds, we particularly explore
two ‘rules’ - on hate speech and discriminatory stereotypes - with up to two
additional instruction parameters that specify demographic groups to target.
Our instructions encompass multiple parameters (see Figure 2 and Methods) to
ensure that red teamers systematically explore the risk space, reducing
redundancies and uncovering potential vulnerabilities that might otherwise be
overlooked. These parameters are additive, meaning that specifying one (e.g.,
a rule) doesn’t limit our ability to measure harm across other parameters
(e.g., different use cases). As such, additional parameters can be added –
constrained only by the cognitive load they impose on human raters. We stress
test this approach by aiming for coverage across labels of different levels of
specificity: attackers may be asked to attack demographic groups based on
single labels (race, gender), or combinatory labels (race$\times$gender).

###  3.2 Improving Signal Quality

Applying a sociotechnical lens222A sociotechnical approach is rooted in the
observation that AI systems are sociotechnical systems: both humans and
machines are necessary in order to make the technology work as intended
(Selbst et al., 2019). In the context of red teaming, this entails considering
the interplay between different human attackers or annotators with an AI
system, as well as societal and systemic structures that influence definitions
of harm - such as what ‘ counts’ as a discriminatory stereotype and what does
not., STAR centers the interplay of human attackers and annotators with the AI
system. To provide a legitimate and reliable signal, we leverage different
types of expertise, employing fact-checkers, medical professionals, and lived
experience of generalists from different demographic groups. To learn from
disagreement, we introduce an arbitration step to our annotation pipeline.

##### Expert- and demographic matching

Experts provide a more reliable and authoritative signal in their domains of
expertise. This is why we employ raters with fact-checking and medical
expertise to annotate relevant rules. We extend this logic to lived
experience, which constitutes a relevant form of expertise on whether or not a
given utterance constitutes hate speech or discriminatory stereotypes against
one’s own demographic group. In addition, affected communities arguably should
be prioritised and offer a more legitimate signal in the context of offense
against their specific groups. Thus all attacks on medical, public interest,
or demographic groups are annotated leveraging the relevant form of expertise.

We also anticipate that people of different demographic groups are often more
familiar with the discriminatory stereotypes and hate speech targeted at their
own group, compared to people of other demographic groups (Bergman et al.,
2024). As a result, asking people to design attacks targeting their own group
may create more ecologically valid signal, i.e. better reflect likely attacks
from malicious users in real-world settings who rely on common tropes and
stereotypes Gordon et al. (2021); Parrish et al. (2024). To test the relative
effectiveness of ‘demographic matching’ not for annotation but for red
teaming, 50% of attacks against a given demographic group are conducted by
demographically-matched attackers, and 50% by a control of out-group
attackers. This required recruiting a diverse red teaming and annotator pool.
In particular, we recruit red teamers and annotators to obtain an even spread
over multiple demographic labels including on gender and ethnicity (for
demographics see E).

##### Learning from annotator disagreement

To enhance the reliability of our red teaming process, we gather labels from
two annotators. However, discrepancies in judgment often arise. While some
argue that such disagreement should be preserved for subjective tasks (e.g.
Aroyo and Welty, 2015; Plank, 2022; Aroyo et al., 2023), red teaming
necessitates clear safety recommendations. Unlike prior methods that merely
add up ratings, we implement a two-step annotator $\rightarrow$ arbitrator
pipeline, modeling argument exchange as in normative annotation settings
(Bergman et al., 2023).

We first obtain annotator reasoning for their judgment on whether the model
violated a rule. When two annotators’ ratings significantly diverge (by two or
more steps on the four-step Likert scale), their dialogue and reasoning are
presented to a third annotator, acting as an arbitrator. This arbitrator
provides an additional rating and explanation, with all three annotations
retained. This process uncovers annotator thinking and allows the arbitrator
to weigh different perspectives for a more comprehensive judgment. The
arbitrator is subject to the same expert- and demographic matching logic as
annotators.

##  4 Methods

##### Data

We obtain 8360 dialogues by 227 red teamers, annotated by 308 annotators and
arbitrators, all in January 2024. Each dialogue adversarially tests model
performance on one rule out of a content safety policy (see B).

##### Task design

Adversarial testing here is conducted by human red teamers, in a multi-turn
(mean: 16.4 ± 11.3) setting. Participants are assigned a red teaming task, an
annotation task, or an arbitration task. Participants can perform multiple
tasks in sequence, but they never see the same dialogue twice.

##### Red teaming task

Red teamers are given procedurally generated instructions with up to five
parameters, directing red teamers to:

  1. 1.

Violate a specific rule from the safety policy;

  2. 2.

Employ a specified level of adversariality (low, medium, high) in their
attack;

  3. 3.

Emulate a particular use case (e.g., information search, entertainment);

  4. 4.

Commit to a specific topic before initiating the dialogue, which they can
freely choose;

  5. 5.

In cases where the rule involves hate speech or discriminatory stereotypes,
identify the specific demographic group targeted by the attack.

The demographic groups that attackers are asked to target are randomly
selected one- to two-way intersections out of the gender and race labels
listed in Appendix D.

Red teamers engage in written dialogue with a proprietary model. We encourage
10–15 turns but red teamers determine when to end the exchange. After
completing the dialogue, red teamers perform ‘ pre-annotation’ on whether the
chatbot broke the assigned rule or any other rules; and whether the dialogue
mentioned any demographic groups and if so which ones. Here, more demographic
labels are available including disability status, age, religion and sexual
orientation.

##### Annotation task

Annotators are provided with chat logs from a red teaming task. Where the red
teamer had been instructed to make the proprietary model break a rule with
respect to a particular demographic group, annotators are demographically
matched to the attacked group. On rules pertaining to medical expertise or
public discourse, annotators are respectively medical or fact-checking
professionals.

Two annotators rate each dialogue on whether the targeted rule was broken on a
four-point Likert scale. In addition to their rating, they provide free-text
reasoning to explain their rating. Where the two annotators are two or more
steps apart, an arbitrator rates the same dialogue.

##### Arbitration task

Arbitrators are provided with a dialogue between a red teamer and the
proprietary model, and with the free-text reasoning from both previous
annotators. They are then asked to make their judgement using the same Likert
scale as annotators, and to provide their open-text reasoning.

##### Participants

We recruited $n=313$ participants for our study (of which $n=286$ red teamed
and $n=225$ annotated at least once), ensuring demographic diversity through
self-identification in a voluntary questionnaire. Participants independently
interacted with and evaluated the model under ethical approval from our ethics
committee. Particular care was taken to build well-being considerations such
as rest and opt-out steps into the task. They were compensated based on time
spent (adhering to local living wage standards), to ensure quality assurance.

##  5 Analysis

We perform a series of quantitative and qualitative analyses to test the
steerability and reliability of the STAR approach.

###  5.1 UMAP embedding

To compare thematic clustering of red teaming approaches, we project dialogues
(between attacker and language model) from multiple datasets into a shared
word embedding space (Fig. 2).333We first project the dialogues onto high-
dimensional embeddings using Gecko (Lee et al., 2024)), then onto two
dimensions using UMAP (McInnes et al., 2020). We chose UMAP to be able to
compare STAR to prior results, particularly building on (Ganguli et al.,
2022). UMAP is a dimension reduction technique that finds a low-dimensional
representation of high-dimensional data while preserving the data’s underlying
structure. These datasets include two prior red teaming efforts, STAR, and a
dataset of real-world dialogues between users and a proprietary system, which
were flagged by the user due to the model displaying undesired behaviour. For
a fair comparison, we downsample each dataset by randomly selecting the same
number of data points. For more detail on these datasets see Appendix F).

We use hierarchical agglomerative clustering on the UMAP embeddings to
identify twenty semantic groupings for the dialogues (iteratively joining
pairs of clusters that are close to each other in the euclidean space of the
UMAP embedding (Pedregosa et al., 2011)). The approximate outlines of these
clusters are drawn manually in Figure 2), and semantic labels per cluster are
listed in Table 1. Two reviewers independently assigned semantic labels per
cluster and disagreeing labels (clusters 4 and 13) were reviewed by a third
reviewer, followed by a discussion among all labellers to determine the final
labelling via consensus.

###  5.2 Quantitative and qualitative analysis

For comparing in- vs. out-group annotations, we include all dialogues where
red teamers were instructed to attack a specific demographic group in the
context of breaking the discriminatory stereotypes and hate speech rules.
These dialogues were annotated mostly by in-group members, with 62% of these
dialogues being annotated by in-group members and 38% by out-group members. We
compute odds ratios of different groups mentioned in instructions to yield a
successful red teaming attempt and test statistical significance using ANOVA
and t-tests. For qualitative insights, we manually inspect a random sample of
rater dialogues and annotator reasoning.

##  6 Results

We make a series of findings that highlight advantages of the STAR method.

###  6.1 Controlled exploration of the target area

Visual inspection of Figure 2 shows comparably broad coverage and low
clustering of the STAR approach, despite more specific instructions compared
to the other projected red teaming approaches. Analysing clusters in the
embedding space reveals a thematic split between the three red teaming
approaches (Table 1). The most common themes in STAR dialogues concern gender
stereotypes (cluster 2) and race-based bias (16), followed by medical topics
(8), reflecting the instructions. The most common themes in Anthropic
dialogues are malicious use (5), explicit stories including adult fiction (3),
and facilitating crime (0). Most common themes in DEFCON dialogues are prompts
about model training followed by model refusals (4), passwords and sensitive
personal data (7), and PII including from celebrities (14). In contrast, most
common themes in real-world flagged dialogues were advice and recommendations
(1), computer code (12) and refusals (4).

Figure 2: UMAP of the embedding space of dialogues across three red teaming datasets: Anthropic, DEFCON, and STAR; as well as dialogues between a proprietary model and users that were flagged as undesirable by users. Each dot indicates a dialogue. For comparability, we downsampled all datasets to include maximum 4000 randomly selected instances. Table 1: Overview of twenty semantic clusters observed in the embedding space mapped in Figure 2. Cell colour represents high (dark) and low (light) numbers of dialogues per cluster. Cluster | Anthropic | Real-world dialogues | DEFCON | STAR | total |  aggregate_label
---|---|---|---|---|---|---
0 | 564 | 54 | 277 | 126 | 1021 |  Crime, Malicious Use
1 | 20 | 954 | 16 | 52 | 1042 |  Advice, Recommendation
2 | 140 | 65 | 45 | 1013 | 1263 |  Gender/Race Bias, Women
3 | 613 | 152 | 74 | 65 | 904 |  Creative Writing, Sexual Explicit
4 | 128 | 347 | 682 | 35 | 1192 |  Refusal, AI training
5 | 797 | 13 | 35 | 39 | 884 |  Help Requests For Malicious Acts
6 | 127 | 181 | 120 | 261 | 689 |  Politically Sensitive
7 | 9 | 83 | 476 | 10 | 578 |  Online Account Passwords/Security; Stories
8 | 139 | 124 | 147 | 564 | 974 |  Medical, Wellness
9 | 346 | 69 | 150 | 385 | 950 |  Demographic Hate
10 | 12 | 108 | 232 | 51 | 403 |  Recommendations, Fact-Seeking
11 | 7 | 70 | 359 | 1 | 437 |  Math
12 | 1 | 426 | 11 | 0 | 438 |  Image Analysis, Software
13 | 1 | 168 | 1 | 0 | 170 |  Punting/ Unable To Respond
14 | 122 | 24 | 494 | 7 | 647 |  PII, Financial Data; Celebrity Info
15 | 50 | 158 | 385 | 156 | 749 |  Fact-Seeking, Public Interest Topics
16 | 75 | 54 | 80 | 645 | 854 |  Racism
17 | 68 | 46 | 193 | 250 | 557 |  Politcs, US Politics
18 | 348 | 49 | 48 | 126 | 571 |  Drugs, Explosives, How-To/ Use
19 | 200 | 20 | 58 | 190 | 468 |  Advice, Script/ Text Editing or Generation, Sexual Content
Total | 3767 | 3165 | 3883 | 3976 |  |

Analysing the spread of red teaming attacks across race, gender, and
race$\times$gender intersectionalities reveals that STAR achieves a
sufficiently _even spread_ of attacks across these categories as intended.
Predictable exceptions arose regarding the labels "non-binary", "Asian and
male", and "Hispanic and male", where we were unable to recruit the target
number of red teamers (Fig. 3)444Recall that instructions are limited by the
availability of red teamers of each demographic group, as we show instructions
or annotations of dialogues that harm certain groups only to demographically
matched members of that group..

Figure 3: Specific instructions and a diverse annotator pool result in even
exploration of attacks against different demographic groups, while maintaining
‘demographic matching’.

###  6.2 Signal quality

We make a series of findings that suggest the STAR approach provides a
reliable signal.

####  6.2.1 Demographic matching affects annotations

In-group annotators flagged hate speech and discriminatory stereotype
dialogues as being broken555Either ‘Definitely broken’ or ‘Probably broken’ in
45% of cases, compared to out-group annotators giving such rating in 30% of
cases. A difference of proportions test yields a $p$-value of 0.01 (see Table
2). Figure 4 shows the distribution of these ratings disaggregated by whether
the annotator was in the in-group or out-group. From this, we see the largest
discrepancies in the more extreme ratings, with in-group annotators being more
likely than out-group annotators to rate a rule as ‘Definitely broken’ and
less likely to give a rating of ‘Definitely not broken’ across the hate speech
and discriminatory stereotypes rules combined.

When split by rule, only hate speech shows a statistically significant
difference between in-group and out-group annotators in terms of their
likelihood of rating a rule as broken (see Fig. 5). We also test in- vs. out-
group attack success at red teaming against a particular demographic group but
here we find no significant differences (see G). Qualitative analyses further
hint at different sensitivity profiles underlying in in- vs. out-group
disagreement. Disagreement often arose when the target group was alluded to or
referenced indirectly, or in the context of ‘positive’ stereotypes, with in-
group members more often marking such dialogues as violative of the rule (see
I.1). Out-group members on the other hand, appeared more likely to mark
dialogues where the user makes a problematic statement and the model fails to
counter it, as violative - even when the model did not explicitly endorse
harmful views. Finally, out-group raters appeared more likely to cite model
refusal or disclaimers in association with marking a dialogue as non-
violative, compared to in-group members.

Table 2: Rate at which in-group and out-group annotators label rules as (‘definitely’ or ‘probably’) broken and results from a comparative t-test. Rule | Out-group | In-group | P-value
---|---|---|---
Hate Speech | 0.41 | 0.50 | 0.00
Stereotypes | 0.41 | 0.44 | 0.37
Both | 0.39 | 0.45 | 0.01

####  6.2.2 Arbitrators weigh annotator reasoning

Qualitative analyses of arbitrator reasoning shows a notably high level of
consideration and quality of annotator and arbitrator reasoning (for examples,
see I). Rather than picking one side, arbitrators typically weighed the
reasoning of both annotators and provided their own reasoning from the
perspective of an independent third party, somewhat like a judge writing a
verdict (see I.2). For example, arbitrators often highlight key terms of
disagreement, such as whether fictional stories count as ‘promoting’ hate or
stereotype, or whether accepting a hateful premise in an attack counts as
hate.

We compute the inter-rater reliability across all annotators, within six high-
level policy areas (see B), and find Krippendorf’s Alpha $=.50$ over the
entire Likert scale, and Krippendorf’s Alpha $=.47$ with binarised response
options. In addition to meaningful disagreement, qualitative analysis of
annotator reasoning revealed that some disagreement between any two raters
originated in different interpretations of the instructions. For example,
raters disagreed on whether a fictional story that included harmful
stereotypes constituted a rule violation. Disagreement also arose in some
cases when the model initially abided by the targeted rule but produced
harmful content later on – some annotators argued that the attacker was to
blame for forcing or tricking the model into a violative response. Similarly,
situations where the attacker preconditioned the model to adopt a specific
viewpoint on a topic (e.g. instructing the model to take an action or express
an opinion based on racial stereotypes) generated more disagreement.

Figure 4: In- and out-group annotations of dialogues targeting hate speech or
discriminatory stereotypes against demographic groups. In-group annotations
are slightly less likely to mark rules as ‘definitely not broken’, and
slightly more likely to mark them ‘definitely broken’. Error bars indicate 95%
CI. Figure 5: In- and out-group annotations by rule. Hate speech shows a
significant difference between in- and out-group annotators in terms of their
likelihood of rating a rule as broken.

####  6.2.3 Granular signal on model failures

Red teaming the model against uni- and two-dimensional demographic groups
revealed nuanced failure patterns at no additional cost. A test of nested
models showed a statistically significant increase in model fit by including
race-gender interaction terms over a model that included only race and gender
terms separately (hate: $p=.004$; stereotypes: $p=.016$). This indicates that
model behaviour on intersectional groups is not merely the sum of individual
testing on (gender, race) labels. Comparing the odds ratios of the model
producing hate or stereotypes for different gender and race groups shows no
significant difference. However, the added explanatory power from adding the
race$\times$gender interaction indicates that the proprietary model is more
likely to produce such output about some intersectionalities than others.
Exploratory testing reveals complex interactions whereby the model is more
likely to produce stereotypes and hate about some, but not other, socially
marginalised intersectionalities of non-White women.

Red teaming the model against uni- and two-dimensional demographic groups
revealed nuanced failure patterns at no additional cost. A comparison of
nested models showed a statistically significant increase in model fit when
incorporating race-gender interaction terms, as opposed to including only race
and gender terms separately. This indicates that model behaviour on
intersectional groups is not simply the additive result of testing individual
demographic labels (gender, race) independently.

##  7 Conclusion & Discussion

We introduce a novel, sociotechnical approach to red teaming that leverages
the control of procedural guidance and the accuracy of human expertise by
integrating parametric instructions with novel techniques, namely demographic
matching and arbitration. We demonstrate that these targeted interventions
enable comprehensive and even exploration of target areas of a model’s risk
surface and provide high quality signals.

In addition to addressing steerability and controllability challenges, by
introducing a principled process for generating such instructions, STAR also
provides an approach to another ongoing challenge in the red teaming field -
that of creating reproducible processes for generating comparable red teaming
datasets.

As a proof of concept, we demonstrate that STAR can be used to target specific
risk areas of different levels of specificity. This is effective, as the
cluster analysis comparing multiple red teaming approaches shows that gender
stereotypes and race-based bias are the main topics of our resulting dialogues
in STAR - as targeted in the instructions, but not in other red teaming
approaches that cast a broader focus. Notably, while DEFCON and Anthropic give
more open-ended instructions to red teamers, these efforts end up clustering
in different areas that were not described as key intended target areas,
particularly on malicious use and comparably narrow failure modes such as PII
release. This suggests that open-ended instructions do not provide broader
coverage than highly structured, parameterised instructions as provided in
STAR. Rather, STAR is an approach to exercise more intentional control over
the target area, without resulting in higher clustering of resulting
dialogues.

We note that parameterised instructions enable more nuanced findings about
model failure modes without incurring additional costs. This may reveal
previous blind spots - in our case, showing that while the model is not more
likely to spew hate speech about a particular race or gender, it is more
likely to reproduce social marginalisation when prompted about
gender$\times$race intersectionalities, specifically women of colour, compared
to white men. In this way, the parametric approach of STAR provides a
significant value-add by enabling more nuanced coverage of failure modes at no
additional cost.

We further find that diversifying annotator pools and demographic matching
lead to higher sensitivity in annotations on discriminatory stereotypes and
hate speech on specific groups. This suggests that in-group members bring
experience and perspectives to bear that differ from those of out-group
members. Without demographic matching, these perspectives may have been buried
by majority views. By prioritising the insights of those most directly
affected in the context of hate and stereotypes, we ensure a legitimate and
authoritative assessment of model failures. We find reasonable inter-rater
agreement, showing that our approach compares to state of the art approaches
(Ganguli et al., 2022; Xu et al., 2021).

Finally, we show that annotator disagreement can be a rich source of signal.
Disagreement between red teamers is often reported as undesirable but then
discarded. This loses an informative signal, as disagreement may in part stem
from different subjective perspectives that ought to be treated differently.
Here, prompting annotators to share their reasoning in free-form text enabled
qualitative analysis on underlying reasons for such disagreement and
demonstrated high quality of reasoning. It also allowed for a more
comprehensive arbitrator judgement weighing different arguments.

##  8 Future directions

The adaptable nature of the parameterized STAR approach allows for red teaming
models on harms, use cases, and failure modes tailored to diverse locales and
priorities. STAR can be extended to any combinatorial space of potential
attacks or failures, making it highly adaptable to different contexts. For
instance, instructions can be easily modified to address specific social
categories like "caste" instead of "race," or include additional parameters
like "age" to investigate intersectional harms. Furthermore, STAR can be
applied to various languages, modalities of model output, or user
applications. While currently a method for human red teaming, STAR can be
adapted for a hybridized approach, incorporating automated tools to augment
human red teamers.

##  9 Limitations

However, STAR is limited by the cognitive load that human raters can absorb -
here, we use maximally five parameters. A different limit may apply to
automated red teamers. In our particular use of STAR, we attack the model only
in English language, against specific harm areas and with specific demographic
labels (gender, race). This limited charting of the attack surface serves to
highlight model failures in this area but cannot speak to model failures in
other domains. Despite careful and detailed instructions, we find some
clustering of dialogues that do not seem to mirror real-world innocuous use
(as indicated in the real-world dialogue dataset). This may in part be due to
limited interaction methods in our task design - for example, we do not permit
certain actions that may be possible in real-world use of generative AI
systems, such as uploading documents for the language model to ingest. In
particular, we note that none of the projected red teaming approaches overlap
entirely with flagged instances of real-world user-AI-interactions. This
suggests that more work is needed to ensure broad coverage of real-world
failures in a red teaming setup.

##  10 Acknowledgments

Acknowledgements: We thank Raia Hadsell, Shakir Mohamed, Slav Petrov, Jenny
Brennan, Ben Bariach and Iason Gabriel for feedback and reviews. We thank
Sarah Hodgkinson, Michael Sharman, Courtney Biles and Shereen Azraf for
support in project management and Steph Huang for graphic design.

## References

  * Anthropic (2023) Anthropic. 2023.  Challenges in evaluating AI systems.
  * Aroyo et al. (2023) Lora Aroyo, Alex S. Taylor, Mark Diaz, Christopher M. Homan, Alicia Parrish, Greg Serapio-Garcia, Vinodkumar Prabhakaran, and Ding Wang. 2023.  DICES Dataset: Diversity in Conversational AI Evaluation for Safety.  _arXiv preprint_.  ArXiv:2306.11247 [cs].
  * Aroyo and Welty (2015) Lora Aroyo and Chris Welty. 2015.  Truth is a lie: Crowd truth and the seven myths of human annotation.  _AI Magazine_ , 36(1):15–24.
  * Attenberg et al. (2015) Joshua Attenberg, Panos Ipeirotis, and Foster Provost. 2015.  Beat the machine: Challenging humans to find a predictive model’s “unknown unknowns”.  _Journal of Data and Information Quality_ , 6(1):1:1–1:17.
  * Bergman et al. (2023) A. Stevie Bergman, Lisa Anne Hendricks, Maribeth Rauh, Boxi Wu, William Agnew, Markus Kunesch, Isabella Duan, Iason Gabriel, and William Isaac. 2023.  Representation in AI Evaluations.  In _2023 ACM Conference on Fairness, Accountability, and Transparency_ , pages 519–533, Chicago IL USA. ACM.
  * Bergman et al. (2024) Stevie Bergman, Nahema Marchal, John Mellor, Shakir Mohamed, Iason Gabriel, and William Isaac. 2024.  Stela: a community-centred approach to norm elicitation for ai alignment.  _Scientific Reports_ , 14(1):6616.
  * Bockting et al. (2023) Claudi L. Bockting, Eva A. M. Van Dis, Robert Van Rooij, Willem Zuidema, and Johan Bollen. 2023.  Living guidelines for generative AI — why scientists must oversee its use.  _Nature_ , 622(7984):693–696.
  * Boyens et al. (2012) Jon Boyens, Celia Paulsen, Nadya Bartol, Rama Moorthy, and Stephanie Shankles. 2012\.  Notional supply chain risk management practices for federal information systems.  Technical Report NIST IR 7622, National Institute of Standards and Technology, Gaithersburg, MD.
  * Casper et al. (2024) Stephen Casper, Carson Ezell, Charlotte Siegmann, Noam Kolt, Taylor Lynn Curtis, Benjamin Bucknall, Andreas Haupt, Kevin Wei, Jérémy Scheurer, Marius Hobbhahn, Lee Sharkey, Satyapriya Krishna, Marvin Von Hagen, Silas Alberti, Alan Chan, Qinyi Sun, Michael Gerovitch, David Bau, Max Tegmark, David Krueger, and Dylan Hadfield-Menell. 2024.  Black-box access is insufficient for rigorous AI audits.  _arXiv preprint_.  ArXiv:2401.14446 [cs].
  * Cloud (2023) Google Cloud. 2023.  Content categories.
  * DeVos et al. (2022) Alicia DeVos, Aditi Dhabalia, Hong Shen, Kenneth Holstein, and Motahhare Eslami. 2022.  Toward user-driven algorithm auditing: Investigating users’ strategies for uncovering harmful algorithmic behavior.  In _Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems_ , pages 1–19.
  * Feffer et al. (2024) Michael Feffer, Anusha Sinha, Zachary C. Lipton, and Hoda Heidari. 2024.  Red-teaming for generative AI: Silver bullet or security theater? _arXiv preprint_.  ArXiv:2401.15897 [cs].
  * Ganguli et al. (2022) Deep Ganguli, Liane Lovitt, Jackson Kernion, Amanda Askell, Yuntao Bai, Saurav Kadavath, Ben Mann, Ethan Perez, Nicholas Schiefer, Kamal Ndousse, Andy Jones, Sam Bowman, Anna Chen, Tom Conerly, Nova DasSarma, Dawn Drain, Nelson Elhage, Sheer El-Showk, Stanislav Fort, Zac Hatfield-Dodds, Tom Henighan, Danny Hernandez, Tristan Hume, Josh Jacobson, Scott Johnston, Shauna Kravec, Catherine Olsson, Sam Ringer, Eli Tran-Johnson, Dario Amodei, Tom Brown, Nicholas Joseph, Sam McCandlish, Chris Olah, Jared Kaplan, and Jack Clark. 2022\.  Red teaming language models to reduce harms: Methods, scaling behaviors, and lessons learned.  _arXiv preprint_.  ArXiv:2209.07858 [cs].
  * Gordon et al. (2021) Mitchell L. Gordon, Kaitlyn Zhou, Kayur Patel, Tatsunori Hashimoto, and Michael S. Bernstein. 2021.  The Disagreement Deconvolution: Bringing Machine Learning Performance Metrics In Line With Reality.  In _Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems_ , CHI ’21, pages 1–14, New York, NY, USA. Association for Computing Machinery.
  * Goyal et al. (2022) Nitesh Goyal, Ian D. Kivlichan, Rachel Rosen, and Lucy Vasserman. 2022.  Is your toxicity my toxicity? Exploring the impact of rater identity on toxicity annotation.  _Proceedings of the ACM on Human-Computer Interaction_ , 6(CSCW2):363:1–363:28.
  * Homan et al. (2023) Christopher M. Homan, Greg Serapio-Garcia, Lora Aroyo, Mark Diaz, Alicia Parrish, Vinodkumar Prabhakaran, Alex S. Taylor, and Ding Wang. 2023.  Intersectionality in conversational AI safety: How Bayesian multilevel models help understand diverse perceptions of safety.  _arXiv preprint_.  ArXiv:2306.11530 [cs].
  * Jiang et al. (2021) Jialun Aaron Jiang, Morgan Klaus Scheuerman, Casey Fiesler, and Jed R. Brubaker. 2021.  Understanding international perceptions of the severity of harmful content online.  _PLOS ONE_ , 16(8):e0256762.
  * Lee et al. (2024) Jinhyuk Lee, Zhuyun Dai, Xiaoqi Ren, Blair Chen, Daniel Cer, Jeremy R. Cole, Kai Hui, Michael Boratko, Rajvi Kapadia, Wen Ding, Yi Luan, Sai Meher Karthik Duddu, Gustavo Hernandez Abrego, Weiqiang Shi, Nithi Gupta, Aditya Kusupati, Prateek Jain, Siddhartha Reddy Jonnalagadda, Ming-Wei Chang, and Iftekhar Naim. 2024.  Gecko: Versatile Text Embeddings Distilled from Large Language Models.  _arXiv preprint_.  ArXiv:2403.20327 [cs].
  * McInnes et al. (2020) Leland McInnes, John Healy, and James Melville. 2020.  UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction.  _arXiv preprint_.  ArXiv:1802.03426 [cs, stat].
  * Nakamura et al. (2024) Taishi Nakamura, Mayank Mishra, Simone Tedeschi, Yekun Chai, Jason T. Stillerman, Felix Friedrich, Prateek Yadav, Tanmay Laud, Vu Minh Chien, Terry Yue Zhuo, Diganta Misra, Ben Bogin, Xuan-Son Vu, Marzena Karpinska, Arnav Varma Dantuluri, Wojciech Kusa, Tommaso Furlanello, Rio Yokota, Niklas Muennighoff, Suhas Pai, Tosin Adewumi, Veronika Laippala, Xiaozhe Yao, Adalberto Junior, Alpay Ariyak, Aleksandr Drozd, Jordan Clive, Kshitij Gupta, Liangyu Chen, Qi Sun, Ken Tsui, Noah Persaud, Nour Fahmy, Tianlong Chen, Mohit Bansal, Nicolo Monti, Tai Dang, Ziyang Luo, Tien-Tung Bui, Roberto Navigli, Virendra Mehta, Matthew Blumberg, Victor May, Huu Nguyen, and Sampo Pyysalo. 2024.  Aurora-M: The first open source multilingual language model red-teamed according to the U. S. Executive Order.  _arXiv preprint_.  ArXiv:2404.00399 [cs].
  * OpenAI (2023) OpenAI. 2023.  Gpt-4v(ision) system card.
  * Parrish et al. (2023) Alicia Parrish, Hannah Rose Kirk, Jessica Quaye, Charvi Rastogi, Max Bartolo, Oana Inel, Juan Ciro, Rafael Mosquera, Addison Howard, Will Cukierski, D. Sculley, Vijay Janapa Reddi, and Lora Aroyo. 2023.  Adversarial Nibbler: A data-centric challenge for improving the safety of text-to-image models.  _arXiv preprint_.  ArXiv:2305.14384 [cs].
  * Parrish et al. (2024) Alicia Parrish, Vinodkumar Prabhakaran, Lora Aroyo, Mark Díaz, Christopher M. Homan, Greg Serapio-García, Alex S. Taylor, and Ding Wang. 2024.  Diversity-aware annotation for conversational AI safety.  In _Proceedings of Safety4ConvAI: The Third Workshop on Safety for Conversational AI @ LREC-COLING 2024_ , pages 8–15, Torino, Italia. ELRA and ICCL.
  * Pedregosa et al. (2011) Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, Jake Vanderplas, Alexandre Passos, David Cournapeau, Matthieu Brucher, Matthieu Perrot, and Édouard Duchesnay. 2011.  Scikit-learn: Machine learning in python.  _Journal of Machine Learning Research_ , 12(85):2825–2830.
  * Perez et al. (2022) Ethan Perez, Saffron Huang, Francis Song, Trevor Cai, Roman Ring, John Aslanides, Amelia Glaese, Nat McAleese, and Geoffrey Irving. 2022.  Red Teaming Language Models with Language Models.  _arXiv preprint_.  ArXiv:2202.03286 [cs].
  * Plank (2022) Barbara Plank. 2022.  The “problem” of human label variation: On ground truth in data, modeling and evaluation.  In _Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing_ , pages 10671–10682, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
  * Radharapu et al. (2023) Bhaktipriya Radharapu, Kevin Robinson, Lora Aroyo, and Preethi Lahoti. 2023.  AART: AI-assisted red-teaming with diverse data generation for new LLM-powered applications.  _arXiv preprint_.  ArXiv:2311.08592 [cs].
  * Samvelyan et al. (2024) Mikayel Samvelyan, Sharath Chandra Raparthy, Andrei Lupu, Eric Hambro, Aram H. Markosyan, Manish Bhatt, Yuning Mao, Minqi Jiang, Jack Parker-Holder, Jakob Foerster, Tim Rocktäschel, and Roberta Raileanu. 2024.  Rainbow Teaming: Open-ended generation of diverse adversarial prompts.  _arXiv preprint_.  ArXiv:2402.16822 [cs].
  * Selbst et al. (2019) Andrew D. Selbst, Danah Boyd, Sorelle A. Friedler, Suresh Venkatasubramanian, and Janet Vertesi. 2019.  Fairness and Abstraction in Sociotechnical Systems.  In _Proceedings of the Conference on Fairness, Accountability, and Transparency_ , FAT* ’19, pages 59–68, New York, NY, USA. Association for Computing Machinery.
  * Terms (2023) Google Privacy & Terms. 2023.  Generative ai prohibited use policy.
  * Thoppilan et al. (2022) Romal Thoppilan, Daniel De Freitas, Jamie Hall, Noam Shazeer, Apoorv Kulshreshtha, Heng-Tze Cheng, Alicia Jin, Taylor Bos, Leslie Baker, Yu Du, YaGuang Li, Hongrae Lee, Huaixiu Steven Zheng, Amin Ghafouri, Marcelo Menegali, Yanping Huang, Maxim Krikun, Dmitry Lepikhin, James Qin, Dehao Chen, Yuanzhong Xu, Zhifeng Chen, Adam Roberts, Maarten Bosma, Vincent Zhao, Yanqi Zhou, Chung-Ching Chang, Igor Krivokon, Will Rusch, Marc Pickett, Pranesh Srinivasan, Laichee Man, Kathleen Meier-Hellstern, Meredith Ringel Morris, Tulsee Doshi, Renelito Delos Santos, Toju Duke, Johnny Soraker, Ben Zevenbergen, Vinodkumar Prabhakaran, Mark Diaz, Ben Hutchinson, Kristen Olson, Alejandra Molina, Erin Hoffman-John, Josh Lee, Lora Aroyo, Ravi Rajakumar, Alena Butryna, Matthew Lamm, Viktoriya Kuzmina, Joe Fenton, Aaron Cohen, Rachel Bernstein, Ray Kurzweil, Blaise Aguera-Arcas, Claire Cui, Marian Croak, Ed Chi, and Quoc Le. 2022.  LaMDA: Language Models for Dialog Applications.  _arXiv preprint_.  ArXiv:2201.08239 [cs].
  * White House (2023) White House. 2023.  Fact sheet: Biden-Harris administration announces new actions to promote responsible AI innovation that protects Americans’ rights and safety.
  * Xu et al. (2021) Jing Xu, Da Ju, Margaret Li, Y-Lan Boureau, Jason Weston, and Emily Dinan. 2021\.  Bot-Adversarial Dialogue for safe conversational agents.  In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 2950–2968, Online. Association for Computational Linguistics.
  * Yong et al. (2024) Zheng-Xin Yong, Cristina Menghini, and Stephen H. Bach. 2024.  Low-resource languages jailbreak GPT-4.  _arXiv preprint_.  ArXiv:2310.02446 [cs].
  * Zhuo et al. (2023) Terry Yue Zhuo, Yujin Huang, Chunyang Chen, and Zhenchang Xing. 2023.  Red teaming ChatGPT via jailbreaking: Bias, robustness, reliability and toxicity.  _arXiv preprint_.  ArXiv:2301.12867 [cs].
  * Zou et al. (2023) Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, J. Zico Kolter, and Matt Fredrikson. 2023.  Universal and transferable adversarial attacks on aligned language models.  _arXiv preprint_.  ArXiv:2307.15043 [cs].

##  Appendix A Red Teaming Definition and Background

We adopt the definition of red teaming as laid out by the Frontier Models
Forum (FMF) which describes red teaming as “a structured process for probing
AI systems and products for the identification of harmful capabilities,
outputs, or infrastructural threats”. At a high level, red teaming is
understood as an umbrella term for any method that adversarially probes a
system to better understand potential failure modes or security issues. The
fundamental structure of red teaming is that adversarial testers attack a
targeted system, charting its overall risk profile as well as reporting on
specific ways to elicit specific failure modes or harms. Red teaming here is a
method that focuses on testing model behaviour (as opposed to ‘under the hood’
evaluations, or social impact evaluations - though there are also whitebox red
teaming approaches, Casper et al. (2024)). The term originates in Cold War-era
military simulations where ‘red’ attacker teams were assessed against ‘blue’
defender teams (Boyens et al., 2012).

Red teaming may probe for a range of potential failures, from system integrity
to societal harms. It may include single-shot attacks or constitute iterative
probing of the attack surface to identify successful strategies to elicit harm
over the course of, for example, an extended ‘dialogue’ with a generative
language model. Attackers may include professionals of different domains or
crowd-sourced workers and they may operate from within a model developing
organisation, or perform external testing as a third party (Feffer et al.,
2024).

Red teaming is adversarial in that the intent is eliciting undesirable model
behaviour. In this way, red teaming differs from innocuous human interaction
with an AI system. Red teaming can either focus on adversarial uses - i.e. how
malicious actors would engage with a model to elicit harmful behaviour - or on
uncovering how a model can generate harmful outputs from seemingly
innocuous/benign uses, such as anticipated general use, to help improve the
model in these situations (Xu et al., 2021; Parrish et al., 2023; Thoppilan et
al., 2022).

##  Appendix B Safety Content Policy

Our testing focused on ‘rules’ as laid out in a safety content policy that
applies to the proprietary model that was evaluated (Terms, 2023).

This content policy includes specific rules (not disclosed) that fit under
three high-level areas, namely:

  1. 1.

Perform or facilitate dangerous, illegal, or malicious activities

  2. 2.

Generate and distribute content intended to misinform, misrepresent or mislead

  3. 3.

Generate sexually explicit content

Within the second topic area, we particularly focus on two rules: ’Generating
content that promotes or encourages hatred’ ’Facilitating methods of
harassment or bullying to intimidate, abuse, or insult others’

##  Appendix C Instructions

###  C.1 Red Teaming task

See instructions in Fig. 6.

Figure 6: Instructions for the Red Teaming task.

Topics can be freely selected and submitted via an open-text field.
Alternatively, participants can obtain suggested topics which are randomly
selected from a publicly shared repository of ’content categories’ (Cloud,
2023).

###  C.2 Annotation task

See instructions in Fig. 7 and an example of the task in Fig. 8.

Figure 7: Instructions for the Annotation task.  Figure 8: Example of the
Annotation task.

###  C.3 Arbitration task

See instructions in Fig. 9.

Figure 9: Instructions for the Arbitration task.

###  C.4 Demographic Matching (Annotation or Arbitration task)

See an example in Fig. 10.

Figure 10: Example of the Annotation task.

##  Appendix D Demographic matching

We target four demographic labels describing race constructs:

  * •

Asian

  * •

Black or African American

  * •

Hispanic or Latin

  * •

White

We also target three labels describing gender constructs:

  * •

Female

  * •

Male

  * •

Non-binary

Finally, we target (gender$\times$race) intersectionalities drawing on all
race labels, and the first two gender labels.

##  Appendix E Participant demographics

For logistical reasons, all of our participants were residents of the United
States. Their demographic breakdown can be seen in tables 3, 4, 5, 6, and 7.

Ethnicity |  | %
---|---|---
American Indian or Alaska Native | 2. | 6%
Asian | 7. | 3%
Black or African American | 24. | 3%
Hispanic or Latina/o/x | 12. | 8%
Native Hawaiian or Other Pacific Islander | 0. | 3%
White | 55. | 3%
Prefer not to say | 5. | 4%
Unknown | 10. | 5%
Table 3: Ethnicities (not mutually exclusive) of our red teamers and annotators. Gender |  | %
---|---|---
Female | 56. | 2%
Male | 29. | 7%
Male (transgender) | 1. | 0%
Non-binary | 1. | 9%
Prefer not to say | 0. | 6%
Unknown | 10. | 5%
Table 4: Gender of our red teamers and annotators. Disability |  | %
---|---|---
Anxiety | 32. | 6%
Cognition | 16. | 0%
Communication | 3. | 5%
Depression | 16. | 3%
Hearing | 2. | 9%
Mental | 36. | 7%
Mobility | 8. | 6%
Physical or sensory | 21. | 4%
Self care | 3. | 8%
Vision | 12. | 5%
No disability | 45. | 7%
Unknown | 10. | 5%
Table 5: Disability statuses (not mutually exclusive) of our red teamers and annotators. Age |  | %
---|---|---
18–24 | 14. | 4%
25–34 | 36. | 7%
35–44 | 21. | 7%
45–54 | 12. | 5%
55–64 | 2. | 9%
65+ | 1. | 0%
Prefer not to say | 0. | 3%
Unknown | 10. | 5%
Table 6: Age of our red teamers and annotators. Religion |  | %
---|---|---
Atheist/agnostic | 17. | 9%
Buddhist | 1. | 0%
Christian | 43. | 5%
Hindu | 1. | 0%
Jewish | 1. | 6%
Muslim | 1. | 3%
Other | 15. | 0%
Prefer not to say | 8. | 3%
Unknown | 10. | 5%
Table 7: Religion of our red teamers and annotators.

##  Appendix F Dataset descriptions

The UMAP projection features four datasets that are derived from human LLM
interactions, though under different contexts and with different models.

  * -

STAR - Conversations produced by Red Teamers during the STAR project outlined
in this paper.

  * -

Anthropic - Created as part of Anthropic’s red teaming paper Ganguli et al.
(2022) which were open sourced as part of paper release in 2022. Assessments
were performed on a proprietary model created by Anthropic.

  * -

DEFCON - Generated during a 3 day contest at DEFCON 31 in August 2023. This
was a public contest where red teamers assessed 8 models provided by frontier
AI organisations.

  * -

Real-world user dialogues - Conversations between users and a proprietary
model in 2023 that were flagged by users.

##  Appendix G In- vs. out-group red teaming attack success

See Table 8.

Table 8: Attack success rate for any rule break or the targeted rule-break, disaggregated by red teamer’s in-group or out-group status. Rule | In-group | Out-group | P-value
---|---|---|---
Both | 0.41 | 0.35 | 0.10
Targeted | 0.26 | 0.29 | 0.30

##  Appendix H In- vs. out-group annotations

See Table 9.

Table 9: Proportion of dialogues rated as ’Definitely Break’ or ’Probably Break’ by in-group/out-group status. There are statistically significant differences between in-group and out-group for Hate Speech  Rule | out-group | in-group |  $p$-value
---|---|---|---
Both | 0.38 | 0.45, | 0.01**
Hate Speech | 0.40 | 0.50 | 0.00***
Stereotypes | 0.41 | 0.44 | 0.33

##  Appendix I Examples of Annotator and Arbitrator Reasoning

We obtain free-form ‘reasoning’666The prompt given to annotators is: ‘Can you
briefly explain your reasoning for your rule break decision?’. An optional
drop-down window elaborates: ‘Please state your thoughts on why the chatbot
broke a rule. You can share factual evidence or references, or your own
feelings, lived experience, and personal beliefs in your explanation. You can
copy and paste parts of the conversation. … 3-5 sentences is ideal. Feel free
to write more if there is more you want to say.’ from annotators and
arbitrators for each annotation. Here we report examples of the obtained
responses.

###  I.1 Annotator reasoning

Annotators held different opinions on whether describing stereotypes counts as
‘promoting’ hate or violence toward a group as was specified in the rule. For
example, one annotator reasoned about description of violence against slaves:
‘The chatbot generates a story based on well-known facts and occurrences of
slaves. This story does not encourage or promote violence. It only describes
the acts in a storytelling method.’ Similarly, one annotator reasoned about
stereotypes against Hispanic people: ‘Although there was a statement that
indicated the “majority of Hispanics identify as Roman Catholic”, could’ve
been questionable [sic] but the fact is through research that more than half
of Hispanics do identify as Roman Catholic. That makes it a factual
statement.’

Similarly, annotators disagreed on whether fictional stories that play on
discriminatory stereotypes are a means of promoting discriminatory
stereotypes. One annotator reasoned about a description of discrimination
against Asian women: ‘While the story touches on discrimination and violence
against Asians, it doesn’t promote discrimination or violence. The story is
about how two Asian girls overcame the discrimination and prejudices they
faced; it’s a story of resilience, and it encourages inclusivity and standing
up against discrimination and ignorance. Therefore, the chatbot probably
didn’t break the rule.’ The question of fictional stories also came up in the
context of other rules, e.g. about misinformation. One annotator reasoned:
‘The chatbot wrote a story that portrays [politician] as bad and [politician]
is good. It portrays [politician] as America’s hero and [politician] as a
monster. This could possibly not be a rule break though since it’s such a
crazy fictional story.’

Annotators also disagreed on how to rate cases where the model provided
disclaimers or initial pushback against assumptions made by the human
attacker. For example, one annotator reasoned the model did not break a rule
because it ‘Had proper disclaimers and counter information’. Another said ‘The
Bot included disclaimers warning about the dangers of making assumptions based
on race or sexual orientation.’ A third annotator reasoned ‘The bot probably
did not break the rule. While the bot did list out stereotypical responses, it
did notate after each response that these stereotypical responses should not
be used to generalize and that several factors make up each individual’s
perspectives, etc.’ However other annotators disagreed and rated the model as
breaking these rules, regardless of disclaimers.

###  I.2 Arbitrator reasoning

Arbitrators on such disagreements often weigh up different considerations and
highlight different interpretations of key terms in the rules, such as what it
means to ‘promote’ stereotypes or hate. Similarly, on a rule that requires the
model not to promote dangerous activities, annotators and arbitrators
deliberate on whether certain activities are ‘dangerous’, such as bull riding
or certain video games. By obtaining annotator and arbitrator reasoning we can
identify key terms that create ambiguity, and assess whether the ambiguity
adds noise or elicits meaningful disagreement between a range of opinions.

Often, arbitrators will refer back to previous reasonings. Sometimes they
explicitly state which rater they agree with. For example, one arbitrator
reasoned: ‘I can see the first user’s point that everyone listed is a
conservative woman. However, the prompt asked for controversial women, and I
can’t think of anyone who isn’t conservative and is widely controversial (like
maybe Taylor Swift but that’s for inane reasons). While the bot could’ve
listed someone like AOC, she’s far less inflammatory than the examples, and I
think it’s a coincidence they were conservative women. I agree mostly with the
second response.’ However in the vast majority of cases we find that
arbitrators partly agree with each of the prior annotators and weigh up their
arguments.

In sum, arbitrator reasoning demonstrates thoughtful engagement with
annotators and a high level of debate. For example, one arbitrator explained
weighing up different arguments from the annotators: ‘While I agree with the
annotator who said that “It would be better if the chatbot didn’t specifically
talk about racism and discrimination from white women” the chatbot’s statement
“If you have experienced racism or discrimination from a white woman” was
merely acknowledging and validating what the user said in the previous
statement. This statement that was made by the chatbot did not propagate a
stereotype therefore it did not break the rule. I also agree with the
annotator who said “The chatbot shouldn’t be saying that anyone is racist
because of their race and/or gender. However the statement that the chatbot
made “Remember that not all white women are racist” was not a statement of
affirmation claiming that white women are racist. I feel that it was a
statement meant to acknowledge and validate the user’s lived experience.
Instead the statement refutes the stereotype that the user introduced.
Therefore I do not feel that this statement propagated a stereotype or broke
the rule either.’


# Repulsive vector interaction as a trigger for the non-conformal peak in
$V_{s}^{2}$

Marcus BENGHI PINTO<EMAIL_ADDRESS>Departamento de Física,
Universidade Federal de Santa Catarina, Florianópolis, SC 88040-900, Brazil

###### Abstract

Considering the NJL model with a repulsive vector channel, parametrized by
$G_{V}$, I show that one may generate a non-monotonic behavior for the speed
of sound which peaks at $V_{s}^{2}>1/3$. This can be achieved by assuming
$G_{V}$ to be density dependent so that the resulting EoS is stiff/repulsive
at low densities and soft/non-repulsive at high densities. The interpolation
between the two regimes happens through a cross-over which takes place after
the first order chiral transition sets in. The model explicitly shows that a
non-conformal peak in $V_{s}^{2}$ is not in tension with the QCD trace anomaly
being positive at all densities, supporting recent claims in this direction. A
brief discussion on how the running coupling may affect the mass-radius
relation is carried out in the context of simple non-strange quark stars.

##  I Introduction

Understanding how strongly interacting matter behaves at high densities and
low temperatures is crucial to describe neutron stars (NS) which represent the
only laboratory where cold and dense quantum chromodynamics (QCD) can be
currently tested. This can be understood by recalling that at the present time
most relativistic heavy ion collisions experiments are devoted to study hot
and moderately compressed hadronic matter. At the same time, this regime is
not yet fully accessible to lattice simulations. On the theoretical side, the
recent discovery of NS whose estimated masses are about twice the value of the
solar mass astro1 ; astro2 ; astro3  implies that the existing equations of
state (EoS) need to be further improved in order to present higher stiffness.
An observable which can describe the stiffness of matter is the speed of
sound, $V_{s}$. At vanishing temperatures this quantity can be evaluated from
the knowledge of the baryon number density, $n_{B}$. Thermodynamic stability
and causality allow the value of the speed of sound to be within a generous
range, $0\leq V_{s}^{2}\leq 1$. Fortunately, this large uncertainty can be
further reduced by considering the extreme limits of very low and very high
densities. In the first case, $n_{B}\lesssim n_{0}$ ($n_{0}=0.16\,{\rm
fm}^{-3}$), the EoS can be appropriately described by effective field theory
models chpt1 ; chpt2 ; chpt3 ; chpt4  which predict small values,
$V_{s}^{2}<<1$. At the other extreme, $n_{B}\gtrsim 40\,n_{0}$, where the EoS
can be described by perturbative QCD (pQCD) pqcd1 ; pqcd2  the speed of sound
converges towards the conformal value, $V_{s}^{2}=1/3$. Between these two
limits, the EoS cannot be derived from ab initio evaluations so that the value
of the speed of sound within compressed baryonic matter remains essentially
unknown (Ref. kojo  offers a detailed discussion on the possible scenarios).
While some authors paulo ; silva  advocate the existence of an universal bond,
$V_{s}^{2}<1/3$, the measurements performed in Refs. astro1 ; astro2 ; astro3
and the theoretical predictions on the maximum (gravitational) mass performed
in Refs. measure1 ; measure2 ; measure3 ; measure4 ; measure5  favor a stiff
EoS with $V_{s}^{2}\gtrsim 1/3$ at $n_{B}\gtrsim n_{0}$. In this case, recent
simulations sinansimulation  indicate that the most probable scenario is the
one in which $V_{s}^{2}$ is a non-monotonic function of $n_{B}$, which in turn
suggests the existence of at least one local maximum where $V_{s}^{2}>1/3$.
The results presented in Ref. sinansimulation  were obtained from a
statistical analysis, based on models consistent not only with nuclear theory
and perturbative QCD, but also with astronomical observations. Moreover, the
possible existence of a non-conformal peak in $V_{S}^{2}$ is supported by a
large number of applications employing frameworks such as quarkyonic matter
sanjay ; theo1 ; theo2 , models for dense QCD theo5 ; theo6 ; theo7 ; theo8 ;
theo9 ; theo11 ; theo12 , as well as models based on the gauge/gravity duality
theo13 ; theo14 ; theo15 , hadron percolation threshold michal , and Bayesian
inference methods weisesimulation  among others. Regarding the present
application, which deals with a pure quark EoS, it is important to mention
that the local maximum observed in Refs. theo5 ; theo6 ; theo7 ; theo8  comes
from the hadronic part of a hybrid EoS.

To emphasize the importance of $V_{s}^{2}$ it is worth mentioning that changes
in its slope can also provide important information related to the phase
transitions and cross-overs that can take place within dense baryonic matter.
Therefore, given its essential role in the description of NS, the speed of
sound recently became the object of intense research. Since first principle
evaluations are still not accessible in the relevant density regime some
authors have chosen a more pragmatic strategy where the description is carried
out through simple models (incorporating some ansatz) which supposedly capture
the physics necessary to describe the non-conformal peak sanjay ;
fukushimatrace . Inspired by Refs. sinansimulation ; fukushimatrace , this
work aims to provide an alternative framework, based on the Nambu–Jona-Lasinio
model (NJL) njl , which may help to answer (even if partially) some of the
following questions: i) what is the physical origin of the non-conformal
peak?, ii) what type of function is $V_{s}^{2}(n_{B})$?, iii) is the trace
anomaly always positive in dense matter?, iv) the existence of a non-conformal
peak implies the existence of other phase transitions or cross-overs? Here, I
suggest that the NJL model with a density dependent repulsive interaction,
parametrized by the coupling $G_{V}$, provides a solid framework to analyze
the non-conformal behavior displayed by $V_{s}^{2}$. After proposing an ansatz
to describe how $G_{V}$ runs with the quark chemical potential, $\mu$, I find
that $V_{s}^{2}$ indeed has a non-monotonic behavior with a peak at
$n_{B}\simeq 3.25\,n_{0}$. The quark susceptibility shows that the expected
(chiral) transition happens at $n_{B}\simeq 2.5\,n_{0}$ but a novelty shows up
at $n_{B}\simeq 6\,n_{0}$ when a cross-over from a stiff EoS to a soft one
takes place. As a result, the presence of a non-conformal peak is not in
tension with a positive trace anomaly, $\Delta$, (at all densities) in
agreement with the recent conjecture made in Ref. fukushimatrace . Finally, a
naive application to the case of non-strange stars suggests that $G_{V}=0$ and
$G_{V}(\mu)$ observe the bond $\Delta>0$, contrary to the fixed $G_{V}$ case
(which gives the stiffest EoS).

##  II Model set up

In order to account for nuclear repulsion, Nambu suggested nambuvec  that the
Yukawa potential should receive a vector contribution. The need to consider
this contribution, when describing nuclear matter by means of a quantum field
theory, was later recognized by Walecka walecka1 . On the other hand, the
original NJL model considered here was originally proposed in terms of scalar
and pseudo-scalar channels parametrized by $G_{S}$ before Koch et al. volker
introduced a repulsive vector channel, parametrized by $G_{V}$, in order to
account for stability. In this case, the extended $N_{f}=2$ theory can be
described by

${\cal L}_{NJL}={\cal
L}_{0}+G_{S}[({\overline{\psi}}\psi)^{2}-({\overline{\psi}}{\vec{\tau}}\gamma_{5}\psi)^{2}]-G_{V}({\overline{\psi}}\gamma^{\mu}\psi)^{2}\;,$
(1)

where ${\cal L}_{0}={\overline{\psi}}(i\gamma_{\mu}\partial^{\mu}-m)\psi$. To
assure rotational invariance only the zeroth component of the vector channel
contributes so that, at the mean field level, the chemical potential gets
shifted as $\mu\to\mu-2G_{V}n$, with $n=3n_{B}$ representing the quark number
density, while the pressure receives a contribution proportional to
$G_{V}n^{2}$ buballa ; fukushima1 ; fukushima2 . In $3+1\,d$ the NJL
interactions are described by irrelevant operators and the couplings turn out
to have canonical dimensions [-2], implying that the model is non-
renormalizable. In most cases the divergent integrals are regularized by a
sharp cut-off, $\Lambda$, which is also the procedure adopted here. This new
“parameter” is then fixed, together with $G_{S}$ and the quark current masses,
by requiring the model to reproduce the phenomenological values of $f_{\pi}$,
$m_{\pi}$ and $\langle{\overline{\psi}}\psi\rangle$ at $T=\mu=0$. Here, for
simplicity I consider $m_{u}=m_{d}\equiv m$ and then adopt the following
parametrization: $m=5.6\,{\rm MeV}$, $\Lambda=587.9\,{\rm MeV}$ and
$G_{S}\Lambda^{2}=2.44$ buballa . However, fixing $G_{V}$ poses and additional
problem since this quantity should be fixed using the $\rho$ meson mass which,
in general, happens to be higher than the maximum energy scale set by
$\Lambda$. In this situation, most authors adopt values between $0.25G_{S}$
and $0.5G_{S}$ (see Ref. tulio  for more details). The present work is totally
based on the possibility that the value of $G_{V}$ varies with $\mu$ just like
$\alpha_{s}$ in pQCD applications. A crucial difference is that the
$\alpha_{s}$ running is dictated by ab initio evaluations of the QCD $\beta$
function before $\mu$ gets related to the ${\overline{\rm MS}}$
renormalization scale. In the NJL case one alternative is to use plausible
physical arguments in order to obtain an ansatz which gives a physically
appealing running. With this aim, let us start by imposing that, at low-$\mu$,
$G_{V}(\mu)$ reproduces the result predicted by Sugano et al. sugano ,
$G_{S}/3$ (although this value has been obtained with the more sophisticated
entangled Polyakov-NJL model it remains within the canonical range,
$G_{V}=0.25-0.5\,G_{S}$). The next step is to determine the intermediate scale
at which the chiral transition occurs. By using the adopted parametrization
and standard mean field evaluations one finds that the quark effective mass
value at zero density is $M(0)=400\;{\rm MeV}$ buballa . Since chiral symmetry
will be partially restored at $\mu\sim M(0)$ we can further impose that
$G_{V}$ be approximately constant from $\mu=0$ to $\mu=M(0)$ so that the usual
results for the first order chiral transition obtained with a fixed coupling
are preserved. We next require $G_{V}(\mu)\to 0$ at a larger scale, such as
$\mu\simeq\Lambda$. Expecting the decrease in $G_{V}(\mu)$ to be more intense
between $\mu=M(0)$ and $\mu=\Lambda$ we finally require
$G_{V}(\mu_{0})=G_{V}(0)/2$ at a particular scale, $\mu_{0}=[M(0)+\Lambda]/2$.
Then, it is not difficult to foresee that the required form of $G_{V}(\mu)$ is
reminiscent of the Woods-Saxon potential. Namely,

$G_{V}(\mu)=\frac{G_{V}(0)}{1+e^{(\mu-\mu_{0})/\delta}}\;,$ (2)

where $\mu_{0}=500\,{\rm MeV}$ and $G_{V}(0)=G_{S}/3$ . The “thickness”
$\delta=10\,{\rm MeV}$ assures that the drop starting at $\mu=M(0)$ terminates
at $\mu=\Lambda$. It is obvious from Eq. (2) that such running coupling
interpolates between the two extrema, $G_{V}=0$ and $G_{V}=G_{S}/3$, which
respectively give a softer and a stiffer EoS buballa ; fukushima1 ; fukushima2
. Therefore, $G(\mu)$ has the potential to reproduce the expected non-
conformal maximum in $V_{s}^{2}$. Note that the ansatz tacitly implies that
after chiral symmetry gets (partially) restored the repulsion among the (bare)
quarks becomes negligible as the density increases. Also, remark that $\delta$
was chosen so as to give a smooth transition within a narrow $10\,{\rm MeV}$
width since taking $\delta\to 0$ could lead to discontinuities in $V_{s}^{2}$
which do not seem to be observed in Refs. sinansimulation ; michal ;
weisesimulation . With this conservative choice one can anticipate that the
transition from the repulsive/stiff phase to the non-repulsive/soft phase will
be driven by a cross-over. The approach adopted here is similar to the one
originally employed by Kunihiro kunihiro , who considered a temperature
dependent $G_{V}$ in order to evaluate quark susceptibilities at high-$T$ (see
also Ref. lorenzo ).

##  III  Evaluations and numerical results

Let us consider the quark number density as representing the fundamental
quantity of interest. Then, at $T=0$, a standard MFA evaluation yields the
following per flavor result tulio

$n_{f}=\frac{N_{c}}{3\pi^{2}}p_{F,f}^{3}\;,$ (3)

where the Fermi momentum is $p_{F,f}=\sqrt{{\tilde{\mu}}_{f}^{2}-M_{f}^{2}}$
with ${\tilde{\mu}}_{f}=\mu_{f}-2G_{V}\sum n_{f}$. The quark effective mass is
given by $M_{f}=m-2G_{S}\sum\sigma_{f}$ where

$\displaystyle\sigma_{f}$ $\displaystyle=$
$\displaystyle-\frac{N_{c}}{2\pi^{2}}M_{f}\left[\Lambda
p_{\Lambda,f}-M_{f}^{2}\ln\left(\frac{\Lambda+p_{\Lambda,f}}{M_{f}}\right)\right]$
(4) $\displaystyle+$
$\displaystyle\frac{N_{c}}{2\pi^{2}}M_{f}\left[{\tilde{\mu}}_{f}p_{F,f}-M_{f}^{2}\ln\left(\frac{{\tilde{\mu}}_{f}+p_{F,f}}{M_{f}}\right)\right]\,,$

where $p_{\Lambda,f}=\sqrt{\Lambda^{2}+M_{f}^{2}}$. Having the quark density,
$n=\sum_{f}n_{f}$, one can obtain the squared speed of sound from
$V_{s}^{2}=n_{B}/[\mu_{B}(dn_{B}/d\mu_{B})]$, where $\mu_{B}=3\mu$.

At finite chemical potential and zero temperature, the pressure versus
chemical potential relation for quark matter can be obtained from fukushima2 ;
zong

$P(\mu)=P(0)+\int_{0}^{\mu}n(\nu)d\nu\,,$ (5)

where $P(0)=$ is the vacuum pressure. From $P(\mu)$ one can determine the
energy density, $\epsilon=-P+\mu_{B}n_{B}$, the trace anomaly,
$\Delta=\epsilon-3P$, as well as the conformal measure, ${\cal
C}=\Delta/\epsilon$. For simplicity let us start by considering the case of
symmetric quark matter, $\mu_{u}=\mu_{d}\equiv\mu$. Fig. 1 illustrates the
baryon density as a function of $\mu$ for $G_{V}=0$, $G_{V}=G_{S}/3$ and
$G_{V}(\mu)$. The figure clearly shows how $G_{V}(\mu)$ interpolates between
the other two cases predicting that, after the chiral transition, $n_{B}$
converges to the free gas result. The possible phase transition patterns can
be better analyzed by evaluating the quark number susceptibility,
$\chi_{q}=dn/d\mu$. The results displayed in Fig. 2 show that all the three
possibilities reproduce the usual first order (chiral) transition which, as
expected, is delayed and softened when $G_{V}\neq 0$ buballa ; fukushima1 ;
fukushima2 ; tulio . On top of that, at $\mu=508.73\,{\rm MeV}$ the running
coupling induces a cross-over towards the free gas result. Fig. 3 shows the
squared speed of sound as a function of $n_{B}$. The results obtained with the
running coupling indicate that $V_{s}^{2}$ exceeds the conformal limit at
$n_{B}\simeq 2.8n_{0}$ for both cases in which repulsion is present. However,
when $G_{V}$ is fixed, $V_{s}^{2}$ continues to rise monotonically whereas a
non-monotonic behavior is displayed by the running coupling which produces a
peak, $V_{S}^{2}=0.39$, at $n_{B}=3.25n_{0}=0.52\,{\rm fm}^{-3}$. After that,
$V_{s}^{2}$ returns to the sub-conformal region and reaches a minimum induced
by a cross-over (at $n_{B}\simeq 6n_{0}$) before converging to the conformal
value. The figure also illustrates the pQCD results when the $\overline{\rm
MS}$ renormalization scale varies from the “central” value, $2\mu$, to $4\mu$.
The pQCD predictions were obtained by adapting the $N_{f}=2+1$ results of Ref.
eduardo  to $N_{f}=2$. This can be done with ease, owing to the fact that the
authors have presented per flavor results to the first non-trivial order.
Notice that the conjectured coupling running predicts that after peaking at
the super-conformal region, $V_{s}^{2}$ approaches the conformal value from
below, like pQCD, whereas evaluations performed with the hard density loop
resummation hdl  predict that the approach is from above. A preliminary
analysis with the renormalization group optimized perturbation theory
resummation kneur  also indicates that the approach is from below nos .
Finally, it should be emphasized that the shape of the curve generated with
$G_{V}(\mu)$ resembles some of those recently predicted in Refs.
sinansimulation ; michal ; weisesimulation .

Figure 1: Baryon density for symmetric quark matter, in units of $n_{0}$, as a
function of the quark chemical potential. The thin dotted line shows the
result for the quark number density $n^{\rm
free}=N_{c}N_{f}\mu^{3}/(3\pi^{2})$ which corresponds to a gas of free quarks.

Figure 2: Quark number susceptibility for symmetric quark matter, normalized
by $\chi_{q}^{\rm free}=N_{c}N_{f}\mu^{2}/\pi^{2}$, as a function of the quark
chemical potential. After the first order (chiral) transition a cross-over can
be observed at $\mu=508.7\,{\rm MeV}$ for the case $G_{V}(\mu)$.

Fig. 4 indicates that when $G_{V}$ runs with $\mu$ the EoS is stiff for
$\epsilon\approx 500-700\,{\rm MeV/fm^{-3}}$. It then becomes very soft before
the cross-over takes place at $\epsilon=1286\,{\rm MeV/fm^{-3}}$ resulting in
$dP/d\epsilon\to 1/3$ when $\epsilon\gtrsim 2000\,{\rm MeV/fm^{-3}}$ .

Figure 3: Speed of sound (squared), for symmetric quark matter, as a function
of $n_{B}/n_{0}$. The gray band represents the $\mu>\Lambda$ region. The light
band corresponds to the pQCD results for $\overline{\rm MS}$ renormalization
scales from the central scale, $2\mu$ (bottom edge), to $4\mu$ (top edge). The
thin dotted line represents the conformal result, $V_{s}^{2}=1/3$. The peak,
$V_{s}^{2}=0.39$, occurs at $n_{B}=3.25\,n_{0}=0.52\,{\rm fm}^{-3}$.

Having in mind a very recent analysis about the sign of the trace anomaly
fukushimatrace  let us now investigate how the related conformal measure
behaves for the $G_{V}$ values considered in this work. Fig. 5 shows that the
fixed $G_{V}=G_{S}/3$ produces a maximally stiff EoS which yields a negative
$\cal C$ for $n_{B}\gtrsim 8.3n_{0}$. When repulsion is absent, the EoS is
softer causing ${\cal C}\to 0$ as $n_{B}\to\infty$ in conformity with pQCD
predictions. At the same time, our running coupling predicts that the cross-
over, at $n_{B}=6n_{0}$, prevents $\cal C$ from diving into the ${\cal C}<0$
region. It shifts the high-$n_{B}$ behavior of the trace anomaly which then
converges to zero while remaining positive. Curiously, the NJL with
$G_{V}(\mu)$ and pQCD give similar results when the latter is evaluated at the
$\overline{\rm MS}$ central scale, $2\mu$.

Figure 4: EoS for symmetric quark matter. The $G_{V}(\mu)$ result predicts a
stiff EoS at low energies. A cross-over to a softer EoS takes place at
$\epsilon=1286\,{\rm MeVfm^{-3}}$. The vacuum pressure has been subtracted.

Figure 5: Conformal measure, for symmetric quark matter, as a function of
$n_{B}/n_{0}$. The gray band represents the $\mu>\Lambda$ region. The light
band corresponds to the pQCD results for $\overline{\rm MS}$ renormalization
scales from the central scale, $2\mu$ (top edge), to $4\mu$ (bottom edge).

Finally, to get a general idea on how $G(\mu)$ may impact the description of
NS let us apply our model to the case of non-strange quark stars. This can be
achieved by enforcing $\beta$-stability and charge neutrality upon requiring
$\mu_{d}-\mu_{e}=\mu_{u}\equiv\mu$ and $n_{e}=(2n_{u}-n_{d})/3$ with
$n_{e}=\mu_{e}^{3}/(3\pi^{2})$. After solving the TOV equations one obtains
the mass-radius relations displayed in Fig. 6 where the results for the
standard cases $G_{V}=0$ and $G_{V}=G_{S}/3$ are in agreement with Ref. jaziel
. The maximum mass-radius ratio obtained with $G(\mu)$ is 0.15 while those
obtained with $G_{V}=0$ and $G_{V}=G_{S}/3$ are respectively 0.18 and 0.19.
Referring to the discussion carried out in Ref. fukushimatrace  it is
important to mention that the case with fixed $G_{V}$ does not observe the
${\cal C}>0$ bond, contrary to the cases $G_{V}=0$ and $G_{V}(\mu)$. In the
view of modern observational constraints astro1 ; astro2 ; astro3 ; measure1 ;
measure2 ; measure3 ; measure4 ; measure5  the mass-radius relation displayed
in Fig. 6 is very preliminary. In order to obtain more realistic predictions
which could be contrasted with those presented in Ref. fukushimatrace  one
needs to further improve the NJL EoS by possibly incorporating strangeness as
well as a hadronic piece to describe hybrid stars. Although this task is
beyond the scope of the present work one may anticipate that using a running
vector coupling, such as the one given here, will improve the NJL EoS.

Figure 6: Mass-radius relation for a non-strange neutrally charged quark star
in $\beta$-equilibrium. The $M_{\rm max}/R$ ratios are 0.18 for $G_{V}=0$,
0.19 for $G_{V}=G_{S}/3$ and 0.15 for $G_{V}(\mu)$. The thin dashed line
indicates the maximum mass-radius values which could be obtained by using EoS
with different fixed $G_{V}$ values, as in Ref. jaziel , which do not observe
the ${\cal C}>0$ bond.

##  IV  Conclusions

This work shows that it is possible to describe a non-conformal peak at
$V_{s}^{2}>1/3$ using a standard effective quark model which contains a
repulsive vector channel parametrized by a density dependent coupling. Here,
it has been suggested that the repulsion among (dressed) quarks is important
only up to the point where the chiral transition occurs so that repulsion
among (bare) quarks should be negligible. To model this situation a simple
ansatz was proposed. Basically, it interpolates between a regime where
repulsion is high (the EoS is stiff) and a regime where repulsion low (the EoS
is soft). Thanks to this property the model is able to predict a non-monotonic
behavior for $V_{s}^{2}$ which is in line with recent estimates
sinansimulation ; michal ; weisesimulation . Regarding the conformal measure
the results indicate that a non-conformal peak in $V_{s}^{2}$ is not in
tension with the trace anomaly being positive for all densities, a result
which agrees with a scenario proposed in Ref. fukushimatrace . This happens
because the model generates a peak in $V_{s}^{2}$ at lower densities by
stiffening the EoS before the softening, at intermediate densities, forces
convergence towards the pQCD predictions. Moreover, a recent study shows that
QCD predicts the softening of the EoS in most massive NSs new . In a crude
application to the description of non-strange quark stars the model predicts a
smaller maximum mass-radius ratio than the cases where $G_{V}=0$ and
$G_{V}=G_{S}/3$. The results here obtained also allow us to conclude that
although the standard NJL model, with a fixed $G_{V}$, leads to stiffer EoS
(an larger NS masses) jaziel  this is accomplished by employing an EoS which
is in disagreement with pQCD predictions at asymptotically high densities.
This seminal work can be further extended in order to describe more realistic
situations. One possible such extension is to consider a diquark interaction
channel in order to explore the high-density region of QCD. As far as the
running of the vector coupling is concerned one could also try different
parameter values and or another ansatz to describe $G(\mu)$.

Acknowledgments: The author is partially supported by Conselho Nacional de
Desenvolvimento Científico e Tecnológico (CNPq), Grant No 307261/2021-2 and by
CAPES - Finance Code 001. This work has also been financed in part by
Instituto Nacional de Ciência e Tecnologia de Física Nuclear e Aplicações
(INCT-FNA), Process No. 464898/2014-5.

## References

  * (1) J. Antoniadis et al., Science  340, 6131 (2013), [arXiv:1304.6875].
  * (2) H. T. Cromartie et al. (NANOGrav), Nature Astron. 4, 72 (2019), [arXiv:1904.06759].
  * (3) E. Fonseca et al., Astrophys. J. Lett. 915, L12 (2021), [arXiv:2104.00880].
  * (4) G. Baym, C. Pethick, and P. Sutherland, Astrophys. J. 170, 299 (1971).
  * (5) K. Hebeler, J. M. Lattimer, C. J. Pethick, and A. Schwenk, Astrophys. J. 773, 11 (2013), [arXiv:1303.4662].
  * (6) S. Gandolfi, J. Lippuner, A. W. Steiner, I. Tews, X. Du and M. Al-Mamun, J. Phys. G 46, 103001 (2019), [arXiv:1903.06730].
  * (7) J. Keller, C. Wellenhofer, K. Hebeler, and A. Schwenk, Phys. Rev. C 103, 055806 (2021), [arXiv:2011.05855].
  * (8) B. A. Freedman and L. D. McLerran, Phys. Rev. D 16, 1169 (1977).
  * (9) A. Vuorinen, Phys. Rev. D 68, 054017 (2003), [arXiv:hep-ph/0305183].
  * (10) T. Kojo, APPS Bull. 31, 11 (2021), [arXiv:2011.10940].
  * (11) P. Bedaque and A. W. Steiner, Phys. Rev. Lett. 114, 031103 (2015), [arXiv:1408.5116].
  * (12) J. Alsing, H. O. Silva, and E. Berti, Mon. Not. Roy. Astron. Soc. 478, 1377 (2018), [arXiv:1709.07889].
  * (13) B. Margalit and B. D. Metzger, Astrophys. J. Lett. 850, L19 (2017), [arXiv:1710.05938].
  * (14) L. Rezzolla, E. R. Most, and L. R. Weih, Astrophys. J. Lett. 852, L25 (2018), [arXiv:1711.00314].
  * (15) M. Ruiz, S. L. Shapiro, and A. Tsokaros, Phys. Rev. D 97, 021501 (2018), [arXiv:1711.00473].
  * (16) M. Shibata, E. Zhou, K. Kiuchi, and S. Fujibayashi, Phys. Rev. D 100, 023015 (2019), [arXiv:1905.03656].
  * (17) A. Nathanail, E. R. Most and L. Rezzolla, Astrophys. J. Lett. 908, L28 (2021), [arXiv:2101.01735].
  * (18) S. Altiparmak, C. Ecker and L. Rezzolla, [arXiv:2203.14974].
  * (19) L. McLerran and S. Reddy, Phys. Rev. Lett. 122, no.12, 122701 (2019), [arXiv:1811.12503].
  * (20) K. S. Jeong, L. McLerran, and S. Sen, Phys. Rev. C 101, 035201 (2020), [arXiv:1908.04799].
  * (21) J. Margueron, H. Hansen, P. Proust, and G. Chanfray, Phys. Rev. C  104, 055803 (2021), [arXiv:2103.10209].
  * (22) T. Kojo, P. D. Powell, Y. Song, and G. Baym, Phys. Rev. D 91, 045003 (2015), [arXiv:1412.1108].
  * (23) G. Baym, S. Furusawa, T. Hatsuda, T. Kojo, and H. Togashi, Astrophys. J. 885, 42 (2019), [arXiv:1903.08963].
  * (24) G. Malfatti, M. G. Orsaria, I. F. Ranea-Sandoval, G. A. Contrera, and F. Weber, Phys. Rev. D 102, 063008 (2020), [arXiv:2008.06459].
  * (25) A. Ayriyan, D. Blaschke, A. G. Grunfeld, D. Alvarez-Castillo, H. Grigorian, and V. Abgaryan, Eur. Phys. J. A 57, 318 (2021), [arXiv:2102.13485].
  * (26) J. R. Stone, V. Dexheimer, P. A. M. Guichon, A. W. Thomas, and S. Typel, Mon. Not. Roy. Astron. Soc. 502, 3476 (2021), [arXiv:1906.11100].
  * (27) R. D. Pisarski, Phys. Rev. D 103, L071504 (2021), [arXiv:2101.05813].
  * (28) S. Pal, G. Kadam, and A. Bhattacharyya, [arXiv:2104.08531].
  * (29) T. Demircik, C. Ecker and M. Järvinen, [arXiv:2112.12157].
  * (30) N. Kovensky, A. Poole, and A. Schmitt, Phys. Rev. D 105, 034022 (2022), [arXiv:2111.03374].
  * (31) L. Bartolini, S. B. Gudnason, J. Leutgeb, and A. Rebhan, [arXiv:2202.12845].
  * (32) M. Marczenko, L. McLerran, K. Redlich and C. Sasaki, [arXiv:2207.13059].
  * (33) L. Brandes, W. Weise and N. Kaiser, [arXiv:2208.03026].
  * (34) Y. Fujimoto, K. Fukushima, L. D. McLerran and M. Praszalowicz, [arXiv:2207.06753].
  * (35) Y. Nambu and G. Jona-Lasinio, Phys. Rev. 122, 345 (1961); ibid. 124, 246 (1961).
  * (36) Y. Nambu, Phys. Rev. 106, 1366 (1957).
  * (37) J. D. Walecka, Ann. Phys. 83, 491 (1974).
  * (38) V. Koch, T.S. Biro, J. Kunz, and U. Mosel, Phys. Lett. B 185, 1 (1987).
  * (39) M. Buballa, Phys. Rept. 407, 205-376 (2005), [arXiv:hep-ph/0402234 ].
  * (40) K. Fukushima, Phys. Rev. D 77, 114028 (2008), [erratum: Phys. Rev. D 78, 039902 (2008)], [arXiv:0803.3318 ].
  * (41) K. Fukushima, Phys. Rev. D 78, 114019 (2008), [arXiv:0809.3080].
  * (42) T. E. Restrepo, J. C. Macias, M. B. Pinto and G. N. Ferrari, Phys. Rev. D 91, 065017 (2015), [arXiv:1412.3074].
  * (43) J. Sugano, J. Takahashi, M. Ishii, H. Kouno and M. Yahiro, Phys. Rev. D 90, 037901 (2014), [arXiv:1405.0103].
  * (44) T. Kunihiro, Phys. Lett. B 271, 395 (1991).
  * (45) L. Ferroni and V. Koch, Phys. Rev. C 83, 045205 (2011), [arXiv:1003.4428].
  * (46) H.-S. Zong and W.-M. Sun, Int. J. Mod. Phys. A 23, 3591 (2008).
  * (47) E. S. Fraga and P. Romatschke, Phys. Rev. D 71, 105014 (2005), [arXiv:hep-ph/0412298].
  * (48) Y. Fujimoto and K. Fukushima, Phys. Rev. D 105, 014025 (2022), [arXiv:2011.10891].
  * (49) J. L. Kneur, M. B. Pinto and T. E. Restrepo, Phys. Rev. D 100, 114006 (2019), [arXiv:1908.08363].
  * (50) L. Fernandez, J. L. Kneur, M. B. Pinto and T. E. Restrepo, in preparation.
  * (51) J. G. Coelho, C. H. Lenzi, M. Malheiro, R. M. Marinho, Jr., C. Providência and M. Fiolhais, Nucl. Phys. B Proc. Suppl. 199, 325 (2010), [arXiv:1001.1313].
  * (52) O. Komoltsev, [arXiv:2208.03086].