# CS-JEDI: Required DEI Education, by CS PhD Students, for CS PhD Students

Bailey Flanigan  Carnegie Mellon University ,  Ananya A Joshi  Carnegie Mellon
University ,  Sara McAllister  Carnegie Mellon University and  Catalina Vajiac
Carnegie Mellon University

(2023)

###### Abstract.

Computer science (CS) has historically struggled with issues related to
diversity, equity, and inclusion (DEI). Based on how these issues were
affecting PhD students in our department (the Carnegie Mellon University CS
Department), we identified required DEI education for PhD students as a
potentially high-impact approach to improving the PhD student experience in
our program. Given that no existing curriculum met the desired criteria, we
(PhD students) — alongside many members of the CMU community — developed and
implemented CS-JEDI: Justice, Equity, Diversity, and Inclusion in Computer
Science. CS-JEDI is a 6-week DEI curriculum that is now taken by all first-
year PhD students in our department. This paper covers CS-JEDI’s motivation
and goals; describes how its evidence-based curriculum is tailored to these
goals and to the CS PhD context; and gives a data-driven evaluation of the
extent to which CS-JEDI’s first offering, in Spring 2022, achieved these
goals.

††journalyear: 2023††copyright: rightsretained††conference: Proceedings of the
54th ACM Technical Symposium on Computing Science Education V. 1; March 15–18,
2023; Toronto, ON, Canada††booktitle: Proceedings of the 54th ACM Technical
Symposium on Computer Science Education V. 1 (SIGCSE 2023), March 15–18, 2023,
Toronto, ON, Canada††doi: 10.1145/3545945.3569733††isbn:
978-1-4503-9431-4/23/03

##  1\. Introduction

It is well-established that the field of computer science (CS) struggles with
pervasive issues related to diversity, equity, and inclusion (DEI). At all
levels, the field of CS persistently under-represents many groups including
women, black and indigenous people of color, queer people, and disabled people
(Zweben and Bizot, 2020; Aspray and Bernat, 2000; Varma, 2006; Holloman et
al., 2021; Rhodes et al., 2010). Underlying this lack of representation are
more complex issues — ranging from field-specific cultural issues to social
issues like systemic racism, sexism, and ableism — that make it difficult,
unwelcome, and sometimes even unsafe for people outside our field’s dominant
social groups to pursue a career in CS (McKinley, 2018; Evans et al., 2018;
Weisshaar, 2017; Hammer et al., 2020; Whitney and Taylor, 2018; Estrada et
al., 2016; Head et al., 2021; Van Veelen et al., 2019). On top of these
barriers, PhD students face high rates of mental health issues due to
academia’s “dark patterns” (Hammer et al., 2020), along with “stress about
productivity and self-doubt,…feeling devalued, [issues with] advisor
relationships,… difficulties with work-life balance, and feelings of isolation
and loneliness” (Satinsky et al., 2021). Despite these struggles, PhD students
often do not seek help (Satinsky et al., 2021). While many of these issues
affect students of all identities, we consider them DEI issues because they
place disproportionate burdens on community members from marginalized and
underrepresented groups (Hammer et al., 2020).

As PhD students in Carnegie Mellon’s department of CS (CSD), we experienced
and witnessed many such DEI issues. Based on our experiences, a survey of our
peers’ experiences, and existing research, we hypothesized that these issues
could be mitigated by an introductory DEI course for first-year PhD students.
In particular, we envisioned a course engineered to (a) be more comprehensive
and self-directed than a standard DEI training, (b) be well-received as a
required course for all students, and (c) connect core DEI topics to the CS
PhD context. While there exist many DEI education programs across industry and
academia, none satisfied these criteria: some are measurably effective but
designed to be optional (e.g., (Association, 2021; Univerity, [n. d.]; ebe,
[n. d.]; University, [n. d.]; of Washington, 2022)); others are required but
can be one-size-fits-all or focused on compliance, limiting their potential to
create lasting knowledge retention or cultural change (Bezrukova et al., 2016;
of Education, [n. d.]; Khalid and Snyder, 2021); and the few programs that are
both required and more in-depth are not tailored to the CS PhD experience
(e.g., (Wood, 2020; O’Donnell, 2020; Leibrock, 2021; Division, 2022)).

In light of this gap, we developed our own open-access111All materials
(referenced here in blue) can be downloaded from our course website.  DEI
curriculum, titled CS-JEDI: Justice, Equity, Diversity, and Inclusion in
Computer Science. CS-JEDI is a 6-week introductory DEI course that is, as of
Fall 2021, required for all new PhD students in our program. As desired, CS-
JEDI is for PhD students in that its content and structure are tailored to the
CS PhD experience. CS-JEDI is also by PhD students: it is designed to be
mainly PhD student-taught, and it was created primarily by a group of 15 CS
PhD students, who contributed their expertise, perspectives, and over 2500
person-hours of work. Beyond this core working group, the curriculum benefited
from multiple rounds of detailed input from the CMU Eberly Center for Teaching
and Learning, plus many students, staff, faculty, and other experts.

##  2\. Origin and Motivations of CS-JEDI

CS-JEDI originated in August 2020 when our mounting experiences with DEI
issues prompted us to informally survey other PhD students about their
experiences in CSD. The $\sim$40 anecdotes we heard from $\sim$25 students
were consistent with the trends documented in the literature: students
described interactions with other students — and sometimes faculty — involving
sexism, racism, xenophobia, homophobia, and harassment. Also in accordance
with the literature, students reported feeling that their experiences were not
understood by others, struggling to set boundaries and communicate needs in
advising relationships, feeling unable to intervene in troubling situations,
experiencing isolation and mental health issues, and lacking knowledge of
resources. Students often kept their struggles to themselves: they didn’t know
they could bring them up, lacked the lexicon to describe them, did not know
where to seek support, or worried they would be stereotyped or viewed as
overly-sensitive for voicing their concerns. These results suggested to us
that CSD could benefit from a course equipping students to (1) actively create
inclusive academic environments, (2) communicate openly about DEI issues, and
(3) practice self-care and self-advocacy within academia’s inherent power
structures.

#### Why a DEI focus?

Being trained primarily in CS and math, incoming CS PhD students may have
little prior education on DEI topics. However, in light of the issues
identified above, DEI competency has great potential to positively shape
students’ PhD experiences. Students can benefit from simply being aware of
struggles commonly faced in graduate school: if they encounter these
struggles, they can name them and know they are not alone. Further,
understanding fundamental topics like intersectionality, privilege, systemic
inequality, and stereotype threat can provide students with language to
articulate how these factors are influencing their own experiences, empowering
them to advocate for themselves and seek support. Such topics can also
increase students’ awareness of others’ — potentially very different —
experiences, engendering more empathy, inclusive behavior, and support between
peers. These benefits can extend to the broader CS community, too: as future
researchers, teachers, and managers, PhD students have great potential to
influence our field’s culture, making it a worthwhile investment to equip them
to exert this influence inclusively.

#### Why a required course?

From the perspective of an institution seeking to prepare students for
successful careers, DEI education is sensible: in order to secure grants and
jobs in academia and industry, applicants are increasingly being required to
demonstrate DEI knowledge and contributions to DEI efforts (for Teaching and
Learning, [n. d.]; NSF, 2022). More importantly, including all students can
have much greater community impact: discussion about DEI as a complete cohort
can help students find supportive connections they would not have otherwise
made. Further, it can build common language between all students on DEI topics
and foster a culture in which awareness, open discussion, and accountability
are norms. Creating cultural change requires involving all of a community’s
members (Yohn, 2021), but optional DEI courses are unlikely to have such
reach, as they are often only attended by those who are already knowledgeable
(Kulik et al., 2007). As a result, optional courses may not reach many
students who are simply unaware of the potential benefits — a hypothesis our
data will support. Moreover, this selection bias is part of a broader pattern
of marginalized groups disproportionately contributing labor toward addressing
DEI issues (Holloman et al., 2021) — a trend which is unlikely to be countered
with an optional course. In contrast, requiring this work of all students
helps to redistribute this labor and establishes through action that creating
an inclusive culture is everyone’s responsibility.

##  3\. CS-JEDI Course Goals

Now, we describe CS-JEDI’s goals in three parts: its key concepts (Sec. 3.1);
its learning objectives, which apply those concepts in day-to-day life (Sec.
3.2); and its high-level teaching goals (Sec. 3.3).

###  3.1. Key Concepts

Each week of CS-JEDI focuses on a different key DEI concept. To give students
a concrete entry point to these often vast concepts, we frame each one through
a core question, whose many possible answers touch on some of the concept’s
central aspects. For example, Week 2 is on sources of inequality, and its core
question is, How can inequality be perpetuated by a policy/criterion that
intends to be neutral to people’s identities? For each week, we define three
to four lenses through which the core question can be considered. Below, we
list each week’s key concepts, plus some of their lenses in parentheses. The
full list can be found in the Syllabus.

Week 1: Introduction and Identity; Week 2: Foundations of Inequality (implicit
bias, intersectionality, privilege, oppression); Week 3: Foundations of
Identity Safety (stereotype threat, identity safety, structural /
interpersonal strategies); Week 4: Intent versus Impact (microaggressions);
Week 5: Well-Being in the PhD Program (mental health, boundaries, self-
compassion); and Week 6: Allyship and promoting positive change.

###  3.2. Learning Objectives

CS-JEDI’s learning objectives describe skills that students can use to apply
the key concepts in their daily lives. These skills can also help address the
DEI issues identified in Sections 1 and 2.

  1. O1.

Discuss key concepts openly and inclusively with others.

  2. O2.

Name instances of key concepts in day-to-day scenarios.

  3. O3.

Advocate for and care for oneself using evidence-based practices and campus
resources.

  4. O4.

Create an intentionally inclusive space for others using evidence-based
practices.

  5. O5.

Apply evidence-based inclusive practices in standard academic contexts (e.g.,
teaching, admissions, hiring).

  6. O6.

Identify new JEDI topics of interest and self-educate on those topics from
multiple perspectives.

###  3.3. High-Level Course Goals

CS-JEDI was designed with the goal of being an enjoyable and valuable
experience for all students. Toward this aim, it pursues three specific,
evidence-based goals:

#### Goal 1: Create an identity-safe classroom.

Colloquially, identity safety occurs when students feel they can bring their
authentic selves to class. More concretely, CS-JEDI aims to treat students’
differences as assets rather than deficits, and to ensure that all students
can find their experiences reflected in the material—two features of an
inclusive class environment according to the Culturally Responsive Teaching
(CRT) framework (America, 2019). Since peer interaction is a core aspect of
CS-JEDI, meeting this goal means that students’ differences are treated as
assets by both instructors and other students.

#### Goal 2: Build community among students.

As discussed in Section 2, PhD students often suffer from isolation and a lack
of support. CS-JEDI, being taken by all first-years, offers an opportunity to
help students form a supportive community early in graduate school.

#### Goal 3: Reduce the sense of forced participation.

One known pitfall of required DEI education is that it can evoke, for some
participants, a sense of forced participation (Williams and Thomas, 2020).
This sense can cause disengagement, create or strengthen anti-DEI convictions
(Williams and Thomas, 2020), and negatively impact the learning environment.

##  4\. CS-JEDI Implementation (Spring 2022)

This section overviews CS-JEDI’s main components (Sec. 4.1) and then details
key features of its design (Sec. 4.2 and 4.3). In particular, we describe the
Spring 2022 version of the curriculum, which was taught over Zoom (future
offerings will be in person). This version is the product of two years of
iterative testing and revision: two previous versions were pilot-tested (once
as a requirement), and the curriculum was fully revamped three times based on
feedback from students, staff, faculty, and the Eberly Center.

###  4.1. Roadmap of Course Components

CS-JEDI consists of six, consistently-structured weeks. Fig. 1 gives a roadmap
of CS-JEDI’s weekly curriculum components. CS-JEDI takes ¡3 hours per week, or
18 hours total, of student time.

Fig. 1. CS-JEDI’s weekly schedule. Gray and blue indicate out-of-class and in-
class time, respectively.

#### Before class.

Students spend 1 hour preparing for class each week through a pre-class
activity. As a precursor to this activity, all students are assigned one of
the current week’s lenses. The activity requires each student to research
their lens using at least two sources (we provide 10-20 sources per lens;
students can also introduce their own) and document anything — evidence,
opinions, questions — that they think would be helpful to share with peers.

#### In class.

Students spend 80 minutes per week in class. Class consists of a 5-minute
mindfulness activity, a 30-minute guest lecture by a domain expert, and then
45 minutes of peer discussion. Peer discussion has two parts: first, students
share ideas among others with the same lens in lens discussions. Then, in
synthesis discussions, students meet in groups of 4-5, with at least one
student per lens, to discuss how their respective lenses inform the core
question.

#### After class.

Students spend 30 minutes on a post-class reflection, which is a free-form
written reflection on that week’s material.

###  4.2. The key: tailoring to the CS PhD context

CS-JEDI is specifically tailored to CS PhD students — a feature enabled by the
fact that students were its primary architects. This tailoring allows CS-JEDI
to leverage students’ common identity and experience as members of a CS PhD
program to increase the material’s relevance and accessibility to all students
at once.

This tailoring comes in two main forms, the first of which is via CS-JEDI’s
content. In the opening lecture, the course is motivated with examples of how
DEI issues affect members of our department and field and how DEI competency
can improve the PhD experience (see Sec. 2). Similarly, pre-class activities
contain many sources connecting key DEI concepts to CS, academia, and the PhD
experience. For example, these sources describe inclusive research lab
cultures (Hammer et al., 2020), inclusive teaching in mathematical contexts
(e.g., (Eberly Center, [n. d.]; Su, 2015; Nordell, 2022)), personal accounts
of DEI issues in CS (e.g., (McKinley, 2018)), connections between CS culture
and inequities in our field (e.g., (Ko, 2021)), and an instructor-curated list
of student resources on campus. When choosing guest-lecturers, instructors
prioritize domain experts from CS-adjacent backgrounds, while also
prioritizing those from underrepresented groups. All scenarios or thought
questions posed to students are similarly specific to the CS context. Students
can also connect a DEI topic to their CS research, or apply CS techniques to
learn more about DEI topics, via an optional course project.

We also tailor the course by adopting structures and content framings that
anticipate specific features and challenges of DEI education in the CS PhD
context. We motivate and describe these structures and framings throughout the
next subsection.

###  4.3. Evidence-Based Design Approaches

Now, we describe the core evidence-based approaches that CS-JEDI employs
toward achieving the goals in Section 3. Many of these approaches follow one
of two frameworks: Universal Design for Learning (UDL) (CAST, 2018), which
focuses on lowering barriers to learning by giving students choice, and
Culturally Responsive Teaching (CRT) (America, 2019), which recognizes and
responds to students’ cultural backgrounds, experiences, and identities.

#### Preparing students for inclusive participation.

Week 1’s class is dedicated to preparing all students — regardless of their
level of experience discussing sensitive topics — to help create an identity-
safe class environment. In week 1, students review the discussion guide, which
provides clear community guidelines, tips for phrasing responses inclusively,
and class procedures for accountability and self-advocacy. They then practice
applying these tools in small groups using provided scenarios. To increase
students’ awareness of identity-based assumptions they might be making, week 1
also covers identity and intersectionality, and includes an “identity iceberg”
activity for exploring how identities can be invisible.

#### Building instructor and peer support into discussion.

Despite this preparation, peer discussion still poses risks to students’
identity safety. In the CS PhD context, we anticipated this risk to be
particularly heightened for students from groups that have been historically
marginalized in the US, as they are likely to be underrepresented in the
course (Zweben and Bizot, 2020). These students could feel pressured to
educate others, or invalidated among peers who share few of their identities
or experiences. To lessen risks of peer discussion, instructors — trained in
inclusive teaching practices and facilitating difficult dialogues — can serve
as moderators. Lens groups always have a moderating instructor. Synthesis
groups are too numerous to be constantly moderated, but instructors often
check in on groups and are available to moderate if needed or asked. Because
synthesis groups are not always moderated, we build additional, peer support
into these groups: first, these groups are fixed, allowing students to build
comfort with their group over time. They are also chosen based on a pre-class
survey, which ensures that students are always placed with someone they have
identified as supportive to them, and are not placed with anyone with whom
they are uncomfortable.

#### Incorporating cooperative learning via the Jigsaw Method.

The Jigsaw Method is a cooperative learning method that, in some contexts, has
been found to support learning and reduce prejudice (Walker and Crogan, 1998;
Aronson, 2002). In CS-JEDI’s implementation of this method, students
individually study their lens and then share what they learned with others,
both within and across lenses. In addition to giving students a chance to
contribute to their group’s collective knowledge, this approach exposes
students to all lens topics in a short time.

#### Supporting students of all knowledge levels.

Students enter CS-JEDI with wide-ranging knowledge levels on all course topics
(see Sec. 5). To support students when they are new to a topic, we provide a
course glossary, which defines the key concepts and several adjacent terms.
Additionally, pre-class activities contain clearly-marked introductory sources
on every lens. In framing the content, instructors take care to normalize the
difficulty and importance of the subjective questions and qualitative evidence
(e.g., firsthand accounts) that are central to studying DEI topics. This may
be particularly important for CS PhD students, who are likely more accustomed
to quantitative evidence and questions with provable solutions. Finally, to
support students with more incoming knowledge, pre-class activities contain
many nuanced sources, and students can request reading recommendations from
instructors based on their interests. Students can also deepen their study by
finding their own sources, completing an optional instructor-advised course
project, or checking out a book from the class library.

#### Creating space for all perspectives and opinions.

As instructors strongly emphasize, the purpose of CS-JEDI is not to prescribe
specific perspectives or opinions; rather, its goal is to help students
develop their own opinions, based on diverse perspectives and forms of
evidence. An essential feature of CS-JEDI is that it is student-taught: this
reduces power dynamics between students and instructors, creating space for
more open questioning. CS-JEDI is also built around peer discussion, which
provides a space for students to — outside the presence of instructors — pose
their own questions to peers and engage in critical discussion about those
questions. Recognizing that any evidence we provide will unavoidably represent
limited perspectives, pre-class activities offer numerous and diverse sources
per lens and invite students to introduce their own sources. To create space
for perspectives from beyond the US context, students can use sources in any
language.

#### Offering diverse modes of learning.

A core tool of the two previous approaches was giving students choice. CS-JEDI
additionally uses this tool in implementing three tenets of the UDL framework:
giving choice over representation of information, engagement, and action and
expression (CAST, 2018). For choice in representation, the sources in pre-
class activities include academic research, first-person narratives, and self-
reflection tools (e.g., self-surveys). These sources also come in diverse
formats (e.g., text, audio, video) to suit different learning styles. For
choice in engagement, students can interact with guest lecturers and engage in
various ways during peer discussion. For choice in action and expression,
students can submit notes, diagrams, or any other representation of what they
learned in pre-class activities. Post-class reflections give students space to
practice self-reflection, self-education, and creative expression.

#### Minimizing mental load on students.

Time cost and mental load are particular concerns in the PhD context, where
students are frequently overworked and strongly incentivized to focus solely
on research (Hammer et al., 2020). To limit CS-JEDI’s mental load on students,
the course is short, has consistent structure week-to-week, and phrases
instructions as simply and consistently as possible. It is graded pass-fail on
the basis of effort, and extensions are given freely upon request. To reduce
the effort required to engage in class, we begin with a mindfulness exercise,
shown to promote engagement (Miralles-Armenteros et al., 2019).

#### Emphasizing empathy, growth, and putting in the work.

At the core of CS-JEDI’s content is its emphasis on (1) understanding that
others may experience the same aspects of our community differently, (2)
engaging with the impacts of one’s actions and learning from mistakes (rather
than focusing on blame (Chavez and Weisinger, 2008)), and (3) taking
responsibility for contributing to inclusion in our community. CS-JEDI not
only emphasizes these messages, but tries to equip students to enact them: it
exposes students to a wide range of others’ experiences in our field, teaches
intent versus impact, offers students a plethora of sources with which to
self-educate in the future, and provides an instructor-curated list of ongoing
campus DEI efforts that students can join. It is important to note that these
messages push against cultural norms of our field (and academia more broadly),
which emphasizes individual merit and achievement, and in which empathy and
service are not always rewarded (Hammer et al., 2020). The student-taught
nature of CS-JEDI is an important asset in overcoming these cultural forces,
because it can increase the impact of instructors teaching by example. To
model these messages, instructors open class by briefly describing their own
processes of putting in the work to learn, unlearn, and advocate for DEI. They
also openly invite students to hold them accountable through many low-barrier
feedback channels, such as post-class reflections, an anonymous feedback
portal, and a DEI-trained staff contact.

##  5\. Evaluation Results (Spring 2022)

#### Evaluation Methods.

We evaluated the Spring 2022 curriculum with an anonymous course evaluation,
which was designed in collaboration with the Eberly Center.222This study was
approved by the Institutional Review Board (IRB). Students had 15 minutes of
class time to complete it (but could stay late). The evaluation asked 15
questions, most asking students to rate their agreement with statements
according to a five-point Likert Scale:

#### Demographics.

30 out of 34 enrolled students submitted the evaluation. We removed two
students’ responses due to concerns about their reliability (these students
reported having lost the maximum possible amount of knowledge throughout the
course).333We will, however, consider their short answer responses in our
analysis, as they conveyed senses of forced participation. Thus, in the
reported data, $n=28$ (or occasionally $n=27$, denoted as †). Per the last
question of the survey, 16 students self-identified as international students
and 11 self-identified as belonging to a marginalized group (2 declined to
answer). We note the possibility that these are underestimates. When most
relevant, we will report statistics for these subgroups in addition to
aggregate results.

#### Evaluation of course components.

Before presenting our main results, we summarize students’ feedback on CS-
JEDI’s components. We call a student positive toward a component if they
agreed or strongly agreed that it supported their learning. On average,
students were solidly positive toward pre-class activities, guest lectures,
synthesis group discussions, and the student-taught nature of the course. They
were neutral on post-class activities, and slightly below neutral on lens
discussions. Most students found the provided instructions clear and felt that
the out-of-class work took a reasonable amount of time.

###  5.1. Main Results

First, CS-JEDI substantially increased students’ comprehension of all key
concepts (Sec. 3.1). This is shown in Figure 2, which compares the
distributions of students over three increasing levels of (self-evaluated)
comprehension from before to after the course. Despite students’ heterogeneous
incoming knowledge, all students reached at least Bin 2 — and 88% - 92%
reached Bin 3, the highest level of comprehension — on all topics.

We also find that CS-JEDI helped students develop skills for applying this
knowledge. As shown in Figure 3, 52% - 72% of students said they were more
equipped, and 48% - 78% were more likely, to utilize the skills specified in
learning objectives O1 - O6 (Sec. 3.2).

Fig. 2. Key concept comprehension pre- (lighter bars) & post-CS-JEDI.

Finally, we evaluate CS-JEDI’s achievement of its high level course goals
(Sec. 3.3) through several other questions, mostly summarized in Table 1. Note
that these questions also directly evaluate many of the specific approaches in
Sec. 4.3. Our findings here will show encouraging points of success, but also
reveal opportunities for improvement, which we will discuss further in Section
6.

Goal 1: Fostering identity safety is most directly addressed by Question (a),
which asked students if they felt they could bring their authentic selves to
class. Responses to (a) suggest that a majority of students felt welcome — and
a strong majority did not feel actively unwelcome — in class. These results
are supported by students’ majority positive/neutral responses to questions
(b) - (g), which evaluate specific aspects of identity safety. Importantly,
despite facing potentially heightened risks, marginalized students actually
responded far more positively than average to these questions. International
students, by contrast, often responded less positively.

Fig. 3. Improvement on learning objectives (O1-O6). Students felt that they… |  all students  ($n=28$) |  marginalized ($n=11$) |  international  ($n=16$)
---|---|---|---
| Agree+ | Neutral+ | Agree+ | Neutral+ | Agree+ | Neutral+
(a) could bring their authentic selves to class | 54% | 79% | 64% | 82% | 38% | 63%
(b) could to alert instructors to concerns † | 59% | 93% | 73% | 91% | 56% | 88%
(c) had the opportunity to voice opinions and questions † | 67% | 89% | 91% | 91% | 50% | 81%
(d) could explore topics relevant to their experiences † | 75% | 89% | 91% | 100% | 75% | 88%
(e) could tailor their learning to the right difficulty | 57% | 79% | 82% | 100% | 50% | 75%
(f) could explore different perspectives / opinions † | 74% | 89% | 82% | 100% | 69% | 88%
(g) could engage with material suited to how they learn | 68% | 82% | 91% | 100% | 50% | 75%
(h) would have taken a course on DEI if not required* | 32% | 32% | 64% | 64% | 25% | 25%
(i) are glad they took it † | 63% | 85% | 81% | 91% | 50% | 81%
Table 1. Percentage of students that responded positively to questions related
to the pedagogical approaches. Agree+ and Neutral+ correspond to responses
$\geq 4$ and $\geq 3$ respectively, per the Likert scale. * denotes a question
that did not have a neutral option, and † indicates $n=27$.

We find that CS-JEDI did promote Goal 2: build community among students. In
response to a set of questions distinct from those in Table 1, 63% of students
reported that after the course, they had more of a common language with their
peers on DEI topics; 48% said they formed at least one supportive connection
with another student; 33% were more likely to ask peers for support; and 48%
of students also felt that they or others would feel more welcome in our field
if more members of the CS community took this course.

Although two students expressed sentiments of forced participation in free
responses (see footnote 3), the majority of responses to all survey questions
were neutral or positive, suggesting that most students were receptive to
taking CS-JEDI. We evaluate Goal 3: Reducing the sense of forced participation
more directly with questions (h) and (i), which together test whether students
found the course valuable even if they wouldn’t have opted into it. We find
that the answer is fairly strongly affirmative: even though only 32% of
students would have taken the course had it not been required, 63% are glad
they did. We see that marginalized students were significantly more likely to
take a DEI course even if not required (64%), and a higher fraction were glad
they did (82%). International students were much less likely to take the
course by choice (25%), but still, 50% were glad they took it. Underlying this
50% was substantial heterogeneity, with 19% answering “5” and 19% answering
“1”.

##  6\. Discussion

Given the potential pitfalls of required DEI education and the challenges of
our context, it was unclear to us at the outset whether a course like CS-JEDI
could be successful. We tried to circumvent these pitfalls and challenges
through careful course design: we tailored CS-JEDI specifically to our
audience, made it peer-taught, and pursued skill-based learning objectives
that could be directly applied in students’ daily lives. We also involved many
students, faculty, and staff in the development process, which helped the
course anticipate many context-dependent challenges, and created community
investment that has propelled the course toward its establishment in our
program’s curriculum.

The results presented in this paper — and our broader experience developing,
piloting, and teaching CS-JEDI — suggest to us that it is possible to design
required DEI education programs whose positives outweigh the negatives. CS-
JEDI fulfilled many of the promises that motivated us to pursue a required
course in the first place: for instance, it not only reached, but benefited,
many students who were otherwise unlikely to take a DEI-related course during
graduate school. These results are consistent with the possibility that
required DEI education reaches students who would appreciate it, but are
unaware of the potential benefits. Moreover, our finding that marginalized
students would have opted into a course like CS-JEDI at far higher rates
suggests that had CS-JEDI not been required, it could have deepened existing
trends of marginalized academic community members undertaking greater service
burdens.

Of course, we see these results as only the beginning: the Spring 2022 version
we study here was only the first of indefinite annual offerings of CS-JEDI.
Future iterations will be taught in person, offering many opportunities for
new innovations. As we will discuss, the curriculum is designed to respond
with agility to student feedback so that it can continue to improve over time.

#### Future improvements to the CS-JEDI curriculum.

Our results already reveal many opportunities for improving CS-JEDI. At the
top of this list, we want the course to do more to strengthen student
community, give all students the option to learn at the right level of
difficulty (Table 1, (f)), and make sure there is space for all students to
bring their authentic selves to class (Table 1, (a)). One particular community
we can better serve in these ways, based on our survey results, is
international students, who were overall less positive than average in
response to the questions in Table 1. We proceed with the recognition that
“international students” are an extremely heterogeneous group, as is reflected
in our results.

Based on student feedback from throughout the class, we now distill some
factors that likely contributed to the negative responses we saw. The material
in week 2 is especially difficult, and may have been overwhelming; many course
aspects primarily engage with DEI issues in the US context, potentially
reducing accessibility or identity safety for students from non-US
backgrounds; and students from different cultures may differ in their comfort
levels with sharing or submitting their own opinions on sensitive topics.

In response, we identify three planned changes to the curriculum. First, we
plan to actively involve students in establishing course motivation, e.g, by
beginning class by anonymously crowd-sourcing students’ ideas about what
marginalization can look like in CS and how greater equity can be achieved.
Second, we will front-load more accessible material. To do so, we will re-
order the key concepts to first cover well-being in the PhD program (week 5)
and identity safety (week 3) before broaching the more challenging (and
sometimes more US-centric) topics of inequality, intent versus impact, and
allyship (weeks 2, 4, and 6). Third, we will try to alleviate pressure on
students to share or document their own perspectives and opinions. We will do
so by more explicitly encouraging students to share and submit information in
any form — e.g., quotes from sources, bullet-point notes, or questions.
Further, we will provide synthesis groups with a weekly scenario-based thought
question. This can help facilitate discussions centered around concrete
experiences, hopefully lessening pressure on students to share their own
opinions as a means of generating discussion.

#### Sustainable and equitable implementation.

Surrounding CS-JEDI’s curriculum are the institutional processes and resources
by which it is sustained over time. These processes were designed with two
goals in mind: the curriculum should evolve in response to feedback while
remaining high-quality and open-source, and its student instructors should be
sufficiently supported, prepared to teach, and compensated. The latter goal is
extremely important, given that the PhD student instructors will have high
turnover, potentially limited experience teaching, and may face complex power
dynamics associated with teaching their peers.

Our sustainable implementation plan describes how we approach these goals —
how materials are documented and archived over time, and how instructors are
selected, compensated, trained, and supported. Additionally, our teaching
manual guides instructors through the process of preparing and teaching the
course with detailed instructions. Finally, the curriculum’s modular structure
permits switching out individual readings, lenses, core questions, and entire
key concepts with only self-contained changes to the materials, facilitating
low-effort changes in response to feedback.

#### Adaptability.

This sustainability-promoting infrastructure also makes CS-JEDI’s curriculum
easily adaptable to new contexts, even outside CS / academia. Included in our
implementation plan are strategies for implementing CS-JEDI while protecting
community members with less institutional power, which are informed by our
experiences implementing the course in our own department.

#### Conclusions.

Although, we (the authors) led various parts of the CS-JEDI initiative, none
of us entered our PhD program planning to become so deeply involved in DEI
efforts. Like many others who engage in DEI work, we created this course out
of frustration and concern about the inequities we saw consistently denying
students (including us) an equal opportunity to succeed in our field. We did
so despite strong incentives not to challenge the status quo.

The fact that so many CS PhD students undertook so much unpaid labor and risk
speaks to the urgency of the inequities in CS. These inequities, while
pervasive and constantly affecting some, may be invisible to others, even
within the same department.444For example, many faculty in CSD were surprised
by the negative experiences students reported in the survey described in
Section 2. This is an argument for educating everyone in our field — students,
staff, faculty, and more — so that the work required to make our community
inclusive can truly be shared among all its members.

Acknowledgements. CS-JEDI would not have been possible without the working
group of PhD students who offered their time, careful thought, and lived
experiences toward the creation of this curriculum: Abhinav Adduri, Alex Wang,
Anson Kahng, Josh Williams, Judeth Oden Choi, Pallavi Koppol, Paul Gölz,
Samantha Reig, Tabitha E. Lee, Valerie Chen, and Ziv Scully. We are also
profoundly thankful for the instrumental curriculum development support
provided by Dr. Alexis Adams of the Eberly Center. We additionally thank Dr.
Chad Hershock, Dr. Michael Melville, Shernell Smith, and Dr. Joanna Dickert
for their consultation. We thank Professor Zico Kolter for assuming the role
of faculty instructor, Deb Cavlovich for constant administrative support, the
CSD DEI, SCS DEI, and SCS CS-JEDI steering committees for their support
throughout this effort. Innumerable additional faculty, staff, and students
from the CMU community also provided invaluable support for CS-JEDI through
proofreading, consulting, advocacy, and more, for which we thank them
earnestly.

Bailey Flanigan is supported by a Hertz Foundation fellowship and NSF GRFP,
Ananya A Joshi and Catalina Vajiac by NSF GRFP, and Sara McAllister by an
NDSEG fellowship.

## References

  * (1)
  * ebe ([n. d.]) [n. d.].  Course 99263 - Centering Diversity, Equity, and Inclusion in Course Design & Teaching.  http://coursecatalog.web.cmu.edu/intercollegeprograms/courses/courses.pdf.
  * America (2019) New America. 2019\.  Understanding Culturally Responsive Teaching.  https://www.newamerica.org/education-policy/reports/culturally-responsive-teaching/understanding-culturally-responsive-teaching/.
  * Aronson (2002) Elliot Aronson. 2002\.  Building empathy, compassion, and achievement in the jigsaw classroom.  In _Improving academic achievement_. Elsevier, 209–225.
  * Aspray and Bernat (2000) William Aspray and Andrew Bernat. 2000.  Recruitment and retention of underrepresented minority graduate students in computer science. In _Report on a Workshop by the Coalition to Diversity Computing_.
  * Association (2021) The Computer Architecture Student Association. 2021.  Advancing and promoting dei in computer architecture –– summer 2021 reading group.  https://www.sigarch.org/advancing-and-promoting-dei-in-computer-architecture-summer-2021-reading-group/.
  * Bezrukova et al. (2016) Katerina Bezrukova, Chester S. Spell, Jamie L. Perry, and Karen A. Jehn. 2016.  A meta-analytical integration of over 40 years of research on Diversity Training Evaluation.  _Psychological Bulletin_ 142, 11 (2016), 1227–1274.  https://doi.org/10.1037/bul0000067
  * CAST (2018) CAST. 2018.  The UDL Guidelines.  http://udlguidelines.cast.org.
  * Chavez and Weisinger (2008) Carolyn Chavez and Judith Weisinger. 2008.  Beyond Diversity Training: A Social Infusion for Cultural Inclusion.  _Human Resource Management_ 47 (05 2008), 331 – 350.  https://doi.org/10.1002/hrm.20215
  * Division (2022) UCSF Graduate Division. 2022\.  GRAD 202: Racism in Science.  https://graduate.ucsf.edu/grad-202-racism-in-science.
  * Eberly Center ([n. d.]) Carnegie Mellon University Eberly Center. [n. d.].  Teaching across cultures.  https://www.cmu.edu/teaching/designteach/teach/teaching-across-cultures.html.
  * Estrada et al. (2016) Estrada et al. 2016\.  Improving underrepresented minority student persistence in STEM.  _CBE—Life Sciences Education_ 15, 3 (2016), es5.
  * Evans et al. (2018) Teresa M Evans et al. 2018\.  Evidence for a mental health crisis in graduate education.  _Nature biotechnology_ 36, 3 (2018), 282–284.
  * for Teaching and Learning ([n. d.]) UT Austin Center for Teaching and Learning. [n. d.].  Drafting a Diversity, Equity, and Inclusion (DEI) Statement.  https://ctl.utexas.edu/drafting-diversity-equity-and-inclusion-dei-statement.
  * Hammer et al. (2020) Jessica Hammer, Alexandra To, and Erica Principe Cruz. 2020\.  Lab counterculture. In _Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems_. 1–14.
  * Head et al. (2021) Erika Head, Emma Anderson, Sepi Moghadam, Elise Dorough, Leslie Sessoms, and Christopher Hovey. 2021\.  Improving Diversity, Equity, and Inclusion in Doctoral Computing Education. 764–765.  https://doi.org/10.1145/3408877.3432577
  * Holloman et al. (2021) T Holloman, JS London, Walter C Lee, et al. 2021\.  Underrepresented and overlooked: insights from a systematic literature review about black graduate students in engineering and computer science.  _IJEE International Journal of Engineering Education_ 37, 2 (2021).
  * Khalid and Snyder (2021) Amna Khalid and Jeffrey Aaron Snyder. 2021.  Don’t Mistake Training for Education.  https://www.insidehighered.com/views/2021/04/29/colleges-should-focus-education-more-training-about-dei-issues-opinion.
  * Ko (2021) Amy Ko. 2021.  Deconstructing CS Culture.  https://faculty.washington.edu/ajko/slides/RESPECT2021DeconstructingCSCulture.pdf.
  * Kulik et al. (2007) Carol T. Kulik, Molly B. Pepper, Loriann Roberson, and Sharon K. Parker. 2007.  The Rich Get Richer: Predicting participation in voluntary diversity training.  _Journal of Organizational Behavior_ 28, 6 (2007), 753–769.  https://doi.org/10.1002/job.444
  * Leibrock (2021) Keegan Leibrock. 2021\.  Rice working to launch diversity and inclusion course for first-year students.  https://www.ricethresher.org/article/2021/09/rice-working-to-launch-diversity-and-inclusion-course-for-first-year-students.
  * McKinley (2018) Kathryn S. McKinley. 2018\.  What happens to US does not happen to most of you.  https://www.sigarch.org/what-happens-to-us-does-not-happen-to-most-of-you/.
  * Miralles-Armenteros et al. (2019) Sandra Miralles-Armenteros, Ricardo Chiva, Alma Rodríguez-Sánchez, and Zina Barghouti. 2019.  Mindfulness and academic performance: The role of compassion and engagement.  _Innovations in Education & Teaching International_ 58 (10 2019), 1–11.  https://doi.org/10.1080/14703297.2019.1676284
  * Nordell (2022) Jessica Nordell. 2022\.  Math is personal.  https://www.theatlantic.com/education/archive/2021/09/bias-math-sexism-racism/620207/.
  * NSF (2022) NSF. 2022.  NSF 101: Five tips for your Broader Impacts statement.  https://beta.nsf.gov/science-matters/nsf-101-five-tips-your-broader-impacts-statement.
  * O’Donnell (2020) Mary Rose O’Donnell. 2020\.  Pitt offers new mandatory anti-racism class for first year students.  https://pittnews.com/article/159354/news/pitt-offers-new-mandatory-anti-racism-class-for-first-year-students/.
  * of Education ([n. d.]) U.S. Department of Education. [n. d.].  Advancing Diversity and Inclusion in Higher Education.  https://www2.ed.gov/rschstat/research/pubs/advancing-diversity-inclusion.pdf.
  * of Washington (2022) University of Washington. 2022\.  Equitable and inclusive computer science pedagogy.  https://courses.cs.washington.edu/courses/cse492t/.
  * Rhodes et al. (2010) Jo Rhodes, Nicole Buzzetto-Hollywood, Ojiabo Ukoha, and Narendra Rustagi. 2010.  Unlocking the Barriers to Women and Minorities in Computer Science and Information Systems Studies: Results from a Multi-Methodolical Study Conducted at Two Minority Serving Institutions.  _Journal of Information Technology Education:Research_ 9 (01 2010), 115–131.  https://doi.org/10.28945/1167
  * Satinsky et al. (2021) Emily N Satinsky, Tomoki Kimura, Mathew V Kiang, Rediet Abebe, Scott Cunningham, Hedwig Lee, Xiaofei Lin, Cindy H Liu, Igor Rudan, Srijan Sen, et al. 2021\.  Systematic review and meta-analysis of depression, anxiety, and suicidal ideation among Ph.D. students.  _Scientific Reports_ 11, 1 (2021), 1–12.
  * Su (2015) Francis Edward Su. 2015\.  Mathematical Microaggressions.  _MAA FOCUS_ (2015), 36–37.
  * Univerity ([n. d.]) Stanford Univerity. [n. d.].  TA Training: Workshops & Courses.  https://ctl.stanford.edu/ta-training/workshops-and-courses.
  * University ([n. d.]) Carnegie Mellon University. [n. d.].  Biasbusters.  https://www.cmu.edu/scs/scs4all/programs/biasbusters.html.
  * Van Veelen et al. (2019) Ruth Van Veelen, Belle Derks, and Maaike Dorine Endedijk. 2019\.  Double trouble: How being outnumbered and negatively stereotyped threatens career outcomes of women in STEM.  _Frontiers in psychology_ 10 (2019), 150.
  * Varma (2006) Roli Varma. 2006\.  Making computer science minority-friendly.  _Commun. ACM_ 49, 2 (2006), 129–134.
  * Walker and Crogan (1998) Iain Walker and Mary Crogan. 1998.  Academic performance, prejudice, and the jigsaw classroom: new pieces to the puzzle.  _Journal of Community and Applied Social Psychology_ 8 (1998), 381–393.
  * Weisshaar (2017) Katherine Weisshaar. 2017\.  Publish and perish? An assessment of gender gaps in promotion to tenure in academia.  _Social Forces_ 96, 2 (2017), 529–560.
  * Whitney and Taylor (2018) Telle Whitney and Valerie Taylor. 2018.  Increasing women and underrepresented minorities in computing: The landscape and what you can do.  _Computer_ 51, 10 (2018), 24–31.
  * Williams and Thomas (2020) Joan C. Williams and David A. Thomas. 2020.  Why diversity programs fail.  https://hbr.org/2016/07/why-diversity-programs-fail.
  * Wood (2020) Sarah Wood. 2020\.  Pitt introduces mandatory anti-racism course for freshmen.  https://www.diverseeducation.com/news-roundup/article/15107611/pitt-introduces-mandatory-anti-racism-course-for-freshmen.
  * Yohn (2021) Denise Lee Yohn. 2021\.  Company Culture Is Everyone’s Responsibility.  https://hbr.org/2021/02/company-culture-is-everyones-responsibility.
  * Zweben and Bizot (2020) Stuart Zweben and Betsy Bizot. 2020.  2020 Taulbee Survey.  https://cra.org/wp-content/uploads/2021/05/2020-CRA-Taulbee-Survey.pdf.