# Generative AI in Computing Education: Perspectives of Students and
Instructors

Cynthia Zastudil  Computer & Information Sciences
Temple University
Philadelphia, PA
<EMAIL_ADDRESS>Magdalena Rogalska  Computer & Information
Sciences
Temple University
Philadelphia, PA
<EMAIL_ADDRESS>Christine Kapp  Computer & Information Sciences
Temple University
Philadelphia, PA
<EMAIL_ADDRESS>Jennifer Vaughn  Computer & Information Sciences
Temple University
Philadelphia, PA
<EMAIL_ADDRESS>Stephen MacNeil  Computer & Information Sciences
Temple University
Philadelphia, PA
<EMAIL_ADDRESS>
###### Abstract

Generative models are now capable of producing natural language text that is,
in some cases, comparable in quality to the text produced by people. In the
computing education context, these models are being used to generate code,
code explanations, and programming exercises. The rapid adoption of these
models has prompted multiple position papers and workshops which discuss the
implications of these models for computing education, both positive and
negative. This paper presents results from a series of semi-structured
interviews with 12 students and 6 instructors about their awareness,
experiences, and preferences regarding the use of tools powered by generative
AI in computing classrooms. The results suggest that Generative AI (GAI) tools
will play an increasingly significant role in computing education. However,
students and instructors also raised numerous concerns about how these models
should be integrated to best support the needs and learning goals of students.
We also identified interesting tensions and alignments that emerged between
how instructors and students prefer to engage with these models. We discuss
these results and provide recommendations related to curriculum development,
assessment methods, and pedagogical practice. As GAI tools become increasingly
prevalent, it’s important to understand educational stakeholders’ preferences
and values to ensure that these tools can be used for good and that potential
harms can be mitigated.

###### Index Terms:

Generative models, large language models, computing education, student
perceptions, instructor perceptions

##  I Introduction

The introduction of Generative AI (GAI) models has led to significant
excitement in the computing education community [1, 2, 3, 4, 5, 6, 7, 8, 9,
10]. These generative models are typically powered by large language models
such as GPT-3 or Codex which are capable of producing code explanations [6, 9,
5] that have been rated as being better than peer-generated explanations [9].
In addition, these models can solve basic programming assignments [1, 7] and
perform slightly worse than students on quizzes with multiple-choice questions
[8]. Understandably, these substantial and rapid advances in the performance
of generative models are causing excitement and consternation among students
and instructors. A recent birds of a feather discussion at the SIGCSE
conference [11] highlighted a few emerging concerns that educators have
related to over-reliance, plagiarism, and other forms of misuse. These
concerns have been echoed by position papers [7, 12], a working group [13],
and an investigation into instructor perspectives [14]. However, these
concerns only scratch the surface, there is an urgent need to understand
students’ and instructors’ preferences about and what they want from these GAI
tools so that future development and regulation can align with the values of
these critical stakeholders.

In this paper, we conduct an interview study with 18 participants including
both students (N=12) and instructors (N=6). The goal of this interview study
is to better understand these two key stakeholders’ awareness of these models
and their preferences about how these models should be used in computing
education classrooms. We are particularly interested in what ways students and
instructors envision the involvement of generative AI in computing education,
both positively and negatively.

In this paper, we investigate three research questions:

  * RQ1: What preferences do instructors and students have regarding how Generative AI should be used in computing education classes?

  * RQ2: What concerns do instructors and students have regarding the use of Generative AI in computing education?

  * RQ3: What implications might Generative AI have on computing education curricula, pedagogy, and assessment methods?

The results from our study suggest that students and instructors believe
computing course curricula and assessment methods should be updated to include
GAI tools; however, there are important challenges which need to be addressed
in order to use them successfully while reducing their potential harm to
instructors and students alike. These results provide a timely and important
snapshot of students’ and instructors’ perspectives and preferences which can
inform the design of pedagogies, tools, and policies that best align with
students’ preferences and needs.

##  II Related Work

###  II-A An AI Phase-Shift in the Classroom

Less than three decades ago, widespread access to the Internet fundamentally
changed the education landscape. As internet access in the United States rose
from 14 to 77 percent, students could suddenly access educational resources,
play educational games, and interact with peers in online learning
communities. This inequitable access to these resources also led to a digital
divide where some students were left behind because they did not have personal
computers or reliable internet access [15]. Recent advances in AI are poised
to create a similar phase shift in the education landscape. AI-powered tools
are becoming ubiquitous both in online and face-to-face learning environments.
From a student’s perspective, these AI-powered tools provide a number of
important benefits such as an increased awareness of their performance,
personalized learning pathways [16], and personalized support in the form of
intelligent tutors [17, 18]. From an instructor’s perspective, learning
analytics tools provide instructors with the ability to monitor student
activity and adapt their teaching [19, 20, 21, 22]. Some of these tools are
even integrated into classroom environments where instructors can use
augmented reality to monitor student learning, metacognition, and behavior in
real-time [23]. Chen et al. discuss additional ways in which AI in the
classroom can be beneficial for instructors such as in improving student
reviews, grading, and feedback, utilizing intelligent tutoring systems, and
improving pedagogical practices and student experiences using AI-powered VR
for experiential or practical learning experiences [24].

However, many of these advances have been limited by the current abilities of
machine learning. Recent advances in machine learning have introduced
generative models capable of understanding and generating code have crucial
implications for computing education. Researchers are already demonstrating
the capabilities of these models to help students generate code [25, 3, 26],
explain code to students [6, 5, 9], and even produce programming assignments
[4, 1]. However, there are also some emerging challenges related to over-
reliance and plagiarism [5, 11]. For instance, a recent study showed how these
models can perform nearly as well as students on multiple-choice questions
[8]. So while a phase shift appears to be underway, it is a critical moment to
ensure that things are changing for the better and not for the worse.

###  II-B Carefully Considering the Design of AI Systems

The introduction of AI systems has led to new design challenges related to
their probabilistic nature, their lack of transparency, and their issues
related to fairness and accountability. These challenges have led to rapidly
evolving research topics such as explainable AI (XAI), human-AI interaction
(HAII), and human-centered machine learning. They have also led to the
creation of new conferences that focus on fairness, accountability, and
transparency. Across these fields, there is a resounding call to focus on
approaches that are human-centered [27]. For instance, XAI has grappled with
questions like explainable to who and for what purpose [28]. Additionally,
people calibrate their trust through interactions and experiences with the AI
system [29]; however, other factors such as age, culture, and personality can
also affect trust in AI systems [30]. Finally, there can be a tendency to
develop an over-reliance on AI systems which is, unfortunately, most common
for low-expertise users [31]. Based on these threads of research,
understanding stakeholders is crucial for developing AI systems that are
comprehensible, trustworthy, and avoid excessive reliance. Conversely, there
are many examples of AI systems that violate these principles and cause harm
[32].

Numerous computing education researchers are already addressing these issues
of explainability, trust, and over-reliance. For instance, researchers are
investigating the importance of AI explainability in computing settings [33].
Khosravi et al. developed a new framework for XAI to guide researchers and
practitioners to develop AI-powered education (AIED) systems that are more
trustworthy, explainable, and best aid in learners’ educational goals [34].
Their framework outlined 6 important factors to consider when building XAI:
who the stakeholders are (e.g., instructors and students), what benefits
stakeholders receive from interacting with the system, what potential issues
stakeholders may encounter when using the system, how explanations are
presented, what AI models are used, and the best ways to design user-friendly
and effective AIED systems. Frameworks that exist to guide the development of
effective AIED systems that mitigate potential harm to all stakeholders are
useful. However, the usage of GAI tools in education, such as ChatGPT and
GitHub Copilot, poses a unique problem, as they were not developed solely for
an educational purpose and their inclusion in the classroom may have unknown
effects. Many researchers have begun to explore the potential negative effects
the inclusion of these models into education may bring, including bias and
fairness of these models, over-reliance, explainability, and trust [7, 11].

##  III Methodology

To investigate our research questions about student and instructor
perspectives, we conducted semi-structured interviews with 18 participants.
This qualitative research method is often used to elicit participants’
perceptions, experiences, and values regarding a particular topic or
technology [35]. The field of computing education has begun adopting interview
studies [36, 37, 38, 39], often to explore the perspectives of instructors or
students with respect to topics such as instructor pain points [40] or student
perspectives about online versus on-site teaching [41]. In this study, we
focus on the perspectives of both instructors and students because it is
critical to include all relevant stakeholders when considering the
implications of new technology such as generative AI.

###  III-A Participant Demographics

We conducted interviews with students (N=12) and instructors (N=6) which
resulted in 18 total interviews. Participants were recruited from the
Department of Computer and Information Sciences at a large R1 University in
the United States. The instructors included a mix of tenure-track and
teaching-track faculty. Students were recruited through student organizations,
flyers, and snowball sampling. Student participants majored in either Computer
Science (11/12) or Information Science and Technology (1/12). All of the
instructors interviewed taught computer science courses.

###  III-B Interview Protocol and Analysis

We conducted 30-minute semi-structured interviews. Interviews were semi-
structured so that participants could partially guide the conversation toward
insights they believed were important and allowed us to follow up on
interesting aspects of the conversation [42]. The main questions from the
semi-structured interview are included in the Appendix; however, we often
asked additional follow-up questions to obtain further insights and
clarification from participants. Interviews were recorded and we thematically
analyzed the interviews [43] by reviewing the transcripts, coding
participants’ insights, and then identifying themes. Two researchers followed
this process independently, then compared codes in order to help mitigate
interpretation biases. Once the thematic analysis was completed for all of the
interviews, both researchers compared and combined themes through mediation.

Figure 1: On the left, summarized themes from student interviews are presented
and, on the right, summarized themes from instructor interviews are presented.

###  III-C Limitations

There is currently an urgent need to understand the implications of generative
AI in computing education. For example, researchers and practitioners are
already raising concerns about over-reliance, academic misconduct, and model
trustworthiness [7, 12]. This work serves as an early probe into understanding
current preferences, values, and concerns. While limited in scope, our work
provides a preliminary investigation in order to help instructors,
researchers, and more quickly adapt to this changing landscape. There are a
few limitations to our study which may impact the generalizability of our
results and that warrant discussion. The participants of this study, both
instructors and students, came from a single R1 university and represent a
relatively small sample size, and as such, these results may not fully
represent different demographics across different universities. However, prior
work has shown that results obtained from qualitative studies with small
sample sizes can still provide valuable insights for the topic being
researched [44, 45].

##  IV Results - Themes from Student Interviews

In this section, we present the themes that emerged from our student
interviews. A summarized version of the results from interviews with both
stakeholders (students and instructors) can be found in Figure 1.Within each
theme, we present sub-themes, our observations, relevant supporting quotes,
and relationships between the theme and our three research questions presented
in the introduction. Despite focusing our questions on Generative AI (GAI)
tools such as GitHub Copilot, Codex, and Grammarly, our participants tended to
focus on ChatGPT.

###  IV-A Perceived Benefits of GAI Tools (RQ1)

Students shared the ways in which they have found GAI Tools to be beneficial
in their computing courses, as well as their potential uses in computing
education.

GAI tools reduce the effort to write code and find learning materials. Most
students (10/12) believed that GAI tools, such as ChatGPT, have the potential
to become, and in many cases are already, valuable learning tools to reduce
the effort for students and instructors to complete some common programming
tasks and find learning materials. Of the remaining two students, one was not
very familiar with GAI tools and was therefore unsure of their value and the
other actively tried to avoid using them.

Students shared a number of ways in which they have found GAI tools useful in
the computing education context:

  * •

Generating skeleton or boilerplate code (S2, S3, S8, S9)

  * •

Generating explanations of code (S2, S6, S7, S10, S11)

  * •

Generating examples of concepts (e.g., linked-lists, sorting algorithms) (S2,
S4, S5, S6, S11)

Reflecting on how GAI tools can generate examples to supplement existing
learning materials, S2 said,

> “[Students] can use [ChatGPT] to find resources online… like, ‘Hey, I need a
> problem related to bubble sort,’ and then it will show them examples.”

Students use GAI tools to avoid busy work. When asked what motivates students
to use these tools, students who frequently use them expressed that they were
more likely to use GAI tools for assisting with, or even completing,
assignments that they perceived as “meaningless” or mere “busy work.” Some
students (4/12) compared it to existing tools, such as the calculator, whose
introduction into the classroom allowed students to progress towards more
advanced topics faster and reduce the amount of time students have to spend on
familiar tasks or could be considered tedious [46]. According to some of the
students, this refers to any work that takes too much of their time for very
little academic benefit or that they aren’t passionate about. Participant S2
remarked on what kind of work they use ChatGPT for the most, saying,

> “ChatGPT could just kind of do all the busy work or anything I could figure
> out, in like a couple of minutes. I’d rather ChatGPT do it, because it
> will.”

GAI tools shift the focus to higher levels of abstraction. Students expressed
(6/12) that GAI tools could help reduce the amount of time spent working at
lower levels of abstraction. Students often gave examples of how GAI tools
could shift the learner’s focus toward higher-level skills (e.g., design, the
analysis and evaluation of code, or advanced software engineering concepts)
which students are more likely to encounter in later computing courses. For
example, participant S9 described how students might focus on design patterns
and software design rather than implementation details. They said,

> “You don’t usually get design patterns until you are in graduate school,
> right? Unless you’d specifically seek it out…we could use ChatGPT to help
> teach design patterns where… You can use ChatGPT as a tool to implement all
> the little pieces. But we are going to try to break down this larger problem
> to figure out what pattern applies to it, and then use whatever you need to
> use to generate the individual pieces of code. But we really care about the
> overall structure and making sure that the students kind of have this
> broader understanding of the piece of software rather than a single
> algorithm or a single piece of code or a smaller scale program.”

GAI tools provide alternative perspectives and sources of assistance. Another
common theme amongst students who regularly use GAI tools is that these tools
provide another source of assistance outside and their instructors and
teaching assistants (4/12). Some reasons why students said they choose to use
a tool such as ChatGPT rather than asking their instructors or teaching
assistants are:

  * •

Some professors are not good at explaining course concepts and code (S4, S5)

  * •

GAI tools can provide multiple and alternative perspectives about programming
concepts (S5, S10)

  * •

GAI tools can be more convenient sources of assistance over instructors or TAs
(S5, S8)

S4 described a time in which they wished they were able to use GAI tools in
order to get more help with a concept in one of their computing courses,
especially when they found themselves searching for a different perspective
when they felt their instructor wasn’t communicating what they needed,

> “With some professors, they don’t know how to communicate the answer that
> you’re looking for… I had a lot of trouble wrapping my head around it when I
> asked in what context will we use a linked list. [My Instructor] just
> explained what a linked list was, and with ChatGPT, I feel like I could say,
> ‘give me an example of a linked list and how I would implement it in this
> language,’ ”

###  IV-B Concerns about GAI Tools (RQ2)

While students largely expressed excitement about GAI tools’ benefits to
computing education, students also expressed an amount of trepidation and
concern regarding their use in computing education.

Concerns about over-reliance. Most students (9/12) mentioned the potential for
students to develop an over-reliance on these tools as a concern. More
specifically, students (6/12) detailed their concerns about the quality of
education students receive if they become over-reliant on GAI tools that don’t
adequately explain their responses. Some of the specific concerns about over-
reliance and the quality of education were:

  * •

Students may not develop an understanding of the material when they get
assistance from GAI tools (S1, S4, S5, S9, S11, S12)

The following quotes from participants further illustrate some of these
student concerns. Participant S1 said,

> “When using them I don’t really fully understand what they’re telling me,
> and it’s just kind of like, ‘Oh! There’s the answer.’ But I’m not really the
> one learning and really digesting what’s happening and how I got to that
> conclusion.”

Participant S4 provided an additional perspective, saying,

> “I feel like a lot of people just heavily rely on it, and they use it to
> just copy and paste the work, and not necessarily understand why. I feel
> like they would just copy the work given to them. It kind of ruins the
> purpose of learning it in the first place.”

Concerns about trustworthiness. The majority of students (7/12) also expressed
concerns about the trustworthiness of the output given by GAI tools. Students
raised concerns about the reliability of information generated by these models
due to their potential to misinform students. Specifically, students outlined
the following concerns about trustworthiness:

  * •

Information sources are missing or incorrect (S1, S8, S9)

  * •

GAI tools tend to hallucinate information (S6, S8, S10)

  * •

Students have to double-check everything, not just information, but things
like false context and tone (S4, S6)

Participant S8 said, due to these models’ tendency to hallucinate students
have to be careful to double-check the information they receive,

> “[ChatGPT] is helpful, but I understand that sometimes it also hallucinates
> as well. It will give you false information. I don’t really know if it is
> yet like a Google replacement…, I feel like it’s only as useful if you
> really know the topic that you’re talking about, and like you can read what
> it’s saying, and be sure that it’s not like [meaningless output], because
> sometimes you do have to search again on Google just to make sure what it
> said is correct.”

Participant S4 also described needing to check responses for correctness,
sharing that it wouldn’t stop them from using GAI tools altogether and it
prevents them from relying on the tools too much,

> “[Incorrect ChatGPT responses] wouldn’t keep me from using it again. It’s
> just a small detail. It keeps me on my toes, and makes sure I double-check
> everything. I can’t rely too much on it.”

Concerns about academic integrity. The majority of students (7/12) also
brought up the issue of academic integrity. These students expressed the
belief that the amount of plagiarism will increase as these tools become more
popular.

However, students did not necessarily believe that AI-aided plagiarism is all
that different from plagiarism that occurred previously. Students claimed that
the only difference between plagiarizing by copying someone else’s work or
paying for a subscription service such as Chegg and using a tool like ChatGPT
is that it’s free and available to everyone.

> “Before I’ve seen people use things like, Chegg, to just straight up just
> finish their homework. I feel like it’s just a difference between you using
> ChatGPT, which is a free service, and it can give you the information you
> want, and you could probably just change up a couple of things with it
> versus paying a fee for a subscription-based service that just hosts the
> work that your professors are giving to you.” (S4)

###  IV-C Predicted Implications of GAI Tools (RQ3)

Students were excited to share their insight about how course curricula and
assessment methodologies should be adapted to accommodate GAI tools.

Instructors are responsible for adapting their courses. In our interviews,
none of the students said that instructors should ban the use of GAI tools. On
the contrary, multiple students (5/12) expressed that instructors are
responsible for adapting their courses in ways to work with this new
technology, as instructors have had to in the past with any innovative and
popular technology, such as calculators, Wikipedia, and the Internet (4/12).
While the majority of students expressed that they were not sure exactly what
steps need to be taken by instructors in order to adapt their classes, a
couple of students indicated updates they hope instructors do not make to
their classes. Participants S3 and S8 specifically mentioned one way to avoid
plagiarism would be to increase the number of exams or the weight of exams in
the final grades of courses. However, both students said they would not want
this,

> “I guess they would have to possibly lower lab weights and maybe increase
> exam weights, or you know things that can be more so tested in person… I
> feel like it shouldn’t be like that… I kind of had the benefit of, like, you
> know, being able to use Google, having like a 40-50% like lab weight. Being
> able to use Google, learn, read a lot of documentation, and you know,
> actually, kind of somewhat suffer through labs, but also I knew that the lab
> grade would save me because I wasn’t as good on exams,” (S8)

Instructors must focus on student engagement. Several students (3/12)
specifically mentioned that, when learning new material, hands-on active-
learning assignments are the most helpful and engaging. Students (4/12)
emphasized that they believed that engaging courses and coursework would more
likely result in engaged students who see value in and want to complete the
assignments and are less likely to plagiarize. Participant S1 said the
following,

> “Instructors need to understand why students use ChatGPT in the first place,
> and some of the reasons why they use it is they just don’t understand the
> content, and they want an easy out and that means that something is going
> wrong with the curriculum that’s being presented to them.”

Instructors should incorporate GAI tools into the classes. Students were
excited about the potential for instructors to include these models in their
curricula. Students suggested that instructors could focus less on
implementation details and more on high-level concepts. Participant S5 pointed
to the importance of other skills that should be learned in early programming
courses, particularly problem-solving.

> “I think learning the language is the easiest part, but I think the
> introductory programming courses offer something a little bit more
> important, which is problem-solving. Which I think will still … be a very
> important part of programming, because that’s really all programming is.
> It’s just problem-solving… But the problem-solving mindset is definitely
> something that cannot just be assumed people have.”

Students (4/12) also suggested that instructors could teach students how to
use GAI tools by teaching students the best ways to prompt tools such as
ChatGPT and the limitations and risks of using these tools. A couple of
students also suggested instructors could use GAI tools to scaffold learning,
as described in more detail by participant S5,

> “It’s like asking a mathematician to do mundane repetitive math equations
> today without a calculator… yeah, you could do it, but do you want to? Not
> really. So, I think instructors should, you know, start off teaching their
> students, you know, how to do it. How to do everything without an AI
> language model to help them. And then, as they get more advanced with it,
> they, you know, introduce the AI to just, you know, make the more difficult
> tasks a bit easier.”

##  V Results - Themes from Instructor Interviews

The following section details the themes and observations discovered through
our instructor interviews. Similar to students, instructors tended to focus
primarily on ChatGPT.

###  V-A Perceived Benefits of GAI Tools (RQ1)

While instructors (5/6) have not used GAI tools frequently, they envision a
number of ways in which they can be beneficial for students in computing
courses. Overall, instructors were familiar with GAI tools like ChatGPT and
GitHub Copilot, but used them much less than students. Consequently, they had
fewer insights about they might be used in the computing education context.
However, every instructor (6/6) thought that these tools could be valuable for
students.

GAI tools can help students understand code and computing concepts. A couple
of instructors (2/6) thought that GAI tools were beneficial for students when
they encounter unfamiliar code and that GAI tools could provide just-in-time
support. While prior work has demonstrated the ability of generative AI to
explain code, participant I2 described how this helps students when they
encounter unfamiliar code online,

> “You can give [ChatGPT] a little piece of a program you got from Github or
> whatever, and have it explain to you what this code is doing, and it would
> do a good job.”

GAI tools can help students find inspiration, brainstorm, and get feedback on
ideas. A majority of the instructors (4/6) detailed ways in which GAI tools
could be used to aid in creative processes such as ideation and brainstorming.
For example, participant I1 described how GAI tools can help with
brainstorming solutions to coding problems. Additionally, some instructors
(2/6) described ways in which they’ve found success in having GAI tools,
specifically ChatGPT, provide students with feedback on their writing for
project-based computing courses. Participant I2 described how they’ve had
students use ChatGPT to try to find weaknesses or areas of improvement for
their essays or project proposals.

GAI tools’ low cost and accessibility can provide students with high quality
learning resources. A couple of instructors (2/6) expressed their excitement
about how the low cost and barrier to entry can provide students with
resources that can be prohibitively expensive, such as private tutors or paid
services such as Chegg or CourseHero. Additionally, GAI tools could act as
high quality learning resources outside of students’ instructors and TAs,
providing another perspective on course topics.

###  V-B Concerns about GAI Tools (RQ2)

Instructors hold many common concerns regarding AI systems for the use of
generative AI in computing education and their use by students.

Concerns about trustworthiness. Most of the instructors (5/6) cited the lack
of trustworthiness of responses from GAI tools as one of their primary
concerns. All of the instructors mentioned that it’s important to double-check
the information received from these models. However, as one instructor pointed
out, students don’t always have the necessary foundational knowledge to know
whether the outputs are right or wrong. Participant I4 said,

> “We don’t know that [ChatGPT] is giving [students] the right information.
> and you know, if I know a topic, I know when I’m getting bogus information.
> They don’t, you know, so that’s a concern.”

Concerns about over-reliance. Several instructors (3/6) also cited over-
reliance as a significant concern when considering students using these models
in their classes. Participant I3 expressed concern about students not fully
learning material if they rely too heavily on GAI tools. Participants I1 and
I3 both expressed concern that students may lose important skills, such as
writing, if they become too reliant on these models. Participant I5 discussed
their concern about GAI tools replacing students’ own effort, saying,

> “I think pedagogically, we’ll have to kind of go back to the drawing board.
> And come up with new ways to assess. Not necessarily just to prevent
> students from using and benefiting from these tools, by all means we should
> embrace change and new methods and new ways of doing things. So find ways
> where students can use these things to assist, to help but not one where it
> replaces their own cognitive load.”

Concerns about academic integrity. All instructors (6/6) expressed concern
about students using GAI tools to plagiarize. Several instructors (4/6)
expressed that, due to the availability and low cost of tools like ChatGPT,
students are likely to plagiarize more. However, one instructor (I5) said,

> “I feel like a student who was never motivated to cheat in the first place
> will not do it now just because ChatGPT is available. Sufficiently motivated
> students would have always cheated. So if I were to lazily assume that there
> is an uptick in the number of students who are now exploiting this tool or
> an uptick in the number of students who are now cheating as a result of this
> tool, it would have been those in the middle who were always willing to, but
> perhaps lack the time, resources, access, or the know-how to properly engage
> in it, but I don’t think just there’s this giant movement towards more
> students doing it, who would not have done it otherwise. I don’t think it’s
> turning students into cheaters. I think it’s just making it more accessible
> to those who already would have done it.”

This participant’s insight aligns with the perspectives shared by the other
instructors in that, the concerns about academic integrity which have arisen
due to the presence of GAI tools in the classroom, are less focused on just
stopping students from cheating, which is unlikely, but rather in developing
ways in which instructors can disincentivize cheating.

###  V-C Predicted Implications of GAI Tools (RQ3)

Instructors shared uncertainty about what exactly needs to change in course
designs and assessment methods; however, they emphasized the importance of
proactively incorporating these models into computing education curricula.

GAI tools should be incorporated into classrooms. Unanimously, instructors
(6/6) emphasized that they believe GAI tools should not be banned, and they
should embrace their existence and include them in the classroom. Instructors
believed it was imperative to understand how these tools work. Some of the
ways they envisioned incorporating GAI tools into their courses included:

  * •

Let students use it but they should be able to explain why the output is
correct or incorrect (I1, I3, I6)

  * •

Clearly articulate what students need to know how to do without using GAI
tools (I4)

Participant I4, detailed how, with any new tools which shift how courses are
taught, clear expectations for students need to be established,

> “… before calculators were a thing we used to expect students to memorize
> multiplication and division tables. And you know, after the calculator came
> into being, of course, you know it’s reasonable not to expect people to
> memorize that when you know they’ve got a tool that they can use to figure
> out how to multiply things, you know. And I could see the same thing
> happening with us with programming now that tools like ChatGPT exist, you
> know, maybe there are certain things that we could expect students to use a
> tool for, and we don’t have to test them on it. But at the same time, you
> know, there has to be some core level of knowledge that we should expect
> from a student, you know, without the use of tools.”

GAI tools can facilitate critical thinking A couple of instructors (2/6) were
excited about the potential to use these models in their courses to help
students further develop their critical thinking and analysis skills.
Participant I6 said,

> “ Let them actually use ChatGPT to come back with a report as a first draft.
> But then the job of the student is not simply to return the report, but
> instead, look at the report ChatGPT wrote, and then decide or criticize
> whether it’s right or wrong, and then, if it’s wrong, then what is wrong
> with it, or try to improve… So in that sense, it is more useful for the
> student to develop their critical thinking… instead of just like the old way
> of just writing a report.”

Additionally, participant I5 said,

> “Just the production of something is not going to be as important, perhaps
> leaning more into analysis, perhaps leaning more into your ability to
> troubleshoot. Versus creation which is one of which is one of the steps that
> you know of many steps in any sort of understanding or building a system or
> an algorithm or a data structure or anything else. …It’s not, give me code,
> it’s, here is some code, give me some sort of analysis that matches or fits
> into something that is not easily template-tized, something that is not
> easily described or describable.”

Instructors need to update assessment methods. In terms of how to update
curriculum to adapt courses to account for the existence of these models,
instructors did not have a very clear idea of what exactly needed to be done,
with many expressing uncertainty about the best way to assess student
progress, prevent plagiarism, and ensure that students do not rely too heavily
on the models, hindering their education. In their interviews, instructors did
mention a few considerations they had for updating their assessment methods:

  * •

Mitigate plagiarism by developing engaging assignments and reducing busy work
(I3)

  * •

Change course grade weights such that easily plagiarized assignments can’t
carry students’ grades (I4, I5)

  * •

Continue to give proctored exams since they can’t be plagiarized like other
out out-of-class assignments (I4)

##  VI Discussion & Conclusion

###  VI-A Converging Themes across Stakeholders and Opportunities

In this section, we present converging themes that emerged through our
stakeholder interviews. Based on these areas of convergence, we present some
opportunities to maximize the benefits of generative models in computing
education settings.

####  VI-A1 Students and instructors are concerned about trustworthiness and
over-reliance

Across most interviews, students and instructors were concerned about the
trustworthiness of GAI tools as well as the potential for students to become
over-reliant on them. Participants highlighted the black-box nature of GAI and
their tendency to hallucinate non-factual information. This raised concerns
about students not understanding the responses they receive from GAI, a lack
of reliability of the information received, and the impact of over-reliance on
students’ learning outcomes.

Opportunity #1: Explicitly teach students when and how to use GAI tools.
Educators can teach courses and modules about how to fact-check the model’s
output in cases where the model is known to perform poorly; calibrating a
healthy skepticism in generative AI. Additionally, instructors should not
assume computing students have AI literacy. It may be necessary to teach
courses in computational thinking and AI literacy [47].

####  VI-A2 Students and instructors believe GAI tools will broaden access

Students expressed that they believed tools like ChatGPT are valuable learning
tools, partially due to the fact that it’s available at any time and it’s free
or inexpensive to use. Opportunity #2: GAI tools democratize access to help.
Similar to when unequal access to the Internet caused some students to be left
behind [15], today, unequal financial access to tutors, unpaid internships,
college prep programs, and paid content can have similar effects. GAI tools
may offer cheaper access to high-quality tutoring.

###  VI-B Diverging Themes across Stakeholders and Challenges

In this section, we present some diverging themes which have the potential to
cause tension between students and instructors. We also describe how these
divergences in preferences may eventually lead to challenges.

####  VI-B1 Students tended to be more familiar and frequent users of GAI
tools than instructors

The majority of the students were either familiar with the capabilities of GAI
tools or use them frequently. On the contrary, some instructors had tried GAI
tools, but they did not use them very frequently.

Challenge #1: Instructors need to become familiar with the functionality and
limitations of GAI tools in order to adapt curricula and assessment methods
adequately. Currently, instructors are at a disadvantage to students who are
much more familiar with these models.

####  VI-B2 Students and instructors are not aligned on how to adapt
assessment

While both stakeholders expressed uncertainty about how to adapt assessment
methods to account for GAI tools, the suggestions they did provide did not
align. Students emphasized that the weight and frequency of hands-on, active
learning assignments, such as programming labs, should not change as they are
valuable to students’ comprehension of new material and learning goals. A few
of the instructors proposed to adapt their assessment methods to combat
academic dishonesty by reducing the weight of some of the lab assignments and
increasing the weight or frequency of in-class, proctored assignments, such as
quizzes or tests. This divergence between the values of students and
instructors has the potential to create significant tension between students
and instructors.

Challenge #2: Instructors wanted to develop assessments that mitigate academic
dishonesty. One-third of interviewed instructors suggested doing this by
increasing the weight of proctored exams. However, students expressed that
they still want hands-on and active learning assignments (e.g., programming
lab assignments) to represent a larger portion of their overall grade,
because, as many students expressed, they learned better when they were
engaged in the tasks they had to complete. Increasing the frequency or weight
of in-class proctored exams moves away from evidence-based active learning
[48], additionally it may negatively affect students.

####  VI-B3 Instructors were not aware of students’ motivations for using GAI
tools

Students shared a variety of reasons as to why they would choose to use GAI
tools to assist with or complete their assignments, such as reducing busy
work, getting another perspective other than their instructor on a concept,
and getting help when their instructor is not available. When instructors were
asked to consider why students use these models, the only common response was
regarding students using GAI tools for busy work.

Challenge #3: Instructors need to better understand students’ motivations for
using GAI tools. Without this understanding, instructors could potentially
make changes to pedagogy and assessment that unintentionally harm students. To
address this concern, researchers and practitioners could co-design solutions
with students. Co-design is useful for curriculum design [49] and has already
been used for integrating AI into classrooms [50]. These techniques may not
only result in better, more equitable solutions but also have the potential to
build relationships and leverage students’ deeper familiarity with these GAI
tools.

###  VI-C Conclusion

This paper presents the first systematic investigation of both students’ and
instructors’ experiences with and preferences for using GAI tools in computing
classrooms. While these results are preliminary, we believe that rapidly
disseminating these critical views can guide researchers to capitalize on
opportunities while mitigating potential challenges.

## Acknowledgment

We would like to thank Dr. Brian McInnis for feedback on the abstract which
helped refine the final framing of the paper.

## References

  * [1] S. Sarsa, P. Denny, A. Hellas, and J. Leinonen, “Automatic generation of programming exercises and code explanations using large language models,” in _Proc. of the 2022 ACM Conf. on Int. Computing Education Research - Volume 1_. ACM, 2022, p. 27–43.
  * [2] S. MacNeil, A. Tran, J. Leinonen, P. Denny, J. Kim, A. Hellas, S. Bernstein, and S. Sarsa, “Automatically generating cs learning materials with large language models,” _arXiv preprint arXiv:2212.05113_ , 2022.
  * [3] M. Wermelinger, “Using github copilot to solve simple programming problems,” in _Proceedings of SIGCSE_. New York, NY, USA: Association for Computing Machinery, 2023, p. 172–178. [Online]. Available: https://doi.org/10.1145/3545945.3569830
  * [4] J. Finnie-Ansley, P. Denny, B. A. Becker, A. Luxton-Reilly, and J. Prather, “The robots are coming: Exploring the implications of openai codex on introductory programming,” in _Australasian Computing Education Conf._ , 2022, pp. 10–19.
  * [5] S. MacNeil, A. Tran, D. Mogil, S. Bernstein, E. Ross, and Z. Huang, “Generating diverse code explanations using the gpt-3 large language model,” in _Proc. of the 2022 ACM Conf. on Int. Computing Education Research - Volume 2_. ACM, 2022, p. 37–39.
  * [6] S. MacNeil, A. Tran, A. Hellas, J. Kim, S. Sarsa, P. Denny, S. Bernstein, and J. Leinonen, “Experiences from using code explanations generated by large language models in a web software development e-book,” in _Proceedings of the 54th ACM Technical Symposium on Computer Science Education V. 1_ , 2023, pp. 931–937.
  * [7] B. A. Becker, P. Denny, J. Finnie-Ansley, A. Luxton-Reilly, J. Prather, and E. A. Santos, “Programming is hard - or at least it used to be: Educational opportunities and challenges of ai code generation,” in _Proceedings of the 54th ACM Technical Symposium on Computer Science Education_ , ser. SIGCSE 2023, 2023. [Online]. Available: https://doi.org/10.1145/3545945.3569759
  * [8] J. Savelka, A. Agarwal, C. Bogart, and M. Sakr, “Large language models (gpt) struggle to answer multiple-choice questions about code,” 2023.
  * [9] J. Leinonen, P. Denny, S. MacNeil, S. Sarsa, S. Bernstein, J. Kim, A. Tran, and A. Hellas, “Comparing code explanations created by students and large language models,” in _Proceedings of the 2023 Conference on Innovation and Technology in Computer Science Education V. 1_ , ser. ITiCSE 2023. New York, NY, USA: Association for Computing Machinery, 2023, p. 124–130. [Online]. Available: https://doi.org/10.1145/3587102.3588785
  * [10] A. Tran, L. Li, E. Rama, K. Angelikas, and S. MacNeil, “Using large language models to automatically identify programming concepts in code snippets,” in _Proc. of the 2023 ACM Conf. on Int. Computing Education Research - Volume 2_ , vol. 1. ACM, 2023, pp. 563–569.
  * [11] S. MacNeil, J. Kim, J. Leinonen, P. Denny, S. Bernstein, B. A. Becker, M. Wermelinger, A. Hellas, A. Tran, S. Sarsa, J. Prather, and V. Kumar, “The implications of large language models for cs teachers and students,” in _Proceedings of the 54th ACM Technical Symposium on Computer Science Education V. 2_ , ser. SIGCSE 2023. New York, NY, USA: Association for Computing Machinery, 2023, p. 1255. [Online]. Available: https://doi.org/10.1145/3545947.3573358
  * [12] E. Kasneci, K. Sessler, S. Küchemann, M. Bannert, D. Dementieva, F. Fischer, U. Gasser, G. Groh, S. Günnemann, E. Hüllermeier, S. Krusche, G. Kutyniok, T. Michaeli, C. Nerdel, J. Pfeffer, O. Poquet, M. Sailer, A. Schmidt, T. Seidel, M. Stadler, J. Weller, J. Kuhn, and G. Kasneci, “Chatgpt for good? on opportunities and challenges of large language models for education,” _Learning and Individual Differences_ , vol. 103, p. 102274, 2023\.
  * [13] J. Prather, P. Denny, J. Leinonen, B. A. Becker, I. Albluwi, M. E. Caspersen, M. Craig, H. Keuning, N. Kiesler, T. Kohn, A. Luxton-Reilly, S. MacNeil, A. Petersen, R. Pettit, B. N. Reeves, and J. Savelka, “Transformed by transformers: Navigating the ai coding revolution for computing education: An iticse working group conducted by humans,” in _Proceedings of the 2023 Conference on Innovation and Technology in Computer Science Education V. 2_ , ser. ITiCSE 2023. New York, NY, USA: Association for Computing Machinery, 2023, p. 561–562. [Online]. Available: https://doi.org/10.1145/3587103.3594206
  * [14] S. Lau and P. J. Guo, “From” ban it till we understand it” to” resistance is futile”: How university programming instructors plan to adapt as more students use ai code generation and explanation tools such as chatgpt and github copilot,” 2023. [Online]. Available: https://www.samlau.me/pubs/cs-instructors-adapting-to-chatgpt-copilot-ai-tools_ICER-2023.pdf
  * [15] J. Margolis, R. Estrella, J. Goode, J. J. Holme, and K. Nao, _Stuck in the Shallow End_. The MIT Press, 2008.
  * [16] J. D. Basham, T. E. Hall, R. A. Carter Jr, and W. M. Stahl, “An operationalized understanding of personalized learning,” _Journal of Special Education Technology_ , vol. 31, no. 3, pp. 126–136, 2016.
  * [17] T. Crow, A. Luxton-Reilly, and B. Wuensche, “Intelligent tutoring systems for programming education: a systematic review,” in _Proceedings of the 20th Australasian Computing Education Conference_ , 2018.
  * [18] J. R. Anderson, C. F. Boyle, and B. J. Reiser, “Intelligent tutoring systems,” _Science_ , vol. 228, no. 4698, pp. 456–462, 1985.
  * [19] P. Leitner, M. Khalil, and M. Ebner, “Learning analytics in higher education—a literature review,” _Learning analytics: Fundaments, applications, and trends: A view of the current state of the art to enhance E-learning_ , pp. 1–23, 2017.
  * [20] B. Rienties, H. Køhler Simonsen, and C. Herodotou, “Defining the boundaries between artificial intelligence in education, computer-supported collaborative learning, educational data mining, and learning analytics: A need for coherence,” in _Frontiers in Education_ , vol. 5. Frontiers Media SA, 2020, p. 128.
  * [21] P. Ihantola, A. Vihavainen, A. Ahadi, M. Butler, J. Börstler, S. H. Edwards, E. Isohanni, A. Korhonen, A. Petersen, K. Rivers _et al._ , “Educational data mining and learning analytics in programming: Literature review and case studies,” _Proceedings of the 2015 ITiCSE on Working Group Reports_ , pp. 41–63, 2015.
  * [22] T. Cerratto Pargman and C. McGrath, “Mapping the ethics of learning analytics in higher education: A systematic literature review of empirical research,” _Journal of Learning Analytics_ , vol. 8, no. 2, 2021.
  * [23] K. Holstein, B. M. McLaren, and V. Aleven, “Student learning benefits of a mixed-reality teacher awareness tool in ai-enhanced classrooms,” in _Artificial Intelligence in Education_ , ser. AIED 2018. Springer, 2018.
  * [24] L. Chen, P. Chen, and Z. Lin, “Artificial intelligence in education: A review,” _IEEE Access_ , vol. 8, pp. 75 264–75 278, 2020.
  * [25] B. Puryear and G. Sprint, “Github copilot in the classroom: Learning to code with ai assistance,” _J. Comput. Sci. Coll._ , vol. 38, no. 1, 2022.
  * [26] P. Denny, V. Kumar, and N. Giacaman, “Conversing with copilot: Exploring prompt engineering for solving cs1 problems using natural language,” 2022.
  * [27] M. O. Riedl, “Human-centered artificial intelligence and machine learning,” _Human Behavior and Emerging Technologies_ , vol. 1, no. 1, pp. 33–36, 2019\.
  * [28] U. Ehsan and M. O. Riedl, “Human-centered explainable ai: Towards a reflective sociotechnical approach,” in _HCI International 2020-Late Breaking Papers: Multimodality and Intelligence: 22nd HCI International Conference, HCII 2020, Copenhagen, Denmark, July 19–24, 2020, Proceedings 22_. Springer, 2020, pp. 449–466.
  * [29] J. D. Lee and K. A. See, “Trust in automation: Designing for appropriate reliance,” _Human factors_ , vol. 46, no. 1, pp. 50–80, 2004.
  * [30] K. A. Hoff and M. Bashir, “Trust in automation: Integrating empirical evidence on factors that influence trust,” _Human factors_ , vol. 57, no. 3, pp. 407–434, 2015.
  * [31] S. Gaube, H. Suresh, M. Raue, A. Merritt, S. J. Berkowitz, E. Lermer, J. F. Coughlin, J. V. Guttag, E. Colak, and M. Ghassemi, “Do as ai say: susceptibility in deployment of clinical decision-aids,” _NPJ digital medicine_ , vol. 4, no. 1, p. 31, 2021.
  * [32] A.-S. Mayer, F. Strich, and M. Fiedler, “Unintended consequences of introducing ai systems for decision making.” _MIS Quarterly Executive_ , vol. 19, no. 4, 2020.
  * [33] K. Fiok, F. V. Farahani, W. Karwowski, and T. Ahram, “Explainable artificial intelligence for education and training,” _The Journal of Defense Modeling and Simulation_ , vol. 19, no. 2, pp. 133–144, 2022.
  * [34] H. Khosravi, S. B. Shum, G. Chen, C. Conati, Y.-S. Tsai, J. Kay, S. Knight, R. Martinez-Maldonado, S. Sadiq, and D. Gašević, “Explainable artificial intelligence in education,” _Computers and Education: Artificial Intelligence_ , vol. 3, p. 100074, 2022.
  * [35] K. Peters and E. Halcomb, “Interviews in qualitative research,” _Nurse researcher_ , vol. 22, pp. 6–7, 03 2015.
  * [36] S. Heckman, J. C. Carver, M. Sherriff, and A. Al-zubidy, “A systematic literature review of empiricism and norms of reporting in computing education research literature,” _ACM Trans. Comput. Educ._ , vol. 22, no. 1, oct 2021\. [Online]. Available: https://doi.org/10.1145/3470652
  * [37] J. Sheard, S. Simon, M. Hamilton, and J. Lönnberg, “Analysis of research into the teaching and learning of programming,” in _Proceedings of the Fifth International Workshop on Computing Education Research Workshop_ , ser. ICER ’09. New York, NY, USA: Association for Computing Machinery, 2009, p. 93–104. [Online]. Available: https://doi.org/10.1145/1584322.1584334
  * [38] A. Lishinski, J. Good, P. Sands, and A. Yadav, “Methodological rigor and theoretical foundations of cs education research,” in _Proceedings of the 2016 ACM Conference on International Computing Education Research_ , ser. ICER ’16. New York, NY, USA: Association for Computing Machinery, 2016, p. 161–169. [Online]. Available: https://doi.org/10.1145/2960310.2960328
  * [39] S. Schulz, S. Berndt, and A. Hawlitschek, “Exploring students’ and lecturers’ views on collaboration and cooperation in computer science courses - a qualitative analysis,” _Computer Science Education_ , 01 2022\.
  * [40] S. Mirhosseini, A. Z. Henley, and C. Parnin, “What is your biggest pain point? an investigation of cs instructor obstacles, workarounds, and desires,” in _Proceedings of the 54th ACM Technical Symposium on Computer Science Education V. 1_ , ser. SIGCSE 2023. New York, NY, USA: Association for Computing Machinery, 2023, p. 291–297. [Online]. Available: https://doi.org/10.1145/3545945.3569816
  * [41] B. T. Jónsson, M. Pischetola, N. Inie, M. Daniels, and C. Brabrand, “Student perspectives on on-site versus online teaching throughout the covid-19 pandemic,” in _2022 IEEE Frontiers in Education Conference (FIE)_ , 2022, pp. 1–9.
  * [42] D. Magaldi and M. Berler, “Semi-structured Interviews,” in _Encyclopedia of Personality and Individual Differences_ , V. Zeigler-Hill and T. K. Shackelford, Eds. Cham: Springer International Publishing, 2020, pp. 4825–4830. [Online]. Available: https://doi.org/10.1007/978-3-319-24612-3_857
  * [43] V. Braun and V. Clarke, “Using thematic analysis in psychology,” _Qualitative Research in Psychology_ , vol. 3, no. 2, pp. 77–101, 2006. [Online]. Available: https://www.tandfonline.com/doi/abs/10.1191/1478088706qp063oa
  * [44] C. Zander, L. Thomas, J. E. Moström, and A. Eckerdal, “Copying can be good: How students view imitation as a tool in learning to program,” in _2020 IEEE Frontiers in Education Conference (FIE)_ , 2020, pp. 1–9.
  * [45] L. van Beek, M. van den Bogaard, and M. de Vries, “Feedback perceptions: preliminary analysis of semistructured group interviews with first-year bachelor students of computer science,” in _2021 IEEE Frontiers in Education Conference (FIE)_ , 2021, pp. 1–8.
  * [46] P. Drijvers and M. Doorman, “The graphics calculator in mathematics education,” _The Journal of Mathematical Behavior_ , vol. 15, no. 4, pp. 425–440, 1996.
  * [47] D. Long and B. Magerko, “What is ai literacy? competencies and design considerations,” in _Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems_ , ser. CHI ’20. New York, NY, USA: Association for Computing Machinery, 2020, p. 1–16. [Online]. Available: https://doi.org/10.1145/3313831.3376727
  * [48] C. C. Bonwell and J. A. Eison, “Active Learning: Creating Excitement in the Classroom. 1991 ASHE-ERIC Higher Education Reports,” ERIC Clearinghouse on Higher Education, The George Washington University, One Dupont Circle, Suite 630, Washington, DC 20036-1183 ($17, Tech. Rep., 1991, iSBN: 9781878380081 ISSN: 0884-0040 ERIC Number: ED336049. [Online]. Available: https://eric.ed.gov/?id=ED336049
  * [49] K. L. Gunckel and F. M. Moore, “Including Students and Teachers in the Co-Design of the Enacted Curriculum,” Tech. Rep., 2005, publication Title: Online Submission ERIC Number: ED498676. [Online]. Available: https://eric.ed.gov/?id=ED498676
  * [50] K. Holstein, B. M. McLaren, and V. Aleven, “Co-Designing a Real-Time Classroom Orchestration Tool to Support Teacher–AI Complementarity,” _Journal of Learning Analytics_ , vol. 6, no. 2, pp. 27–52, Jul. 2019. [Online]. Available: https://learning-analytics.info/index.php/JLA/article/view/6336

## Appendix

Semi-Structured Interview Script
Personal Background Information:

  1. 1.

(Students) What is your major? What year are you? What kinds of classes have
you taken?

  2. 2.

(Instructors) What courses do you teach? How many years have you been
teaching? What kinds of assignments do you to tend to have students complete?

Awareness of GAI tools:

  1. 1.

Are you familiar with GAI tools like GitHub CoPilot, Amazon CodWhisperer,
ChatGPT, Grammarly, or Google Smart Compose?”

Prior Experiences with GAI tools:

  1. 1.

Have you ever used a GAI tool for an educational reason? If so, describe a
time when you used a GAI tool to help with an assignment.

  2. 2.

What kind of assignment was it for? How was it helpful/unhelpful? Would you
use it again? Why did you choose to use it?

Values Held for GAI tools:

  1. 1.

(Students) What types of coursework/assignments do you find yourself needing
assistance on?

  2. 2.

(Students) When would you use a tool like these in addition to or instead of
getting assistance from your instructor, teaching assistants, or peers?

  3. 3.

(Students) What types of assignments do you think are the most beneficial to
your ability to comprehend new material?

  4. 4.

What situations do you think these tools could be beneficial for students? For
instructors?

  5. 5.

What are your concerns for their usage by students? By instructors?

  6. 6.

How often do you believe students use tools like these to plagiarize work on
assignments?

  7. 7.

What curriculum changes do you expect will be required as a result of these
models?