# Impact of virtual REU experiences on students’ psychosocial gains during the
COVID-19 pandemic

Dina Zohrabi Alaee<EMAIL_ADDRESS>Micah K. Campbell  Benjamin M. Zwickl
School of Physics and Astronomy, Rochester Institute of Technology, Rochester,
New York 14623, USA

###### Abstract

In the Summer of 2020, due to COVID-19, institutions either canceled or
remotely hosted their Research Experience for Undergraduates (REU) programs.
We carried out a 16-week longitudinal study examining the impact of these
fully remote research experiences on mentees’ psychosocial gains (e.g.,
identity). We studied the phenomenon of a remote research experience from the
standpoint of the mentees ($N$=10) and their mentors ($N$=8), who were each
interviewed seven and three times, respectively (94 total interviews). All
mentees reported that this experience was highly beneficial through different
factors and that they developed a sense of belonging, self-efficacy, and
identity despite working remotely. Then, we synthesize these results with
prior literature and develop a framework showing how different experiences and
constructs affect the development of the physics and researcher identity.
Gaining greater understanding regarding factors leading to the growth of
psychosocial gains may help REU coordinators and REU mentors to re-design
their undergraduate research program and provide the support that their
mentees’ needed to have a positive undergraduate research experience.

††preprint: APS/123-QED

##  I Introduction

Undergraduate research is an important experience that affects student
education and career development [1, 2, 3, 4, 5]. Across many studies,
undergraduate research programs that include in-person interactions between
the research mentor and mentee, access to physical equipment, and supportive
lab environments have been documented as beneficial [5, 6, 7]. Substantial
prior research has examined the influence of in-person undergraduate research
experiences (UREs) on promoting outcomes, such as retention [8, 9, 10, 11, 12,
13, 14, 15, 16, 17, 18], and persistence [19, 20, 21, 22] in STEM career
pathways. For instance, Bauer and Bennett [13] contend that undergraduate
research may help mentees clarify future career goals and are more likely to
attend graduate school.

Beyond the benefits of undergraduate research programs to help students to
clarify career goals [15, 18, 17], facilitate their research-based skill
development [23, 24, 25, 15], learn a wide variety of content knowledge [24,
26], and improve their critical thinking skills [27], these research programs
can also help students to enhance psychosocial gains, such as increased self-
confidence [18, 15, 17, 28], communication skills [24, 17], identity [29, 30,
31], and sense of belonging (SoB) to their research community [29, 32, 33,
34]. For instance, Seymour et al. conducted interviews with students at end of
one undergraduate research program to understand the benefits of engaging in
research [18]. Based on their report, almost all students gained more
confidence in doing research and presenting their work. These students also
reported greater gains in career-related decisions and understanding the
research process. The primary purpose of the U.S. National Science Foundation
(NSF) Research Experience for Undergraduates (REU) program is to immerse
students in a research environment and prepare them for future careers in
research [35]. Each REU site recruits 10-16 diverse U.S. scholars from across
the country for 8-10 weeks of residential research opportunities each summer.
Students become members of a research group and learn a variety of practical,
academical, and professional skills. Additionally, they may attend research
seminars, professional development workshops, and social events organized for
REU students.

There are several limitations in existing studies of undergraduate research
experiences. For instance, these studies are mostly derived from a combination
of self-reported surveys, one single follow-up survey, end-of-program formal
evaluation [14, 36, 29, 23], and multiple interviews [17, 18]. Hunter et al.
[17] and Seymour et al. [18], both collected data through in-depth interviews
three times over the 3 years of the study (summer 2000, before their
graduation in spring 2001, and a third time as graduates in 2003-2004). In
addition, they interviewed faculty advisors once in the summer of 2000. In
Seymour et al.’s work students commented on a checklist of possible benefits
derived from the previous literature as part of their protocol. Additionally,
while a variety of research studies have explored the positive benefits of in-
person undergraduate research experiences, very little research has yet
focused on remote research experiences [37]. We tried to fill a gap in prior
research by conducting longitudinal in-depth interviews with both mentors and
mentees throughout the duration of the remote REU program and after it
finished. The purpose of this paper is to present mentees’ experience within
the remote REU program that impacts psychosocial gains including their SoB,
self-efficacy, and physics, and researcher identity. This study is a part of a
larger analysis that describes a holistic view of the mentees’ remote REU
experience over a 10-week duration and its impact on learning outcomes and
career options.

##  II Literature review around psychosocial gains

Some literature identifies that in-person undergraduate research experiences
positively affect students’ psychosocial gains such as sense of belonging
(SoB) [38, 39, 40], self-efficacy [41, 30], and identity [38, 42, 43, 30]. The
following section describes some studies on these three constructs that we
explored in our interviews to understand how previous literature has framed
them.

###  II.1 Sense of belonging (SoB)

The literature on SoB contains two key aspects that influence students’ SoB.
First, some literature identifies the students’ perception of social
acceptance (other-recognition), which refers to the feeling of connectedness,
being accepted, cared, and valued within different contexts (e.g., peers,
professors, and community of scientists) [44, 45, 46, 47]. Thomas and
Galambos’s study revealed that having a supportive environment is necessary
for positive student outcomes [48]. Their study revealed that high-quality
interaction between students and professors can create greater overall
satisfaction which contributes to more SoB, which is then related to other
outcomes such as retention. Although some studies has been focused on
students’ feeling of belonging to various academic contexts and its
relationship to academic achievement (retention within the education system)
[49, 45, 50, 51, 32, 48], other studies have been focused on students’ SoB
through self-recognition (e.g., personal traits, self-worth) [52, 44, 53, 54].
This self-recognition aspect of SoB, although less obvious, involves students’
self-perception around their beliefs and emotions and the expectancy of
success within a context, which could lead them to greater academic
satisfaction (self-recognition). The SoB in college contexts is often referred
to as a dynamic construct, which can change over time. However, many studies
focus on a growth in students’ “general SoB” and their overall college
experiences [55, 26, 56, 57]. For instance, Hurtado and Ponjuan [56] collected
two sets of longitudinal data from students in ethnic minority groups on nine
college campuses to understand the general college experience (e.g.,
experiencing discrimination) as a predictor of students’ SoB.

In times such as summer 2020 where the whole world was in a state of lockdown
due to COVID-19, many students were isolated in their own homes and
experienced the REU program remotely. Thus, feeling belonging may have been a
particularly challenging outcome for those who were doing research in a remote
format. Part of the present study was designed to explain how REU
participants’ SoB to multi-space research contexts (e.g., REU lab group,
researcher community) would change while there was no in-person contact with
the other members of the community.

###  II.2 Self-efficacy

Bandura’s social cognitive theory [58, 59] asserts that self-efficacy is a set
of beliefs about an individual’s own capacity that impact an individual’s
choices and the effort that they put forth to complete a task and accomplish
goals. He argued that self-efficacy is influenced by different factors, such
as personal performance experiences, vicarious experiences, social persuasion,
and physiological and affective states. To gain a better understanding of
students’ self-efficacy, a considerable number of studies have investigated
students’ beliefs in their own ability to succeed [60, 61, 62, 63] and the
role of higher self-efficacy in regarding future career choices [64, 65, 66,
67, 68, 69, 70, 71, 72, 73, 74]. Self-efficacy is a dynamic cognitive process
that is likely to change and develop during skill development opportunities
[60, 61, 62]. A broad range of benefits stemming from undergraduate research
including boosting self-efficacy [75, 6, 76, 24] which support mentees’ self-
motivation to believe that they can do research. Part of the present study was
designed to observe changes in self-efficacy as a result of this remote REU
program.

###  II.3 Identity

Identity and SoB can be seen as part of the same psychosocial concepts’
family. Gee [77] defines identity as self-recognition of being a certain “kind
of person” within the specific context. SoB, on the other hand, mean to feel a
sense of acceptance to someone or something such as a particular context [44,
45, 46]. Identity is context specific (e.g., a disciplinary identity). For
instance, science identity is how students may perceive themselves in science
specific context [78, 79, 80] (e.g.,“I am a science person”; adapted from
Hazari et al. [81]). Carlone and Johnson developed a grounded model for
science identity which includes three interrelated dimensions: Competence,
performance, and recognition [82]. Their model assumes that one’s science
identity is influenced by one’s other identities (e.g., gender, racial, and
ethnic). Hazari et al. [81] used Carlone and Johnson’s framework [82] on
science identity in order to develop a discipline-specific framework for
physics identity. Their framework is composed of four fundamental constructs:
performance, perceptions of competency, perceptions of others, and interest,
all of which influence a physics identity. However, Carlone and Johnson’s
framework, which omits interest, was based on research with women scientists
who already had a prior interest in science careers.

A number of studies using survey and interview data found that students who
experienced undergraduate research programs were more likely to feel stronger
disciplinary identity (e.g., a member of the STEM community and a researcher)
and to persist in STEM fields [81, 83, 84, 85, 86, 87, 88, 11, 89]. For such a
reason, in this study we intended to identify mentees’ experiences that may
impact the development of their physics and researcher identity during the
remote REU program.

##  III Methods and context

We emailed the coordinators of 64 physics REU programs and asked if their REU
would be taking place in a remote format in summer 2020. Our respondents
provided us with 3 varieties of answers; $N$=8 hosted a remote REU, $N$=18
were canceled, and $N$=38 provided no response and the status of their REU
program was unclear from their website. To recruit students, we sent an email
to the REU coordinator to forward to their REU students through their program,
and then after students volunteered to participate in this study, we sent an
email to their paired mentors. We conducted interviews with paired
participants; mentees ($N$=10) and mentors ($N$=8) from six different REU
programs. Approval of the study through our institutional Human Subjects
Research Office was granted before recruitment and data collection began. In
addition, the consent form that was signed by all participants included
information about the study purposes and a request for permission of video
recording. All the participants were given the right to terminate their
participation at any point. Participation incentives were offered in the form
of $20 gift cards for each interview that were sent weekly. Our sample of
mentees were gender and ethnically diverse. Just over half 60% ($N$=6) of
mentees were men, and 40% ($N$=4) were women. The population of mentees
discussed here was 50% ($N$=5) white, 30% ($N$=3) Asian, and 20% ($N$=2)
identified as two-or-more races/ethnic groups. Two mentees were not U.S.
citizens and were supported financially from sources other than the REU
program. Table 1 summarizes the mentees demographics characteristics and Table
2 briefly outlines their projects. All mentees were physics majors, and many
had double majors that showed their broad range of interests, such as Spanish,
music, computer science, and math.

Table 1: Mentees population (N=10)

.  Categories N=10 Gender Women 4 Men 6 White 5 Race/Ethnicity Asian 3 More
than one 2 Rising Senior 7 Year of college Rising Junior 2 Rising Sophomore 1
Doctoral Universities 4 Type of home Master’s Colleges and 2 institution
Universities Baccalaureate Colleges 4

Table 2: Projects’ characteristics. Name | Attribute | Description of project
---|---|---
Andrew | Simulation | Characterize the efficiency of a detector, learn about details of nuclear reactions simulations,
|  | and refine them
Bruce | Computational | Numerically model quantum optical devices, learn PyBoard coding, construct a circuit with
| and experimental | equipment that was shipped to his home
Caleb | Computational | Look at the atomic structure and different spectra to make a model for electrical conductivity
|  | and glass transition temperature
David | Simulation | Make a basic resonator model to learn the modeling program and then make an acoustic model
|  | of a reed instrument
Emma | Experimental | Began physics education research and transitioned to a second project building a 3D-printed
|  | particle trap using parts that shipped to her home to make circuits
Freida | Computational | Use models in high-energy physics to predict the probability of decay modes in collisions
Grace | Simulation | Learn density functional theory, model certain molecules, and look at the dynamics of the system
Helen | Simulation | Simulate the decay processes of short lived isotopes
Ivan | Simulation | Learn about CMS and LHC and use simulations and experimental data to refine the codes for
|  | detection of the charged particles in a large collider experiment
Joshua | Computational | Learn neutron mirror model and add new equations into the old code to solve problems related
|  | to nuclear physics

All the mentors were men ($N$=8). They were professors at diverse
institutions, including 75% ($N$=6) from institutions granting Doctoral
degrees and 25% ($N$=2) from institutions granting Bachelor’s degrees. There
was only one paired mentor who was a woman and she did not respond to the
invitation to participate in this study. It is possible that the lack of women
mentors in our small sample is due to the fact that only about 20% of US
physics faculty are women. However, another contributor could have been the
COVID-19 pandemic, which impacted every aspect of life and added many
challenges, particularly for women who traditionally have taken on more care-
giving responsibilities on top of their work responsibilities. To explore the
participants’ perspectives in-depth, mentees were individually interviewed at
six points throughout the REU program and one time after it finished
(interview nine), and mentors were interviewed two times during the REU
program and once after it finished (interview ten). Figure 1 shows the main
topics of the interviews for each week. We used a semi-structured interview
format [90, 91, 92], relying on the set of planned questions that revolved
around the research experience. We video-recorded all interviews via Zoom with
the permission of interviewees. Mentor interviews took between 30 and 45
minutes, while mentee interviews took between 60 and 90 minutes. Overall, 94
interviews were conducted over summer 2020.

Figure 1: Overview of Interview Study Design and Protocol Content.

Each interview was recorded and auto-transcribed within Zoom. After all of the
interviews were completed, we corrected the automated transcription errors and
the reformatted transcripts became the focus of our phenomenographic analysis
[90]. Analysis of the transcripts were executed using Dedoose software [93].
We began the data analysis process by reading and carefully re-reading each
transcript to become familiar with the data. Once we immersed ourselves in the
data, we broke down the transcripts according to our research questions. We
then looked at the different sections of the transcripts and created the
initial major categories around three psychosocial constructs (e.g., SoB,
self-efficacy, and identity). The initial major categories for each constructs
mostly came from our interview protocol, while sub-categories emerged from the
participants’ narratives to identify variation in the statements about the
mentees’ psychosocial gains. In our data analysis, we focused on growth during
the REU program. We grouped mentees’ responses into different levels based on
their wording and the context of the quotes (e.g., lower-level). Part of this
process includes the development of Fig. 2-4 on each construct and then Fig.
5, which consolidates all the constructs together. Overall, a total of 1397
segments were coded including mentees’ quotes around SoB ($N$=46), Self-
efficacy ($N$=32), and identity ($N$=76) and many additional indirect
references to those same constructs.

##  IV Results

This section presents the findings that emerged related to developing an
understanding of mentees’ psychosocial gains. Each subsection focuses on a
specific psychosocial construct, beginning with sense of belonging.

###  IV.1 Sense of belonging

Figure 2: Process of growth in mentees’ sense of belonging as a function of
time. The areas shaded in lighter color denote the lower-level of SoB while
the areas shaded in darker color denote the higher-level of SoB. The text in
the boxes are paraphrases of mentees’ quotes.

Gains in the SoB construct describe growth in mentees’ feeling of acceptance
by their research group, which is distinct from feelings of acceptance by the
broader scientific community or research community. We asked questions around
mentees’ SoB several times throughout the REU program. Figure 2 shows the
process of growth in mentees’ SoB as a function of time. We use a gradient
scheme with lighter shades representing lower-level of SoB in the bottom rows
of the figure and darker shades indicating a higher-level of SoB. The text in
the boxes are paraphrases of mentees’ quotes.

Our analysis revealed that one way that mentees’ experienced a SoB was by
having “communication and social interactions” with other members of their lab
community and other REU student (Fig. 2, left side). During interviews three
and four, some mentees felt connected and valued because their mentors
provided a welcoming and supportive group environment for them. Grace, who
worked with other two REU students and had a couple of weekly meetings with
her mentor and other lab members, said “Everyone is very success-oriented.
They want to know how you’re doing. They want to know where you are in your
work. They ask personal questions too; How are you? What have you done today?
Are you guys being active? Like making sure you’re okay to feel like it’s a
very welcoming community.”

Some other mentees expressed a higher-level of SoB during interviews three and
four. David said, “I feel like a part of the group. My mentor does a good job
of making sure everyone feels included in each of the meetings.” He felt
accepted and welcomed from his research lab community. Andrew commented,
“We’ve been engaging more in communication like Slack and stuff. I didn’t
really feel that way at the beginning, I felt really detached, but now that
we’re getting more comfortable with what we’re doing, we are communicating
more.” Bruce and Helen both did not feel belonging during week three because
of a lack of community and social interactions. During interview nine, Andrew
expressed high-level SoB, noting that, “the sort of dynamic we had over just
discussing things back and forth throughout the day kind of made me more
sociable, I wasn’t worried about getting on a Zoom call with them and talking
to them and like discussing things and helping someone if they had like a
problem with their code or something. So I think it advanced me.”

Another way for mentees to feel belonging to their lab community was through
making “scientific contributions” and recognition for those scientific
contributions (Fig. 2, right side). During interview three Freida said, “I
think that I need to prove to myself that I can do it and that I belong in
this community. I think that they help just by being there, by helping me
succeed, and by supporting me. Like by teaching me and I think that something
that they do really well is they don’t like talk down to me. They understand
that I don’t know because I’ve never learned, not because I can’t learn. And
that’s such an important thing for education everywhere. I think making me
feel like I can do it.” David felt high-level SoB since, “I’m actually
contributing to the group and I understand what’s going on across the group’s
projects.” Interestingly, Helen who did not feel belonging in interview three,
by interview four began feeling like a member of her research lab and part of
the team, “Because when my mentor asked me to talk more about what I was doing
for a part of the project at our research meeting today, it was more valuable,
like my mentor wants me to contribute what I’ve been doing and inform the
group on the process and the progress that we’ve made in this part of it. Then
be able to ask everyone else questions about what I’m doing too and receive
their feedback.” Again, Bruce did not feel SoB in his research group. He said,
“I don’t really feel like I’ve [earned?] my place yet. I think I would have to
get some results to be a winner in my place.” One possible explanation is that
he mentioned in the first week feeling imposter syndrome, while another
explanation is he did not have enough scientific contribution and did not
produce enough results to feel as though he belonged to his lab group.

During interview nine, which took place after the REU program finished, almost
all mentees reported a higher-level of SoB than they felt at the beginning of
the summer. For instance, David thought his REU experience over the summer
definitely helped him to feel like a part of the bigger physics community. He
said, “Just because I’ve been able to experience and contributed as well as
being able to look through the work that other people have done in the field a
lot more in-depth. And having done my own little bit of work in the field
helps me to understand better the work that others have done in it.” He added,
“Even now that I’m not doing that research full time. I feel much more in part
of that than I did back when I was only taking classes before.” Similarly,
Andrew felt stronger SoB, “Because I’m active in asking questions and trying
to answer what I can and engaging in our discussions.” Freida felt she had
more knowledge about physics and research, and also felt she had made a
scientific contribution. She said, “I definitely consider myself part of the
physics community. Probably a lot more than I did at the beginning, because
now I feel like I know a lot about it and I’ve gotten a lot done and I’m
informed and I just feel more in it, like I’m more submerged in it personally.
Maybe you could say my physics identity has gone from amateur to beginner, to
intermediate now. I feel like I’m actually part of the field.” Both Helen and
Bruce felt more belonging after the program finished and they came back on
their campus and communicating with friends and professors.

In summary, findings from analyzing the SoB construct indicate that SoB grows
out of community and social interactions (e.g., feeling welcomed by the
research lab, having an approachable mentor) and scientific contribution
(e.g., producing new results, asking questions, getting more knowledge about
physics).

###  IV.2 Physics and research related self-efficacy

Figure 3: A representation of the growth in mentees’ self-efficacy. The areas
shaded in lighter color denote the lower-level of self-efficacy while the
areas shaded in darker color denote the higher-level of self-efficacy. The
text in the boxes are paraphrases of mentees’ quotes.

Mentees were asked about their confidence around doing research in physics at
two points throughout the REU program (interviews six and nine). Figure 3
briefly illustrates changes in mentees’ confidence between interview six and
interview nine. During interview six, we asked mentees to rate their self-
efficacy within their current project, supposing zero is “not at all
confident” and 10 is “very confident” and also how they would characterize
their self-efficacy beyond the scope of the REU program. Lower-level self-
efficacy (any score of 7 or less) is associated with mentees feeling less
comfortable in achieving all their projects’ goals and doing research
independently. Higher-level self-efficacy (any score of 8 or higher) is
associated with positive changes in mentees’ ability to do research, to
produce a positive project outcome, and to feel more comfortable with doing
research. Within their current REU project, seven out of ten mentees felt very
confident around achieving their project’s goals and rated between 8-10. For
instance, David said, “I’d say probably an eight or nine. I’m fairly confident
that I’ll be able to get at least my goal out of it. I’m not 100% sure if
we’ll be able to complete every all of our goals for the research. But we’ve
gotten some interesting results so far. I feel we’ll be able to extend those
results towards our eventual goal. I think part of the confidence also comes
from the fact that since it’s remote, there’s not as hard of the end deadline.
I’ll have to do end presentation at the end of the summer, but, if there’s
still work that we want to do, I’ll be able to keep in communication with my
professor and keep doing the research.” For David having a scientific
contribution and producing potentially new results for his project contributed
to his self-efficacy. Three other mentees rated their confidence between 5-8.
For instance, Andrew believed he was on the right track to achieve the goals
but hadn’t gotten there yet. He said, “The way I’ve been putting data and
looking at it. I think I’ve come up with some good results already. It’s just
a matter of buckling down and doing a one big final trial and getting the
final result out of it, instead of just the smaller trials and sort of seeing
like a general trend.” Besides, during the same interview, mentees talked
about their self-efficacy for broader pursuits in doing research alone after
the REU program finished. Six mentees stated that they felt pretty independent
and were able to direct their project, but they still preferred to work with
other researchers. Helen said, “I feel more confident since I’ve worked so
independently (due to the remote REU format) but it’s definitely nice to have
a mentor to direct you.”

During interview nine after the REU program finished, several mentees noticed
improvements in their self-efficacy as a result of participating in the REU
program. There were no lower-level self-efficacy statements in interview nine.
For David, his confidence stemmed from getting more physics knowledge and
understanding his research group projects which might impact his future career
decisions. He said, “I was just trying to learn to catch up and to understand
what was going on, whereas after I’d been doing it for about a month or so, I
felt a lot more confident in what I’ve been able to learn and I had a better
understanding of what was going on. So I was able to contribute a lot more… I
feel a lot more comfortable than before, because I had a chance to work, even
if it’s just for summer full time doing physics. Since I enjoyed that, I feel
a lot more confident proceeding with that as a career goal.” Freida felt
confidence to learn about different sub-fields outside of her work hours now.
Helen said she had confidence to do independent research and “What being part
of a larger research group will look like when I’m thinking about grad
school.”

###  IV.3 Identity as a researcher and a physicist

Figure 4: Process of growth in mentees’ physics and researcher identity as a
function of time. The areas shaded in lighter color denote the lower-level of
identity while the areas shaded in darker color denote the higher-level of
identity.

In this study, belonging is focused on a sense of connectedness with the lab
group, while identity refers to mentees’ perception of self within the larger
physics or research community beyond their lab group. For most mentees,
developing a stronger physics and researcher identity followed after a
stronger SoB and stronger self-efficacy. We grouped mentees’ responses around
their physics and researcher identity into three categories (lower-level, mid-
level, and higher-level). Figure 4 shows the changes in mentees’ identity as a
function of time. We use different shades of color to represent various level
of identity (e.g.,lighter shades representing lower-level of identity). The
text in boxes were paraphrased from mentees’ quotes.

During the sixth week of interviews, mentees were asked to describe their
physics and researcher identity. Five mentees expressed a low-level of
researcher identity. They described themselves as a “not very productive
researcher” or said they would feel like a real physicist “maybe after doing
more research.” During interview six, Joshua did not recognize himself as a
researcher as a result of his research experience. He said, “I have to repeat
lots of things from former researchers.” Interestingly, by the final
interview, his identity had changed to a higher-level. He expressed that not
only he felt that he belongs to the physics community, but also he felt he
“was born for doing research.” He commented, “I think REU has taught me
something about confidence in doing research. I think the REU has made me
think of myself as a physicist.” There were growing evidences that the
relationship he had with his mentor (e.g., spoke the same language) impacted
how he recognized himself as a physicist.

During interview six, Freida believed that she could be “a physicist”, and
that she is “getting there”. She added, “not because I can’t be a physicist,
but because I’m not done with my training. I’m like a physics apprentice.” She
felt she had more knowledge about physics and her research group now. She
said, “I am a five. I’m trying to build my skills, and I believe that someday
I will be more, but right now, I’m still learning.” During interview nine, she
said, “I am much more interested in high energy physics, like keeping up with
news and things. I feel much more sure that graduate school is what I want to
do. I feel more informed about it and committed kind of to physics. I
definitely consider myself part of the physics community. Probably a lot more
than I did at the beginning. Because, now I feel I know a lot about it and
I’ve gotten a lot done and I’m informed and I just feel more in it, like I’m
more submerged in it personally, maybe you could say my physics identity has
gone from an amateur to a beginner, to intermediate. Now I feel like I’m
actually part of the field.” This example indicates how good REU experiences
form a higher-level of identity as a physicist and as a future researcher.
Specifically, we observe that more knowledge about physics, the research
group, and self-recognition related to a greater sense of being a member of
her lab group and the broader researcher community.

In addition, mentees who reported having good research performance and
competence experienced a higher-level of physics and researcher identity. For
instance, Emma described how her “comfort level” changed after she saw the
final virtual presentations. She said, “It made me feel a little bit more
confident and outgoing in the world of physics. Because there’s just kind of
this feeling of, well, we don’t really belong to physics, like I do physics
education stuff and not doing real research, but after this REU experience,…
it made me feel that I do have something to offer, and I do belong in the
physics field.”

During interview eight, mentees were asked to describe any changes around
their identity as a physicist or as a researcher. Three mentees said they
hadn’t noticed any changes since the last interview. Four mentees felt a
stronger identity. For instance, comments included, “The group treated me as a
researcher”, “Feel more comfortable in doing research as a career”, and “See
myself more like a physicist and a researcher.” For additional examples see
the blue parts in Fig. 4. Interestingly, Bruce, who was not sure about his
future career steps during the first weeks of the REU program, said he thinks
his focus has changed since he moved back to his home institution for Fall
classes. He said, “I’m a lot more focused now and I’m a lot more driven to do
the research. And because of that, I’m actually looking at different graduate
schools now in the process of applying to them.” It seems high-level identity
had a positive impact on his future career decision-making.

###  IV.4 Mentors’ perspectives on growth

To understand each mentee’s growth in light of their mentor’s perspective, we
asked mentors to explain their mentoring philosophy and how they relate to
mentees’ psychosocial constructs.

#### Giving mentees ownership and autonomy.

Five out of eight mentors explained that their mentoring philosophy includes
helping mentees increase their sense of ownership. Interestingly, sense of
ownership was also noted by mentees as a factor that linked to their SoB
through scientific contribution. Quotes included phrases such as, “get some
results” and “feel that what they are doing is important” to their research
group. To achieve the ownership construct, mentors gave mentees some autonomy
and also encouraged them to solve problems in different ways. One mentor said,
“I think really trying to have them put things together and describe how it’s
working. They feel that they have done it, and they can actually tell us in
detail about how things are operating and functioning and we’re not the ones
who always know at all. What is going to happen? So that way. I think that is
how one tries to impart that sense of ownership to students. I think because
they quickly become the experts compared to us.” Two mentors provided mentees
with the bigger picture of the project and their expectations. As Freida’s
mentor explained, “She had a project and it was very well specified that this
was her project. There was no one else working on it and again, I’m working
with graduate students and postdocs, I make sure they were not working on the
project. So it was hers and hers alone. We would sit down and talk about
things but if you run into problems, I would often help her move forward. But
I tried to always make sure to have her regurgitate things. It was her job to
take, understand, and do it herself.” Some mentors described how their mentees
experienced freedom and trust. Caleb’s mentor said, “By respecting and by
giving them great freedom, that is, say if he had ideas on how to do things I
would say. Do it and show me. I did not have to tell them this is how you do
step by step. In the beginning, yes. But it didn’t take long to get going on
his own and then come up with his own improvements and extensions and
research. And he was very good at that. So I certainly think that encouraged
him. I think he liked that. I think he liked the research.” This emphasizes
the way in which mentors can be open and honest with their mentees and build a
strong and trusting relationship.

#### Supporting mentees’ confidence and self-efficacy.

Several mentors described that promoting mentees’ confidence is part of their
mentoring philosophy.Confidence was a term used by most of the mentees and
mentors and is very close to the construct of self-efficacy in our study
design. For instance, David’s mentor said, “Part of my mentoring method is to
get students to be confident in themselves… Sometimes I want students to do
things that I can’t do, or I’m not really good at so that in itself gives them
a lot of independence, because then they are the expert in doing whatever and
I’m the person. So they’re kind of working with me as a real collaborator,
rather than just sort of an assistant. And then the other part of my mentoring
philosophy is to try to have kind of independence, where I have an idea where
this research is going to go. But sometimes the direction changes a little bit
when I see what the students can do and what really gets them interested in
what they’re good at. I think that’s always kind of fun because I always start
out to summer with an idea of what the project is going to be like, and
sometimes it turns out to be somewhat different.” This approach was a very
important part of supporting mentees’ meaningful contribution and recognition
which lead to establishing self-efficacy among mentees. It is worth
acknowledging how autonomy, self-efficacy and making scientific contributions
all interrelate in some mentors’ statements.

Some mentors explained how they helped mentees to become more connected to the
STEM community (e.g., professional presentations at the conferences) For
instance, Caleb’s mentor reached out to his colleagues and introduced Caleb to
the field.

##  V Discussion

Figure 5: Various factors contributing to boost SoB and self-efficacy and
respectively physics and research identity and SoB to the larger physics and
research community.

A key takeaway from our study is that there were multiple factors in the
remote REU program that impacted students’ psychosocial gains and possibly
their future career outcomes. For the purposes of conceptualizing this study,
we synthesize the findings from results section with the broader literature on
the positive impact of the undergraduate research programs on students’
outcomes (e.g., interest in the STEM field and career decision-making [94, 15,
95, 96, 97]). Figure 5 portrays an integrated model that shows how various
elements of the REU experience contributed to growth in mentees’ SoB, self-
efficacy, and leads to physics and research identity, which culminates in a
positive view of a research career and future career decisions.

The lack of any of these factors can usually result in a mediocre
undergraduate research experience. For instance, if mentees feel no freedom
and autonomy in their project (e.g., “just keep repeating from former
researcher”), they might interpret this as low recognition from the mentor,
which leads to feeling less trustworthy as a researcher and is unlikely to
build their identity as a researcher.

#### Self- and other-recognition.

Bandura’s theory of self-efficacy involves a set of beliefs about an
individual’s capability to perform the actions needed to attain a specified
level of performance [58]. This definition is aligned well with the component
of Fig. 5 which focuses on self-recognition and other-recognition. As bubbles
on the center of Fig. 5 shows, we find that SoB and self-efficacy changes with
mentees’ feeling of recognition by self and others. Furthermore these findings
are consistent with the evidence in multiple studies that found
performance/competence and other-recognition have a direct effect on physics
identity [81, 98]. First, this perspective gives a structure to know what
factors impact self-recognition construct. The left-upper bubbles in Fig. 5
feed directly into the self-recognition. As we discussed earlier in sense of
belonging section and in Fig. 2, mentees who had a lower-level of self-
recognition reported lower-level of SoB to their lab community. They mentioned
their need to produce potentially new results for their project to feel
stronger performance/competence, which is followed by a higher-lever of self-
recognition.

Many quotes showed that self-recognition was supported by gaining more
knowledge about physics and research groups, knowing about their strengths
(e.g., fast learner) and weaknesses (e.g., need more background knowledge),
and producing new results for their project. After they completed their
projects and final presentations, during interview nine, quotes included
phrases such as, “I feel like I understand a lot of the physics of what’s
going on”, “able to learn and understand the software and the simulation
relatively quickly”, and “able to present my results to people and show them,
hey, I did this over the summer… these are the results and this is how it’s
going to be helpful to my mentor and his lab group.”

In the context of our study, other-recognition refers to how mentors and other
members of the lab community recognize a mentee as a physics person and as a
researcher. Other-recognition was supported by conversations between the
mentee and other group members, which also built trust and a comfortable
communicating space between them. These are the bubbles on the right-upper
side of Fig. 5 that feed directly into the other-recognition. When Helen
talked about her lab community, she said, “They’re all willing to talk to me
about my project and research. They easily bounce ideas off each other and it
seemed their goal was building knowledge not who’s right, who’s wrong, and
whose name is on what thing.”

Another example is Joshua’s mentor who shared some computer code with him to
work on them. Joshua said, “I feel I am trusted, so I will try my best to
solve that problem… I see when a physicists revise the code, they will added
their name in the code. So when I finish debugging the code, I can add my name
on it and this can be occurs when I belong to the physics community.” Emma
said, “My mentor treated me like as equal which is kind of uncommon when I
found working with anyone with a doctorate degree when you’re an undergrad.
That was very nice. He also really trusted me to get things done, to have my
own ideas. He kind of believed in my ability to do any physics-related things
which is not super common when people see that I’m doing instructional
physics… He was just like, here’s the project. Let’s do it.” She added any
part that she wasn’t sure of, she asked and her mentor taught her. We have
pointed out that this is a good example of how other-recognition for
scientific contribution (Reliable and trustworthy) and self-recognition
(Produce potentially new results for their project) both can lead to a greater
SoB and self-efficacy and respectively physics and research identity.

During interviews with mentors, we classified their mentoring philosophy
responses into two categories:“Giving mentees ownership and autonomy” and
“Supporting mentees’ confidence and self-efficacy”, which interestingly fit
within the factors that mentees mentioned throughout their REU experience in
Fig. 5. As we discussed earlier in mentors’ perspectives on growth section,
mentors supported and respected mentees’ autonomy to guide and encourage them
enough with their project (e.g., increase their sense of ownership), which
also stems from their trust in mentees’ ability to do research (Other-
Recognition). On the other hand, from the mentees’ perspective, there was a
connection between establishing a trustful relationship and a higher-level of
other-recognition, which lead to a higher-level of SoB and self-efficacy.
Caleb described how his mentor trusted him to let him be an independent
researcher, “He just guides you enough that you’re on the right track, but
then gives you enough freedom to kind of explore and figure it out on your
own… You feel like a good sense of accomplishment, like, I really did this
instead of he’s telling me what to do the entire time.” According to mentors’
statements, giving mentees autonomy and involving them in scientific
contributions often occurred together and could lead to more self-efficacy and
SoB.

#### Supportive mentors and lab groups.

Our data showed that mentees’ SoB is fostered through a positive relationship
and interpersonal connection with a mentor. These are the bubbles on the left-
lower side in Fig. 5 that feed directly into the SoB, separate from other-
recognition and self-recognition. This result is consistent with evidence in
multiple studies that found mentor-mentee interaction were significantly
related to SoB [99, 26, 44, 56, 54]. For instance, Hoffman et at., found that
students who had more interactions with their mentors are more likely to
“count on” their mentor for support, guidance, and feedback which impacted
students’ SoB [99]. David described how his mentor’s attitude made him feel
belonging to the research community: “the way my mentor treats us, is he does
it in a very inclusive way, and in a way that he recognizes and supports the
accomplishments that we’ve been able to do.”

Our results suggest that approachable mentors with shared interests can help
mentees to feel more connected to the lab community. For instance, Emma, who
spent part of the summer working on a physics education research project, felt
she had a common interest with her mentor around “getting more people involved
in physics and helping people see that physics can be a path for them, even if
they come from a background with very little science education. I think that I
kind of connect more with my REU mentor than I did with my previous (in-
person) mentor who was also a little less approachable. He was a little more
intimidating and I don’t know if it’s just because he doesn’t talk as much.
But I’m like my REU mentor and I get along a lot better.” Bruce’s supportive
mentor helped him overcome his imposter syndrome and have an improved SoB in
the community. During the first weeks of the program, he was reluctant to ask
questions and share his results with his lab group due to “fear of looking
stupid.” By interview nine, he stated that after experiencing the REU program
and communicating with his mentor, “I’ve become more comfortable and… become
less stressed about interacting with the community… I was also able to ask
questions online, essentially, so I’ll be able to ask my mentors for help when
I got stuck on something, they were able to help me out a lot. They’re just
able to give advice that worked really well.” Besides, Bruce’s mentor added
that in order to provide a welcoming environment for REU students, he tried to
“reply back to him quickly.”

Some mentees stated that because of the friendly and welcoming lab group
community, they became more comfortable with the level of their knowledge
which led them to become involved in scientific work and made them feel a
higher-level of SoB. Quotes in this part included phrases such as, “feel open
and comfortable”, “willing to talk to me about my project”, “bounce ideas off
each other easily”, and “people in the group were all asking me questions, but
they were in such a friendly tone I felt very welcomed into the group.” For
these reasons, most mentees had a positive feeling about their remote lab
group community, both emotionally and practically. In particular, two mentees
spoke about how everyone in their group “wants them to succeed”. Grace said,
“They’re trying to help each other. It’s very collaborative, which I enjoy. It
feels a little bit more safe and open. I think it’s very success oriented.”
Such a supportive lab environment can provide a space with more “other-
recognition” particularly through scientific conversations.

#### Outcomes of improved SoB and self-efficacy.

Literature on Social Cognitive Career Theory (SCCT) seeks to understand how
career paths develop among students [100, 101]. Self-efficacy is one of the
constructs in the SCCT map that can predict the link between the higher self-
efficacy and career choices. It is assumed that students with a higher-level
of self-efficacy in a specific field are more likely to approach a particular
field. On the other hand, Hazari’s physics identity framework [81] asserts
that performance, competence, other-recognition, and interest impact students’
physics identity, which is a predictor of their future career decisions. In
our study (Fig. 5), as mentees’ self-efficacy and SoB increase, their physics
and researcher identity increases, which translates into a positive view of a
research career and their future career decisions. During the last interview,
a majority of mentees ($N$=8) said the REU experience helped them understand
the nature of research work and aided them in figuring out their interests.
For instance, Ivan said he is more interested in nuclear physics and astronomy
now, but “before the REU program, I did not even know what I was interested in
because there were too many fields.” Some mentees remarked that their
understanding of the research process was improved after the REU program. They
also believed this experience increased their readiness for future career
decision-making. Freida thought it was a positive experience because, “It
helps me figure out where my interests or figure out what type of research I
would be happiest doing and really reaffirm that this research is something
that I want to do. And it made me much more informed as to what kinds of
research options there are.” Andrew and Grace both said that the REU
experience has confirmed their interest in graduate school and further
research. Andrew said, “Because beforehand I was kind of like, I don’t know if
that’s gonna be a lifestyle I want to get into because I didn’t know. I
haven’t done research before so I didn’t really know if it was going to be
something I enjoyed or something I was going to absolutely hate. So after
doing that I was kinda like, yeah, I really enjoyed this. I can see myself
doing that.” In addition, Grace added “It opened some doors for me, some
thought doors and also doors to other people.”

It is also important to emphasize the shifts in the definition of a SoB from
the small REU lab group to the larger physics-research community as one of the
results of the good remote REU experience. For instance, Helen said, “I think
REU experience just gives me more confidence to know what being part of a
larger research group will look like when I’m thinking about grad school. I
can definitely like feel more confident because then I have something to
compare against or talk about and reference as an experience. I actually know
if I was to pursue that, what it would look like.”

#### Comparison between remote format and in-person format.

Despite the unusual format of a remote REU, most of the mentors and mentees
found this remote experience to be beneficial in the mentees’ psychosocial,
academic, and future career development. We found mentees developed a greater
SoB, self-efficacy, and identity after this experience, which is likely to
improve their academic performance and possibly enhance persistence in the
field of physics. These outcomes are consistent with evidence from multiple
studies of in-person research experiences that found higher gains in several
areas (e.g., self-efficacy or ability to work independently [29]).

In the early weeks of the program, several mentees with prior in-person
research experience expressed concern about the remote format and how it might
limit developing relationships with other REU students, lab members, and their
mentor. However, the findings indicate that a number of novel forms of
communication happened during the remote format to assuage these concerns
(e.g., use more instant communication such as Slack, text messaging, social
hours). Results from our study around how mentors can help the mentees to gain
more self-efficacy and feel a stronger SoB to the community are similar to
previous studies around the role of the mentor in students’ desired research
experience outcomes [102, 103, 104, 105].

One goal of any undergraduate research program is to develop mentees’ content
knowledge and research skills. Our longitudinal study found that many times
during interviews, mentees talked about learning more from the literature and
learning how to ask questions. Indeed, half of the mentors specifically
mentioned seeing growth in their mentee’s scientific skills. These outcomes
are consistent with several studies around in-person UREs that reported higher
gains in mentees’ research skills after partaking in an undergraduate research
program [6, 106, 107, 24, 108, 75, 15, 109, 29, 18]. Interestingly, several
mentees appreciated the remote format of the program (family time, flexible
work hours, being more independent researchers). For instance, having a
flexible work setting at home helped them to become more independent learners.
Both Helen and Grace learned a lot about how to be a remote independent
learners and how to monitor their own time as a beneficial outcome of the
remote REU experience. Grace said, “responsibility and time management into my
own hands, which I think is also very beneficial to learn. That is the upside
to the online. It’s a lot more personal time management, being able to sit
down and say okay, this is a workspace. I’m working right now. Because we’re
all sitting at home. I’m not leaving and going to an office desk.” These
outcomes are consistent with several studies that reported higher gains in
students’ research skills to work independently after the undergraduate
research program [6, 18, 15, 108].

Most mentees ($N$=9) gained clearer ideas about their future career goals.
Some of them reported that participating in this research program helped them
to gain insight into applying to graduate school. These outcomes are
consistent with evidence in multiple in-person studies that found higher gains
in students’ decisions around attending to graduate school after undergraduate
research experience [110, 111, 14, 107, 108, 15, 29, 112, 23, 113]. Some of
mentees mentioned that this remote research experience helped them to
understand the specific sub-field of physics and the area of research better
and gained interest in related careers which are similar to results of some
previous literature [114, 115, 116].

##  VI Conclusion and Implications

This qualitative study was designed to provide a perspective on psychosocial
and career decision outcomes of mentees who participated in a remote
undergraduate research program in summer 2020. We created a psychosocial
growth framework based on our findings from this study around sense of
belonging, self-efficacy, and identity and synthesize them with the broader
literature on the effect of research programs on students’ outcomes. This
model illustrates the important factors that contribute the most to students’
psychosocial growth and to a positive undergraduate research experience.

Overall, comparisons between the outcomes of these remote REUs with published
outcomes of in-person REUs revealed many similar benefits of undergraduate
research. While the number of students participating in undergraduate research
has increased [4, 112] there still may be significant barriers to access for
some students. REUs often require students to travel long distances, which may
pose significant challenges for students with geographic constraints, family
obligations, or health concerns that limit travel. Our study suggests that
remote REU programs could be considered as a means to expand access to
research experiences for some students even after COVID-19 restrictions are
lifted. It is crucial to think about how continuing some remote opportunities
could expand the demographic of students who would be able to participate and
experience the manifold benefits of undergraduate research.

###### Acknowledgements.

We would like to thank all the mentors and mentees who participated in this
study. This work was supported by the National Science Foundation under Grant
No. 1846321.

## References

  * Council [1999] N. R. Council,  _Transforming Undergraduate Education in Science, Mathematics, Engineering, and Technology_ (The National Academies Press, Washington, DC, 1999).
  * Council [2002] N. R. Council,  _Improving Undergraduate Instruction in Science, Technology, Engineering, and Mathematics: Report of a Workshop._ (The National Academies Press, Washington, DC, 2002).
  * Wenzel and Karukstis [2004] T. J. Wenzel and K. K. Karukstis, J. Chem. Educ. 81, 468 (2004), publisher: American Chemical Society.
  * Boyer [2003] E. Boyer,  _The Boyer Commission on Educating Undergraduates in the Research University, Reinventing Undergraduate Education: Three Years after the Boyer Report_ (Stony Brook, NY, 2003).
  * Kuh [2008] G. D. Kuh,  _High-Impact Educational Practices: What They Are, Who Has Access to Them, and Why They Matter_ (Association of American Colleges and Universities, Washington, D.C., 2008) oCLC: 423513415.
  * Mabrouk and Peters [2000] P. A. Mabrouk and K. Peters, Council on Undergraduate Research (CUR) Quarterly , 20 (2000).
  * Pascarella and Terenzini [2005] E. T. Pascarella and P. T. Terenzini,  _How College Affects Students: A Third Decade of Research_ , Vol. 2 (Jossey-Bass, San Francisco, CA, 2005).
  * Russell  _et al._ [2007a] S. H. Russell, M. P. Hancock, and J. McCullough, Science 316, 548 (2007a).
  * Cuthbert  _et al._ [2012] D. Cuthbert, D. Arunachalam, and D. Licina, Stud. High. Educ 37, 129 (2012), publisher: Taylor & Francis.
  * Nagda  _et al._ [1998] B. A. Nagda, S. R. Gregerman, J. Jonides, W. von Hippel, and J. S. Lerner, Rev. High. Ed. 22, 55 (1998).
  * Shanahan [2007] M. C. Shanahan,  _Playing the Role of a Science Student: Exploring the Meaning of the Science Student Role and its Relationship to Students’ Identification in Science_ , Ph.D. thesis, University of Toronto (Canada) (2007), iSSN: 9780494279359.
  * Rodenbusch  _et al._ [2016] S. E. Rodenbusch, P. R. Hernandez, S. L. Simmons, and E. L. Dolan, CBE Life Sci. Educ. 15 (2016).
  * Bauer and Bennett [2003] K. W. Bauer and J. S. Bennett, J. Higher. Educ. 74, 210 (2003), publisher: Routledge.
  * Hathaway  _et al._ [2002] R. S. Hathaway, B. R. A. Nagda, and S. R. Gregerman, J. Coll. Stud. Dev. 43, 614 (2002).
  * Lopatto [2004a] D. Lopatto, CBE Life Sci. Educ. 3, 270 (2004a).
  * Adedokun  _et al._ [2012] O. Adedokun, D. Zhang, L. Parker, A. Bessenbacher, A. Childress, and W. Burgess, J. Coll. Sci. Teach. 42, 82 (2012).
  * Hunter  _et al._ [2007] A. B. Hunter, S. L. Laursen, and E. Seymour, Sci. Educ. 91, 36 (2007).
  * Seymour  _et al._ [2004] E. Seymour, A.-B. Hunter, S. L. Laursen, and T. DeAntoni, Sci. Educ. 88, https://doi.org/10.1002/sce.10131 (2004).
  * Chemers  _et al._ [2011] M. M. Chemers, E. L. Zurbriggen, M. Syed, B. K. Goza, and S. Bearman, J. Soc. Issues 67, 469 (2011), publisher: Wiley-Blackwell.
  * Jones  _et al._ [2010] M. T. Jones, A. E. L. Barlow, and M. Villarejo, J. High. Educ. 81, 82 (2010).
  * Schultz  _et al._ [2011] P. W. Schultz, P. R. Hernandez, A. Woodcock, M. Estrada, R. C. Chance, M. Aguilar, and R. T. Serpe, Educ. Eval. Policy Anal. 33 (2011).
  * Pascarella  _et al._ [1996] E. T. Pascarella, E. J. Whitt, A. Nora, and M. Edison, J. Coll. Stud. Dev. 37, 182 (1996).
  * Russell  _et al._ [2007b] S. H. Russell, M. P. Hancock, and J. McCullough, Science 316, 548 (2007b), publisher: American Association for the Advancement of Science Section: Education Forum.
  * Kardash [2000] C. M. Kardash, J. Educ. Psychol. 92, 191 (2000), place: US Publisher: American Psychological Association.
  * Halstead [1997] J. A. Halstead, J. Chem. Educ. 74, 1390 (1997).
  * Johnson  _et al._ [2007] D. R. Johnson, P. Alvarez, S. Longerbeam, M. Soldner, K. K. Inkelas, J. B. Leonard, and H. Rowan-Kenyon, J. Coll. Stud. Dev. 48, 525 (2007), publisher: Johns Hopkins University Press.
  * Ahlm [1997] K. Ahlm, Council on Undergraduate Research (CUR) Quarterly , 70 (1997).
  * Creighton and John [2013] J. Creighton and J. John, J. Furth. High. Educ. 37, 750 (2013).
  * Lopatto [2007] D. Lopatto, CBE Life Sci. Educ. 6, 297 (2007).
  * Estrada  _et al._ [2011] M. Estrada, A. Woodcock, P. R. Hernandez, and P. W. Schultz, J. Educ. Psychol. 103, 206 (2011).
  * Graham  _et al._ [2013] M. J. Graham, J. Frederick, A. Byars-Winston, A. B. Hunter, and J. Handelsman, Science 341, 1455 (2013).
  * Hausmann  _et al._ [2007] L. R. M. Hausmann, J. W. Schofield, and R. L. Woods, Res. High. Educ. 48, 803 (2007).
  * Dolan and Johnson [2009] E. Dolan and D. Johnson, J. Sci. Educ. Technol. 18, 487 (2009).
  * Eagan  _et al._ [2013] M. K. Eagan, S. Hurtado, M. J. Chang, G. A. Garcia, F. A. Herrera, and J. C. Garibay, Am. Educ. Res. J. 50, 683 (2013).
  * NSF [1990] NSF,  _NSF’s Research Experiences for Undergraduates (REU) Program: An Assessment of the First Three Years. National Science Foundation Report_ (National Science Foundation, Washington, DC, 1990).
  * Blockus [2016] L. Blockus, Washington, DC: National Academies of Sciences, Engineering, and Medicine , 31 (2016).
  * Forrester [2021] N. Forrester, Nature (2021).
  * Johri and Olds [2011] A. Johri and B. M. Olds, J. Eng. Educ. 100 (2011).
  * Pryor  _et al._ [2007] J. H. Pryor, S. Hurtado, V. B. Saenz, J. L. Santos, and W. S. Korn,  _The American Freshman: Forty Year Trends_ (Higher Education Research Institute, Los Angeles, 2007).
  * Barker [2009] L. Barker, ACM Trans. Comput. Educ. 9, 1 (2009).
  * Healy and Rathbun [2013] N. Healy and L. C. Rathbun, in  _Paper Presented at the 120th ASEE Annual Conference_ , ASEE Annual Conference (Atlanta, GA., 2013).
  * Prior and Bilbro [2012] P. Prior and R. Bilbro, in  _University Writing: Selves and Texts in Academic Societies_ , Studies in Writing, edited by M. Castello and C. Donahue (Emerald, 2012) pp. 19–31.
  * Mendoza  _et al._ [2015] N. Mendoza, J. Richard, and T. D. Wickliff, in  _Paper Presented at the 7th First Year Engineering Experiences Conference_ (Roanoke, VA., 2015).
  * Strayhorn [2012] T. Strayhorn,  _College Students’ Sense of Bbelonging_ (Routledge, New York, NY, 2012).
  * Goodenow [1993a] C. Goodenow, Psychol. Sch. 30, 79 (1993a).
  * Baumeister and Leary [1995] R. F. Baumeister and M. R. Leary, Psychol. Bull. 117, 497 (1995), place: US Publisher: American Psychological Association.
  * Tovar and Simon [2010] E. Tovar and M. A. Simon, Meas. Eval. Couns. Dev. 43, 199 (2010), publisher: SAGE Publications Inc.
  * Thomas and Galambos [2004] E. H. Thomas and N. Galambos, Res. High. Educ. 45, 251 (2004).
  * Goodenow [1993b] C. Goodenow, J. Early Adolesc. 13, 21 (1993b), publisher: SAGE Publications Inc.
  * Goodenow and Grady [1993] C. Goodenow and K. E. Grady, J. Exp. Educ. 62, 60 (1993), place: US Publisher: Heldref Publications.
  * Voelkl [1997] K. E. Voelkl, Am. J. Educ. 105, 294 (1997).
  * Osterman [2000] K. F. Osterman, Rev. Educ. Res. 70, 323 (2000).
  * Pittman and Richmond [2008] L. Pittman and A. Richmond, J. Exp. Educ. 76, 343 (2008).
  * Freeman  _et al._ [2007] T. M. Freeman, L. H. Anderman, and J. M. Jensen, J. Exp. Educ. 75, 203 (2007), place: US Publisher: Heldref Publications.
  * Hurtado and Carter [1997] S. Hurtado and D. F. Carter, Sociol. Educ. 70, 324 (1997), place: US Publisher: American Sociological Assn.
  * Hurtado and Ponjuan [2005] S. Hurtado and L. Ponjuan, J. Hispanic. High. Educ. 4, 235 (2005).
  * Maestas  _et al._ [2007] R. Maestas, G. S. Vaquera, and L. M. Zehr, J. Hispanic. High. Educ. 6, 237 (2007), place: US Publisher: Sage Publications.
  * Bandura [1977] A. Bandura, Psychol. Rev. 84, 191 (1977).
  * Bandura [2001] A. Bandura, Annu. Rev. Psychol. 52, 1 (2001).
  * Bandura [1982] A. Bandura, Am. Psychol. 37, 122 (1982).
  * Bandura [1993] A. Bandura, Educ. Psychol. 28, 117 (1993).
  * Bandura [1994] A. Bandura, Encyclopedia of Human Behavior 4, 71 (1994).
  * Jinks and Morgan [1999] J. Jinks and V. Morgan, A Journal of Educational Strategies, Issues and Ideas 72, 224 (1999).
  * Lent  _et al._ [1984] R. W. Lent, S. D. Brown, and K. C. Larkin, J. Couns. Psychol. 31, 356 (1984).
  * Lent  _et al._ [1986] R. W. Lent, S. D. Brown, and K. C. Larkin, J. Couns. Psychol. 33, 265 (1986).
  * Lent  _et al._ [1996] R. W. Lent, S. D. Brown, and G. Hackett, in  _Career Choice and Development_ , 3rd (Jossey‐Bass, San Francisco, 1996) pp. 373–422.
  * Luzzo  _et al._ [1999] D. A. Luzzo, P. Hasper, K. A. Albert, M. A. Bibby, and E. A. Martinelli, J. Couns. Psychol. 46, 233 (1999), publisher: American Psychological Association.
  * Lent  _et al._ [2000] R. W. Lent, S. D. Brown, and G. Hackett, J. Couns. Psychol. 47, 36 (2000), place: US Publisher: American Psychological Association.
  * Zimmerman [2000] B. J. Zimmerman, Contemp. Educ. Psychol. 25, 82 (2000).
  * McGarty [2002] C. McGarty, in  _Stereotypes as Explanations: The Formation of Meaningful Beliefs about Social Groups_ (Cambridge University Press, New York, NY, US, 2002) pp. 16–37.
  * Dalgety and Coll [2006] J. Dalgety and R. K. Coll, Int. J. Sci. Math. Educ. 4, 97 (2006).
  * Lent and Brown [2006] R. W. Lent and S. D. Brown, J. Career Assess. 14, 12 (2006), publisher: SAGE Publications Inc.
  * Gibbons and Borders [2010] M. Gibbons and L. Borders, Prof. Sch. Couns. 13, 234 (2010).
  * Morales [2014] E. E. Morales, Int. J. High. Educ. 3, 92 (2014), publisher: Sciedu Press.
  * Lopatto [2003] D. Lopatto, Council on Undergraduate Research (CUR) 23, 139 (2003).
  * Gandara [1999] P. Gandara,  _Priming the Pump: Strategies for Increasing the Achievement of Underrepresented Minority Undergraduates_ (College Entrance Examination Board, New York, 1999).
  * Gee [2000] J. P. Gee, Rev. Res. Educ. , 99 (2000).
  * Brickhouse  _et al._ [2000] N. W. Brickhouse, P. Lowery, and K. Schultz, J. Res. Sci. Teach. 37, 441 (2000), place: US Publisher: John Wiley & Sons.
  * Brown [2004] B. A. Brown, J. Res. Sci. Teach. 41, 810 (2004), publisher: John Wiley & Sons, Inc.
  * Hazari  _et al._ [2013] Z. Hazari, P. M. Sadler, and G. Sonnert, J. Coll. Sci. Teach. 42, 82 (2013), publisher: National Science Teachers Association.
  * Hazari  _et al._ [2010] Z. Hazari, G. Sonnert, P. M. Sadler, and M.-C. Shanahan, J. Res. Sci. Teach. 47, 978 (2010).
  * Carlone and Johnson [2007] H. B. Carlone and A. Johnson, J. Res. Sci. Teach. 44, 1187 (2007).
  * Barton  _et al._ [2013] A. C. Barton, H. Kang, E. Tan, T. B. O’Neill, J. Bautista-Guerra, and C. Brecklin, Am. Educ. Res. J. 50, 37 (2013), publisher: American Educational Research Association.
  * Crowley  _et al._ [2015] K. Crowley, B. Barron, K. Knutson, and C. K. Martin, in  _Interest in Mathematics and Science Learning_ , edited by K. A. Renninger, M. Nieswandt, and S. Hidi (American Educational Research Association, (AERA), Washington, DC, 2015) pp. 297–313.
  * Harrison  _et al._ [2011] M. Harrison, D. Dunbar, L. Ratmansky, K. Boyd, and D. Lopatto, CBE Life Sci. Educ. 10, 279 (2011), publisher: American Society for Cell Biology (lse).
  * Ovink and Veazey [2011] S. M. Ovink and B. D. Veazey, Res. High. Educ. 52, 370 (2011).
  * Barton and Yang [2000] A. C. Barton and K. Yang, J. Res. Sci. Teach. 37, 871 (2000).
  * Chinn [2002] P. W. U. Chinn, J. Res. Sci. Teach. 39, 302 (2002).
  * Pierrakos  _et al._ [2009] O. Pierrakos, T. Beam, J. Constantz, A. Johri, and R. Anderson, in  _2009 39th IEEE Frontiers in Education Conference_ (2009) pp. 1–6.
  * Marton [1981] F. Marton, Instr. Sci. 10, 177 (1981).
  * Bowden  _et al._ [1992] J. Bowden, G. Dall’Alba, E. Martin, D. Laurillard, F. Marton, G. Masters, P. Ramsden, A. Stephanou, and E. Walsh, Am. J. Phys. 60, 262 (1992).
  * Marton [1994] F. Marton, International encyclopedia of education. London: Pergamon 8, 4424 (1994).
  * www.dedoose.com [2018] www.dedoose.com, Dedoose version 8.0.35, web application for managing, analyzing, and presenting qualitative and mixed method research data (2018).
  * Zydney  _et al._ [2002] A. L. Zydney, J. S. Bennett, A. Shahid, and K. W. Bauer, J. Eng. Educ. 91, 151 (2002).
  * Yaffe  _et al._ [2014] K. Yaffe, C. Bender, and L. Sechrest, J. Coll. Sci. Teach. 44, 25 (2014), publisher: National Science Teachers Association.
  * Junge  _et al._ [2010] B. Junge, C. Quinones, J. Kakietek, D. Teodorescu, and P. Marsteller, CBE Life Sci. Educ. 9, 119 (2010).
  * Carpi  _et al._ [2017] A. Carpi, D. M. Ronan, H. M. Falconer, and N. H. Lents, J. Res. Sci. Teach. 54, 169 (2017).
  * Hazari  _et al._ [2020] Z. Hazari, D. Chari, G. Potvin, and E. Brewe, J. Res. Sci. Teach. 57, 1583 (2020), publisher: John Wiley & Sons, Ltd.
  * Hoffman  _et al._ [2002] M. Hoffman, J. Richmond, J. Morrow, and K. Salomone, Journal of College Student Retention https://doi.org/10.2190/dryc-cxq9-jq8v-ht4v (2002).
  * Lent  _et al._ [1994] R. W. Lent, S. D. Brown, and G. Hackett, J. Vocat. Behav. 45, 79 (1994), place: Netherlands Publisher: Elsevier Science.
  * Lent  _et al._ [2002] R. W. Lent, S. D. Brown, and G. Hackett, in  _Career Choice and Development (4th Ed.)_ (2002) pp. 255–311.
  * Chapman [2003] D. W. Chapman, Chronicle of Higher Education 50 (2003), publisher: Chronicle of Higher Education, Inc.
  * Kinkead [2003] J. Kinkead, New Dir. Teach. Learn. , 5 (2003).
  * Merkel [2003] C. A. Merkel, New Dir. Teach. Learn. 2003, 39 (2003).
  * Paul [2006] E. L. Paul, Peer Review 8, 12 (2006), num Pages: 4 Place: Washington, United States Publisher: Association of American Colleges and Universities Section: Practice.
  * l. Gafney [2001] l. Gafney, Council on Undergraduate Research (CUR) Quarterly 21, 172 (2001).
  * Ward  _et al._ [2002] C. Ward, J. Bennett, and K. Webber, Retrieved January, 2, 2008. (2002).
  * Kardash  _et al._ [2008] C. M. Kardash, M. Wallace, and L. Blockus, in  _Creating Effective Undergraduate Research Programs in Science_ (Teachers College Press, New York, NY, US, 2008) pp. 191–205.
  * Lopatto [2004b] D. Lopatto, in  _Paper presented at the annual meeting of Project Kaleidoscope, Richmond, VA._ (2004).
  * Jonides  _et al._ [1992] J. Jonides, W. von Hippel, J. S. Lerner, and B. A. Nagda, in  _Paper Presented at the American Psychological Association Annual Meeting, Washington, DC._ (1992) p. 27.
  * Ishiyama [2002] J. Ishiyama, in  _Paper Presented at the Annual Meeting of the American Political Science Association_ (ERIC, 2002).
  * Russell [2006] S. H. Russell,  _Evaluation of NSF Support for Undergraduate Research Opportunities: Follow-up Survey of Undergraduate NSF Program Participants: Draft Final Report_ (SRI International, Arlington, VA, 2006).
  * Russell [2008] S. H. Russell, in  _Creating Effective Undergraduate Research Programs in Science_ (Teachers College Press, New York, US, 2008) pp. 53–80.
  * Campbell and Skoog [2004] A. Campbell and G. Skoog, J. Coll. Sci. Teach. 33, 24 (2004), publisher: National Science Teachers Association.
  * Campbell and Skoog [2008] A. Campbell and G. Skoog, in  _Creating Effective Undergraduate Research Programs in Science: The Transformation of Student to Scientist_ (Teachers College Press, New York, 2008) pp. 206–214.
  * Foertsch  _et al._ [1997] J. A. Foertsch, B. B. Alexander, and D. L. Penberthy, Madison, WI: The LEAD Center , 34 (1997).