2021

[1]Philipp Bitzenbauer

[1]Professur für Didaktik der Physik, Friedrich-Alexander-Universität
Erlangen-Nürnberg, Staudtstr. 7, 91058 Erlangen, Germany

2]Institut für Didaktik der Physik, Universität Leipzig, Prager Str. 36, 04317
Leipzig, Germany

3]Institut für Mathematik und Angewandte Informatik, Stiftung Universität
Hildesheim, Samelsonplatz 1, 31141 Hildesheim, Germany

4]Institut für Didaktik der Naturwissenschaften, Universität Bremen, Otto-
Hahn-Allee 1, 28359 Bremen, Germany

# (How) Do pre-service teachers use YouTube features in the selection of
instructional videos for physics teaching?

<EMAIL_ADDRESS>Tom Teußner  Joaquin Veith<EMAIL_ADDRESS>hildesheim.de Christoph Kulgemeyer<EMAIL_ADDRESS>*  [  [  [

###### Abstract

This mixed-methods study examines how pre-service teachers select
instructional videos on YouTube for physics teaching. It focuses on the role
of surface features that YouTube provides (e.g., likes, views, thumbnails) and
the comments underneath the videos in the decision-making process using videos
on quantum physics topics as an example. The study consists of two phases: In
phase 1, N = 24 (pre-service) physics teachers were randomly assigned to one
of three groups, each covering a different quantum topic (entanglement,
quantum tunnelling or quantum computing, respectively). From eight options
provided, they selected a suitable video for teaching while their eye
movements were tracked, and think-aloud data was collected. Phase 2 allowed
participants to freely choose one YouTube video on a second quantum topic
while thinking aloud. The results reveal a significant emphasis on video
thumbnails during selection, with over one-third of the fixation time directed
towards them. Think-aloud data confirms the importance of thumbnails in
decision-making. A detailed analysis identifies that participants did not rely
on (content-related) comments despite they have found to be significantly
correlated with the videos’ explaining quality. Instead, decisions were
influenced by surface features and pragmatic factors such as channel
familiarity. Retrospective reflections through a questionnaire support these
observations. Building on the existing empirical evidence, a decision tree is
proposed to help teachers identify high-quality videos considering duration,
likes, comments, and interactions. The decision tree can serve as a hypothesis
for future research and needs to be evaluated in terms of how it can help
systematize the process of selecting high-quality YouTube videos for science
teaching.

###### keywords:

Instructional videos, YouTube, Physics education, Quantum physics, Mixed-
Methods

##  1 Introduction

Prior research has highlighted the benefits of YouTube for learning (e.g., see
Rosenthal, 2018; Jackman, 2019), including increased engagement, better
comprehension, and the flexibility to control the learning experience (Jebe .,
2019; Kay, 2012; Stockwell ., 2015). Consequently, educators and teachers have
recognized the educational value of YouTube, employing it for instructional
purposes (Manca Ranieri, 2016; Jung Lee, 2015). One of such purposes lies in
the integration of YouTube explanatory videos into formal learning
environments.

In the last years, there has been extensive science education research on the
explaining quality of YouTube explanatory videos: Kulgemeyer (2020) developed
a comprehensive framework for effective explanatory videos, based on
guidelines published earlier in the literature (e.g., see Brame, 2016;
Findeisen ., 2019). For details we refer the reader to the research background
section 2 of this article. Further studies have investigated the relationship
between surface features, such as likes and views, and the explaining quality
(i.e., the instructional quality) of YouTube explanatory videos: These studies
have revealed that the surface features provided by YouTube may not serve as
reliable indicators of the explaining quality of a specific video, while a
statistically significant correlation was found between the number of content-
related comments and the explaining quality (Kulgemeyer Peters, 2016;
Bitzenbauer ., 2023). Based on the above findings, Bitzenbauer . (2023)
emphasize that it is crucial to support teachers “in selecting videos with
high explanation quality from the plethora of (online) resources” (p. 2)
through evidence-based selection criteria.

However, to date, there is a dearth of studies investigating the video
selection practices of (pre-service) teachers on YouTube, particularly
concerning their decision-making factors. It remains unclear whether teachers
rely on YouTube’s provided metrics, such as likes, views, or the age of the
video, or if they consider the comments section influential. This article
addresses this research gap by presenting the findings of a mixed-methods
study that explores the decision-making processes of (pre-service) physics
teachers when selecting instructional videos on YouTube to be included in
learning environments (see methods section 4). The study employs a combination
of eye-tracking, think-aloud interviews, and a retrospective questionnaire
survey to gain comprehensive insights into the thought processes and
strategies employed by pre-service teachers during the video selection
process.

##  2 Research background

Explanatory videos, also referred to as instructional videos, play a vital
role in science education research, for example serving as concise
introductions and explanations of specific topics of interest (Wolf Kratzer,
2015). Explanatory videos typically do not exceed 10 minutes in length and
have garnered increased attention in both formal and informal learning
environments, especially through platforms such as YouTube (e.g., see
Beautemps Bresges, 2021; Pattier, 2021).

###  2.1 Quality criteria of instructional YouTube videos

Recent scholarly investigations have focused on understanding the factors
contributing to the success and popularity of explanatory YouTube videos,
particularly in the field of science (Beautemps Bresges, 2021; Welbourne
Grant, 2016). Notably, the video structure has emerged as a crucial
determinant in this regard (Beautemps Bresges, 2021).

However, the primary objective of explanatory videos is to support student
learning, making the quality of explanations of utmost importance (Kulgemeyer
Wittwer, 2023; Pekdag Le Marechal, 2010). Researchers have explored various
frameworks and guidelines to enhance the effectiveness of explanatory videos.
For example, Kulgemeyer (2020) proposed a comprehensive framework for creating
effective explanatory videos that aligns with guidelines established earlier
by Brame (2016) and Findeisen . (2019). Furthermore, Kulgemeyer’s framework
incorporates insights from multimedia learning research and draws upon studies
related to instructional explanations conducted by Geelan (2012) and Wittwer
Renkl (2008). The framework encompasses seven factors comprising 14 features
that collectively influence the effectiveness of explanatory videos. These
factors include video structure, language-level adaptation, minimal
digressions, as well as consideration of prior knowledge, misconceptions, and
student interest (Kulgemeyer, 2020).

Kulgemeyer (2020) empirically tested the effectiveness of the framework by
comparing student achievement when exposed to videos developed in accordance
with the framework against those that did not strictly adhere to the
guidelines. The results demonstrated that students exposed to videos closely
aligned with the framework exhibited significantly higher levels of
declarative knowledge in post-tests ($d=0.42$), although no statistically
significant difference was observed in post-test scores related to conceptual
knowledge.

The correlation between video metrics provided by YouTube, such as the number
of views or likes, and the videos’ explaining quality has yielded mixed
results: Kulgemeyer Peters (2016) conducted an exploratory study focusing on
instructional YouTube videos on mechanics topics and found that the number of
content-related comments posted by users below a specific video was the only
variable that correlated significantly with the explaining quality.
Conversely, the number of views, likes, and dislikes did not exhibit
significant correlations. Similar findings have been brought forth by Kocyigit
Akaltun (2019) who evaluated 53 online videos using the the Global Quality
Scale and found the YouTube metrics did not significantly differ across
quality groups. Bitzenbauer . (2023) conducted an additional exploratory study
that specifically examined explanatory YouTube videos on quantum topics,
namely quantum entanglement and quantum tunnelling. In contrast to earlier
findings, the authors observed a small but significant correlation between the
number of likes and the quality of explanations in their sample of quantum
topic videos ($r=0.37$, $p<0.01$).

###  2.2 Selection processes of YouTube explanatory videos

The increasing abundance of low-quality educational content on YouTube has
become a matter of concern for researchers (e.g., see Tan, 2013; Neumann
Herodotou, 2020; Bohlin ., 2017) highlighting the crucial role that teachers
play in selecting explanatory videos of high quality (Jones Cuthrell, 2011;
Chtouki ., 2012). This issue is further exacerbated by the reliance on
popularity-based rankings in search systems: For instance, Chelaru . (2012)
observed that the top-10 videos in the YouTube search results received a
disproportionately higher number of views, likes, and comments. Additionally,
the study by Chavira . (2021) revealed that out of the ten most-viewed videos
analyzed in their study, only four were deemed satisfactory in terms of
quality.

Despite existing research on YouTube video selection, to the best of our
knowledge, no studies have specifically examined the process by which teachers
select videos from the list provided by YouTube based on search queries.
However, several studies have shed light on user behavior, indicating that
individuals often sequentially view the returned videos until they find one
that aligns with their needs (e.g., see Fyfield ., 2021; Tan Pearce, 2011).
The abundance of choices available on YouTube may contribute to choice
overload, making it challenging to identify high-quality content (Toffler,
1984). Choice overload describes the phenomenon of increased difficulty in
decision-making when faced with a large number of choices (Schwartz, 2016),
potentially resulting in decreased motivation to engage with individual
options (Iyengar Lepper, 2000).

Against this backdrop, it becomes even more apparent that it is essential to
support teachers in the process of selecting explanatory videos for classroom
practice. Two main measures have been at the center of the debate so far:

  1. 1.

Ranked lists of educational channels have been published to “help Internet
users to narrow down their search space by recommending channels” (Tadbier
Shoufan, 2021, p. 3079). However, “there is no reason to assume that the
extensive offer of ranked lists would not lead to choice overload” (Tadbier
Shoufan, 2021, p. 3079).

  2. 2.

To tackle these challenges, scholars have suggested the utilization of
decision-assistance tools like meta-search engines, which employ aggregation
techniques (Meng ., 2002; Dwork ., 2001; Haveliwala, 2002) as described by
Tadbier Shoufan (2021).

##  3 Research rationale

While we agree that pre-made lists or similar resources might not optimally
support (science) teachers in the process of selecting YouTube explanatory
videos for classroom practice, we believe that the existing empirical evidence
on the explaining quality of YouTube explanatory videos might indeed be useful
to facilitate the systematization of teachers’ decision-making process. As
sketched in section 2.1, several studies have brought forth hints for
instructional quality of online explanatory videos and might, hence, provide
evidence in this regard:

  * •

Bitzenbauer . (2023) found a statistically significant correlation ($r=0.46$,
$p<0.001$) between explaining quality and the number of content-related
comments in YouTube videos on quantum topics. Similarly, Kulgemeyer Peters
(2016) reported a significant correlation ($r=0.38$, $p<0.01$) between
explaining quality and the number of relevant comments for videos on Newton’s
third law and Kepler’s laws, respectively.

  * •

Bitzenbauer . (2023) discovered a significant correlation ($r=0.37$, $p<0.01$)
between the number of likes and video explaining quality.

  * •

It is important to exercise caution when considering additional metrics
provided by YouTube, such as the number of views, as previous research has not
found stable correlations with explaining quality.

Of course, reviewing comments under each video in search of high-quality
explanatory videos is practically unfeasible and time-consuming. Moreover,
based on the available evidence, it is challenging to determine a quantitative
threshold indicating an adequate number of content-related comments or likes.
Nonetheless, considering the current state of research, it appears feasible to
explore ideal (i.e., efficient) decision-making processes by systematically
analyzing the order in which the different criteria can be employed by
teachers: As a starting point for future research aimed at supporting teachers
in their decision-making processes when selecting YouTube explanatory videos
for science teaching, we propose the decision tree presented in Figure 1 as a
representation summarizing the hints of instructional quality of explanatory
videos according to the current state of research described above.

This decision tree suggests teachers to first ask quick initial questions
during their search process such as whether a given video has an appropriate
duration for classroom use or whether it has already received user likes. If
both of these surface-level criteria are positive, it recommends teachers to
explore the comments section. The presence of not only superficial but also
content- or video-related comments indicates cognitive stimulation of viewers,
as “videos that accumulate plenty of those relevant comments are more
successful in catching viewers’ attention as these videos might use either a
more stimulating explanation or the explanation delivered is considered as a
starting point for further learning progress” (Kulgemeyer Peters, 2016, p.
12). Moreover, if there are even interactions among users, including responses
to content-related comments, this may provide additional evidence of a high-
quality video. Finally, teachers are then encouraged to assess the
instructional quality of the video by viewing it themselves.

Figure 1: Decision tree to support (pre-service) teachers selection process
when searching explanatory videos suitable for physics teaching as
hypothesized based on the current state of the literature. The proposed order
is not to be considered strict.

The suggested decision tree serves as a hypothesis for future studies
examining teachers’ selection processes of instructional videos for science
teaching as stated above. Due to the current lack of empirical studies on
(pre-service) physics teachers’ decision-making processes when choosing
YouTube explanatory videos for the physics classroom, we have two main
objectives in this article: First, we aim to explore how pre-service teachers
utilize YouTube metrics when selecting instructional videos for physics
teaching. Secondly, we intend to compare their selection and decision-making
processes with the procedure recommended by the existing literature, as
represented through the decision tree in this article.

Hence, with our research we aim to approach the following research question:
How – if at all – do prospective teachers use the features and comments
sections provided by YouTube when selecting YouTube explanatory videos for
teaching purposes?

We decided to address this question in the context of quantum physics YouTube
videos because (a) as mentioned above, there have been related studies
published previously we can build upon with our findings and (b) quantum
physics deals with difficult-to-grasp topics and different visualizations or
explanations are common to describe the same phenomena due to their abstract
nature. Thus, a highly varying degree of explaining quality is to be expected
when exploring explanatory videos for topics like quantum tunneling or quantum
entanglement.

##  4 Methods

###  4.1 Study design

We investigated our research question by conducting a mixed methods study
comprised of eye-tracking, think aloud interviews and a concluding
questionnaire. The selection of precisely these methods as well as their
interrelation will be explained more thoroughly in the following subsections
4.2.1-4.2.3. The mixed methods study consisted of three phases (P1 to P3, for
an overview see Figure 2):

  1. [ ]

  2. P1:

In the first phase, the participants were presented with a pre-constructed
image chart showing eight different YouTube video suggestions for a specific
topic via the original surface features provided by YouTube (e.g., thumbnail,
length, title, views, upload date, channel name, number of subscribers). As
additional information, we added the corresponding number of likes the videos
have already received (cf. Figure 4). The participants were then asked to
select one of the offered explanatory videos that they deemed suitable for use
with learners without prior knowledge in physics teaching on this topic. The
videos displayed in the chart, however, could not be opened or watched and,
instead, the selection had to be based solely on the provided features. In
addition to tracking the eye-movements during the selection process (cf.
Section 4.2.1), the participants were prompted to voice their thoughts at all
times in the sense of a think aloud interview (cf. Section 4.2.2).

  3. P2:

In the second phase, the image chart was removed and the participants were now
allowed to freely explore a previously opened YouTube search tab concerned
with a second specific quantum topic. Again, the task was to select one of the
videos suggested by YouTube for a hypothetical physics classroom lesson
covering the specific topic with learners without prior knowledge. In contrast
to the first phase, the participants’ eye-movement was no longer tracked but
they were now also allowed to open and watch the videos as well as scroll
through the comment section. This way, the selection process could place a
more pronounced focus on content related reasons to substantiate the rather
superficially guided process in phase 1. Similar to phase 1, all thought
processes had to be conveyed verbally at all times.

  4. P3:

After the initial combination of eye-tracking and think aloud, the study
concluded with a questionnaire given to students in retrospective which asked
the study participants to reflect on the importance of the different surface
features provided by YouTube in their selection processes (cf. Section 4.2.3).

To ensure that the results of our study are not directly dependent on (and
hence, restricted to) a specific (quantum) topic covered throughout the
phases, we randomly assigned each study participant to one of three different
groups A, B or C prior to starting the data collection. Each participant took
part in the study individually and was – depending on the group assignment –
given the task of selecting explanatory YouTube videos on two different
quantum topics (namely, either quantum tunneling, quantum entanglement, or
quantum computing) in study phases 1 and 2 as described above. A subsequent
one-way ANOVA comparing the different eye-tracking metrics investigated in
this study (cf. Section 4.4) across the three study groups revealed no
statistically significant differences among the groups. This indicates that
our results are not directly linked to a specific topic but it is sensible to
analyze the data collected in this study across the groups – we took advantage
of this observation in our study, as we analyzed the data from all 24
participants simultaneously, as if they had been collected under exactly the
same conditions. An overview of our study design is presented in Figure 2.

Figure 2: Study design visualized using a flowchart. The different topics
covered in the first two phases are color-coded and, as indicated, were
switched among the groups A to C between both phases.

###  4.2 Data collection

####  4.2.1 Eye-Tracking

Eye-tracking data was collected in study phase 1 using a stationary head-free
eye-tracking system from Tobii (Tobii Pro Fusion) alongside their respective
software (Tobii Pro Lab). The eye tracker operates at a sampling frequency of
60 Hz and a nominal spatial accuracy of $<0.3^{\circ}$ visual angle. The
stimuli were presented on a 24-inch computer screen (1920 $\times$ 1080 pixel
resolution and 60 Hz frame rate). Prior to the study, a nine point calibration
process was utilized to ensure accurate eye-tracking and the participants were
introduced to the basics of eye-tracking. The instructor verified the
agreement between the measured gaze positions and the actual points on the
screen. If the calibration results were not deemed satisfactory, the
calibration was repeated. In instances where the eye tracker failed to detect
sufficient calibration data, participants were repositioned in front of the
eye tracker. Additionally, potential factors that could have interfered with
pupil detection were examined. On average, the distance between each
participant and the tracker was 60 cm.

####  4.2.2 Think aloud

Since previous research has identified the need to complement eye-tracking
data with additional verbal data we amended the eye-tracking results by
incorporating think aloud interviews into our study design (Chien ., 2015;
Smith ., 2010). During think aloud interviews, “participants think out aloud
while performing a given task, or recall thoughts immediately following
completion of that task” (Eccles Arsal, 2017, p. 514). The participants’
verbalizations were recorded as well as transcribed and subsequently subjected
to further analysis (cf. Section 4.4). The goal of this method lies in
uncovering cognitive processes that are not as accessible with the other
methods used (Rios ., 2019). Thus, even though it might interfere with the
study objective due to the verbalizations resulting in an overall higher
cognitive demand, think aloud studies are often used as an introspective annex
(Sasaki, 2013; McKay, 2009). We leveraged this method by asking the
participants to articulate their thought processes at any point in time in
both, study phases 1 (image chart) and 2 (free exploration). To use thinking
aloud as effectively as possible in the study, the participants were provided
with an instruction on thinking aloud, which was formulated following
Mackensen-Friedrichs (2004) to ensure a standardized procedure (Sandmann,
2014). Some of the cues given to the participants were: (1) Speak your
thoughts aloud, (2) There should be no pauses in speaking, so verbalize your
thoughts without pauses, (3) Do not organize your thoughts before speaking,
imagine you are alone in the room, (4) Thinking aloud may be a bit unfamiliar.
Therefore, you will be supported and repeatedly prompted to express your
thoughts.

The additional verbal data obtained from those interviews provides further
insights into the cognitive processes, motivations, and decision-making that
underlie the observed eye movements, offering a more complete picture of
participants’ experiences and interpretations (cf. Brückner ., 2020).
Furthermore, eye-tracking data alone can identify moments of attention shifts
or fixations on specific elements, but it may not explain the reasons for
these shifts. Since our study addresses selection processes based on visual
stimuli, supplemental verbal data can clarify whether a shift in gaze was
triggered by interest, confusion, or any other factors, shedding further light
on the nature of the participants’ attentional patterns.

####  4.2.3 Questionnaire

To further enhance cross-validity we concluded our study with a final
questionnaire in phase 3. Here, participants were asked to rate whether they
considered the different (surface) features provided by YouTube important to
their decision-making processes on a four-point rating scale (strongly
disagree, disagree, agree, strongly agree). The addressed features were number
of views, likes, comments and subscriptions as well as thumbnail, channel,
video title, video length, video description, upload date, order determined by
YouTube’s search algorithm and specific comments. One the one hand, these
ratings allow to establish a ranking among all surface features in terms of
their importance. On the other hand, the retrospective view obtained from the
concluding questionnaire contrasts the introspective view from phases 1 and 2,
enabling a triangulation with the findings from both the eye-tracking and the
think aloud interviews.

###  4.3 Sample

A total of $N=24$ German pre-service physics teachers ($9$ female, $15$ male)
who were at least in their second year of study participated in this research.
None of the participants relied on strong glasses or contact lenses (diopter
$>1$). The participation in our study was voluntary, not financially
recompensed and informed consent was obtained from all participants.

###  4.4 Data analysis

The eye-tracking data were evaluated in terms of three metrics that reflect
attention allocation and cognitive demand: First, we analysed the total
fixation duration, which can be described as “the total duration of all
fixations on a specific stimulus” (van der Laan ., 2015, p. 1). High values of
this metric indicate a more pronounced focus on certain areas (Hahn Klein,
2022), thus it is the commonly reported measure of visual attention (Shruti
Goyal Miyapuram, 2015). Second, we investigated the metric fixation counts
that often accompanies the total fixation duration as a measure of attention
allocation (cf. Wang ., 2014; Just Carpenter, 1976). Lastly, we analysed the
mean fixation duration, which is often interpreted as a measure of cognitive
processing demand (Negi Mitra, 2020). Consequently, higher values of mean
fixation duration indicate a “higher cognitive effort to process information”
(Hahn Klein, 2022, p. 10). The areas of interest (AOIs) required for
quantitative analysis were matched with the surface features provided by
YouTube (cf. Section 4.2.1) as is indicated in Figure 3. However, for the data
analysis, the individual AOIs for each of the proposed videos shown in the
image chart in phase 1 were not considered: Instead, so-called aggregated tags
were created that combine the eye-tracking metrics for several related AOIs
(e.g., all Like-AOIs). For instance, we combined all the data from the Like-
AOIs using an aggregated tag “Likes”. This allowed us to analyze how
frequently and for how long participants viewed the number of likes across
different video options.

Figure 3: The AOIs were defined covering all the surface features given for
the eight video options shown to the participants. The eye-tracking metrics
regarding the related AOIs were summarized using aggregated tags as described
in the body text.

For the subsequent think aloud interviews we conducted a qualitative content
analysis. To this end, we (a) associated the participants’ verbal expressions
with the corresponding surface features provided by YouTube and (b)
categorized their decisions for or against each video. The categories for
these decisions were developed based on both inductive and deductive
procedures (Forman Damschroder, 2001). An overview of all categories and their
descriptions is provided in the appendix (cf. Table 5). The category system
was applied by two independent raters and dissenting judgements were resolved
through discussion. The interview data were analysed threefold: First, we
calculated the relative speaking time allocated to each (surface) feature and
visually displayed the resulting share of each feature in a bar chart. Second,
we visualized the temporal trajectory of each interview through the various
categories and plotted them alongside a common axis, resulting in a temporal
topography graphic for each of the two phases. Lastly, we counted the most
frequently used arguments among the participants’ reasonings and how often
they lead to a decision for or against a video. This insight were used to
provide an overview of the (surface) features provided by YouTube for each
video that are most influential during the decision-making process of pre-
service physics teachers.

The responses of the concluding questionnaire were summarized using a
diverging stacked bar chart are constructed from the participants’ responses
by aligning the bars from a stacked bar chart relative to the scale’s centre
(Robbins Heilberger, 2011). In addition, each of the response options was
color coded and equated with a number (strongly disagree $\widehat{=}\ -2$,
disagree $\widehat{=}\ -1$, agree $\widehat{=}\ 1$, agree completely
$\widehat{=}\ 2$) so that a mean agreement value for each surface feature
could be calculated, resulting in a ranking among all surface features
provided by YouTube (cf. Veith ., 2022).

##  5 Results

In the following, we present the results of our study, separated by
methodology. First, we provide an overview of the assessed eye-tracking
metrics (cf. Section 5.1) and second, we enrich those findings with the
results of the think-aloud data (cf. Section 5.2) as well as the subsequent
questionnaire study (cf. Section 5.3).

###  5.1 Eye-tracking results

The eye-tracking data were collected in study phase 1 where participants were
presented with a carefully constructed chart containing eight search results
from YouTube (for details see methods section 4). These options were
specifically chosen to exhibit a range of surface features. The participants’
task was to determine which of the eight explanatory videos would be suitable
for inclusion in a learning environment related to the topic being
investigated. Figure 4 presents an illustrative heat map generated from the
eye-tracking data obtained from one of the participants in the study. The heat
map provides visual information regarding the areas that received the highest
attention during the task.

Figure 4: Exemplary heat map created from the eye-tracking data from one of
the study participants in phase 1 of our study. Qualitatively, participants’
spots of attention are represented through red color. In the following, we
provide a quantitative analysis of the data underlying the participants’ eye
movements.

Table 1 provides the descriptive statistics for the metric total fixation
duration for each area of interest. With a mean percentage of 35.39% of the
total fixation duration, the Thumbnail was by far the most compelling AOI and
the only one with a share above 10%. While the AOIs Title (9.58%) and Channel
(6.17%) were also able to captivate the participants’ attention to some
extent, the remaining AOIs played a seemingly negligible role during the
selection process. This discrepancy is visualized via boxplots in Figure 5.

Table 1: Descriptive statistics for the total fixation duration (in %) for each area of interest. In addition to the minimum, maximum and mean values, we report the lower ($Q_{1}$), middle ($Q_{2}$) and upper ($Q_{3}$) quartiles. | Subs | Views | Thumbnail | Date | Length | Channel | Likes | Title
---|---|---|---|---|---|---|---|---
Max | 6.48 | 5.48 | 57.03 | 4.85 | 3.66 | 14.53 | 5.24 | 19.10
Min | 0.19 | 0.57 | 12.38 | 0.00 | 0.00 | 1.47 | 0.00 | 2.39
$Q_{1}$ | 1.28 | 1.73 | 29.84 | 0.29 | 0.46 | 4.73 | 0.32 | 6.69
$Q_{2}$ | 1.89 | 2.20 | 37.39 | 0.56 | 0.71 | 5.90 | 0.95 | 9.45
$Q_{3}$ | 3.61 | 3.72 | 41.73 | 0.94 | 1.50 | 7.84 | 1.79 | 12.39
Mean | 2.55 | 2.47 | 35.39 | 0.78 | 1.08 | 6.17 | 1.56 | 9.58
Figure 5: Boxplots for the total fixation duration for each area of interest.
The whiskers indicate $1.5\times\text{IQR}$, where IQR is the interquartile
range.

Descriptive statistics for the metric mean fixation duration are provided in
Table 2. The data in this regard paint a different picture by taking similar
values for each AOI. The mean fixation durations average between 176 ms (Subs)
and 240 ms (Date) and, thus, lie in the typical range of 100-600 ms reported
by Hahn Klein (2022). The boxplots illustrating these data substantiate this
relationship – the middle 75% of data can be located between 200ms and 300ms
across almost all AOIs (cf. Figure 6).

Table 2: Descriptive statistics for the mean fixation duration (in seconds) for each area of interest. In addition to the minimum, maximum and mean values, we report the lower ($Q_{1}$), middle ($Q_{2}$) and upper ($Q_{3}$) quartiles. | Subs | Views | Thumbnail | Date | Length | Channel | Likes | Title
---|---|---|---|---|---|---|---|---
Max | 0.31 | 0.41 | 0.31 | 0.31 | 0.44 | 0.35 | 0.37 | 0.32
Min | 0.16 | 0.14 | 0.16 | 0.08 | 0.09 | 0.13 | 0.13 | 0.13
$Q_{1}$ | 0.21 | 0.20 | 0.21 | 0.18 | 0.20 | 0.24 | 0.17 | 0.23
$Q_{2}$ | 0.23 | 0.23 | 0.25 | 0.22 | 0.23 | 0.26 | 0.21 | 0.25
$Q_{3}$ | 0.27 | 0.26 | 0.26 | 0.26 | 0.31 | 0.28 | 0.27 | 0.27
Mean | 0.17 | 0.19 | 0.21 | 0.24 | 0.18 | 0.23 | 0.21 | 0.23
Figure 6: Boxplots for the mean fixation duration for each area of interest.
The whiskers indicate $1.5\times\text{IQR}$, where IQR is the interquartile
range.

Lastly, we investigated the metric fixation counts for each AOI. With there
being striking differences in the total fixation durations of each AOI but
similar values for mean fixation duration, it becomes apparent that the number
of fixations must vary in a manner similar to the total fixation duration. The
data provided Table 3 paint a coherent picture in this regard. Again, with a
mean percentage of 45.59% of the total number of fixations, there is a
predominant focus on the AOI Thumbnail, with Title (11.60%) and Channel
(7.12%) being second and third, respectively. Consequently, the boxplots for
the metrics total fixation duration and fixation count are somewhat congruent
(cf. Figure 7).

Table 3: Descriptive statistics for the fixation count (in %) for each area of interest. In addition to the minimum, maximum and mean values, we report the lower ($Q_{1}$), middle ($Q_{2}$) and upper ($Q_{3}$) quartiles. | Subs | Views | Thumbnail | Date | Length | Channel | Likes | Title
---|---|---|---|---|---|---|---|---
Max | 6.50 | 5.34 | 69.30 | 5.20 | 2.93 | 14.93 | 5.60 | 22.63
Min | 0.37 | 0.94 | 18.34 | 0.00 | 0.00 | 2.60 | 0.00 | 5.14
$Q_{1}$ | 1.83 | 2.14 | 36.39 | 0.46 | 0.59 | 5.25 | 0.64 | 8.69
$Q_{2}$ | 2.29 | 3.04 | 47.29 | 0.93 | 1.04 | 6.55 | 1.00 | 11.29
$Q_{3}$ | 4.67 | 4.21 | 50.84 | 1.25 | 1.59 | 8.41 | 2.83 | 14.96
Mean | 3.04 | 3.11 | 45.59 | 1.06 | 1.12 | 7.12 | 1.82 | 11.60
Figure 7: Boxplots for the fixation counts for each area of interest. The
whiskers indicate $1.5\times\text{IQR}$, where IQR is the interquartile range.

###  5.2 Think-aloud interview results

To analyze the selection process more thoroughly, the results from the eye-
tracking study are now complemented with data from a think-aloud study, as
described in Section 4. Figure 8 offers an initial comprehensive overview of
the content aspects addressed in the argumentation provided by the study
participants: It shows the relative proportion (of total speaking time) of
each surface feature in participants’ utterances in both study phases. In
phase 1, the most pronounced focus was placed on the thumbnail with 30.9%,
followed by Channel and Title with 23.0% and 14.2%, respectively. Regarding
the free exploration in phase 2, however, the data convey a more
differentiated impression: Here, the surface feature channel emerges on the
top with 22.0%, with the thumbnail (20.0%) and length (17.4%) taking second
and third places. In addition, the surface features likes, subs, date, order,
description and comments are almost negligible with an allocated speaking time
of below 5% throughout both phases.

Figure 8: Bar chart visualizing the relative proportion (of total speaking
time) of each surface feature in participants’ utterances in both study
phases. Phase 1 indicates the allocated speaking time regarding the pre-
constructed image chart, while phase 2 (yellow) indicates the allocated
speaking time during free exploration (cf. Section 4).

A more comprehensive insight into the structure of the participants’ selection
process is offered by the bar charts in Figures 9 and 10.

Figure 9: Topography of the think-aloud interviews in the first phase, one for
each study participant P1 to P24. The red strokes indicate a decision for or
against a video. The upper row indicates the color coding for each surface
feature. White sections represent small breaks where the participants did not
address a specific surface feature. Figure 10: Topography of the think-aloud
interviews in the second phase, one for each study participant P1 to P24. The
red strokes indicate a decision for or against a video. The upper row
indicates the color coding for each surface feature. Hatched sections
represent parts of the interview where the participants watched a video.

Here, each individual interview is presented as a bar and the sections
dedicated to the different surface features are color coded respectively.
Hence, these visualizations allow for a deeper insight into the temporal
topography of each interview. Analyzing this topography, it becomes apparent
that blue (thumbnail) and violet (channel) cover the most area during the
first phase, in accordance with the findings presented in Figure 8. In the
second phase, where participants were allowed to click on and even watch
videos, this dynamic changes: On the one hand, the violet sections increase,
indicating a greater focus on the surface feature channel. On the other hand,
the participants had access to more surface features such as comments or the
video description. Decisions for or against a video are indicated by red
strokes. For example, a red stroke after a blue section displays a decision
against a video because of the thumbnail. A summary of all decisions and the
surface features they were based on is presented in Table 4.

Table 4: Overview of the decisions for or against a video based on the respective (surface) features, sorted by study phases.  Surface Feature |  Decision for (+) and against ($-$) a video in phase 1 |  Decision for (+) and against ($-$) a video in phase 2 |  Total decisions for both phases combined
---|---|---|---
| + | $-$ | + | $-$ |
Thumbnail | 3 | 12 | 2 | 9 |  26 (5+, 21$-$)
Length | 3 | 12 | 2 | 6 |  23 (5+, 18$-$)
Channel | 3 | 4 | 0 | 2 |  9 (3+, 6$-$)
Title | 1 | 4 | 1 | 3 |  9 (2+, 7$-$)
Views | 1 | 1 | 1 | 3 |  6 (2+, 4$-$)
Content | 0 | 1 | 2 | 0 |  3 (2+, 1$-$)
Likes | 1 | 1 | 0 | 0 |  2 (1+, 1$-$)
Subs | 0 | 1 | 0 | 0 |  1 (0+, 1$-$)
Description | 0 | 0 | 1 | 0 |  1(1+, 0$-$)
Order | 0 | 0 | 0 | 0 |  0
Date | 0 | 0 | 0 | 0 |  0
Comments | 0 | 0 | 0 | 0 |  0
Not related to a specific surface feature | 13 | 1 | 17 | 0 |  31 (30+, 1$-$)

With a total of 26 and 23 decisions, the surface features thumbnail and video
length are by far the most influential ones. In congruence to the bar chart
presented in Figure 8, the channel, the video title and the number of views
views can also be regarded guiding for the selection process, while more
specific features such as likes, subs or comments seem almost entirely
irrelevant for decision-making. Lastly, it is noticeable, that the most
decisions could not be attributed to a specific surface features. In
particular, 30 out of the 51 positive decisions do not seem to be related to
surface features provided by YouTube. We will elaborate on this finding in
more detail in the discussion section 6.

To obtain a more in-depth view on the decisions used by participants that
could be related to a surface feature, their arguments during the interviews
were categorized (cf. Section 4). Since the thumbnail feature lead to the most
decisions in both phases, we present a bar chart for the most frequently used
arguments in Figure 11.

Figure 11: Bar charts visualizing the relative share of each category (T1 to
T5) addressing the thumbnail that was used as an argument for or against a
video. The categories are summarized in the category system provided in the
appendix of this article (cf. Table 5).

With the most frequently used arguments being T3 (the thumbnail indicates an
interesting video) and T4 (the thumbnail indicates a boring or nonprofessional
video), it becomes apparent that arguments addressing affective aspects
dominate over content-related reasonings. An overview of the overall top 5
most frequently used arguments is provided in Figure 12. We discuss these
findings in section 6.

Figure 12: Top five of the most frequently used arguments for or against a
video during the think-aloud interviews. The respective category system is
provided in the appendix of this article (cf. Table 5).

###  5.3 Questionnaire results

In the final part of our study we, in retrospection, asked the participants to
evaluate the extent to which they agree with the respective features having
been relevant for their selection process. The responses are visualized using
a diverging stacked bar chart in Figure 13. In congruence to our previous
findings, the thumbnail takes a sole first place with a rating of 1.80 (where
2 corresponds to “agree completely” and $-2$ to “strongly disagree”). With 21
out of 24 participants agreeing completely with the thumbnail being important
for their selection process, the thumbnail even exceeds the video length in
terms of relevance for decision-making in the participants’ retrospective
views. In contrast, the number of comments (average rating $-1.88$) and
comments themselves (average rating $-1.52$) solidify last places and do not
seem to contribute meaningfully to the decision-making process.

Figure 13: Diverging stacked bar chart visualizing the participants’ agreement
that the respective surface feature is important for decision making. The
respective average ratings for each (surface) feature (cf. Section 4) are
provided in labels on each bar where 2 corresponds to “agree completely” and
$-2$ to “strongly disagree”. The abbreviation “nr. Comments” stands for the
number of comments under a video.

##  6 Discussion

###  6.1 The process of selecting instructional videos for physics teaching

In phase 1 of our study, pre-service physics teachers were given the task of
selecting one explanatory video on quantum physics from a set of eight
options. Participants were provided with excerpts from the YouTube search
results and had access to various metrics such as views, likes, and channel
information. The analysis of participants’ eye movements revealed a
significant emphasis on video thumbnails, with more than one-third of their
total fixation time and counts directed towards this area of interest (AOI).
Surprisingly, there were no statistically significant differences in mean
fixation duration between different AOIs, despite this measure “is often
considered an indicator of cognitive processing demand” (Hahn Klein, 2022, p.
10). This finding, hence, contrasts with the results of Hsieh Chen (2011), who
suggested that viewing content with different information types requires
varying cognitive resources.

To gain further insight into the participants’ selection processes, we
examined their think-aloud data. This analysis, consistent with the eye-
tracking data, revealed that during both phase 1 (selection of one out of
eight options based on surface features) and phase 2 (free exploration),
participants predominantly voiced their considerations in relation to the
thumbnail AOI. An in-depth categorization of argumentation structures and
decision-making uncovered four key observations:

  1. 1.

The video duration played a significant role in participants’ choices,
aligning with didactic perspectives as they were selecting videos for
instructional purposes.

  2. 2.

The video content had a minor influence on participants’ decisions during the
free exploration phase. Instead, choices were primarily guided by thumbnail,
duration, channel, and title features, indicating a reliance on surface
features and pragmatic decision-making among pre-service physics teachers.
This tendency to select videos they already had a connection with or were
familiar with, such as those from known channels, is in accordance with
findings from cognitive psychology (Chen, 2016).

  3. 3.

A notable portion of the decisions made during the study could not be
attributed to surface features, comments, or video content based on either
eye-movements or verbalizations. Hence, in these cases, the study participants
either struggled to explicitly articulate their decisions due to multiple
considerations or did just not articulate them at a deeper level. Similar
cases have been observed in physics education research on teachers’
professional competences, where prior research has found that experienced
teachers’ actions in the classroom are guided by informed decisions and
teaching routines that cannot be easily verbalized (e.g., see Borko
Livingston, 1989; Livingston Borko, 1989; Stender, 2014). To clarify whether
similar principles contribute to an explanation of the observations made in
our study requires further investigation.

  4. 4.

Although the participants had the opportunity to view comments associated with
each YouTube video during the free exploration phase 2, surprisingly, none of
the participants explicitly based their decisions on comments. This
observation contrasts with findings from previous research reporting that
students relied on comments as an indicator of video quality (e.g., see
Fyfield ., 2021; Tan Pearce, 2011) and, instead, indicates that the selection
process tends to be less systematic. However, previous studies have
consistently shown a strong and statistically significant correlation between
the explanatory quality of YouTube videos and the number of content-related
comments (Kulgemeyer Peters, 2016; Bitzenbauer ., 2023). It is therefore
noteworthy that these comments did not play a significant role in the
decision-making process of our participants.

The eye-tracking and think-aloud data were complemented by retrospective
questionnaire responses: When asked about the features that influenced their
video selection for instructional purposes, the majority of respondents
indicated thumbnail, duration, channel, or title, while video descriptions and
the quantity or quality of comments played a minor role, even in retrospective
evaluation.

The finding that participants primarily explored the top results of the
YouTube search list aligns with previous studies (e.g., see Fyfield ., 2021;
Tan Pearce, 2011): Over half of the participants reported approaching video
selection in a sequential manner based on the order of videos in the search
list (cf. figure 13). This pragmatic approach leads to quick decisions (made
in a time frame of less than 10 minutes in the free exploration phase 2 of
this study) that are mainly based on surface features (e.g., thumbnails) or
familiarity (e.g., channel). The analysis of individual argumentation
categories (see figure 11 and figure 12) supports this assumption.

In conclusion, our findings suggest that the decision-making process of (pre-
service) physics teachers when searching for suitable YouTube explanatory
videos (on quantum topics in this study) for instructional purposes is
primarily driven by pragmatism, efficiency, and reliance on familiar features.
The availability of empirical evidence regarding the explanatory quality of
online videos seems to be overlooked by (pre-service) physics teachers,
representing a missed opportunity to streamline the selection process.
Therefore, in section 7, we will synthesize existing empirical evidence and
propose a preliminary decision tree that may assist teachers in efficiently
identifying high-quality explanatory videos on YouTube.

###  6.2 Contrasting the selection processes with the proposed decision tree

The observations made in this study indicate that the selection processes of
(pre-service) physics teachers when searching explanatory videos suitable for
physics teaching are predominantly unsystematic, relying on superficial or
familiar aspects, and characterized by pragmatic choices. These tendencies
give rise to two interconnected issues when it comes to real instructional
preparation, where videos with high explaining quality are sought:

  1. 1.

Teachers may require significant time to find suitable videos due to the
unsystematic approach.

  2. 2.

There is a probability that teachers may select videos of lower quality.

It is obvious that in in the study reported in this article the decision-
making process of the participants in most cases diverged from the proposed
decision tree. In light of this, it becomes necessary to support teachers in
systematizing their selection process to overcome the identified problems in
practice. The decision tree proposed in section 3 might be a valuable tool in
this regard as it reflects the sate of the literature. While we are aware that
future studies are required and might lead to a refined version of the
decision tree, the significance of the decision tree in its current form lies
in its capacity to systematize the selection process without imposing
quantitative guidelines or thresholds. This acknowledges the absence of
empirical evidence supporting such measures and recognizes that decisions
should be made by teachers on a case-by-case basis, taking into account the
specific topic. Future studies should investigate whether the use of the
decision tree indeed facilitates efficient and successful identification of
high-quality explanatory videos on various topics. Additionally, it will be
crucial to determine whether decision steps need to be supplemented or
specified.

###  6.3 Limitations

The present study has several limitations that should be considered when
interpreting the results. First, the focus on explanatory videos related to
three specific quantum topics may restrict the generalizability of the
findings. Although we designed the study with three separate groups, each
tasked with selecting videos for instructional situations on two different
quantum topics, this control measure may not account for potential variations
that could arise if explanatory videos on further (physics) topics, e.g.,
classical mechanics topics, were included. Further research is needed to
validate the reported results in a broader range of topics. Second,
understanding the cognitive processes of prospective physics teachers during
video selection is a complex empirical endeavor, and the chosen data
collection methods – even though they might complement each other – come with
inherent limitations. While the analysis of eye-tracking data is based on the
eye-mind assumption (Just Carpenter, 1980), previous research has emphasized
the importance of complementing eye movement analysis with additional verbal
data to gain a comprehensive understanding (Brückner ., 2020; Chien ., 2015;
Chiou ., 2022; Mason ., 2013; Smith ., 2010; Wu Liu, 2021). To address this
concern within our mixed-methods approach, we employed introspective thinking-
aloud in our study. Additionally, the retrospective questionnaire used for
internal validation allows participants to reflect on their experiences;
however, it may also trigger ad-hoc generated associations and thoughts
regarding the different YouTube surface features (for similar arguments see
Winkler ., 2021, 2023). Third, it is important to consider that while this
study focuses on the selection processes employed by (pre-service) physics
teachers in finding YouTube explanatory videos on quantum physics suitable for
teaching, the selection situations created within the study design differ from
real classroom planning scenarios. Particularly, in our study, participants
had unlimited time for decision-making, whereas real instructional planning is
significantly influenced by time constraints. However, the analysis of think-
aloud data reveals that decision-making occurred within a time frame of less
than 10 minutes in phase 2 of the study (free exploration), which aligns with
a reasonable time frame in natural classroom planning situations. Lastly, the
time-consuming nature of the study led to a relatively small sample size.
However, the primary aim of this study was to gain in-depth insights into the
video selection process rather than to produce generalizable findings on a
surface level. Future research with larger sample sizes could provide a
broader perspective on the topic.

##  7 Conclusion

This mixed-methods study explored how pre-service teachers select
instructional videos on YouTube for physics teaching, focusing on the role of
surface features (likes, views, thumbnails) and comments. The results indicate
that the decision-making processes of (pre-service) physics teachers when
searching for suitable YouTube explanatory videos is primarily driven by
pragmatism, efficiency, and reliance on familiar features.

Based on the current sate of research into the explaining quality of online
explanatory videos, we proposed a decision tree which reflects how an
efficient and successful selection process might look. Although the decision-
making process of the study participants often differed from the proposed
decision tree in this study, it serves as a hypothesis for future research
aimed at supporting teachers in systematizing their selection process: Further
studies should explore whether the decision tree facilitates efficient and
successful identification of high-quality explanatory videos on various
topics, and how it might be adapted and refined, e.g., with regards to
different subject areas and teaching contexts. Also, future studies might
examine how the decision tree works as a tool for preparing teaching, e.g., in
related courses in science teacher education. Lastly, it seems particularly
crucial to consider the evolving nature of online platforms in future
research: For example, research could examine how the decision tree (or an
evolved version thereof) can be adapted to accommodate changes in platform
features and the emergence of new video metrics. Collaborative research
involving educators, researchers, and platform developers may further enhance
the decision tree’s practicality and usability for (pre-service) teachers,
facilitating their video selection process and ultimately benefiting student
learning experiences in physics and beyond.

## Declarations

Funding No funding was received for conducting this study.

Competing interests The authors have no competing interests to declare that
are relevant to the content of this article.

Financial Interests The authors have no relevant financial or non-financial
interests to disclose.

Ethics approval Ethical review and approval was not required for the study on
human participants in accordance with the local legislation and institutional
requirements. The patients/participants provided their written informed
consent to participate in this study.

Informed consent of participants Not applicable.

Data availability The data presented in this study are available on request
from the corresponding author.

## References

  *   * Beautemps Bresges (2021) beautemps2021comprisesBeautemps, J. Bresges, A.  2021\.  What comprises a successful educational science YouTube video? A five-thousand user survey on viewing behaviors and self-perceived importance of various variables controlled by content creators What comprises a successful educational science YouTube video? A five-thousand user survey on viewing behaviors and self-perceived importance of various variables controlled by content creators.  Frontiers in Communication5600595.
  * Bitzenbauer . (2023) bitzenbauer2023Bitzenbauer, P., Höfler, S., Veith, J., Winkler, B., Zenger, T. Kulgemeyer, C.  2023\.  Exploring the Relationship Between Surface Features and Explaining Quality of YouTube Explanatory Videos Exploring the relationship between surface features and explaining quality of youtube explanatory videos.  International Journal of Science and Mathematics Education1–24.  https://doi.org/10.1007/s10763-022-10351-w
  * Bohlin . (2017) bohlin2017conceptualBohlin, G., Göransson, A., Höst, G.E. Tibell, L.A.  2017\.  A conceptual characterization of online videos explaining natural selection A conceptual characterization of online videos explaining natural selection.  Science & Education26975–999.
  * Borko Livingston (1989) borko1989cognitionBorko, H. Livingston, C.  1989\.  Cognition and improvisation: Differences in mathematics instruction by expert and novice teachers Cognition and improvisation: Differences in mathematics instruction by expert and novice teachers.  American educational research journal264473–498.
  * Brame (2016) brame2016Brame, C.J.  2016\.  Effective educational videos: Principles and guidelines for maximizing student learning from video content Effective educational videos: Principles and guidelines for maximizing student learning from video content.  CBE—Life Sciences Education154es6.
  * Brückner . (2020) bruckner2020epistemicBrückner, S., Schneider, J., Zlatkin-Troitschanskaia, O. Drachsler, H.  2020\.  Epistemic network analyses of economics students’ graph understanding: An eye-tracking study Epistemic network analyses of economics students’ graph understanding: An eye-tracking study.  Sensors20236908.
  * Chavira . (2021) chavira2021educationalChavira, A.M., Rivas, J.F., Molina, A.P.R.F., de la Cruz, S.A., Zárate, A.C., Musa, A.B. Osorio, V.J.C.  2021\.  The educational quality of the critical view of safety in videos on YouTube® versus specialized platforms: Which is better? Critical view of safety in virtual resources The educational quality of the critical view of safety in videos on YouTube® versus specialized platforms: Which is better? critical view of safety in virtual resources.  Surgical Endoscopy1–9.
  * Chelaru . (2012) chelaru2012canChelaru, S.V., Orellana-Rodriguez, C. Altingovde, I.S.  2012\.  Can social features help learning to rank y YouTube videos? Can social features help learning to rank y YouTube videos?  International Conference on Web Information Systems Engineering International conference on web information systems engineering ( 552–566).
  * Chen (2016) chen2016thingsChen, L.  2016\.  The things attracting our attention: Evidence from text reading The things attracting our attention: Evidence from text reading.  Advances in Cognitive Neurodynamics (V) Proceedings of the Fifth International Conference on Cognitive Neurodynamics-2015 Advances in cognitive neurodynamics (v) proceedings of the fifth international conference on cognitive neurodynamics-2015 ( 573–578).
  * Chien . (2015) chien2015learningChien, KP., Tsai, CY., Chen, HL., Chang, WH. Chen, S.  2015\.  Learning differences and eye fixation patterns in virtual and physical science laboratories Learning differences and eye fixation patterns in virtual and physical science laboratories.  Computers & Education82191–201.
  * Chiou . (2022) chiou2022exploringChiou, GL., Hsu, CY. Tsai, MJ.  2022\.  Exploring how students interact with guidance in a physics simulation: Evidence from eye-movement and log data analyses Exploring how students interact with guidance in a physics simulation: Evidence from eye-movement and log data analyses.  Interactive Learning Environments303484–497.
  * Chtouki . (2012) chtouki2012impactChtouki, Y., Harroud, H., Khalidi, M. Bennani, S.  2012\.  The impact of YouTube videos on the student’s learning The impact of YouTube videos on the student’s learning.  2012 international conference on information technology based higher education and training (ITHET) 2012 international conference on information technology based higher education and training (ithet) ( 1–4).
  * Dwork . (2001) dwork2001rankDwork, C., Kumar, R., Naor, M. Sivakumar, D.  2001\.  Rank aggregation methods for the web Rank aggregation methods for the web.  Proceedings of the 10th international conference on World Wide Web Proceedings of the 10th international conference on world wide web ( 613–622).
  * Eccles Arsal (2017) Eccles-17Eccles, D.W. Arsal, G.  2017\.  The think aloud method: what is it and how do I use it? The think aloud method: what is it and how do I use it?  Qualitative Research in Sport, Exercise and Health9514–531.
  * Findeisen . (2019) findeisen2019Findeisen, S., Horn, S. Seifried, J.  2019\.  Lernen durch Videos: Empirische Befunde zur Gestaltung von Erklärvideos Lernen durch Videos: Empirische Befunde zur Gestaltung von Erklärvideos.  MedienPädagogik16-36.
  * Forman Damschroder (2001) Forman-07Forman, J. Damschroder, L.  2001\.  Qualitative Content Analysis Qualitative Content Analysis.  L. Jacoby L.A. Siminoff (), Empirical Methods for Bioethics: A Primer Empirical methods for bioethics: A primer ( 39–62).  Bingley, UKEmerald Group Publishing Limited.  10.1016/S1479-3709(07)11003-7
  * Fyfield . (2021) fyfield2021navigatingFyfield, M., Henderson, M. Phillips, M.  2021\.  Navigating four billion videos: teacher search strategies and the YouTube algorithm Navigating four billion videos: teacher search strategies and the YouTube algorithm.  Learning, Media and Technology46147–59.
  * Geelan (2012) geelan2012teacherGeelan, D.  2012\.  Teacher explanations Teacher explanations.  Second international handbook of science education987–999.
  * Hahn Klein (2022) hahn2022eyeHahn, L. Klein, P.  2022\.  Eye tracking in physics education research: A systematic literature review Eye tracking in physics education research: A systematic literature review.  Physical Review Physics Education Research181013102.
  * Haveliwala (2002) haveliwala2002topicHaveliwala, T.H.  2002\.  Topic-sensitive pagerank Topic-sensitive pagerank.  Proceedings of the 11th international conference on World Wide Web Proceedings of the 11th international conference on world wide web ( 517–526).
  * Hsieh Chen (2011) hsieh2011differentHsieh, YC. Chen, KH.  2011\.  How different information types affect viewer’s attention on internet advertising How different information types affect viewer’s attention on internet advertising.  Computers in human Behavior272935–945.
  * Iyengar Lepper (2000) iyengar2000choiceIyengar, S.S. Lepper, M.R.  2000\.  When choice is demotivating: Can one desire too much of a good thing? When choice is demotivating: Can one desire too much of a good thing?  Journal of personality and social psychology796995.
  * Jackman (2019) jackman2019youtubeJackman, W.M.  2019\.  YouTube usage in the university classroom: An argument for its pedagogical benefits YouTube usage in the university classroom: An argument for its pedagogical benefits.  International Journal of Emerging Technologies in Learning (Online)149157.
  * Jebe . (2019) Jebe-2019Jebe, F., Konietzko, S., Lichtschlag, M. Liebau, E.  2019\.  Studie: “Jugend/ YouTube/Kulturelle Bildung. Horizont 2019”’ Studie: “Jugend/ YouTube/Kulturelle Bildung. Horizont 2019”’.  Essen, GermanyRat für Kulturelle Bildung e.V.
  * Jones Cuthrell (2011) jones2011youtubeJones, T. Cuthrell, K.  2011\.  YouTube: Educational potentials and pitfalls YouTube: Educational potentials and pitfalls.  Computers in the Schools28175–85.
  * Jung Lee (2015) jung2015Jung, I. Lee, Y.  2015\.  YouTube acceptance by university educators and students: A cross-cultural perspective YouTube acceptance by university educators and students: A cross-cultural perspective.  Innovations in education and teaching international523243–253.
  * Just Carpenter (1976) Just-76Just, M.A. Carpenter, P.A.  1976\.  Eye fixations and cognitive processes Eye fixations and cognitive processes.  Cognitive Psychology8441–480.
  * Just Carpenter (1980) just1980theoryJust, M.A. Carpenter, P.A.  1980\.  A theory of reading: from eye fixations to comprehension. A theory of reading: from eye fixations to comprehension.  Psychological review874329.
  * Kay (2012) kay2012Kay, R.H.  2012\.  Exploring the use of video podcasts in education: A comprehensive review of the literature Exploring the use of video podcasts in education: A comprehensive review of the literature.  Computers in Human Behavior283820–831.
  * Kocyigit Akaltun (2019) kocyigit2019doesKocyigit, B.F. Akaltun, M.S.  2019\.  Does YouTube provide high quality information? Assessment of secukinumab videos Does YouTube provide high quality information? assessment of secukinumab videos.  Rheumatology international3971263–1268.
  * Kulgemeyer (2020) kulgemeyer2020frameworkKulgemeyer, C.  2020\.  A framework of effective science explanation videos informed by criteria for instructional explanations A framework of effective science explanation videos informed by criteria for instructional explanations.  Research in Science Education5062441–2462.
  * Kulgemeyer Peters (2016) kulgemeyer2016Kulgemeyer, C. Peters, C.H.  2016\.  Exploring the explaining quality of physics online explanatory videos Exploring the explaining quality of physics online explanatory videos.  European Journal of Physics376065705.
  * Kulgemeyer Wittwer (2023) kulgemeyer2023misconceptionsKulgemeyer, C. Wittwer, J.  2023\.  Misconceptions in physics explainer videos and the illusion of understanding: An experimental study Misconceptions in physics explainer videos and the illusion of understanding: An experimental study.  International Journal of Science and Mathematics Education212417–437.
  * Livingston Borko (1989) livingston1989expertLivingston, C. Borko, H.  1989\.  Expert-novice differences in teaching: A cognitive analysis and implications for teacher education Expert-novice differences in teaching: A cognitive analysis and implications for teacher education.  Journal of teacher education40436–42.
  * Mackensen-Friedrichs (2004) mackensen2004forderungMackensen-Friedrichs, I.  2004\.  Förderung des Expertiseerwerbs durch das Lernen mit Beispielaufgaben im Biologieunterricht der Klasse 9 Förderung des expertiseerwerbs durch das lernen mit beispielaufgaben im biologieunterricht der klasse 9 .  Christian-Albrechts Universität Kiel.
  * Manca Ranieri (2016) manca2016Manca, S. Ranieri, M.  2016\.  Facebook and the others. Potentials and obstacles of social media for teaching in higher education Facebook and the others. Potentials and obstacles of social media for teaching in higher education.  Computers & Education95216–230.
  * Mason . (2013) mason2013eyeMason, L., Pluchino, P., Tornatora, M.C. Ariasi, N.  2013\.  An eye-tracking study of learning from science text with concrete and abstract illustrations An eye-tracking study of learning from science text with concrete and abstract illustrations.  The Journal of Experimental Education813356–384.
  * McKay (2009) McKay-09McKay, S.L.  2009\.  Introspective Techniques Introspective Techniques.  J. Heigham R.A. Croker (), Qualitative Research in Applied Linguistics. Qualitative research in applied linguistics.  Springer.
  * Meng . (2002) meng2002buildingMeng, W., Yu, C. Liu, KL.  2002\.  Building efficient and effective metasearch engines Building efficient and effective metasearch engines.  ACM Computing Surveys (CSUR)34148–89.
  * Negi Mitra (2020) Negi-20Negi, S. Mitra, R.  2020\.  Fixation duration and the learning process: an eye tracking study with subtitled videos Fixation duration and the learning process: an eye tracking study with subtitled videos.  Journal of Eye Movement Research13.
  * Neumann Herodotou (2020) neumann2020evaluatingNeumann, M.M. Herodotou, C.  2020\.  Evaluating YouTube videos for young children Evaluating YouTube videos for young children.  Education and Information Technologies254459–4475.
  * Pattier (2021) pattier2021sciencePattier, D.  2021\.  Science on Youtube: successful edutubers Science on youtube: successful edutubers.  TECHNO REVIEW. International Technology, Science and Society Review/Revista Internacional de Tecnología, Ciencia y Sociedad1011–15.
  * Pekdag Le Marechal (2010) pekdag2010Pekdag, B. Le Marechal, J.F.  2010\.  Movies in chemistry education Movies in chemistry education.  Asia-Pacific Forum on Science Learning and Teaching11115.
  * Rios . (2019) Rios-19Rios, L., Pollard, B., Dounas-Frazer, D.R. Lewandowski, H.J.  2019\.  Using think-aloud interviews to characterize model-based reasoning in electronics for a laboratory course assessment Using think-aloud interviews to characterize model-based reasoning in electronics for a laboratory course assessment.  Physical Review Physics Education Research15010140.
  * Robbins Heilberger (2011) Robbins-11Robbins, N. Heilberger, R.  2011\.  Plotting Likert and other rating scales. Plotting Likert and other rating scales.  Paper presented at the Proceedings of the 2011 Joint Statistical Meeting, Miami Beach, FL, USA
  * Rosenthal (2018) rosenthal2018Rosenthal, S.  2018\.  Motivations to seek science videos on YouTube: Free-choice learning in a connected society Motivations to seek science videos on youtube: Free-choice learning in a connected society.  International Journal of Science Education, Part B8122–39.
  * Sandmann (2014) sandmann2014lautesSandmann, A.  2014\.  Lautes Denken–die Analyse von Denk-, Lern-und Problemlöseprozessen Lautes denken–die analyse von denk-, lern-und problemlöseprozessen.  Methoden in der naturwissenschaftsdidaktischen Forschung179–188.
  * Sasaki (2013) Sasaki-13Sasaki, M.  2013\.  Introspective Methods Introspective Methods.  A. Kunnan (), Companion to language assessment. Companion to language assessment.  Wiley.
  * Schwartz (2016) Schwartz-2016Schwartz, B.  2016\.  The paradox of choice: Why more is less: How the culture of abundance robs us of satisfaction The paradox of choice: Why more is less: How the culture of abundance robs us of satisfaction.  New YorkEcco Press.
  * Shruti Goyal Miyapuram (2015) Goyal-15Shruti Goyal, K.P. Miyapuram, U.L.  2015\.  Predicting Consumer’s Behavior Using Eye Tracking Data. Predicting Consumer’s Behavior Using Eye Tracking Data.  Paper presented at the Second International Conference on Soft Computing and Machine Intelligence, Hong Kong
  * Smith . (2010) smith2010eyeSmith, A.D., Mestre, J.P. Ross, B.H.  2010\.  Eye-gaze patterns as students study worked-out examples in mechanics Eye-gaze patterns as students study worked-out examples in mechanics.  Physical Review Special Topics-Physics Education Research62020118.
  * Stender (2014) stender2014unterrichtsplanungStender, A.  2014\.  Unterrichtsplanung: Vom Wissen zum Handeln: Theoretische Entwicklung und empirische Überprüfung des Transformationsmodells der Unterrichtsplanung Unterrichtsplanung: Vom wissen zum handeln: Theoretische entwicklung und empirische überprüfung des transformationsmodells der unterrichtsplanung ( 170).  Logos Verlag Berlin GmbH.
  * Stockwell . (2015) stockwell2015Stockwell, B.R., Stockwell, M.S., Cennamo, M. Jiang, E.  2015\.  Blended learning improves science education Blended learning improves science education.  Cell1625933–936.
  * Tadbier Shoufan (2021) tadbier2021rankingTadbier, A.W. Shoufan, A.  2021\.  Ranking educational channels on YouTube: Aspects and issues Ranking educational channels on YouTube: Aspects and issues.  Education and Information Technologies263077–3096.
  * Tan (2013) tan2013informalTan, E.  2013\.  Informal learning on YouTube: Exploring digital literacy in independent online learning Informal learning on youtube: Exploring digital literacy in independent online learning.  Learning, media and technology384463–477.
  * Tan Pearce (2011) tan2011openTan, E. Pearce, N.  2011\.  Open education videos in the classroom: Exploring the opportunities and barriers to the use of YouTube in teaching introductory sociology Open education videos in the classroom: Exploring the opportunities and barriers to the use of YouTube in teaching introductory sociology.  Research in learning technology19.
  * Toffler (1984) toffler1984futureToffler, A.  1984\.  Future shock Future shock.  New YorkBantam.
  * van der Laan . (2015) Laan-15van der Laan, L.N., Hooge, I.T.C., de Ridder, D.T.D., Viergever, M.A. Smeets, P.A.M.  2015\.  Do you like what you see? The role of first fixation and total fixation duration in consumer choice Do you like what you see? The role of first fixation and total fixation duration in consumer choice.  Food Quality and Preference3946–55.
  * Veith . (2022) Veith-22Veith, J., Bitzenbauer, P. Girnat, B.  2022\.  Assessing Learners’ Conceptual Understanding of Introductory Group Theory Using the CI2GT: Development and Analysis of a Concept Inventory Assessing learners’ conceptual understanding of introductory group theory using the ci2gt: Development and analysis of a concept inventory.  Education Science12376.  10.3390/educsci12060376
  * Wang . (2014) Wang-14Wang, Q., Yang, S., Liu, M., Cao, Z. Ma, Q.  2014\.  An eye-tracking study of website complexity from cognitive load perspective An eye-tracking study of website complexity from cognitive load perspective.  Decision Support Systems621-10.
  * Welbourne Grant (2016) welbourne2016scienceWelbourne, D.J. Grant, W.J.  2016\.  Science communication on YouTube: Factors that affect channel and video popularity Science communication on youtube: Factors that affect channel and video popularity.  Public understanding of science256706–718.
  * Winkler . (2021) winkler2021quantumWinkler, B., Bitzenbauer, P. Meyn, JP.  2021\.  Quantum physics $\neq$ quantum physics. A survey of researchers’ associations Quantum physics $\neq$ quantum physics. a survey of researchers’ associations.  Physics Education566065031.
  * Winkler . (2023) winkler2023europeanWinkler, B., Veith, J. Bitzenbauer, P.  2023\.  Classical Mechanics Key Topics in Physics Teacher Education: Results of an Exploratory Mind Map Study Classical mechanics key topics in physics teacher education: Results of an exploratory mind map study.  European Journal of Educational Research1231247–1255.
  * Wittwer Renkl (2008) wittwer2008instructionalWittwer, J. Renkl, A.  2008\.  Why instructional explanations often do not work: A framework for understanding the effectiveness of instructional explanations Why instructional explanations often do not work: A framework for understanding the effectiveness of instructional explanations.  Educational Psychologist43149–64.
  * Wolf Kratzer (2015) wolf2015erklarstrukturenWolf, K.D. Kratzer, V.  2015\.  Erklärstrukturen in selbsterstellten Erklärvideos von Kindern Erklärstrukturen in selbsterstellten erklärvideos von kindern.  Jahrbuch Medienpädagogik 12: Kinder und Kindheit in der digitalen Kultur29–44.
  * Wu Liu (2021) wu2021eyeWu, CJ. Liu, CY.  2021\.  Eye-movement study of high-and low-prior-knowledge students’ scientific argumentations with multiple representations Eye-movement study of high-and low-prior-knowledge students’ scientific argumentations with multiple representations.  Physical Review Physics Education Research171010125.

## Appendix

Table 5: Category system used for the think aloud study. Surface Feature |  Category
---|---
Thumbnail |  1\. The thumbnail indicates a thematically well-suited video
2\. The thumbnail indicates that the video might be off-topic
3\. The thumbnail indicates that the video might not be exhaustive, e.g. it is
part of a longer series
4\. The thumbnail seems interesting
5\. The thumbnail seems unappealing or non-professionell
6\. The thumbnail indicates that the topic is explained well
7\. The thumbnail indicates that the topic is not explained well
8\. Neutral examination of the thumbnail
Title |  1\. The title indicates a thematically well-suited video
2\. The title indicates that the video might be off-topic
3\. The thumbnail indicates that the video might not be exhaustive, e.g. it is
part of a longer series
4\. The title seems interesting
5\. The title seems unappealing or non-professionell
6\. The title indicates that the video might delve too deeply into the topic
7\. Neutral examination of the title
Subscriptions |  1\. The respective channel has many subscriptions
2\. The respective channel has few subscriptions
3\. The respective channel has more subscription than other channels
4\. The respective channel has fewer subscriptions than other channels
Channel |  1\. The channel is known
(a) The channel’s name is known
(b) The channel is known for high-quality videos
(c) The channel is known for scientifically inaccurate videos
(d) The channel is known for low-quality videos
2\. The channel is unknown
3\. Neutral examination of the channel
4\. The channel seems fitting
5\. The channel seems unfitting
6\. The channel is verified (blue checkmark)
Likes |  1\. The video has many likes
2\. The video has few likes
3\. The video has more likes than other videos
4\. The video has fewer likes than other videos
Views |  1\. The video has many views
2\. The video has few views
3\. The video has more views than other videos
4\. The video has fewer views than other videos
5\. Neutral examination of views
Length |  1\. The video length seems too long
2\. The video length seems accurate
3\. The video length seems rather short
4\. The video length seems too short
5\. Netrual examination of video length
Upload |  1\. The video is too old
2\. The video is new
3\. Neutral examination of upload date
Order |  1\. The videos suggested first are fitting
2\. The videos suggested first are unfitting
Description |  1\. Neutral examination of video description
2\. The video description is appeealing
Content |  1\. The video content could be fitting
2\. The video content seems rather unfitting
3\. Neutral examination of video content
4\. The video content is contingent
Comments |  1\. Checking the feedback emerging from comments
2\. The comments are rather positive