On one hand, most of these design activities were based on authentic situations, which are close to the real-life experience of students. Challenges were carried out around topics related to social life, such as to identify and redesign systems that existed at school [38]; the design of an escape room for the local fire department to allow participants to playfully and interactively improve awareness of fire safety in and around the house [30]; working revolved around interviewing senior citizens and creating prototypes that met their needs on the background of the aging society [32]; to create a secure environment for the elderly without taking away their freedom [40]; or designing a heat retaining food container for street food vendors at a taxi depot [46]; designing a water filter system for the city's wastewater management plant to help prevent the pollution of a local river [34]; to develop sustainable food products for peer group [55]; designing and constructing shoes [21]; the design theme on preventing bullying in the social context of the class [58]; or targeting on more local or social problems that could be solved with new products, services, or other solutions [33,50,66], etc.

On the other hand, for all design activities, there were no unified answers. Students were encouraged to design as many creative solutions as possible and choose the relatively optimal solution to solve the problem. Many tasks were engineering, such as the marshmallow tower activity and the trebuchet design activity [45], the design of toys [39,43], the engineering design activities of musical instrument, simple machines, and bio-inspired flower [47], or the design of water filters, bridges, circuits in maglev vehicles and windmills, and pollinators and knee braces [62], etc. In another study, students were asked to design a 3D model and print a 3D artefact [57].

In addition, the design activities were usually challenging and cooperative. The problems addressed by designers are wicked problems, such as the design of complex systems or environments for living, working, playing, and learning, and the wicked problems theory of design was explained in [67]. In school education, an important goal is to develop cooperative citizens, which helps to equip students with the ability to collaboratively solve wicked problems in social life. Therefore, to help students solve these complex and wicked problems, some activities were game-based and aimed to create interesting design situations for learners. Especially for the younger children, the fun of activity design is especially important. Thus, some activities were role-playing, such as the design of castle according to an imaginary engineering situation [64]; or storytelling, such as reading the narrative content of the books and solving the engineering problems presented in the books, and finally rewriting the story [52]. Others also employed computer games to study students' learning based on the design process [60,68].

#### 3.4.5. Design Tools and Materials

Among the 44 studies, 20 mentioned the design tools or materials used. Six provided students with conventional design tools or materials, such as paper, pencil, glue, tapes, scissors, etc., [36,42,44,46,50,51]. Specially, one of these studies employed a reverse engineering teaching context, in which students learned from already designed products rather than started from scratch [51]. By dissecting the product's components and structures, it helps students understanding how those components function and work together, and then students can redesign or build their own product to properly solving a design challenge with micro-innovation or applying the scientific concept they had learned during the learning process.

Fourteen studies employed digital platforms as teaching aids of designing and making in the class. For example, some studies organized design activities with open-source hardware and software. Kim, Seo, and Kim [52] provided students with Arduino Leonardobased device to solve the engineering problems, by which artifacts can be made through assembly of various blocks with sensors and actuators, and also be driven by the programming language based on Scratch and app inventor. In [49], Raspberry Pi with programmable microelectronics, adapted versions of SNaP and Tiles toolkits, and Google's Design Sprint Kit techniques were combined to help IoT design across generations. Another platform, such as the mobile telepresence robot called KT, which is controlled over the internet through a web interface, was used to design human-centred robots that served a need in the local environment and allowed remote peers to explore the local spaces [61]. Christensen et al. [40] also explored middle school students' design literacy who had experience with digital technology in maker settings (e.g., FabLabs).

Lin, Chang, and Li [31] studied virtual reality (VR) teaching application on engineering design creativity. In their study, the junior high school students experienced design teaching with VR devices used as teaching tools. The VR box (for a 360◦ view), zSpace built-in software Franklin's Lab (for detailing motor operation), Cyber Sciences-3D (for explaining circuits), and Leopoly-3D (for modelling simulation) (created by zSpace, Inc., San Jose, CA, USA) were used to design an electric model vehicle capable of automatically avoiding obstacles.

There were other tools, meant for prototyping or concept understanding, for example, the 3D design and printing tools, that were employed in some reviewed papers. In [26], the design stage used computer-aided design to create a three-dimensional model of the toy. Forbes et al. [27] studied children' STEM learning performance in 3D technology-enhanced makerspaces by using iPad and 3D design software and print device (Makers Empire, created by Makers Empire Pty Ltd., Adelaide, Australia). Leinonen et al. [57] discussed the digital fabrication in elementary school education. The 3D model design by Tinkercad software (created by Autodesk, Inc., San Francisco, CA, USA) and 3D artefact printing by Ultimaker printer were adopted to aid the design activities, such as designing and printing a name tag, a floor plan, and a game piece. In [43], the teacher explained the mechanical

functions and STEM knowledge via physical and 3D virtual simulation models, and then in the inquiry experiment stage, students were asked to assemble the LEGO to design a movable toy with various mechanical structure types.

Besides 3D modelling tools, other design platforms were also employed. Four studies involved students in the poster design activities, and three of them used poster design games on computer [60,65,68], while another one used WPS Writer® to complete the new year poster design [48].

A tool dedicated to assisting design communication was also mentioned. Won et al. [56] reported on the ways that middle school students appropriated a social networking forum (Edmodo) as a part of the iterative design process in an informal learning environment. In their study, the forum was utilized for the purpose of collaborative design by interaction in progression of the design process.

#### **4. Discussion**

#### *4.1. The Current Status of DTIL*

#### 4.1.1. Research Trend

Among the selected 43 empirical papers, most were published after 2017 and most (12) in 2021 (see Figure 1). It was concluded that the growing trend of high-quality empirical studies on DTIL in K-12 education was significant in the past five years. "The proper study of mankind is the science of design, not only as the professional component of a technical education but as a core discipline for every liberally educated man" [3]. According to the rapid development of empirical research, we can predict that more evidence-based research will focus on this topic in the future, and the potential value of design thinking in K-12 education will be further explored.

#### 4.1.2. Educational Level

Considering the uneven distribution of the age groups in K-12 education, more research is needed to engage the younger (like kindergarten and primary school students) and older learners (high school students) into DTIL. As shown in Section 3, the research covered students of all levels in K-12 education, aged between 3 and 18 years. Students from kindergarten to high school were all involved, and among them, the papers studied middle school students' learning accounted for the largest proportion (23 or 52.3%). It shows that the distribution of existing studies was uneven across educational levels. As a methodology for innovative problem solving, the design thinking intelligence is not necessarily something that comes naturally, it needs to be nurtured and developed in pedagogical approach. According to [69], design must be nurtured from early beginnings, as building design capital is a visionary and long-term job. Wells [70] also pointed out that, similar to language learning, design thinking is something that should be carefully nurtured from an early age and be included in all areas of education. However, in comparison with the studies conducted in university or other adult learning scenarios, there is still a lot of room for research in K-12 education.

#### 4.1.3. Samples and Duration

Among the 44 studies extracted from the 43 papers, many selected small samples or short periods to carry out the empirical research. The reason may be that the activities integrating design thinking had higher requirements for teachers and students. First, teachers need to spend more time preparing various sources for class teaching, such as the design tools and materials. Second, during the lesson, support and guidance are required for student activities. Especially in the hands-on activities, teachers are also concerned about the safety of their students. Third, the diversification of evaluation, such as the evaluation of the design process and design works, also requires teachers to spend more time on observation, interviews, surveys, etc., to obtain and analyse multi-dimensional learning data. The limited energy of teachers makes it difficult to carry out large-scale class

teaching for a long time. Fourth, for students, the design thinking tasks are usually more challenging, which require a longer cycle to constantly iterate and optimize.

However, for DTIL in K-12 education, empirical studies with longer periods and larger samples are still expected to provide more sufficient and convincing evidence to demonstrate the application effect. Despite the difficulties, researchers and teachers could try to solve these problems with relevant technologies, thereby expanding the sample size and extending the experimental period. For example, teachers could try to use intelligent teaching assistants to assist in monitoring students' learning, real-time feedback and forewarning of learning, etc., so that they can better communicate with students, guide them according to students' needs, and realize personalized teaching. Students could use intelligent technologies such as intelligent learning partners to carry out learning, reduce their learning burden and increase their interest, so that they can actively participate in longer-term learning activities. In addition, multiple subjects are encouraged to participate in teaching and research related to DTIL in K-12 education. In addition to research institutions (e.g., university), K-12 schools, and government departments, more social forces are also expected to join, such as communities, science and technology museums, enterprises, etc., to jointly provide support for DTIL, thus ensuring larger scale and longer cycles teaching practice.

#### *4.2. Curriculum Domain in DTIL*

#### 4.2.1. Curriculum

More research is needed to explore the integration of design thinking with single or multiple disciplines from a wider subject range. As the social movement of design thinking has called for the involvement of design thinking into K-12 classrooms, we put together the disciplines mentioned in the reviewed empirical studies. The results revealed that the implementation of design thinking in non-design classrooms is increasingly concerned. Although design thinking by its nature originated from design field in an effort to encourage people to think and practice like an expert and prepare future designers [70], it is now become a simplified version of "designerly thinking" or a way of describing a designer's methods that is integrated beyond the design context [7,9,15]. In addition to design-related courses, most researchers employed the design thinking integrated teaching and learning in the STEM related subjects, such as science, STEM/STEAM, engineering, technology, etc., which is consistent with previous findings [11]. While other researchers have extended design thinking to non-STEM subjects, such as geography [38]. This has a lot to do with the rise of STEM education in recent years. There is societal recognition of the role played by STEM education in preparing students for college and career readiness, and design challenge exactly provides a pathway for the STEM disciplines to work together [14]. Additionally, it has adopted the stance that design and technology should be embedded in various school subjects [71].

How is design thinking integrated into these curriculums? Despite the differences in the curriculums themselves, there are some common threads of DTIL. First, for the purpose of DTIL, it mainly responds to the need for interdisciplinary and innovative talent cultivation in the age of intelligence. Therefore, design thinking is integrated by enabling students to learn and connect multidisciplinary knowledge and skills via involving the design process/skills. Second, for the learning content, it is usually embedded inside the open-ended and authentic problem context, in which students' learning content stems from real life, not those hypothetical questions. That is, the integration of design thinking provides ideas for the reframing of teaching content or topics. Third, for the organization of learning, the curriculums adapted DTIL are characterized by student-centred and co-creative. Teachers act as coach to enable active learning, while students can communicate and collaborate with peers and even with stakeholders so that be able to solve problems creatively.

#### 4.2.2. Design Thinking Model

The design thinking models should be aligned with the complex teaching situations. It was found that the core concepts of design thinking are valued and pursued in the field of K-12 education. However, in terms of the design thinking model, there is no panacea for all situations, that is, there is no unified framework or process in academic. For example, when talking about the design process and characteristics of a certain design activity (e.g., the technology and engineering design), it does not mean that there is just one process [14]. Researchers have also pointed out that if teachers rely too much on the confined procedural, pre-determined process structure, it may strangle the qualitative forms of intelligence inherent in design thinking [70]. In fact, in the human-made world, different fields or disciplines have their applicable design processes, the process of design thinking is surely not deterministic [44]. Therefore, the key ideas of the existing studies for design thinking provide a foundation for understanding the capabilities related to design. On this basis, teachers can adjust the design thinking model reasonably according to the actual situation.

In this study, based on the evidence of the reviewed empirical papers, it reveals that the highly concerned aspects of design thinking are the following: prototype, ideate, define, test, explore, empathize, evaluate, and optimize. These key elements provide a holistic outline for students to solve problems with design thinking. Overall, the results of this study are consistent with previous studies. For example, ITEEA [14] elaborated eight key ideas of design in the PreK-12 technology and engineering education, including a fundamental human activity, open-ended and can always be improved and refined, iterative, a range of skills, universal principles and elements, making, optimized by criteria and constraint, and diversity of approaches. In [11], design thinking in the K-12 educational context was noticed, and they organized the multiple codes of design thinking competences into six areas, including reasoning, problem setting, empathy, ideation, modelling, and process management.

#### *4.3. Learning Evaluation in DTIL*

#### 4.3.1. Dependent Variables

Knowledge, skills, and dispositions are all covered by the dependent variables reported from the reviewed papers. For example, the existing empirical studies measured students' knowledge of relevant subject concepts, e.g., physics concepts of acceleration, Newton laws of motion, velocity, motion, energy, etc., [37], or the understanding of the mechanical concepts [43]. Some studies also checked students' understanding of design thinking (e.g., [39]). Various skills, like creativity, productive thinking, problem solving, and critical thinking, are evaluated as crucial learning effects. Except for the concepts and skills, students' performance on design thinking practice (including the design process and design work) has been paid close attention, and it is the most frequently evaluated one. This exactly confirms the practical orientation of design, or the non-verbal competence [15] that is required. Besides, many studies surveyed students' dispositions, as there exists a bi-directional relationship between emotion and design-based learning [12]. To sum up, in the DTIL context, design thinking is considered in many studies as both a dependent variable and partly as an independent one. Results of the learning effects indicate optimistic learning outcomes of students; additionally, the competences cultivated by the design thinking activities were often associated with the 21st century skills, like communication, collaboration, creativity, and critical thinking [11,27,29,40].

#### 4.3.2. Evaluation Instruments

Considering the benefits of applying design thinking into other subjects, more objective and apt evaluation instruments need to be developed to emphasize the alignment between design thinking competences and domain knowledge. The evaluation instruments used in the existing empirical studies include survey/questionnaire, portfolio, interview, observation, protocol analysis, design work evaluation, test, video recording, audio recording, and journal. Qualitative and quantitative assessments are both included, and the

qualitative assessments are used more frequently. However, it was found that most of them were self-designed measures, such as portfolio, survey, interview, observation, design work evaluation. These assessments were highly subjective. To better serve the trending integration of design thinking into STEM and non-STEM subjects in K-12 education, it calls for more objective and apt design thinking-embedded evaluation instruments that evaluate comprehensive performance for researchers and educators to adapt in their interventions or classrooms. Researchers have pointed out the shortcomings of current evaluation methods, for example, Blom and Bogaers [46] indicated that current verbal protocol analysis methods and theoretical frameworks did not explain how internal and external information sources contribute to novice designers' moment-to-moment thought processes, and they employed a protocol analysis method—Linkography—to investigate the nature of novice designers' thought processes. Furthermore, given that DTIL is usually suggested to be involved in the complex problem-solving situations, more timely evaluation and feedback are needed to help students gain a better learning experience. In the era of intelligence, there is also an urgent need to explore assessment supported by various intelligent technologies.

#### *4.4. Interventions in DTIL*

#### 4.4.1. Study Design

Most of the studies tended to report the DTIL via non-experimental case study in K-12 context [16], the number of which far surpassed that of the quasi-experimental and experimental studies. In the educational context, samples were usually restricted by the original class settings, therefore, it was relatively more difficult to carry out rigorous experimental research that emphasizes the randomization of samples. In this instance, the application of case studies does have value and facilitates the introduction of concrete practical experiences. However, the problem that ensues is the difficulty in identifying the certain competences can be developed by participating in design thinking integrated activities [11], so that a compromise approach is to conduct quasi-experimental study design. According to [63], a study with no random assignment but a control group, multiple measures can be classified into the type of quasi-experimental design to examine whether such a design thinking integrated intervention, as the independent variable, influences students' learning, as the dependent variable. In the future, more rigorous research with quasi-experimental or experimental study design is needed to strengthen the learning effect verification.

#### 4.4.2. Course Type

Most studies were conducted in a formal educational context (i.e., classrooms). The high proportion of studies conducted in the formal educational setting appeared to align with the move of involving design thinking into educational context. Besides, it is also consistent with the finding of large number of research conducted in standard courses, such as science, engineering, technology, etc., (see Section 3). In the future, more studies should be conducted for design thinking integrated interventions or activities applied in informal educational scenarios, such as the design workshops or makerspaces, so that it can be a complementary way to highlight the educational value of design thinking.

#### 4.4.3. Grouping

Most research defaults to a collaborative learning approach to design thinking embedded activities. Collaborative learning has the potential to foster students' understanding of various subject domains, help complex problem solving, and master social skills [58,72]. The group sizes typically employed were between two and four people and were usually randomly grouped or arbitrarily appointed by the teachers. It lacked detailed research to explain the specific reasons or rules. A few studies have looked at the impact of gender differences on design thinking embedded learning [31,34,37]. More in-depth studies are needed to explore applicative collaborative strategies, such as group size, grouping methods like role assignments, dynamic grouping or fixed grouping, gender combination, etc.

#### 4.4.4. Design Thinking Task/Challenge

The design thinking embedded activities were usually comprehensive, transdisciplinary, and integrate a variety of learning methods, such as project-based learning [55], problem-based learning [21], inquiry-based learning [36], game-based learning [60], storytelling [64], etc. It was in line with the characteristics of transdisciplinary learning such as STEM. Additionally, process-oriented design (design-no-make) and product-oriented design (design-and-make) were both adopted to produce various creative solutions, object models, smart products, engineering devices, etc. For example, while involved in the activities of poster design [48,65], playground design [41,42], or the design of smart-things ideas for an outdoor park environment [49], students were encouraged to propose creative ideas to meet certain needs. When participated in the product-oriented design tasks, students were also required to construct artifacts, such as designing and constructing a device that was engineered to provide electricity to a third world community scenario [36]. Results of this study should inspire researchers and teachers to develop design thinking embedded activities that are reality situated, open-ended, challenging, and cooperative, to engage younger students to better develop their domain knowledge and thinking skills, activities that are fun and hand-and-brain on will be in greater need. In future, the integration of design thinking in project-based learning and STEM/STEAM education can be more diverse, and it would be more expected if it can be combined with local cultural characteristics. For example, in one study [73], the "Chinese Wooden Arch Bridges' Intelligent Monitoring system " project was developed to explore the integration of cultural education, scientific inquiry and design thinking. In this STEAM project, the integration of design thinking is believed to not only promote transdisciplinary learning, but also help cultural understanding and inheritance.

#### 4.4.5. Learning Tools and Materials

The tools and materials employed are expected to help students formulate and express creative design ideas. For example, the design cards are convenient for students to sort out the design process and carry out reasoning [49]. The 3D design and printing platforms can help students design and visualize prototypes [27,57]. Conventional design and construct tools, as well as the digital fabrication kits, provide students with hands-on opportunities to materialize ideas. According to the reviewed articles, there were more applications of conventional tools and materials than intelligent ones. Researcher argued that the functions of design are making better what is inadequate and delivering a value that has not yet been delivered, thus technology is a key driver of the project as a whole and will play an important role in that [74–76], likewise, design can promote technological literacy. Future study could be conducted to further explore the technology-enabled design thinking activities in K-12 context.

Especially, in the emerging digital lean manufacturing paradigm [77], the advent of Industry 4.0 (I4.0) technologies such as Artificial Intelligence (AI), Internet of Things (IoT), Virtual Reality (VR), and Mixed Reality (MR), etc., have provided significant enlightenment to education. On one hand, new technologies can be used to enable better performance. For example, during the COVID-19 pandemic, technologies make such large-scale online teaching activities possible. In particular, for experimental or hands-on courses, the technology application like the online simulation platform (i.e., IRobotQ3D) [78], the integration of VR and motion capture system [79], etc., can solve the problems of interactivity and presence to a certain extent. On the other hand, the essence of I4.0 is also to better meet the needs of users, which coincides with the human-centred concept emphasized by design thinking. As we enter the era of intelligence, customers require new and customised products more and more often, and companies have hence to be able to easily follow these requirements in order to survive in today's competitive market [79]. The implication for education is that to integrate technology into DTIL activities may help students collect and analyse data more scientifically, open up broader design space, and enrich the ways and forms of design work

presenting, so as to promote the cultivation of innovative intelligent manufacturing talents that meet the needs of the era.

In conclusion, the implications and suggestions for DTIL in K-12 context are: (a) The potential value of design thinking in K-12 education needs to be further explored with larger samples and longer durations, in particular, more attention needs to be paid to students at the preK-elementary and high school levels. (b) Design thinking models be aligned with the domain learning situations are needed to expand the application of DTIL in a wider range of disciplines, especially non-STEM disciplines. (c) DTIL is expected to facilitate the acquisition of higher-order learning achievements, correspondingly, more objective and apt evaluation instruments are needed. (d) The educational value of design thinking does not arise spontaneously, more targeted interventions are expected to reach its full potential, such as rigorous study design in the pre-post mode or with control groups, more focus on learning in informal settings, intensive grouping strategies that consider gender, group size, role, etc., pertinent design thinking tasks/challenges what are both fun and challenging, and finally, support design activities with appropriate technical tools or materials.

#### **5. Conclusions and Implication**

In the wake of the popularity in design thinking, this literature review analyses 43 high-quality empirical papers on DTIL in K-12 education. The current status and future directions are identified to narrow the gaps. The major findings are summarized as follow.

First, as deduced from the publication trend (see Figure 1), a positive picture of design thinking emerges. However, the empirical evidence to support the effectiveness of DTIL is still rather limited. The results indicate that although samples from preschool to high school are all involved, it shows an uneven distribution of the age groups in K-12 education, which calls for more research to engage the younger learners (like kindergarten and primary school students) into DTIL. If possible, studies with longer periods and larger samples are expected.

Second, DTIL has the potential to deepen students' learning with diversified models in various subject domains, including that of STEM and non-STEM, while the most mentioned applications are the fields of STEM, which is consistent with the result of previous study [12]. In addition, the design thinking models applied varied among studies, which suggests inconsistency in current perceptions of design thinking. On one hand, a prevailing view was to apply design thinking from a thinking model of designer or designer's mindsets (e.g., [40,49–52]), in other words, think of it as a unique problem-solving methodology (e.g., [33,35,37,42,44,48,53]). Our study also supports this view. The theoretical basis of design thinking has been propositioned to be interconnected to Dewey's notion of pragmatist inquiry and aesthetic experience [23,36]. The corresponding training goals were the abilities required to solve the complex and open-ended problems, such as communication skills, cooperation skills, creativity, and critical thinking, etc. On the other hand, however, there were differences in the focus of studies on the application of design thinking, leading to the emergence of different models or competence frameworks. For example, some studies focused on inquiry and experimentation in the early stages of problem solving (such as discover, define, empathy), others focused on modelling or prototyping during the problem-solving process, and still others discussed the impact of emotions, etc.

How do teachers choose or construct a design thinking model or framework that meets the needs of the teaching situation? In fact, it is difficult to construct a universal design thinking model to deal with a variety of complex educational scenarios. Therefore, to meet the teaching needs of more subject domains, future study needs more exploration of the design thinking models that could be compatible with teaching practice in K-12 education. Referring to the existing empirical research, it may be necessary to comprehensively consider related factors such as students, teachers, and the school environments. In addition, although existing research claims that design thinking has great educational value, it is difficult to cover all aspects at once. Therefore, it is a feasible way to develop

various competencies by stages or disciplines. Future study could bring design thinking into both STEM and non-STEM curriculums to further verify how design thinking will help students build knowledge and skills in a wider range of domains. Furthermore, teachers can make targeted combinations or adjustments of elements in a design thinking model or framework to better meet teaching needs.

Third, in response to the many existing design thinking models and subject domains, various dependent variables were measured by different instruments. Mostly, the empirical studies evaluated students' performance on design thinking, such as the design processes and design works. Another two types of learning performances that were paid close attention were emotional/social aspects and subject concepts. Besides, the design thinking concept and other skills like creativity, problem solving, critical thinking, etc., were also involved in the measured variables. For assessment, qualitative and quantitative assessments were both included, and the qualitative assessments were used more frequently, which highlights the practice-orientation and process-orientation of design thinking.

Implications for instruction include: (1) for the dependent variables, both process performance and learning outcomes should be emphasized. In particular, it is helpful to explore the development process of students' thinking by analysing the process of students' design activities. (2) Besides, the culture competency also needs to be included in the assessment. Design was seen as an important force driving the development of the artificial world, so one of the core characteristics of design thinking is that it is human-centred [1,7,32,80,81]. Every citizen should have a certain quality of cultural understanding and inheritance, which emphasizes the practical implementation of these values contained in the excellent culture at the behavioural level [82]. On this basis, we can better solve the problems faced by human beings through design thinking activities, to promote the benign development of human society. (3) To verify the design thinking embedded learning more scientifically and efficiently, more objective and apt evaluation instruments are needed. In addition to conventional subject knowledge and skills, for the evaluation of design thinking, existing research has provided some experience for reference. For the understanding of design thinking concepts, it can be assessed by test [35], survey [39], questionnaire [40], or other qualitative method, such as interview, video/audio recording, etc. [38]. For design thinking, evaluation instruments that can be used are portfolio (e.g., [21,33,42,55,61]), observation (e.g., [45,64]), survey (e.g., [31,33,55,56]), protocol analysis (e.g., [21,42,44,46,59,62]), design work evaluation (e.g., [31,35,45,47,58,60]), interview (e.g., [55]), etc. (4) Furthermore, there is more room for development of relevant evaluation methods supported by technology. In the existing studies, evaluation instruments supported by technology are rarely used. In the future, the application of artificial intelligence technology, multimodal data collection and analysis, real-time evaluation and feedback technology in the classroom still needs to be explored to promote the improvement of teaching quality.

Finally, interventions including study design, course type, grouping, design task/challenge and teaching tools and materials are summarized from the empirical studies. Results show that most research reported DTIL by conducting non-experimental studies in formal classrooms. In the reviewed studies, collaborative learning was frequently used, through which students were usually arranged in groups of 2–4 people. Except for cooperative, the design tasks or challenges employed in the empirical studies were also characterized as real situated, open-ended, and challenging. Moreover, conventional tools or materials were used more often, technical tools and materials were relatively lacking. Overall, at present, the development of design thinking intervention in K-12 education is not sufficient. We are inspired to pay more attention to the application of rigorous study design, balance between formal and informal classrooms, intensive grouping strategies, applicative tasks or challenges, and technical materials in the future. For instruction, because the situations vary among schools, the DTIL intervention should vary accordingly. Moreover, in the selection of design materials or tools, it is also necessary to pay attention to the application of technical materials or tools, to help students understand the technological world in which they live and learn to use technology to innovate and shape the world.

#### **6. Limitations**

As this study mainly focuses on students' learning in a design thinking integrated context of K-12 education, there are some limitations. First, we selected the reviewed papers only from SSCI journals, which ensured the representativeness and quality of sample but lacked comprehensiveness. Second, the school levels were limited to preschool to high school, while colleges and other adult learners were not included, so that the conclusions are only applicable to K-12 education. Third, teachers were considered crucial for learning; separate study is needed to amply elaborate the role of teachers in DTIL. In future, it is expected that more research on teaching, learning and evaluation of DTIL will be carried.

**Author Contributions:** Conceptualization, T.L. and Z.Z.; methodology, T.L. and Z.Z.; writing original draft preparation, T.L.; writing—review and editing, T.L. and Z.Z. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research was financially supported by Ministry of Education in China Project of Humanities and Social Sciences "The Collaborative Innovation Mechanism for Guangdong-Hong Kong-Macao Greater Bay Area Education based on C-STEAM" (22YJC881901), the Major Project of Social Science in South China Normal University (ZDPY2208), the Major basic research and applied research projects of Guangdong Education Department (#2017WZDXM004). The author gratefully acknowledges the support of "2022 Guangdong-Hong Kong-Macao Greater Bay Area Exchange Programs of SCNU".

**Institutional Review Board Statement:** Not applicable.

**Informed Consent Statement:** Not applicable.

**Data Availability Statement:** Not applicable.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **Appendix A**

**Table A1.** General information of the papers.

**Table A1.** *Cont.*

Note: N/A—Not available.

**Table A2.** Study design and

intervention.

**Table**

**A2.**

*Cont.*

#### **References**

## *Article* **Factors Influencing Student Satisfaction toward STEM Education: Exploratory Study Using Structural Equation Modeling**

**Jingbo Zhao 1,†, Tommy Tanu Wijaya 2,\*,†, Mailizar Mailizar 3,\* and Akhmad Habibi <sup>4</sup>**

**Abstract:** Learning satisfaction has a relationship with student outcomes. Furthermore, this has prompted many governments to increasingly implement STEM education-based learning. Many studies have examined the improvement of STEM education by teachers. However, the studies have not analyzed STEM education's effect on students' learning satisfaction. Extending the planned behavior theory, this study aimed to predict high school students' learning satisfaction with STEM education. The questionnaire developed from the TPB model was filled out by 174 high school students in Indonesia. Furthermore, AMOS and SPSS 23 software were used for structural equation model analysis. The results showed that seven of twelve hypotheses were supported. Subjective norm and playfulness factors of STEM education positively relate to students' attitudes toward STEM education. Attitude is the most important factor influencing student satisfaction and acceptance toward STEM education. Therefore, this study provides a theoretical and practical contribution to improving learning satisfaction in technology-based STEM education.

**Keywords:** STEM education; senior high school; TPB; learning satisfaction

### **1. Introduction**

The study of Science, Technology, Engineering, and Mathematics (STEM) education has increased in the last decade, showing its importance in improving students' abilities in the 21st century [1,2]. However, its implementation in Indonesia has many challenges. First, the teacher does not understand STEM education and how to implement it for students. Second, it combines interdisciplinary science, a challenge for schools and teachers accustomed to single-subject teaching. Other problems are class hours, communication on interdisciplinary science, teaching materials, and its relationship with Indonesia's standard curriculum [3].

Indonesia's learning curriculum is centralized, where the government monitors and evaluates teachers' school activities. The government has advised some schools in big cities to use STEM education-based learning. STEM education entered Indonesia in 2014, with Syiah Kuala University becoming one of the largest centers. In collaboration with SEAMEO and QITEP (https://www.qitepinscience.org/, accessed on 30 June 2022), the STEM education research center focuses on improving STEM education quality. They conduct various workshops and develop education-based learning to assist its implementation in schools. Examples of experiments include electroplating, aquaponics technology, electrical installation, and others related to technology and engineering from elementary to high school levels. Although STEM implementation has been running for more than

**Citation:** Zhao, J.; Wijaya, T.T.; Mailizar, M.; Habibi, A. Factors Influencing Student Satisfaction toward STEM Education: Exploratory Study Using Structural Equation Modeling. *Appl. Sci.* **2022**, *12*, 9717. https://doi.org/10.3390/app12199717

Academic Editor: Krzysztof Koszela

Received: 6 August 2022 Accepted: 24 September 2022 Published: 27 September 2022

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

five years, only a few studies measure its success [4–6]. The previous study primarily used cognitive learning performance to measure the quality of its implementation [7,8]. Additionally, several college studies have examined the effects of STEM education, though there is limited literature that focuses on K-12 students.

The success of implementing STEM education is measured in many ways. Previous studies used the perspective of student academic performance [9,10]. Other studies stated that academic performance is less effective in determining the success of implementing a new learning approach. Furthermore, several studies recommend analyzing student satisfaction as an alternative to successfully implementing a new learning model [11,12]. Student satisfaction reflects their perceptions of the learning experience with STEM education [13]. Student satisfaction is also an important outcome affecting motivation, ability, and the desire to participate in learning activities [14,15]. Furthermore, this study considered student satisfaction with STEM education as the dependent variable because it strongly relates to the perception of learning quality. Besides the importance of STEM education and its integration of various fields of science, unsatisfied students may be influenced by its difficulty to implement, impact, and inadequate facilities. Therefore, further studies should identify the predicted determinants related to student satisfaction with STEM education.

This study aimed to develop a new model to explore factors related to student satisfaction toward STEM education through attitude and acceptance. The following section elaborates on the theoretical background of predictors that may relate to student satisfaction with STEM education.

#### **2. Literature Review**

#### *2.1. STEM Education in Indonesia*

Common teaching and learning approaches used by teachers in Indonesia are problemsolving. Where learning activities are teacher-centered, students have less opportunities to participate. Furthermore, teachers rarely use technology-based learning media to explain the material. When STEM education was implemented in Indonesia, the shifting was established. Teachers with STEM education connect science, technology, engineering, and mathematics materials in instruction. The increase in the use of technology-based learning media influences pedagogical and technological knowledge.

STEM education was introduced in Indonesia by SEAQIS in 2015 through the 2015 IBSE-STEM Policy concerning cooperation agreements to organize training in STEM assisted by ATSE Australia. Cooperation between countries is encouraged to implement projects to strengthen the curriculum based on STEM education [9]. SEAQIS is one of the institutions appointed by UNESCO partners to make this program a success [16].

At the beginning of 2018, batch one STEM education training began to be pursued, containing teachers in West Java, Indonesia. In mid-2018, the achievements of technologybased experimental STEM products from batch one became the start and initial guideline for batch two training. This training is currently growing with the support and cooperation of PPPPTK IPA.

From August to October 2018, technology-based STEM experiments were implemented at various school levels. The experiments are mostly at the secondary school level, where students are more mature, careful, and knowledgeable than in elementary school. The experience of implementing teachers and product results are displayed in the STEM workshop. In 2019 and 2020, large-scale training was conducted to learn STEM-based education evenly distributed in Indonesia.

The Indonesian Ministry of Education monitors and supports collaborations and big plans to socialize STEM education [17]. It also hopes that the learning steps could be gradually combined with the national curriculum [9]. Therefore, teaching and learning activities in the future would be more scientific and fun, avoiding the lecture learning model mostly used by teachers [18,19].

Indonesia's STEM education training that produces pre-service and in-service teachers is technology- and project-based correlated with everyday life. Many studies have shown that project-based learning is student-centered and makes students more participative. This approach is suitable to be combined with STEM education, where learning is experimentbased. In line with this, several studies found that STEM education learning in the first week was filled with engineering design and theory cycles. It provided students with preparation for using STEM education experiments with technology, as well as discussing and formulating hypotheses about the product. The teacher provides brainstorming, experimental plans, and product goals before the scientific unit learning with engineering design begins.

Since 2014, several universities in Indonesia have been working to form lesson plans for experimental schools. Some studies conducted are the development of workbookbased STEM education, which improves students' abilities in every meeting, technology literacy, and outcomes. In contrast, this study focused more on the importance of student satisfaction in learning with STEM education than only technology literacy and outcomes.

After introducing the STEM education for more than five years and its implementation running in schools for approximately three years, it is necessary to understand students' acceptance, attitude, and satisfaction. Therefore, this study aimed to explore the factors related to students' attitudes and satisfaction toward STEM education. The results may help the Indonesian government develop STEM education-based learning. They could also provide deeper knowledge about important considerations when implementing learning in schools.

#### *2.2. Theoretical Foundation and Hypotheses Development*

This study proposed a comprehensive model to solve the problem and the need for further study on student satisfaction with STEM education. Figure 1 shows that the proposed model examines perceived usefulness, perceived convenience, facilitating conditions, subjective norm, playfulness, attitude, and collaborative learning. These predictors may affect student satisfaction with STEM education at the high school level. They are the combined results of previous studies derived from the theory of planned behavior (TPB) [20] and several theories about the acceptance model affecting student satisfaction [11,12,21]. The technology acceptance model theory was used because this study defined STEM education as a learning approach that integrates technology with other sciences. According to Zou [22] and Zobair [23], this theory could be used to analyze a person's intentions and acceptance of learning models related to new technology.

**Figure 1.** The proposed model with initial hypothesis.

#### *2.3. Perceived Usefulness (PU)*

PU with technology is how people believe that learning with technology-based STEM education helps them achieve learning goals. Previous studies found that PU positively and significantly relates to student attitudes and acceptance of the new learning approach [24,25]. Similarly, there is a positive relationship between PU and student satisfaction [26]. High school students may be highly satisfied with STEM education with technology-based learning when they believe technology helps them gain new knowledge.

#### *2.4. Perceived Convenience*

For students, determining convenient learning depends on time and effort [27]. Learning is convenient when it is fast, short, clear, and reduces the students' emotional and physical burden [28]. Joo [29] stated that convenience is similar to the TAM model's ease of use. It is felt by students when conducting STEM education to gain new knowledge.

From the self-determination theory perspective, perceived convenience is how people believe a model could help them achieve the desired goal [30]. Liao [31] found that perceived convenience affects one's motivation and intention to use technology-based learning models. Additionally, Yoon and Kim [31] extended TAM by adding a perceived convenience variable to analyze the acceptance of new technology.

#### *2.5. Facilitating Conditions*

Facilitating conditions (FC) are facilities and environmental factors affecting the perception of doing a task [32]. FC facilitates support, including technical assistance, teacher knowledge, and knowledge about STEM education. It could influence students' attitudes toward technology-based STEM education learning. According to Wijaya [33], technical support was the highest factor influencing teachers to implement new technology. Other studies also found FC related to the intention to use technology-based learning models [34–36]. Teo [37] stated that FC affects user satisfaction, and Ngai et al. [38] found that it affects attitudes toward computer use. Therefore, the FC factor may influence student satisfaction toward STEM education.

#### *2.6. Subjective Norm*

Subjective norm is the acceptance of social pressure to perform a behavior [39]. It is also a perception that important people support or do not support one's behavior [40]. In this study, the subjective norm is where people around students believe technology-based STEM education is important learning. Previous studies found that these norms affect technology acceptance and student attitudes [37,41]. Therefore, subjective norms relate to student attitudes and acceptance toward technology-based STEM education.

#### *2.7. Playfulness*

Perceived playfulness may also influence students' attitudes toward STEM education. Studies on technology show that user playfulness is the strongest determinant associated with using new technology-based learning approaches [42,43].

Perceived playfulness is the mindset with three dimensions, including how individuals feel their attention is focused on STEM education with technology experiments, curiosity when conducting experiments, and learning is fun and interesting. Students have a pleasant experience when conducting the experiments in groups. Experiment-based learning is sometimes liked by students and provides intrinsic motivation.  In contrast, intrinsic motivation is the desire to be seen in an activity driven by passion [45].

Perceived playfulness when using learning or objects is related to technology in previous studies. Davis [46] found that playfulness positively influences a person's attitude toward new systems and learning. Therefore, the higher the perceived playfulness students feel in learning technology-based STEM education, the better their attitude [47].

#### *2.8. Attitude toward STEM Education*

Attitude is the desire to do or use something and positive or negative feelings toward an action or system [48]. This study defined attitude toward STEM education as the level at which students have negative or positive feelings toward technology-based STEM education lessons. Previous studies found that attitude toward using embodies successful implementation [24,49,50]. Attitudes toward conformity affect behavior intention, technology acceptance, and perceived learning [51,52]. Therefore, this study hypothesized that attitude affects student satisfaction with technology-based STEM education.

#### *2.9. Collaborative Learning*

Compared to traditional classroom settings, collaborative learning in STEM education transforms the learning environment with ICT-assisted experimentation. It enables students to encourage collaboration and build higher knowledge [53]. Collaborative learning embeds the integrated power of many sciences into learning through large-scale networked education [54,55]. It also encourages students to work together to solve life problems [56,57]. In this case, the teacher is only a tutor helping students overcome difficulties conducting experiments and group discussions. Therefore, collaborative learning may relate to student acceptance of technology-based STEM education.

#### *2.10. STEM Education Acceptance*

STEM education acceptance is the willingness to learn using STEM education. Many people have used and developed acceptance, such as Venkatesh [58] and the TAM model [59], to understand the technology acceptance phenomenon. This study synthesized the technology acceptance theory in the UTAUT model [32] into technology-based STEM education. Many studies have modified the UTAUT model in psychology, information systems, marketing and banking, and education [22,60–62]. In the UTAUT model [32], technology acceptance is influenced by performance expectancy, perceived convenience, subjective norms, and facilitating conditions adopted in this study. Therefore, this study hypothesized that STEM education acceptance is influenced by PU, PC, SN, and collaborative learning, and it affects student satisfaction with technology-based STEM education.

#### *2.11. Satisfaction toward STEM Education*

Student satisfaction is the emotions after having a learning experience with technologybased STEM education. This implies the importance of helping students' post-adoption behavior [63,64]. Student satisfaction is an important factor affecting learning interest and outcomes [65,66]. It is widely used to measure the success or failure of a learning model or approach, especially in education, where student satisfaction is prioritized [67]. Student satisfaction toward technology-based STEM education strongly relates to their learning outcomes. Therefore, various studies examine the factors affecting student satisfaction [14,68,69].

This study used the perceived dimensions of internal factors and educational dimensions as variables to predict the factors related to student satisfaction with STEM education. It used six independent variables, two intermediate variables, and one dependent variable connected into twelve initial hypotheses shown in Figure 1 and Table 1.

**Table 1.** Initial hypotheses about the factors related to student satisfaction toward technology-based STEM education.

#### *2.12. Participants*

This study aimed to build a model to determine the factors related to student satisfaction toward STEM education at the high school level. Based on the theoretical foundation, eight determinants were examined for their relationship with student satisfaction. All students were informed that this online questionnaire was only used for study data, and their identity would be protected. The questionnaire was distributed at the STEM education experimental school guided by the SK university. Furthermore, this study used purposive sampling and the google questionnaire platform for the online questionnaire distribution. Therefore, the identity of the questionnaire filler was ascertained anonymously.

A total of 174 valid responses were collected from May to June 2022. The respondents comprised 118 female and 56 male students participating in teaching and learning activities with STEM education at least once. Furthermore, most teachers were divided into groups of more than three for STEM education. Complete data are shown in Table 2.

**Table 2.** Basic student information.

#### *2.13. Instrument*

This study used online google questionnaires (see Appendix A) and conducted a voluntary and anonymous survey. The first part of the questionnaire indicated students' basic information, experiences with STEM education and the predictors related to satisfaction with STEM education. It used a five-point Likert scale ranging from one (strongly disagree) to five (strongly agree) [70]. The scale indicated the student's level of agreement with the statement on the questionnaire. The original questionnaire had nine latent variables, including perceived usefulness (three items), perceived ease of use (three items), and facilitating condition (three items). Other latent variables were playfulness (three items), subjective norm (three items), attitude toward STEM education (three items), behavior intention toward STEM education (three items), collaborative learning (four items), and student satisfaction toward STEM education (three items), resulting in twenty-eight items.

The questionnaires were sent to three experts on STEM education studies to check the appropriateness and clarity of all items and constructs. They were corrected, discarding two items (play two and SN1) because they were unclear and inappropriate in the construct.

#### *2.14. Data Analysis*

An initial analysis was conducted using AMOS and SPSS software to show whether the data could answer the study objectives. Structural equation modeling (SEM) was also applied because it allows simultaneous analysis of many variables needed and tests the relationship with factor analysis [71]. Confirmatory factor analysis (CFA) testing was first performed to confirm the variables' validity and reliability, which estimated the instrument's internal consistency before processing data using SEM [72]. The Cronbach alpha value must exceed 0.6 to meet the convergent validity criteria [73]. Second, the factor loadings of observed items must exceed 0.5 [74]. Furthermore, the overall fit model in SEM is usually assessed based on the goodness-of-fit indices (GFI) value exceeding 0.90. The parsimonious goodness-of-fit index (PGFI) must exceed 0.50 [73]. RMR must be smaller than 0.08, including the minimum Chi-square value (CMIN) and the ratio of Chi-square to degrees of freedom (where 2/df must be less than 5.0) [75]. The hypothetical model is supported when all the value requirements for SEM are met.

#### **3. Results**

This section presents the proposed SEM model's verification to determine the factors related to student satisfaction with STEM education. First, it displays descriptive statistical data and normality tests, followed by the reliability and validity tests. The last section presents the results of SEM and initial hypothesis testing.

#### *3.1. Descriptive Statistics and Normality Test*

Table 3 shows descriptive statistics of all observed items in this study. Based on the definition from Kline [76], the data normality is measured based on the skewness and kurtosis, which must be at the limit of ±3. In this study, the highest and lowest kurtosis values were 2.985 and 0.198, respectively, while the skewness ranged between −1.229 and −0.280. Therefore, the skewness and kurtosis values met the normal distribution criteria and were useful parameters in SEM.

**Table 3.** Descriptive statistics and normality test.

#### *3.2. Reliability Analysis*

Data reliability was seen from the Cronbach alpha value to measure each construct's internal consistency. Table 4 shows that the Cronbach alpha value is between 0.705 and 0.889. According to Hair et al. [73], the Cronbach alpha value must be greater than 0.70. Therefore, the questionnaire items have high reliability and internal consistency between variables. The next step checked the loading factor, composite reliability (CR), and AVE value to analyze convergent validity. The loading factor, CR, and AVE must be higher than 0.5, 0.7, or 0.5 [74]. In this study, the lowest factor loading, CR, and AVE values were 0.75, 0.90, and 0.76, respectively. Therefore, the proposed model has good convergent validity.

**Table 4.** Measurement construct validity with the factor loading, CR, and AVE.

#### *3.3. Validity Analysis*

Hair [73] suggested checking convergent and discriminant validity. Convergent validity is seen in the average variance extracted (AVE) value. Table 5 shows that all AVE values exceed 0.5, indicating that all constructs are acceptable and useful for further analysis.

**Table 5.** Discriminant validity result and root of average variance extracted.

**Table 5.** *Cont*.

The discriminant validity was analyzed by assessing the square root of AVE for all variables. Table 5 shows that the AVE value is greater than the inter-construct correlations, implying that discriminant validity is accepted [77].

#### *3.4. Model Fit Assessment*

The initial model tested was concluded to identify and interpret the results of the fit index from the model estimation. There are many ways to interpret the fit model tested by looking at CMIN/df, RMSEA, GFI, and AGFI [78]. The model is also tested by evaluating TLI, NFI, and CFI, as well as parsimonious fit with PNFI and PGFI reference indicators [61]. This study used the three methods to test model fit (Table 6).

The data processing results using AMOS software showed that the Chi-square value is 429,521 with 248 degrees of freedom. This indicates that the CMIN/df value is 1.75 in the limit between 1 < x < 3. Furthermore, many studies suggest checking the goodness-of-fit index by considering the variance and covariance predicted in the reproduced matrix [65,79]. A higher GFI value of 0.80 is acceptable because it indicates a better model fit. In this study, the GFI value reached 0.88. Unlike the Chi-square, the RMSEA reference indicator considers the estimated parameters but not the sample size. Therefore, an RMSEA value less than 0.08 indicates an accepted model [80], and an RMSEA value smaller than 0.05 indicates a perfect model. This study's RMSEA value is 0.06, implying an acceptable model. For incremental fit measurement, the values of NFI, CFI, and AGFI must be greater than 0.8 to achieve an acceptable fit model [81]. In this study, the values of NFI, CFI, and AGFI exceeded 0.8. Some of the literature uses parsimonious fit measurement to determine model fit. A parsimonious model is the simplest and sharpest model to explain the analyzed phenomenon [82]. In this study, the entire parsimonious fit index exceeded 0.05, signifying the model is suitable. These three fit model tests confirm that the proposed structural model in Figure 2 is acceptable and appropriate for analyzing and interpreting the factors related to student satisfaction toward STEM education.

**Table 6.** Absolute, incremental, and parsimonious fit measurement.

The linear correlation coefficient (R2) [83] is the most important value for determining whether the proposed structural model strongly explains the factors influencing student satisfaction toward STEM education. Cohen [84] stated that an R2 value greater than 0.26 (26%) was strong enough to explain a model. In this study, the R-value for acceptance toward STEM education was 78.4%, student attitude toward STEM education was 73.0%, and student satisfaction toward STEM education was 70.4%. Therefore, the proposed model explains the factors related to student satisfaction with STEM education.

**Figure 2.** Whole set observation of the measurement model.

#### *3.5. Structural Model and Hypothesis Testing*

This study examined the statistical significance of the 12 initial hypotheses about the relationship with student satisfaction with STEM education. It determined the standardized regression coefficients between the dependent and independent variables. Furthermore, the study analyzed the significance of the *p*-value on each coefficient derived from the SEM output. The accepted hypothesis is where a statistically significant relationship in the predicted direction is confirmed. Table 7 and Figure 3 show three hypotheses with a significant level reaching 0.01 and at a significant level of 0.05 and 0.10 against seven hypotheses.

**Table 7.** Coefficient and hypothesis testing. \*\*\*: *p* < 0.001.

**Table 7.** *Cont*.

**Figure 3.** Final model with a coefficient (β) and *p*-value; a dotted line means an insignificant relationship. \*\*: *p* < 0.01; \*\*\*: *p* < 0.001.

The findings showed that PU did not significantly correlate with STEM education acceptance or student attitude (*p* > 0.05), hence H1 and H2 were rejected. PEU had a relationship with STEM education acceptance (β = −0.539, *p* < 0.05), supporting H3 (PEU → ACC). Furthermore, FC had no significant relationship with attitude toward STEM education (*p* > 0.05), denoting H4 was rejected. SN does not significantly affect STEM education acceptance (*p* > 0.05), thereby H5 was rejected. In contrast, SN significantly correlates with attitude toward STEM education (β = 0.398, *p* < 0.05), supporting H6. Collaborative learning on STEM education has a relationship with STEM education acceptance (β = 0.295, *p* < 0.05) and attitude toward STEM (β = −0.324, *p* < 0.05), supporting H7 and H8. Moreover, the playfulness variable had the strongest relationship with students' attitudes toward STEM (β = 0.796, *p* < 0.01), supporting H9. The attitude had significant relationships with STEM education acceptance and student satisfaction toward STEM education (β = 1.149, *p* < 0.01) and (β = 1.101, *p* < 0.01), also supporting H10 and H11. Based on the coefficient, students' attitude strongly correlates with STEM education acceptance and satisfaction. Additionally, STEM education acceptance has no significant relationship with student satisfaction toward STEM education, meaning H12 was rejected.

#### **4. Discussion**

This study focused on integrating high school STEM education by predicting factors related to student satisfaction. It proposed a model based on the theory of planned behavior (TPB) added with predictors from the literature review.

Students might assume that when they feel the STEM education with technology experiment is relatively easy, their acceptance decreases, as shown by the PEU coefficient of acceptance of −0.184. High school students think experimental STEM education with technology should integrate several science fields with certain difficulties. When the STEM education experiment is too easy, high school students may feel they cannot improve their abilities. They are not challenged to experiment with technology-based STEM education. Moreover, students feel they are more accepting of learning commonly used by teachers. This finding supports previous studies which showed that easy activities did not improve student outcomes [85,86]. However, further understanding using qualitative studies is needed to explain these findings.

Subjective norms affect students' attitudes toward STEM education. The success of implementing learning aids is inseparable from the help of the people around the users [41,87]. Similarly, the success of technology-based STEM education depends on the role of the government, schools, and teachers. Students assume that their attitude cannot be separated from the support of the people that think it provides many opportunities to develop 21st century skills and knowledge to apply in life daily. The stronger the subjective norm, the better the student's attitude toward technology-based STEM education.

Collaborative learning in STEM education encourages students to communicate and increase their satisfaction in teaching and learning activities [88]. It allows them to talk, discuss, and convey their ideas in groups. Furthermore, collaborative learning relates to the acceptance of technology-based STEM education. Students assume that learning has a relationship with their acceptance of technology-based education. This education is a series of experiments difficult to conduct individually. Furthermore, students think that success in conducting experiments on technology-based STEM education requires collaboration. This implies they think collaborative learning is related to their acceptance of technology-based STEM education. However, students believe collaborative learning reduces their attitude toward STEM education. This is because learning in Indonesia mostly used a scientific approach or individual- and teacher-centered methods before STEM education. Therefore, students are not accustomed to working in groups and prefer individual- to experimentbased learning that requires cohesiveness. In some cases, groups with more members increase the possibility of conflicts of ideas, making the students emotional. This aspect provides input for teachers to guide and monitor experiments and student discussions. The teacher must mediate and provide a way out with deliberation when there are differences in opinions within the group during technology-based STEM education experiments.

Perceived playfulness is an advantage of technology-based STEM education. The findings showed that perceived playfulness has the strongest relationship with student attitudes. It has the largest positive indirect effect on student satisfaction toward technologybased STEM education. The model allows students to experiment inside and outside the classroom, making the learning process flexible. Unconsciously, students learn to solve complex problems related to everyday life as a team. The joy of experimenting increases their attitude toward technology-based STEM education, supporting previous studies [89,90].

Students' attitudes toward technology-based STEM education are the strongest determinants compared to acceptance and satisfaction. This implies the importance of improving the effectiveness of STEM education implementation in Indonesia. When students' attitudes improve, they easily accept STEM education-based learning. Similarly, they are more satisfied with experimental STEM education-based learning with technology. This supports a previous study finding that attitude toward a learning model or new technology significantly affected the intention to use the technology [91,92].

Students' acceptance of technology-based STEM education indirectly significantly increased their satisfaction. High school students feel that their satisfaction is more influenced by their attitude. However, further studies should examine other predictors affecting student satisfaction with technology-based STEM education.

Facilitating conditions did not significantly correlate with high school students' attitudes toward technology-based STEM education-based learning. These include teachers ready to help students overcome difficulties experimenting and schools that provide the needed facilities. The schools in question are equipped with complete laboratories to conduct STEM experiments. Furthermore, high school students feel confident and able to conduct their STEM experiments, indicating that their attitude is not influenced.

#### **5. Theoretical and Practical Implications**

This study provided theoretical and practical implications for increasing student satisfaction with technology-based STEM education. First, it is to develop a model to explain the factors related to student satisfaction with technology-based STEM education. The proposed model provides empirical evidence of student satisfaction. Second, this study provides new knowledge about the theory of student satisfaction in STEM education. The information from this new model's development contributes to STEM education's future development. This is useful for school governments and teachers to identify the factors to be considered when implementing STEM education in schools.

Practically, this study showed the relevance of student attitude and satisfaction by analyzing the factors significantly affecting the two variables. Student attitude could be improved by increasing the effectiveness of STEM education learning. This study found that subjective norms significantly affect student attitude toward STEM education. Therefore, educational institutions and teachers should create a conducive environment and spearhead the use of STEM education-based learning. The finding of playfulness implies that schools and teachers should integrate fun and exciting activities that generate pedagogically satisfying interest among students when learning with STEM education. It is also important to incorporate new innovations in experiments conducted by students. Although STEM education is rarely applied to learning activities in Indonesia, the existing experiments may always be interesting for students.

Learning using STEM education has challenges and problems. However, this approach is accepted by students because they are happy with learning activities that require teamwork. In today's era, knowledge should be balanced with the ability to work in a team. The important collaborative skills should be possessed in the 21st century. Therefore, this finding is expected to spur teachers to continue using STEM education-based learning. It could also promote students to collaborate when experimenting in STEM education than traditional or individualized teacher-centered learning.

Student attitude significantly affects their acceptance and satisfaction with STEM education. They feel that they recognize the benefits of STEM education on their learning outcomes. Furthermore, this study is beneficial to the government and academics that focus on developing STEM education in Indonesia. It could help them understand various important factors concerning student acceptance and satisfaction with STEM education. The findings provide a basis for evaluating student attitude, acceptance, and satisfaction with STEM education. Additionally, they contribute to the development of a model to analyze student acceptance.

#### **6. Conclusions**

This study tested the proposed model developed from the modified TPB model with literature review predictors to understand the determinants related to student satisfaction toward technology-based STEM education in Indonesia. Increasing student satisfaction may increase the effect of STEM education learning on student outcomes. Therefore, this study provides a better understanding of the factors to be considered to increase student satisfaction toward STEM education for governments, institutions, and teachers. Although STEM education is sustainable, it does not automatically increase student acceptance and satisfaction. The most important thing is to improve students' attitudes through support from schools and teachers. This could help students become more familiar with technologybased STEM education. The success of increasing student satisfaction is influenced by internal and external factors.

#### **7. Limitations and Future Directions**

Although this study achieves the research objectives and provides several valuable implications, this study has several limitations. First, the study was limited to the school experiment at the senior high school level in collaboration with the Syiah Kuala University in Indonesia. Therefore, the findings should be generalized with caution. Future studies should use larger samples from various countries. Second, this study also used TPB as the base model, while student satisfaction toward STEM education could still be analyzed in many models. Additionally, it was a quantitative study, implying the need for qualitative and longitudinal studies in the future.

The results showed how attitude and collaborative learning in STEM education affect student satisfaction. However, future studies could consider constructs such as student academic achievement, intention to use, and actual use of STEM education. The findings could provide additional knowledge on marginal processes and situations that describe STEM education's effect on student satisfaction.

**Author Contributions:** Conceptualization, T.T.W. and M.M.; methodology, T.T.W.; software, A.H.; validation M.M.; formal analysis, T.T.W. and A.H.; investigation, M.M.; funding acquisition, J.Z.; project administration, J.Z.; writing—original draft, T.T.W., M.M. and A.H.; writing—review and editing, T.T.W., M.M., A.H. and J.Z. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research was supported by the Hainan Provincial Natural Science Foundation of China (622RC670); Project of the 13th Five Year Plan of Educational Science in Hainan Province of China (QJY20191071).

**Institutional Review Board Statement:** The study was conducted in accordance with the Declaration of Helsinki and approved by the Institutional Review Board (or Ethics Committee) of Syiah Kuala University (protocol code 3806/UN11.1.6/TU/01.01/2022 and 1 May 2022).

**Informed Consent Statement:** Not applicable.

**Data Availability Statement:** The data presented in this study are available on request from the corresponding author. The data are not publicly available due to privacy.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **Appendix A**

**Table A1.** Questionnaire items.

#### **References**

## *Article* **How K12 Teachers' Readiness Influences Their Intention to Implement STEM Education: Exploratory Study Based on Decomposed Theory of Planned Behavior**

**Pengze Wu 1, Lin Yang 2,\*, Xiaoling Hu 1, Bing Li 1, Qijing Liu 3, Yiwei Wang <sup>1</sup> and Jiayong Huang <sup>3</sup>**

**Abstract:** Teachers are the key factors in ensuring the effectiveness of STEM education, and their intentions deeply influence their teaching practices. The existing research about the influencing factors of teachers' intentions to implement STEM education has some problems, such as small sample sizes, being limited to teachers of a single subject, and the need for optimization of the theoretical model relied on. This research, based on the decomposed theory of planned behavior combined with the readiness of teachers, formed an assumption model of the factors influencing teachers' STEM education intentions from the aspects of attitudes, subjective norms, and perceived behavioral control. Questionnaires were sent to 532 K12 general teachers in China. A structural equation model (SEM) was used to analyze recycled data and verify the assumption model. The results show the following: (1) The educational readiness of K12 teachers in China was at an upper–middle level. Among them, the level of emotional readiness was the highest, while the level of behavioral readiness was the lowest. (2) The STEM behavioral intention of teachers was at an upper–middle level, and attitudes and perceived behavioral control had direct significant impacts on teachers' intentions to engage in STEM education. Perceived usefulness, self-efficacy, and behavioral readiness were the three strongest indirect impact factors. (3) The emotional readiness of the teachers directly affected their intentions to implement STEM education. Behavioral readiness and cognitive readiness indirectly had an impact on teachers' intentions to implement STEM education by influencing self-efficacy.

**Keywords:** STEM education; teachers' readiness; teachers' intention; decomposed theory of planned behavior

#### **1. Introduction**

STEM is the abbreviation of the initials of four disciplines, including science, technology, engineering, and mathematics. STEM education aims at solving problems such as technological competitiveness and manufacturing dilemmas [1]. STEM education, which has an interdisciplinary integration orientation, has attracted the attention of international research institutions and scholars [2]. STEM education requires students to use knowledge and skills from multiple disciplines to complete learning tasks in the background of complex thematic life [3]. Proven by many empirical studies, STEM education helps students develop creative problem-solving skills and competencies to adapt to the future [4]. These skills and competencies include research inquiry, problem solving, critical and creative thinking, entrepreneurship, collaboration, teamwork, and communication [3,5,6], which are helpful for personal study and work, as well as long-term development. After STEM education was put forward, STEAM education appeared, as well as I-STEAM, STEAMx, and so on derived from STEAM education. For ease of expression, this research uses STEM education to refer to STEM, STEAM, and so on, which are the same types of concepts as STEM education.

**Citation:** Wu, P.; Yang, L.; Hu, X.; Li, B.; Liu, Q.; Wang, Y.; Huang, J. How K12 Teachers' Readiness Influences Their Intention to Implement STEM Education: Exploratory Study Based on Decomposed Theory of Planned Behavior. *Appl. Sci.* **2022**, *12*, 11989. https://doi.org/10.3390/ app122311989

Academic Editor: Francesco Caridi

Received: 5 November 2022 Accepted: 20 November 2022 Published: 23 November 2022

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

Under the background of fierce global competition, STEM education contributes greatly to the development of innovative talents with high-order thinking skills and 21stcentury skills [7,8]. STEM education can help students acquire 21st-century skills such as problem solving [9] and creativity [10,11]. It is considered an important cornerstone for the realization of 21st-century skills [12]. Therefore, STEM education has been widely promoted by countries around the world, including China. The importance attached to STEM education has been reflected in national policies and researched in practical areas. The Ministry of Education of the People's Republic of China put forward that the country should actively explore the application of information technology in new educational models such as interdisciplinary learning (STEM education) [13]. The China STEM Education White Paper suggested that STEM education should be involved in the national strategy for training innovative talents. Teachers are a crucial medium in the implementation of STEM education in schools [14]. Their understanding of STEM, knowledge reserve of related disciplines, STEM teaching competence and experience, etc., directly determine the effectiveness of STEM teaching.

Behavioral intention is described as an important indicator of performing given behaviors. It is also the core predictive variable of behavior [15]. Unless the teacher has both the skills to use the most complex method and the desire to implement them, they cannot use the most complex method to help students learn [16]. As front-line education practitioners, teachers play a vital role in implementing STEM education [17]. Nikolopoulou et al. [18] put forth that teachers' acceptance of and disposition for instructional technology indicated their interest and deeply influenced their teaching practice. As a result, professional development programs for STEM teachers are a core issue of future STEM research [19]. Exploring the teaching intention of teachers is urgent because it directly decides their STEM teaching practice.

Relevant research has shown that the majority of teachers in China strongly agreed with the concept of STEM education. However, teachers who were willing to implement STEM education were in the minority [20] because STEM education is difficult. Li et al. [21] investigated the past behaviors and behavioral intentions of physical education teachers to incorporate STEM education into their teaching and found that the proportion of teachers who regularly incorporated STEM education into their teaching was low. Therefore, some questions remain: Why are teachers in China not so willing to implement STEM education? What are the basic factors that influence their willingness? How can we promote their willingness to implement STEM education? We need to identify which variables dominate in determining teachers' intentions to implement STEM education and put forward suggestions that may strengthen these factors more effectively. This will help to find out the factors that affect the actual performance of STEM teachers from the source and solve problems of source power that bridge the gap between reality and the theory of STEM.

#### **2. Hypothesis Development**

Research on the influencing factors of teachers' intentions to implement STEM education has some problems, such as small sample sizes, being limited to teachers of a single subject, and the need for optimization of the theoretical model relied on [22,23]. To predict an individual's behavioral intentions, a series of models have been constructed and adopted. Among them, the more impactful models include: the technology acceptance model (TAM) [24], the theory of planned behavior (TPB) [15], and the decomposed theory of planned behavior (DTPB) [25]. The DTPB was put forward by Taylor and Todd after they compared TAM theory and the TPB. In the DTPB, attitudes, subjective norms, and perceived behavioral control, the three influencing factors of the TPB, are decomposed into lower-level belief structures.

Attitudes include perceived usefulness, ease of use, and compatibility. Subjective norms include peer influence and superior influence. Perceived behavioral control includes self-efficacy, resource-facilitating conditions, and technology-facilitating conditions. The DTPB model and other models have been verified and compared in different fields, proving that the DTPB has more accurate predictions and explanatory ability by adding decomposed variables [26,27]. This model has been widely used in research on, for example, the intentions of pre-service teachers to use Web 2.0 tools [28,29] and implement ICT [30], and the intentions of junior high school teachers to use online learning platforms [31]. Therefore, this research selected the DTPB model as the basic theoretical framework to comprehensively forecast and interpret the influencing factors of the intention to implement STEM education by Chinese K12 teachers.

Readiness is the level of ability and willingness shown by an individual in a particular job [32]. The readiness of a teacher has a great impact on the quality of classroom instruction, the effectiveness of classroom instruction, and the acquisition of students' abilities [33]. Hata, Nur Fatahiyah Mohamed et al. [34] found that teachers who had more knowledge and better attitudes toward STEM education were better prepared for STEM education.

As a result, this research proposes that readiness should be included in the influencing factors of STEM educational intention; thus, readiness was divided into three dimensions: cognition, emotion, and behavior [35–37] Certain extensions of the DTPB model were established: nine independent variables, four intervening variables, and one dependent variable were used to construct a comprehensive model. Furthermore, corresponding hypotheses were formulated, as shown in Figure 1. Three factors, namely attitudes, subjective norms, and perceived behavioral control, serve as variables that predict teachers' behavioral intentions; perceived usefulness, perceived ease of use, and compatibility serve as variables that predict teachers' attitudes; superior influence, peer influence, student influence, and parental influence serve as variables that predict teachers' subjective norms; self-efficacy and facilitating conditions serve as variables that predict teachers' perceived behavioral control; and cognitive readiness and behavioral readiness serve as variables that predict teachers' self-efficacy.

The relevant hypotheses are explained as follows:

**Figure 1.** The proposed model with the initial hypotheses.

#### *2.1. Attitudes*

Emotional readiness explains how emotions affect teachers' achievements in performing their duties [36]. Instructional emotional readiness reflects the impact of teachers' emotions on their teaching performance. In the DTPB model, attitude is considered to be the degree to which an individual is positive or negative about a particular behavior [15,38],

and it has a corresponding impact on whether to implement the target behavior [39], which is similar to the concept of emotional readiness. As a result, to reduce the conceptual overlap of influencing factors and simplify the model, attitudes and emotional preparedness were studied as the same dimension. The attitude of teachers was mainly evaluated from whether they could generate positive emotions in their implementation of STEM education and achieve good results in STEM education. Attitudes' influence on an individual's behavioral intentions has been strongly supported by previous studies [25,40]. Therefore, the following hypothesis was proposed:

**Hypothesis 1:** *The attitudes/emotional readiness of teachers towards STEM education have a significant impact on teachers' intentions to implement STEM courses.*

According to the deconstruction of the DTPB model for attitude, attitude was also decomposed into three aspects: perceived usefulness, perceived ease of use, and compatibility. Perceived usefulness mainly concerns the perceived STEM benefits of individual teachers when they choose to offer STEM courses. Perceived ease of use refers to the degree to which a person finds it easy to use a specific system [41], as well as the degree of difficulty for teachers who implement STEM courses. Compatibility refers to the degree of fit of technology and existing potential value and experience [42]. It also refers to the degree of adaptation between STEM education and the teacher's teaching concepts, existing teaching experience, etc.

Therefore, the following hypotheses were proposed:

**Hypothesis 1a:** *Perceived usefulness has a significant impact on teachers' attitudes toward engaging in STEM education.*

**Hypothesis 1b:** *Perceived ease of use has a significant impact on teachers' attitudes toward engaging in STEM education.*

**Hypothesis 1c:** *Compatibility has a significant impact on teachers' attitudes toward engaging in STEM education.*

#### *2.2. Subjective Norms*

Subjective norms refer to social pressures that lead individuals to perform specific behaviors, which reflects the cognition of important reference disciplines that want individuals to perform or not perform certain behaviors [25]. In this research, subjective norms refer to the teachers' perceptions of the related groups' encouragement and acceptance of their behaviors when they implement STEM education activities. Subjective norms of behavior are usually found to be highly accurate in predicting behavioral intentions [15].

Therefore, the following hypothesis was proposed:

#### **Hypothesis 2:** *Subjective norms have a significant impact on teachers' intentions to implement STEM courses.*

As different groups may have different opinions on adopting the same specific behaviors, Taylor and Todd believe that there are three important reference groups in an organizational environment, namely peers, superiors, and subordinates. They suggested breaking down the whole population into these three types of reference groups. In the field of education, in addition to teachers, teachers' superiors, teachers' colleagues, and students, the parents of students are also involved in primary and secondary education [43]. Equally, parents also have some influence on teachers. If teachers are not trusted and respected by the parents of their students, they may develop a sense of vulnerability [44]. Thus, whether parents influence the subjective norms of teachers is also a question worth verifying. We mainly used superiors, colleagues (other teachers), students, and parents as

reference groups. Taylor considered that reference groups may influence the regulation of individual subjective norms. Therefore, the following hypotheses were proposed:

**Hypothesis 2a:** *Superiors' views of STEM education have a significant impact on teachers' subjective norms.*

**Hypothesis 2b:** *Colleagues' views of STEM education have a significant impact on teachers' subjective norms.*

**Hypothesis 2c:** *Students' views of STEM education have a significant impact on teachers' subjective norms.*

**Hypothesis 2d:** *Parents' views of STEM education have a significant impact on teachers' subjective norms.*

#### *2.3. Perceived Behavioral Control*

Perceived behavioral control reflects the beliefs about the acquisition of required abilities, resources, and opportunities, or the perception of possible internal and external factors that may hinder behavior execution [15,45]. In this research, perceived behavioral control includes the perception of the internal and external constraints of individual teachers when implementing STEM education. Research has shown that perceived behavioral control is an important determinant of intentions [25]. For example, through empirical investigations, Lin and Williams [19] verified that perceived behavioral control of a higher level was related to stronger STEM instructional intentions. Therefore, the following hypothesis was proposed:

**Hypothesis 3:** *Teachers' perceived behavioral control over STEM education has a significant impact on teachers' intentions to engage in STEM education.*

The DTPB model divides perceived behavioral control into internal and external constraints, which specifically indicate self-efficacy, resources, and technical factors. In this research, self-efficacy reflects teachers' self-evaluation of their abilities to implement STEM courses, indicating the degree of confidence in implementing STEM education. Greater selfefficacy leads to higher behavioral intentions [25,46]. Sadaf [28] found that self-efficacy was the strongest predictor of teachers' intentions. Convenience refers to technology, resources, and other objective factors available to teachers when implementing STEM courses. A lack of convenience may negatively affect the behavioral intentions of teachers [25].

Readiness is a significant predictor of an individual's propensity to use new technologies [47,48]. Teachers' preparation of knowledge and their self-efficacy in STEM are the key to successful STEM implementation [35]. Furthermore, for different topics such as technology applications, jobs, careers, teaching, and learning, relevant research has confirmed that there is a significant correlation between the degree of readiness and selfefficacy [49–51]. Although no research has been conducted to show that the degree of teachers' STEM readiness directly affects their teaching efficacy, it can be deduced from the relevant literature that the former could significantly predict the latter. To reduce concept repetition and simplify the models, this research chose the dimensions of cognitive and behavioral readiness from the degree of readiness to investigate and discuss whether these two could predict the self-efficacy of teachers. The cognitive readiness of teachers refers to the improvement of thinking and the ability to make cognitive choices to solve STEM problems or complete STEM teaching tasks [33]. Behavioral readiness refers to the attitudes towards STEM, changing emotional reactions, and the changes that can be seen in STEM teaching behaviors [36].

Based on this, we proposed the following hypotheses:

**Hypothesis 3a:** *The self-efficacy of teachers' STEM education has a significant influence on perceived behavioral control.*

**Hypothesis 3b:** *Facilitating conditions of teachers' STEM education have a significant influence on perceived behavioral control.*

**Hypothesis 3c:** *The behavioral readiness of teachers' STEM education has a significant influence on self-efficacy.*

**Hypothesis 3d:** *The cognitive readiness of teachers' STEM education has a significant influence on self-efficacy.*

#### **3. Methods**

#### *3.1. Research Design*

Based on the DTPB, this research constructed a model that could explain and predict teachers' STEM teaching preparation extent and behavioral intentions. Based on this model, we designed a questionnaire concerning K12 teachers' intentions to participate in STEM education and analyzed the collected questionnaires with a quantitative research method, i.e., a structural equation model, to verify the hypothetical model.

A total of 532 K12 in-service teachers of all disciplines from different regions in China were randomly contacted to take part in the survey. Informed consent was provided in the survey so that all the participants knew they were participating in an evaluation study and the data they provided were anonymous. Since some of them had selected the same options in the scale selections or wrote that they "never understood STEM education" in open questions, their questionnaires were considered invalid. Having removed invalid questionnaires, there were 479 valid questionnaires left, with an efficiency rate of about 90%. The samples covered the eastern, central, and western regions of China, as well as some other regions. From a gender perspective, more female teachers (59.9% of the total samples) took part in the survey. Young teachers (under 45 years old) accounted for a high percentage of about 73%, and middle-age teachers (over 46 years old) accounted for about 17%. Most of the teachers (78.9%) were undergraduates, and a small percentage (19.2%) had a master's degree. More than half of the teachers (57%) were fairly experienced (with a seniority of more than 10 years). The participants mainly consisted of information technology teachers (32.6%), and they mainly taught in junior and senior high schools.

#### *3.2. Research Instrument*

The questionnaire consisted of three parts with 79 questions. The first two parts were compulsory. Teachers' background information was collected in the first part with 7 questions including gender, age, and the school grades they taught. The second part was a scale for the survey of readiness and the factors that influenced their behavioral intention to implement STEM teaching in K12. The scale was mainly adapted from the DTPB scale constructed by Taylor et al. [25] and the scale constructed by Abdullah et al. to measure the readiness of STEM teachers [36]. Altogether, there were 61 adapted DTPB scale contents and 10 readiness scale contents, among which 67 were originally coded scale items and 4 were reverse-coded scale items. These 71 items constructed 15 potential variables, including behavioral intention, attitudes, perceived usefulness, perceived ease of use, compatibility, subjective norms, superior influence, peer influence, student influence, parental influence, perceived behavioral control, self-efficacy, facilitating conditions, behavioral readiness, and cognitive readiness. In order to avoid the antagonistic psychology of the participants, and to reduce the possibility of distortion of the questionnaire contents, we applied reverse-coded scale items for items of perceived ease of use. Most of these variables have been proven to be reliable and efficient in previous research on teachers' behavioral intentions [28–30,52]. At the same time, we applied a 5-point Likert scale for grading, ranging from 1 ("strongly disagree") to 5 ("strongly agree"). The third part was

1 optional item. Here, we applied open-ended questions to collect teachers' opinions about STEM education. Detailed information about the research instrument can be found in Appendix A.

In order to ensure the content validity of the instrument, we invited experts and high-performance teachers of STEM education research to check the appropriateness and clarity of the items and structure of the questionnaire. Modifications were made according to their suggestions. Cronbach's α reliability coefficient was used to evaluate the internal consistency of the research instrument, and the test results can be found in Table 1. With regards to the 15 latent variables in the scale, their Cronbach α value range was 0.848–0.947, all meeting the standard of α > 0.7. Moreover, Cronbach's α of the whole scale was 0.980, indicating a high reliability and a reliable measurement index content. In order to verify the applicability of the factor analysis, we conducted the KMO measure test and Bartlett's test. The KMO value was 0.969 (greater than 0.9), which meant that the questionnaire data were appropriate for the factor analysis. Bartlett's test value was 0.000 (less than 0.05), which meant that it met the standard, the data were distributed spherically, and the variables were inter-independent to a certain degree.

**Table 1.** Reliability results.

#### **4. Analysis and Results**

Since the emotional readiness, behavioral readiness, and cognitive readiness in the constructed model of the research did not have a theoretical framework, we applied a method that combined both exploratory factors and confirmatory factors to test the model. We aimed to extract three main dimensions in the factor analysis, and the minimum coefficient that can be displayed was set to 0.5. The rotated component matrix is shown in Table 2. We can see that items CR1–CR5 were concentrated under the first dimension, BR2–BR5 under the second, and BI1–BI5 under the third. These items were reserved, since they accorded with the original expectation. However, since BR1 was originally anticipated to belong to the behavioral readiness dimension together with BR2–BR5, while it turned out to be concentrated under the cognitive readiness dimension, we deleted measurement item BR1 because it did not accord with the expectation.

**Table 2.** Rotated component matrix.

Note: Factor loadings <0.50 are omitted, and factor loadings are sorted. CR: cognitive readiness; BR: behavioral readiness; AT: attitudes/emotional readiness.

#### *4.1. Data Analysis of Teachers' STEM Education Readiness*

The analysis results of the descriptive statistics of readiness-relevant items showed that the averages of teachers' cognitive, emotional, and behavioral readiness were 3.64, 4.03, and 3.47, respectively, and the average points of each dimension (variable) were all between 3 and 4, which means that the object groups of the research all had a cognitive, emotional, and behavioral readiness level above the medium level. The emotional readiness level was above the others, while the behavioral readiness level was the lowest. Moreover, the correlations r among the three dimensions were 0.564, 0.505, and 0.769, respectively, and all greater than 0. This means that there was a significant positive correlation among different variables.

We conducted a demography variable difference analysis through an independent *t*-test and ANOVA and found that there was no significant correlation between teachers' genders, teaching disciplines, ages, majors, school grades, seniorities, etc., and the overall level of STEM readiness and the three dimensions. Differences were found only in parts of the items. For example, this research took mathematics, physics, chemistry, geography, information technology, comprehensive practices, and scientific disciplines as STEM-relevant disciplines, while Chinese, English, politics, music, etc., were considered STEM-irrelevant disciplines. The results of the independent *t*-test show that the emotional readiness dimension items AT1, AT2, AT3, and AT4, the behavioral readiness dimension items CR4 and CR5, and the cognitive readiness dimension item BR4 all had Sig values of less than 0.05, indicating a significant difference.

#### *4.2. Data Analysis of Teachers' STEM Education Intention*

In order to analyze teachers' STEM education intention level and influence factors, we firstly applied SPSS 26.0 to obtain descriptive statistics about the behavioral intention dimension. This was carried out as part of the survey of the current level of behavioral intention of China's K12 teachers to implement STEM education. Secondly, AMOS 26.0 was applied to correct the constructed structural equation model, and to test its reliability

and validity. Finally, based on this model, we tested the influence factors of the behavioral intentions of China's K12 teachers to implement STEM education.

As shown in Table 3, the total average level of teachers' behavioral intentions was 3.724, which means that the STEM behavioral intentions of the research subject groups were at a medium to upper level.

**Table 3.** Descriptive statistics of each item of behavioral intention.

Note. BI: behavioral intention.

#### 4.2.1. Reliability and Validity Tests

This research applied AMOS 26.0 to test the original model and found that the SMC values of CO2 and CR5 were less than 0.5, indicating a weaker corresponding relation between the two measurement items and their dimensions. Therefore, we removed them. We also corrected the model according to the MI (modification indices) standard and found some unreasonable measurement items, including PU1, PEU2, SUI1, SUI3, CI3, CI4, STI4, STI5, PA4, PBC1, SE5, SE6, FC1, and BR5; therefore, they were also deleted. In order to make sure the corrected model could effectively assess teachers' behavioral intention to implement STEM education, this research first conducted the analysis at the measurement level to weigh the measurement effect of the observed variables on the latent variables, and then conducted structural model analysis to test the structural relationships among the latent variables.

#### 1. Measurement-Level Analysis

Through our test, both parts of the research, i.e., the experiment and corrected model, had a Cronbach alpha value greater than 0.8 (Tables 1 and 4), which proves their good internal consistency. Usually, the standardized factor loading, average variance extracted (AVE), SMC, and its composite reliability estimation are used to test the convergent validity of a model [53]. With regards to the AVE, the suggested value is 0.5 or above [54]. The suggested value for the standardized factor loading is 0.50 or above; in an ideal situation, it would be 0.70 or above [54]. The suggested value for the SMC is greater than or equal to 0.5. As shown in the results in Table 4, each value reached the stipulated threshold, showing that the model had good convergent validity.

In order to test the discriminant validity of the model, we compared the index of the square root of the AVE and the absolute value of correlation coefficients among the potential variables. As shown in Table 5, the square root of the AVE of each potential variable was higher than the correlation among the latent variables, which indicates that the discriminant validity of the model was good.

**Table 4.** Convergent validity analysis in CFA (confirmatory factor analysis).

Note. AVE: average variance extracted values; SMC: squared multiple correlations.

Note. AT: attitudes/emotional readiness; PU: perceived usefulness; PEU: perceived ease of use; CO: compatibility; SN: subjective norms; SUI: superior influence; PEI: peer influence; STI: student influence; PAI: parental influence; PBC: perceived behavioral control; SE: self-efficacy; FC: facilitating conditions; CR: cognitive readiness; BR: behavioral readiness; BI: behavioral intention.

#### 2. Structural model analysis

In order to evaluate the model fit, according to the research of Hu and Bentler [55], Kashy et al. [56], Widaman [57], and Ejaz Ahmed [58], we applied the comparative fit index (CFI), incremental fit index (IFI), Tucker–53Lewis index (TLI), chi-square (x2)-to-degree of freedom (df) ratio, root mean square error of approximation (RMSEA), and standardized root mean residual (SRMR) values. The traditional rule of thumb in CFA (confirmatory

factor analysis) stipulates that when the x2/df value is less than 3, the SRMR is less than 0.08, the IFI, TLI, and CFI are all greater than 0.9, and the RMSEA is less than 0.08, the model fit can be considered a good one. This research applied the maximum likelihood (ML) estimation technique to evaluate the model parameters, and the model fit indices we obtained are shown in Table 6, all of which met the CFA stipulated index. The x2/df (1.954), SRMR (0.0485), and RSMEA (0.045) were all lower than the maximum requirement value, and the IFI (0.953), TLI (0.948), and CFI (0.953) were all higher than the minimum requirement value, which proves that the model fitted well with the data.

**Table 6.** Statistics for the final models.

Note. The fit indices are based on the work of Hooper et al. [59], Hu and Bentler [55], and Kline [60]; x2/df: chi-square-to-degree of freedom ratio; SRMR: standardized root mean square residual; IFI: incremental fit index; TLI: Tucker–Lewis index; CFI: cumulative fit index; RMSEA: root mean square error of approximation.

#### 4.2.2. Assessment of Hypothesized Relations

According to the model fit test, the fit of this model was good. Through the path coefficients and hypothesis testing, we obtained the model of teachers' intentions to implement STEM education, as shown in Table 7. When the value of the critical ratio (C.R.) was 1.96 or higher, the value of the coefficient (P) was considered significant at the level of 0.05 [53]. In this research, we used the path coefficients (β) of the fitted model and the *p*-value in the analysis and verified that 12 hypotheses, including H1, H3, H1a, H1b, H1c, H2a, H2b, H2d, H3a, H3b, H3c, and H3d were valid, while two hypotheses, namely H2 and H2c, were rejected. Therefore, we constructed the model of teachers' intentions to implement STEM education and its factor path coefficients (as shown in Figure 2). Every path constructed from one latent variable to another showed their relationships, and the path coefficient points of the valid hypotheses were between 0.093 and 0.890, which indicates a positive correlation among the exogenous variables and endogenous variables of the twelve valid hypotheses. In other words, the paths had statistical significance and supported the model.

The value of the SMC (squared multiple correlations) represents the strength of the structural relationships [61]. The value of the SMC ranges between 0 and 1, where stronger relationships are indicated by values close to 1 [62]. The explained maximum variance of the three exogenous variables (perceived behavioral control, subjective norms, and attitudes) to behavioral intention was 0.90. The SMC value of the main endogenous latent construct, i.e., behavioral intention, was 0.53. That is, 53% of the variance among behavioral intention, attitudes, subjective norms, and perceived behavioral control could be explained by behavioral intention.

**Table 7.** Path coefficients and hypothesis testing; \*\*\* *p* < 0.001.

Note. S.E.: standard error; C.R.: critical ratio.

**Figure 2.** Final model with the coefficients (β) and *p*-values. Note. Dotted lines indicate an insignificant relationship; \* *p* < 0.05, \*\* *p* < 0.01, and \*\*\* *p* < 0.001.

#### **5. Discussion**

The data reported here indicate that teachers' readiness for STEM education was generally in the high to average range, among which emotional readiness was the highest. This finding demonstrates that K12 teachers have positive attitudes towards STEM education, and that they were satisfied with the effects of STEM education and were willing to apply

STEM education in their classes. Research conducted in Indonesia evaluating teachers' attitudes towards STEM education found similar results [63]. From a cognitive aspect, the teachers' moderate level of readiness indicates that they had a basic understanding of STEM education. Though the teachers had a certain understanding of STEM education in terms of knowledge and methods as well as the teachers' role, their understanding of comparatively sophisticated STEM programs was still scant. Research conducted by DanyiZheng [33] obtained similar results. The level of teachers' readiness from a behavioral aspect was relatively low compared to that of cognitive readiness and emotional readiness. According to the questionnaire data, teachers' behavioral readiness focused on making elaborate plans before STEM education, but most teachers lacked enough time to implement STEM education in practice. This finding is similar to that of Abdullah [36], which suggested that teachers' behavioral readiness was at an intermediate level and most teachers did not have ample time to implement STEM education. Variables including the teachers' genders, disciplines, ages, majors, teaching stages, and seniorities did not have a significant correlation with the overall level of teachers' readiness and its three aspects, namely cognitive, behavioral, and emotional readiness, for STEM education. More relevant variables should be further explored.

The statistics of intention demonstrated that teachers' behavioral intentions were in the high to average range. This finding, although different from that of Peng [20] and Li [21], is consistent with that of Tse [22]. It was shown that with the introduction of relevant policies and the development of STEM education practices, many K12 teachers realized the importance of interdisciplinary teaching and were willing to implement STEM education in practice.

The data collected from SEM indicated that teachers' behavioral intentions to implement STEM education were mainly impacted by their attitudes and perceived behavioral control, with strong positive effects, while subjective norms did not generate prominent effects. Relevant studies have verified that attitudes, subjective norms, and perceived behavioral control are impact factors on behavioral intention [64–67]. Furthermore, some researchers have also found that subjective norms were normally a weak predictor of behavioral intention [28,68,69]. In addition, research by Atsoglou [70] and Haya Ajjan [71] argued that attitudes and perceived behavioral control were comparatively strong positive impact factors, while subjective norms had no impact on behavioral intention. Haya Ajjan explained that this was because teachers had a high degree of independence in improving the classroom environment.

Negative attitudes may cause teachers to be reluctant to work in STEM education. Three factors, namely ease of use, perceived usefulness, and compatibility, all had impacts on teachers' attitudes, with perceived usefulness having the strongest. Research by Ayesha Sadaf [28,72,73] and Mdutshekelwa Ndlovu [30] pointed out that perceived usefulness, compared with ease of use and compatibility, exerted the greatest impacts on teachers' attitudes. Consequently, to improve teachers' attitudes towards STEM, corresponding interventions should mainly focus on enhancing teachers' awareness that teachers' actions contribute to results, and on guiding teachers to positively evaluate results. Only when teachers realize that STEM education is helpful to their work and can improve students' abilities in all aspects will they be willing to adopt STEM education.

The greater the sense of control teachers have over STEM education, the more willing they are to apply STEM education. This study found that both self-efficacy and facilitating conditions had positive impacts on perceived behavioral control for teachers in STEM education, of which self-efficacy was the strongest impact factor. In previous studies, researchers such as Ayesha [28,73], Mdutshekelwa Ndlovu [30], and Kuan-Chuan Tao [31] all confirmed that self-efficacy and facilitating conditions were impact factors of perceived behavioral control, and most of these studies [28,30,73] showed that, compared with facilitating conditions, self-efficacy had the most significant impacts on behavioral control, which is consistent with the conclusions drawn in this study. This indicates that if schools can provide effective support with resources, it will be easier and more convenient for

teachers to carry out teaching activities, and, more importantly, it will improve teachers' confidence in teaching. Confidence is built on success, and thus, a reliable way to improve self-efficacy is to repeatedly experience a sense of success in the middle of completing a certain task. Therefore, to improve their proficiency in STEM education, teachers should strive for successful experiences and belief in conducting STEM education, and thus believe that they are qualified for success. After that, teachers should have high "self-efficacy" with confidence in their capabilities of conducting STEM education and consequently be willing to implement STEM education.

The significant impacts of teachers' readiness on their intentions to implement STEM education were tested in the hypotheses established by this model. Currently, there is no research exploring the impacts of teachers' readiness on their intentions to implement STEM education, but some studies have confirmed that teachers' readiness for AI significantly influenced their intentions to implement AI teaching [48]. Highly consistent with other studies [21,22], this study found that emotional readiness had a significant correlation with teachers' intentions to implement STEM education. In addition, both behavioral readiness and cognitive readiness had positive impacts on the self-efficacy of teachers in implementing STEM education, among which behavioral readiness was the strongest predictor. This shows that the more teachers understand and prepare for STEM education, the more they will enhance their self-efficacy, which has been verified to be an effective factor in improving teachers' perceived behavioral control in STEM education. Although there are no studies directly exploring the impact of teachers' cognitive readiness and behavioral readiness on teachers' self-efficacy, researchers have found that TPACK (technological pedagogical content knowledge) had an indirect impact on teachers' STEM educational intention [22]. In addition, teachers who fail to prepare themselves well with knowledge of STEM [74] such as science, mathematics, and engineering [75] may be afraid of teaching design-related engineering content to students [76], thus having a low self-efficacy in STEM education. For many primary and secondary school teachers in China, STEM education is still a relatively unfamiliar teaching method. For example, in this study, some primary and secondary school teachers mentioned that they did not understand "what STEM education is". Though the Chinese government encourages and guides colleges and enterprises to commence STEM training for teachers, such training still hardly attracts teachers to participate and receives unsatisfactory results due to the incomplete training mechanism. Teachers' self-efficacy in teaching is usually highly intertwined with the corresponding knowledge and positive experience [77–79]. Insufficient cognitive and behavioral preparation would naturally undermine teachers' confidence in implementing STEM education. Therefore, with the aim of promoting STEM education, the top-level design of STEM professional development for teachers should be strengthened from an emotional aspect (attitudes), cognitive aspect, and behavioral aspect.

#### **6. Conclusions and Further Developments**

#### *6.1. Conclusions and Suggestions*

This study established and tested a research model of the behavioral intention of K12 teachers to implement STEM education based on the DTPB (decomposed theory of planned behavior) and teachers' readiness. The results of this study demonstrate that the model fit the data well and was suitable for the research of teachers' behavioral intentions to implement STEM education. Constructs including self-efficacy, perceived behavioral control, perceived usefulness, behavioral readiness, attitudes, cognitive readiness, compatibility, facilitating conditions, and perceived ease of use had direct or indirect significant impacts on K12 teachers' behavioral intentions to implement STEM education. Amid the above-mentioned constructs, self-efficacy, perceived behavioral control, perceived usefulness, behavioral readiness, attitudes, and cognitive readiness had comparatively stronger impacts. However, there was no evidence that subjective norms had significant impacts on teachers' intentions to implement STEM education. In addition, based on the analysis of Chinese K12 teachers' readiness from emotional, cognitive, and behavioral aspects, this

study found that teachers' readiness for STEM education was a strong predictor of teachers' behavioral intentions, which is consistent with the results of Ayanwale et al. [48]. This study also indicated that Chinese K12 teachers' readiness for and intentions to implement STEM education were both in the high to average range. "Behavioral readiness" was a strong predictor of teaching intention to implement STEM education, but it maintained at a relatively minimal level.

The findings of this study have certain reference significance for policymakers and educators, and serve to promote STEM education and truly improve students' STEM literacy and problem-solving abilities in an interdisciplinary and innovative manner [2]. Compared with studies of its kind, this study achieved breakthroughs in terms of the sample size, the scope of the research objects, and the basic theoretical model and creatively proposed adding teachers' readiness into the DTPB model. Through analyses of K12 teachers' readiness and impact factors of behavioral intentions to implement STEM education, this study provides a reference for the development of STEM education in primary and secondary schools. Policymakers can follow the findings of this study to formulate practical top-level strategic plans and effective STEM teacher professional development programs around key impact factors to strengthen teachers' interdisciplinary teaching literacy.

Given the results of this study, how to improve teachers' teaching intention to implement STEM education can be discussed from two aspects: attitudes and perceived behavioral control. Firstly, only when teachers hold positive attitudes towards the practicability of a certain action can they adopt positive attitudes towards such behavior. Perceived usefulness was the most significant predictor of attitudes, which demonstrates that the teachers paid great attention to the effects of the implementation of teaching behavior; thus, it was crucial to improving teachers' sense of value in teaching. This inspires us to encourage more teachers to develop STEM projects and implement STEM education in real practice so that teachers can compare their original teaching methods with practical cases and truly experience the value of STEM education. Secondly, self-efficacy was the strongest predictor of perceived behavioral control, meaning it is a necessity to enhance teachers' self-efficacy in STEM education. The authors suggest that corresponding interventions should be implemented based on cognitive readiness and behavioral readiness. In terms of cognitive readiness, teachers are expected to pay more attention to relevant requirements, learn from excellent cases, and spend time analyzing them in detail to deepen their understanding of STEM education and their cognition of teachers' roles. In addition, schools can organize relevant lectures and competitions for teachers. With regard to behavioral readiness, most teachers noted that they lack enough time to implement STEM education. Thus, the workload of teachers on non-teaching tasks should be reduced to ensure that they have enough time to prepare for STEM courses. Teachers should also improve teaching strategies, integrate STEM education into discipline education, and determine specific teaching contents tailored to students with different characteristics to make the class more interesting. Considering that support, guidance, and leadership are critical for teachers to shift from traditional teaching styles [80], schools should strengthen their pedagogic support for STEM education and provide opportunities for cooperation among teachers of different disciplines. Finally, favorable facilitating conditions can also improve teachers' perceived behavioral control, thereby enhancing teachers' teaching intentions. The state and government can provide certain funds or formulate policies so as to provide corresponding educational spaces, resources, technologies, and expert guidance, among others, for conducting STEM education. By establishing a cross-school and cross-regional STEM community of practice, a balanced layout of "facilitating conditions" can be encouraged.

#### *6.2. Limitations and Further Development*

Despite the contributions of this study, some limitations remain to be addressed. Firstly, since the data collected in this questionnaire were not balanced from different regions, i.e., there were relatively small sample sizes from central and western China, this study did not explore the impact of regions on teachers' readiness for and intentions to

implement STEM education. Previous studies have proven that data from different regions could predict behavior more accurately and meaningfully [81]. Therefore, more studies investigating whether the educational development of regions has impacts on teachers' readiness and teaching intentions to implement STEM education should be carried out. Secondly, the significant impacts of subjective norms on teachers' intentions to implement STEM education have yet to be tested, but this does not indicate that there is no significant impact between them. Research on non-STEM education has proven that subjective norms are among the impact factors of behavioral intention [67,82]. Further studies can focus on the correlation between subjective norms and teachers' intentions to implement STEM education. Thirdly, most of the previous studies focused on explaining and predicting behavioral intention and actions. Since the DTPB model can intervene in behavior, further studies can explore how to intervene in STEM teaching behavior based on the DTPB.