*Article* **Development and Psychometric Properties of a Scale to Measure Resilience among Portuguese University Students: Resilience Scale-10**

**Jacinto Jardim 1,\* , Anabela Pereira 2 and Ana Bártolo 2**

**Abstract:** Higher education students lack skills to deal in a positive way with the crises inherent to the developmental phase in which they find themselves. However, the complexity of the current global context adds an urgent need for them to be resilient. For this purpose, it is necessary to evaluate their capacity for resilience, which requires the use of instruments that are easy to access, useful, simple, and fast. In this sense, this study aimed to develop and validate a scale to assess students' resilience in the face of adversity. The scale was administered to a sample of 2030 Portuguese higher education students. The results obtained pointed towards a factorial structure composed of two factors named "self-determination" and "adaptability", which showed good internal consistency. Therefore, this scale proved to be a valid measure to assess resilience among the university population. Future studies may consider this variable as an intervention target since it can be a predictor of success in phases of change and crisis.

**Keywords:** resilience; validity; higher education; resilience scale-10

#### **1. Introduction**

Given the continuous transformation of society and the unpredictability of its future, knowing how to deal with adversities has become an essential competence to survive healthy and with quality. This capacity, called resilience, has gained relevance in scientific research [1–6].

The resilience has been defined from different perspectives: As an interactive concept which is concerned with the combination of serious risk experiences and a relatively positive psychological outcome despite those experiences [7]; as a universal capacity that allows a person, group or community to prevent, minimize or overcome the harmful effects of adversity [8,9]; as the capacity of a dynamic system to adapt successfully to disturbances that threaten the system function, viability, or development [10]; as the outcome from negotiations between individuals and their environments for the resources to define themselves as healthy amidst conditions collectively viewed as adverse [11]; and also as the "ability to use knowledge, attitudes and skills in order to prevent, minimize or overcome the harmful effects of crises and adversities" [2] (p. 167). Therefore, a resilient person, facing a stressful or adverse situation, manages to use his or her personal resources by adopting behaviors that help being successful in that circumstance.

If the complexity of the current global context calls for psychological support for the general population [12], in the case of higher education students, the need for this competence is further justified by the fact that they have to face the crises inherent to the developmental stage in which they find themselves, and the obstacles of the education system itself, as well as the unpredictability of the labor market, in which they will soon be integrated [13–16].

**Citation:** Jardim, J.; Pereira, A.; Bártolo, A. Development and Psychometric Properties of a Scale to Measure Resilience among Portuguese University Students: Resilience Scale-10. *Educ. Sci.* **2021**, *11*, 61. https://doi.org/10.3390/ educsci11020061

Academic Editor: Maria José Sousa Received: 14 January 2021 Accepted: 1 February 2021 Published: 7 February 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

Previous studies point towards the need to promote this capacity among higher education students as it facilitates adaptation to changes and ambiguous situations, the recovery from exhausting and tiring conditions, as well as the proficiency in maintaining a calm, clarity of objectives and guidance in hostile situations. Additionally, it enables the student to think strategically and make appropriate decisions in times of pressure, to perform complex tasks, and to adopt flexible behaviors in problem solving. In this domain, several intervention programs have been evaluated as effective [17–19]. For example, the program You Can Do It! [18], which has been widely applied in multiple Australian schools, presents resilience as the basis for social and emotional learning. Nevertheless, research indicates that the effectiveness of a program depends a lot on the type of activities implemented [19]. Therefore, resilience is a concept that should continue to be investigated, so that the programs made available to young students include pedagogical strategies and resources more oriented to the promotion of this capacity.

In order to assess resilience and determine the criteria for the functioning of the resilient person, several scales have been developed and validated [3,20–25]. Most of the existing scales use mainly clinical criteria, as proposed by Anaut [17,18]: The level of anxiety and depression, the level of social competence, school and intellectual success, and clinical symptoms. Therefore, they focus on behavioral and cognitive assessments, personality traits, and psychological disorders. In turn, Sætren et al. [20] (p. 3) proposed an evaluation through the "three-factor model of personal resiliency", that reflects aspects of the individual's personal experience in three dimensions of his or her own development: "Sense of mastery, sense of relatedness, and emotional reactivity". Another instrument also used to assess resilience is the one by Prince-Embury [25], called resiliency scales for children and adolescents (RSCA), which assesses psychological symptoms and psychological status. RSCA comprises three scales: Sense of control—assesses personal attributes such as optimism, self-efficacy, and adaptability; relationship capacity—assesses trust, support, comfort, and tolerance; and emotional reactivity—assesses sensitivity, recovery, and injury.

Therefore, most of the aforementioned instruments address resilience at the clinical level, with predominance to traumatic and health situations of children and adolescents. Additionally, they are often complex scales in terms of completion and interpretation, which makes both application and accessibility difficult in contexts where a large number of individuals need to be evaluated, as in the case of higher education. In order to bridge this gap, we aimed to develop and assess the psychometric proprieties of a new self-assessment scale—resilience scale-10 (RS-10)—to measure resilience in a non-clinical community. More specifically, the purposes of this study were to (i) develop a brief measure of resilience appropriate for university students; and (ii) examine the factor structure and reliability of this scale.

#### **2. Methods**

#### *2.1. Development of the Portuguese Version of the RS-10*

This study consisted of two phases, beginning with the scale development phase aiming at identifying the appropriate items to include in a self-report measure. A set of items related to resilience was collected, considering the following parameters: Knowledge obtained in the literature; objectives of the instrument to be built; population to be evaluated; dimensions of the construct to be evaluated; and behavioral aspects of the construct to be included in the evaluation [26]. Additionally, in order to define the RS-10 items, we used as a reference the questionnaires developed by Grotberg [8,26] and Anaut [1,19]. Based on this process, 30 initial items were obtained. These 30 items were presented to a panel of experts with seven researchers/university professors in the areas of psychology, education, and management, obtaining a reduced final version including only 10 items. The 10 selected items were included since they were more directly related to the ability to deal with adversities in daily life, without a focus on traumatic situations. This version went through a process of spoken reflection carried out with a class of 3rd year students from a private university in Porto, consisting of 22 students (7 male and 15 female). These

participants were instructed to report their subjective perception of personal skills to face adversity. Each item was answered using a 5-point (1 = never; 2 = rarely; 3 = sometimes; 4 = almost always; 5 = always) response scale, so possible scores ranged from 10 to 50. Higher scores indicated a higher level of resilience. The application of the questionnaire lasted approximately 10 min and, subsequently, questions related to the instructions for completing the instrument, suitability of the included items, order of the items, and eventual doubts related to the semantics were discussed with the students (see the final version in Table 1; see also Appendix A). Thereafter, the scale's psychometric properties were assessed using a sample of Portuguese higher education students.

**Table 1.** Description of the items of the resilience scale-10 (RS-10).

#### *2.2. Validating the RS-10: Sample*

The sample included 2030 Portuguese university students over 18 years old (M = 21.11; SD = 2.00; range: 18–26 years) and the majority were female (77.1%). Students were recruited from several Portuguese Higher Education institutions. Most university students were based in higher education institutions located in the North of Portugal (51.5%). Participants attended different training areas, namely social sciences (59.8%), education, health and tertiary services (33.1%), and exact sciences such as engineering, computer science, mathematics, building and agriculture, and transformation industry (7.1%). Among the students, 71.8% were undergraduates and 28.2% were master students. This study has been carried out in accordance with the Declaration of Helsinki and informed consent was obtained from all the participants.

#### **3. Data Analysis and Results**

#### *3.1. Properties of Items*

A preliminary analysis was conducted in order to assess descriptive statistics, normality, and non-multicollinearity at the item level (see Table 2). The mean response for the 10 items was 3.62 (SD = 0.20). No deviations from the normal distribution were found, as the kurtosis and skewness scores for each item fell within −2 and 2. All the items presented significant positive corrected item-total correlations (≥0.42). The internal consistency of

the total scale was good (α = 0.866) and there was a low variation in reliability if items were deleted. All inter-correlations among all the items were below 0.65 suggesting no multicollinearity. Based on this, all the items were retained from subsequent analyses.

#### *3.2. Factorial Validity*

To test the factorial structure of the RS-10, a combination of the principal components analysis (PCA) and confirmatory factor analysis (CFA) was conducted. The sample was randomly split into two samples through the randomization function on the Statistical Package for Social Sciences, version 24 (SPSS Inc., Chicago, IL, USA), resulting in 2030 observations in total, 791 for the PCA and 1239 for the CFA.

PCA using the oblique rotation method suggested a two-factor structure. The Kaiser-Meyer-Olkin (KMO) measure presented a value of 0.897 and Barlett's test of sphericity was significant (χ 2 = 3412.112, *p* < 0.001). Factor 1 comprised items 1, 2, 3, 4, and 5 (5 items), which we termed as "self-determination", based on the three sources of resilience features labelled by Grotberg [8]. Factor 2 included five items (6, 7, 8, and 9, 10), which we termed as "adaptability". These two factors explained together 60.70% of the total variance. Table 3 shows the factor loadings (>0.60) and communalities (>0.40). Additionally, a good internal consistency for each factor was estimated (Factor 1: Cronbach's α = 0.843; Factor 2: Cronbach's α = 0.813).

The two-factor model derived from PCA was then cross-validated. Mardia's kurtosis coefficient of 20.95 with a critical ratio of 23.80 indicated that the data were multivariate non-normal. Therefore, CFA was performed using the maximum likelihood estimation (ML) with bootstrapping (1000 resamples) to generate accurate estimations of standard errors with accompanying confidence intervals (bias-corrected at the 95% confidence level). To assess the overall model fit, we use the following parameters: Root mean square error of approximation (RMSEA), comparative fit index (CFI), and the standardized root mean

square residual (SRMR). The criteria for an acceptable model fit were CFI ≥ 0.95; RMSEA ≤ 0.10 [27], and SRMR ≤ 0.08 [28]. The chi-square test (χ 2 ) was reported, but not used to check the model fit due to its sensibility to large samples [29]. Data obtained from the CFA showed that the two-factor model fitted well descriptively (CFI = 0.957; RMSEA = 0.068 (90% CI 0.059–0.076); SRMR = 0.040). All the standardized factor loadings (see Figure 1) of the item parcels were statistically significant showing that question items were good indicators for each factor. ≥ 0.95; RMSEA ≤ 0.10 [27], and SRMR ≤ 0.08 [28]. The chi-square test (χ2) was reported, but not used to check the model fit due to its sensibility to large samples [29]. Data obtained from the CFA showed that the two-factor model fitted well descriptively (CFI = 0.957; RMSEA = 0.068 (90% CI 0.059–0.076); SRMR = 0.040). All the standardized factor loadings (see Figure 1) of the item parcels were statistically significant showing that question items were good indicators for each factor.

7 0.730 0.568 8 0.792 0.673 9 0.786 0.607 10 0.705 0.437

The two-factor model derived from PCA was then cross-validated. Mardia's kurtosis coefficient of 20.95 with a critical ratio of 23.80 indicated that the data were multivariate non-normal. Therefore, CFA was performed using the maximum likelihood estimation (ML) with bootstrapping (1000 resamples) to generate accurate estimations of standard errors with accompanying confidence intervals (bias-corrected at the 95% confidence level). To assess the overall model fit, we use the following parameters: Root mean square error of approximation (RMSEA), comparative fit index (CFI), and the standardized root mean square residual (SRMR). The criteria for an acceptable model fit were CFI

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 5 of 8

**Figure 1.** Confirmatory factor analysis: Two-factor solution using a part of the sample (database 2; *n* = 1239). **Figure 1.** Confirmatory factor analysis: Two-factor solution using a part of the sample (database 2; *n* = 1239).

#### **4. Discussion and Conclusions**

**4. Discussion and Conclusions** Universities normally have to prepare students to face the multiple difficulties they have to face, both in academia and in the subsequent period of integration in the labor market. However, there are deficiencies regarding the acquisition of skills appropriate to face these challenges [16,30–32]. In times of crisis, as experienced globally today, these are Universities normally have to prepare students to face the multiple difficulties they have to face, both in academia and in the subsequent period of integration in the labor market. However, there are deficiencies regarding the acquisition of skills appropriate to face these challenges [16,30–32]. In times of crisis, as experienced globally today, these are even more evident.

even more evident. In order to carry out an effective intervention in this domain, it is important to evaluate each student individually in order to activate different strategies for them. This argument supported the need to develop a brief and accessible measure for the assessment of resilience in higher education students identifying the degree of their ability to cope with adversity. Our analysis showed that RS-10 is a valid measure and can contribute to the understanding of the perception of personal skills to face adversity among university students. The scale demonstrated a meaningful and strong factor structure, with items effectively measuring the factors, as shown by the loadings. According to the results, the two-factor model identified in the factor analysis indicated that RS-10 can capture the dimensions of self-determination and adaptability that adequately summarize resilience.

With self-determination, despite the difficulties, obstacles, discouragements, and failures, the individual is sure to complete the previously outlined tasks. Therefore, selfdetermination allows goals to be achieved and personal projects and ambitions to be realized. In addition, according to several authors, both from the perspective of clinical psychology, organizations and human resources management, self-determination is decisive in carrying out projects, despite the obstacles [33–37].

In turn, adaptability means the ability to change, whenever necessary, the strategy in order to achieve the previously determined objective. As such, it presupposes mental flexibility and emotional plasticity in order to better face unpredictable and unexpected situations. This is a construct that explains the success of those entrepreneurs who identify opportunities and implement projects. Furthermore, it is a characteristic of societies where innovation predominates, such as the current one we live in. Moreover, according to recent research, flexibility is an essential construct to integrate work teams, to lead people in organizations, and to maintain adequate levels of health and well-being [36,38,39]. Similar to other scales, RS-10 has good internal consistency (α = 0.866), with the factor referring to self-determination showing a Cronbach's α = 0.843 and the factor relating to adaptability revealed a Cronbach's α = 0.813. For example, the Portuguese version of the resilience scale of Wagnild and Young [19], composed of 25 items, which evaluates adolescents presented a Cronbach's alpha of 0.86, and the Brazilian version presented a Cronbach's alpha of 0.80 [23]. In turn, "The Multidimensional Teachers' Resilience Scale" [40] presents the following Cronbach's alpha values: Motivational = 0.78; emotional = 0.72; social = 0.74; professional = 0.75. Furthermore, the original version of the Brief Resilience Scale [18] demonstrated good internal consistency, with Cronbach's alpha values between 0.80 and 0.91. Although the RS-10 shows promising psychometric properties, the scale's overall validity needs to be interpreted in the light of potential limitations. First, the main limitation of this study has to do with the sample, which is only Portuguese, thus we suggest a transcultural validation. Furthermore, the Portuguese version of this self-reporting tool must be translated and validated in other languages. Second, there was no assessment of the reproducibility/repeatability. In addition, an important next step is to assess the convergent validity of the RS-10 through the correlations of the scale with theoretically related constructs such as self-efficacy [33,40] and self-esteem [41,42].

Perceived self-efficacy while as a positive belief allows you to carry out new or difficult tasks in order to obtain desired results. This vision of self-confidence, associated with self-control allows us to deal with stress-inducing situations and promote resilience.

Overall, the RS-10 was developed with input from university students and is a quickresponse instrument appropriate for use across different training areas and degree levels. However, the generalizability of the study results should be better established in future studies in additional settings. Despite this, the study presents opportunities for empirical studies in the field of resilience and we believe that the use of this instrument in clinical practice may contribute to intervene effectively next to students, promoting their capacity for self-determination and adaptability, which are crucial to face the adversities inherent to human nature and the evolutionary dynamics of the history of society.

**Author Contributions:** Conceptualization, J.J., A.P., and A.B.; investigation, J.J., A.P., and A.B.; writing—original draft, J.J., A.P., and A.B.; writing—review and editing, J.J., A.P., and A.B. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research received no external funding.

**Institutional Review Board Statement:** The study was conducted according to the guidelines of the Declaration of Helsinki, and approved by a ethics committee.

**Informed Consent Statement:** Informed consent was obtained from all subjects involved in the study.

**Data Availability Statement:** The data that support the findings of this study are available from the corresponding author [J.J.], upon reasonable request.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **Appendix A**

**Table A1.** Resilience scale-10 (RS-10) (Jacinto Jardim, Anabela Pereira, and Ana Bártolo, 2021) \*.

The following sentences refer to a variety of skills one may think to possess to a smaller or larger degree. Please rate each sentence considering how frequently you think in the way it describes, using the following rating scale: 1 = never; 2 = rarely; 3 = sometimes; 4 = almost always; 5 = always. \* Authorization to use this instrument should be asked from<EMAIL_ADDRESS>
#### **References**

### *Article* **Higher Education and Employability Skills: Barriers and Facilitators of Employer Engagement at Local Level**

**Fátima Suleman 1 , Pedro Videira 1,\* and Emília Araújo 2**

**Abstract:** This research examines the barriers and facilitators to employers' engagement with higher education institutions. The data were collected through interviews with a set of employers (*n* = 19) in the Northern region of Portugal, V.N.de Famalicão, in 2019. We begin by exploring employers' engagement activities as a potential solution to address local-level skill problems. Empirical evidence suggests that the engagement activities are mostly passive as firms use higher education largely as a recruitment channel. The differences in organizational goals and culture are the most cited barriers to the lack of more active engagement. Some efforts have recently been made to strengthen the ties between higher education and employers, notably through a local multi-stakeholder partnership as a potential broker. However, it will take time for this to bear fruit and contribute to reducing skill gaps and shortages. The data show that despite employers' apparent willingness, more effort must be made to encourage active engagement.

**Keywords:** higher education; employers' engagement; employability; multi-stakeholder partnership

#### **1. Introduction**

In recent decades, the collaboration between higher education (HE) and industry has grown significantly [1] fueled by marked changes in the modes of knowledge production and innovation [2,3] as well as increasing societal demands and expectations from higher education institutions (HEIs). This collaboration is based on the recognition that HEIs are an essential source of technology, innovation, and human capital [4]. Under the label of a third mission, HEIs are expected to interact with firms at the national and regional levels to promote innovation and competitiveness in the knowledge economy [5].

Only more recently have academics started to pay attention to other forms of collaboration regarding graduates' skills and training [4,6], namely in curriculum design and delivery [7], and in promoting employability skills [8]. Our study adds insights to this research agenda by examining the employers' perception of the barriers and facilitators to engaging with HEIs. More specifically, this research focuses on the engagement activities implemented by employers to develop employability skills and explores the factors that facilitate or hinder such engagement. These issues are raised by HEIs but also by employers themselves in the literature. This is a timely issue in the context of massification of HE and the skill shortages and gaps reported by employers worldwide; however scientific research is still scarce and geographically limited.

Graduate employability has become one of HE's institutional missions and it has been pursued through different means both inside and outside the classroom [9,10]. HEIs have been pressured to provide graduates with skills that match the economic and labor market skill needs and foster their employability [11,12] in a context of rising graduate unemployment levels and persistent skill problems and gaps reported by employers in different settings. The employers' engagement with HE in education and training has been promoted as a vehicle to bridge the divide between the supply and demand of higher-level

**Citation:** Suleman, F.; Videira, P.; Araújo, E. Higher Education and Employability Skills: Barriers and Facilitators of Employer Engagement at Local Level. *Educ. Sci.* **2021**, *11*, 51. https://doi.org/10.3390/ educsci11020051

Academic Editor: Eleanor Dommett Received: 30 December 2020 Accepted: 27 January 2021 Published: 30 January 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

skills [13,14], foster graduates' employability, and reduce skill mismatch, thus providing employers with a timely supply of required skills in the economy.

However, in spite of the growing interest in both graduate employability and the engagement between employers and HEIs in the development of skills and training, relatively little academic research has been undertaken on the types of engagement activities developed or the factors that facilitate or hinder them [13–15]. In addition, the literature focuses on few national cases, such as the UK and Australia; research from other European countries is remarkably scarce. Furthermore, the perception of employers deserves proper scrutiny since it provides HEIs with information on the barriers and facilitators to developing engagement. This paper contributes to this literature by attempting to answer the following questions: In which activities do employers engage with HE? What are the major obstacles and facilitators of this engagement?

The empirical analysis draws on qualitative data collected in 2019 through interviews with a set of employers (*n* = 19) from a city that has developed a multiple stakeholder partnership aimed at connecting different local stakeholders to promote employability, entrepreneurship, and innovation in the region. We have therefore studied employer engagement at a local/regional level in order to ascertain the extent to which this type of partnership fosters closer ties between HE and employers. This qualitative material was examined through content analysis which allowed us to categorize the skill problems faced by these employers at the graduate level, the engagement activities which they develop with HE, and the drivers and barriers to those engagement activities.

The rest of the paper is organized as follows. The next section provides an overview of the literature on employers' engagement with HE in education and training and the role of HE in developing employability skills, a brief overview of the national and regional contexts regarding HE as well as of the local partnership implemented by the city council. Section 3 is devoted to the methodology before reporting the empirical findings in Section 4. Finally, Section 5 provides a discussion of those findings as well as some policy implications.

#### **2. Theoretical Background**

#### *2.1. Tackling Skill Problems through Employers' Engagement with Higher Education*

The cooperation between higher education institutions (HEIs) and firms has grown significantly in recent times [1,4] fueled by increasing global competition, changes in modes of knowledge production and innovation as well as in the roles attributed to higher education (HE) by policy-makers and societies. This collaboration is expected to provide benefits to different stakeholders [16] by increasing firms' innovation and competitiveness in knowledge economies [5], improving graduates' employability [8], and providing new streams of financing to HEIs in the face of decreasing budgets [17]. Academic interest in university–business collaboration has thus increased dramatically [1] along with the recognition of HEIs as an essential source of technology, innovation, and human capital [4] in the triple-helix model of knowledge production [2] and its subsequent theoretical revisions [3].

However, academic interest has focused mostly on R&D and related activities [1], while collaboration with the development of skills has received less attention [4,6]. The literature on employer engagement attempts to fill this gap but it is mostly limited to a set of countries. Although people may interpret employer engagement with HE in training and education in different ways [18], it essentially comprises responses that help the upskilling of workers or the development of tools that enhance employability [19]. It is reported in the literature that employability is often assumed to be synonymous with work-readiness [12] and HEIs are trying to involve employers to prepare graduates for the world of work. The move towards making employers insiders and key actors in transforming employability into graduate employment [20] has only emerged since the 1990s and it aims to both improve graduates' employability prospects and provide a better response to economic and labor market imperatives [9] by promoting closer ties between the supply of skills and labor market demand.

Employer engagement may be enacted in different forms and through various activities, notably information exchange, internships and using HE as a recruitment channel, participation in job fairs, assessment of the quality of graduates, training opportunities, advice on curriculum and courses, co-design of certain courses, participating in the governance bodies of HEIs and co-funding as a partner.

Some authors further distinguish between "active" and "passive" forms of engagement [13]. A more "passive" engagement involves a simple market transaction and focuses on the acquisition of HEIs' products and services. Examples of the latter may be found in information exchange activities where employers turn to HE to provide information to access the best graduates [21], participation in job fairs [22], or through graduate recruitment [18]. In the context of information exchangers and recruiters, employers are viewed as passive stakeholders that provide information about skills and take advantage of channels to access the most skilled candidates.

Other initiatives imply a more "active engagement", which involves collaboration and the definition of what is being delivered by higher education [13] both at the undergraduate and graduate levels [23] as well as in lifelong learning [4]. A more active engagement allows firms to have far greater influence on the supply of skills and thus contributes to reducing skill problems. Active forms of engagement include, for example, work-based learning, which is often incorporated in the HE curriculum [14,24]. Work-based learning is essentially a partnership between employers and HE institutions to deliver courses and provide students with work experience opportunities [25] and has been found to provide all stakeholders with the most potential benefits [24].

The design, delivery, and assessment of tailored courses [26] is a further example of active engagement. Ref. [27] (p. 2010) label it an "anticipative strategy", which indicates that employers participate in the supply of skills; they work with universities to conceive courses and expect HEIs to provide them with bespoke graduates. Finally, some employers participate in one or more of the governing bodies of HEIs. Once again, this helps employers influence the supply of skills as advisers [28], but according to ref. [29] (p. 2007), it should also involve their financial contribution. Co-funding aims to reduce public expenditure and fosters higher education expansion. However, [10] found evidence of employers' persistent unwillingness to make financial contributions to the development of higher education programs.

#### *2.2. Barriers to Engagement: Higher Education Institutions and Employers*

Another stream of the literature examines the employers' engagement with higher education in order to make this interaction more efficient with a win-win outcome. Under the label of barriers and facilitators [14,15,25], this stream of literature focuses on issues that stakeholders from HE and employers raise in order to facilitate the engagement activities and outcomes.

Available literature provides a set of cultural and structural barriers faced by both HEIs and employers [13,18,30]. Barriers within HE refer to the cultural mismatch that often reduces the willingness to engage with employers to ensure the supply of suitable skills. It is said that HE has a certain disdain for business-like activities and places a strong focus on academic activities. Communication difficulties are another widely reported as employers and HE lack a common language.

Some barriers seem to be structural because they relate to the HE system, notably: the disciplinary-based curriculum; the adaptation of pedagogical methods; lack of flexibility to provide tailor-made courses or curriculum; timing of the response to requirements; or poor customer services. In sum, higher education's response needs to meet, rather than conflict with, the employers' and learners' needs in order to obtain a strategic fit.

Additionally, studies have cited the need for an appropriate learning package, that is, higher education should build or adapt training to better fit the employers' specific needs. This often involves bespoke courses and of course rejects the one-size-fits-all programs. However, HE often has insufficient information about employers' skill needs and engagement involves a trade-off between costs. Ref. [18] note that engagement is costly and risky for HEIs so they may prefer to look for alternatives or prioritize other activities. Moreover, academics' engagement activities are currently not generally acknowledged in career progression within academia [4] and thus often result from sporadic and informal contacts [31] that are not institutionalized.

Employers also impose barriers to engagement. Firstly, the nature and intensity of employers' engagement with HEI vary, notably in line with their workforce composition and specific skill needs, their product or service, and the market competition [13]. Secondly, it is necessary to have a culture of learning and perceive engagement as an investment; whereas the business strategy is often to pursue immediate profit, the benefits of engagement take time to become evident [18]. Thirdly, not only do employers find it difficult to enumerate skill requirements but these may change over time [9], and their main demand is for soft skills and sound work attitudes [23]. Finally, engagement is easier when employers need graduates from certain fields of education, notably engineering [32], chemistry, or health [13].

The reported barriers indicate that a cultural fit between organizations is required from the outset, and it not only calls for changes in work practices and mindsets on both sides but also a share of values. Ref. [19] (p. 2013) highlights the tension between entrepreneurial and academic languages and this creates barriers to collaboration. The literature therefore also examines what facilitates engagement.

#### *2.3. Facilitators of Engagement: Higher Education Institutions and Employers*

The HEIs are faced with two interrelated issues. On the one hand, they must proactively encourage engagement by approaching employers, involving them in learning, and providing information about the benefits [33]. It is essential that HEI and the employer develop proper communication and a shared understanding [18]. On the other hand, the system itself needs to be adapted. Ref. [25] (p. 2015) stress that staff from all levels must be enthusiastic about the engagement, in particular in the case of workplace learning.

Employers have a selective approach to HEIs and trust is one of the major issues. For example, the willingness to invest in skill development depends on the employer's knowledge about the programs supplied by the HEI [30]. Alumni can also be used to build a bridge with the HEI to ensure that collaboration is beneficial. The literature documents the fact that the gratitude towards the academy felt by alumni impacts their willingness to engage [34]. Employers may also develop collaboration with individual academics and follow them across HEIs [13]; that is, they mostly interact informally and this means alumni play a decisive role.

On the other hand, employers prioritize the geographical proximity of HEIs [35]. Geographical and social proximity between HE and firms may help in informal relations and information collection [13]. Furthermore, ref. [15] note that engagement is facilitated by the ability to sustain the partnership. This entails equality among partners, but the complexity of this varies in line with the number of members involved; small partnerships are easier to manage, while larger ones require more clearly defined roles.

Employers' engagement with HE is therefore far from straightforward [19] and involves a continuum that goes from an understanding of specific employers' skill requirements as well as the different levels of skills of students and employees to the implementation of mechanisms to develop such skills. All this process requires a culture of trust and commitment to the defined goals; engagement of skilled staff to interact with employers; flexible systems and working methods adapted to the diverse workforce. Only when these conditions are fulfilled do they work as facilitators; otherwise, they function as barriers and invalidate or contribute to making the engagement inefficient. Ultimately, both the decision taken by employers to meaningfully engage with HE and the potential results of that engagement will depend on their assessment of the perceived costs and benefits of that engagement [13]. Some argue that engagement should involve public investment [36], while others suggest there should be multiple funding sources as they facilitate engagement and acknowledge the relevance of the activities [37].

In sum, in spite of the growing interest shown by different stakeholders in graduates' employability and particularly in the collaboration between employers and HEIs in actual skill development and training, relatively little academic research has been conducted on the types of engagement activities developed, the factors that facilitate or hinder those activities, as well the perceptions of both HE professionals and particularly of employers about these relationships [10,38,39].

A profusion of policy reports have been written at the behest of governments [40] or employers' associations [14] and in some countries, namely the UK and Australia, a number of academic studies have already been made on this subject [41]. However, literature is scarce for other European countries, although some have made huge investments in HE. Furthermore, to the best of our knowledge, there are still no studies in the academic literature on the employers' perceptions of engagement or the factors that facilitate or impede engagement or make it more efficient.

#### *2.4. The National and Regional Contexts*

At the national level, Portugal has made substantial investments and progress in both HE and vocational training in recent decades. There has been a trend of massification in HE since the 1990s, with a sharp increase in enrolment rates from around 157,000 students in 1990 to almost 400,000 in 2019. Over roughly the same period, enrolment rates among 30–34-year-olds have also more than doubled, going from 15.1% in 1992 to 36.2% in 2019 and drawing close to the EU average which stood at 40.3% in 2019 [42]. Other recent trends have seen the diversification and differentiation of the system with the expansion of the private and polytechnic subsystems, and the adoption of the Bologna model in 2006 and of a managerialist reform in 2007 that brought Portuguese HE more in line with the trends in Europe and beyond [43].

Over this period, post-graduate and doctoral enrolment [44], as well as research activities, also increased substantially along with research outputs at the system level [45]. On the other hand, the Portuguese education and training system has been found to be excessively centralized, leaving little room to accommodate regional needs [46,47]. Recent legislation has acknowledged the potential benefits of decentralization and has tried to promote the greater involvement of local and regional authorities and stakeholders in addressing the specific skill needs of sub-national economies. However, up until now, the engagement of these local actors has been mostly limited to advice or consultations initiated by the central government [47].

The relatively sparse and fragmented literature on employer engagement with HE in education and training in Portugal has reported somewhat mixed findings. Ref. [39] (p. 107) acknowledge that the collaboration between employers and HEIs is at an embryonic stage, and this is a pervasive feature. Some authors report examples of a variety of activities of engagement, including the design and delivery of courses and shared governance [28]. Nevertheless, passive forms of engagement, such as hiring candidates or job advertisements [28] and internships [48], are widespread among employers in Portugal.

Given this low level of cooperation, a regional multi-stakeholder partnership, Famalicão Made IN, was formed by the city council of V. N. Famalicão with the aim of connecting different local stakeholders to promote employability, entrepreneurship, and innovation in the region through suitable responses from education and training systems. V. N. Famalicão is in a small but vibrant industry-based region, ranking third in the country's export volume and 2nd in gross added value in manufacturing industries [49]. The main industrial sectors include textiles, metallurgy/machinery and polymers (namely for use as automobile components), and agri-food industries and the unemployment levels are traditionally below the national average. Although the region has a few large industrial firms, namely in the textile and automobile components sectors, the majority are small and medium-sized firms in keeping with the typical profile of firms in Portugal [50].

In the region, there are four HEIs, two universities, one public and other private, and two public polytechnics. The universities are more generalist in what they offer (but with a strong component of engineering degrees) while the polytechnics are more vocational in nature (as is usually the case for polytechnics in the Portuguese system) and with a stronger focus on industry-related degrees. Not surprisingly, these institutions in the more immediate vicinity are the ones with whom the firms in our sample will develop greater ties both for recruitment and R&D purposes. Other institutions in the wider Northern region with whom our firms also develop some ties include also public and private universities and polytechnics.

The primary role of Famalicão Made IN is to build bridges between employers and other actors, such as education and training institutions at all levels, local employment services, municipal and intermunicipal institutions, and ultimately to mediate the relationship between local actors and national policymakers. In the context of employer engagement with HE, this partnership can be viewed as an agent that facilitates the cooperation with firms, especially medium and small enterprises [51].

This program was officially launched in 2013 and was initially focused primarily on VET provision as key to boosting employability and addressing the persistent skill shortages experienced by firms in the regional ecosystem. Over time, it has diversified both its scope of action and the stakeholders engaged in it. The collaboration with local HE is thus more recent and targets joint R&D projects conducted by firms as well as initiatives to reduce skill shortages and gaps.

#### **3. Methods and Data**

The data were gathered in the course of an ongoing project undertaken in Portugal to analyze and reflect on employers' engagement with HEIs as a possible avenue for the reduction of persistent skill mismatches at the graduate level. The paper draws on primary qualitative data gathered from face-to-face semi-structured interviews with human resource managers and owners of 19 industry-based firms located in the northern region of Portugal (county of V.N. de Famalicão). The interviews were conducted by two members of the research team, lasted between 1 and 2 h and were fully transcribed. The questions were intended to draw data on (i) recruitment strategies; (ii) perceptions on the preparation of graduates for the world of work and skill problems faced by these firms; (iii) solutions for skill problems (among which training and recruitment policies, relations established with schools and HE and barriers and facilitators to the engagement with these training organizations); and (iv) characterization data of these firms.

This qualitative material was examined through content analysis in order to gain a better understanding of the actual engagement activities developed by local firms with HEIs and the major obstacles and facilitators of those relations. Following the literature review and content analysis of the interviews, we defined several categories for each analytical dimension: skill problems (shortage and gaps); engagement activities regarding the acquisition, training, and assessment of skills, as well as R&D activities; barriers to that engagement, namely cultural/organizational barriers, and facilitators related to personal/social, geographical proximity. A single table containing the selected analytical dimensions was then produced and all the relevant excerpts from the interviews were coded into those categories. This has allowed a more systematic and comparable overview of the analytical data from the interviews. Smaller excerpts were selected for use throughout the paper to illustrate the employers' perceptions of the subjects under analysis.

The firms were selected by convenience sampling and the sample is therefore not statistically representative. However, we have included firms from the most representative sectors (namely textiles, metallurgy, and agri-food) of this strongly industry-based region, with different characteristics in terms of size and years of activity (Table 1). However, the sample is somewhat skewed in relation to size as it includes a higher proportion of large firms than is present in the region which, as noted, is comprised largely of small and medium firms. Most of the firms in the sample have been actively recruiting graduates in

the past three years, notably in different engineering fields, ICT professionals, and, in some cases, also from management and product design areas.

**Table 1.** Characterization of firms.

#### **4. Results**

This section will start with a brief overview of the main skill shortages and gaps faced by employers in this regional setting. As employers' engagement with HE does not take place in a vacuum, the forms and intensity of these activities are likely to be influenced by either the skill problems at the graduate level or the perceived costs and benefits associated with that collaboration. We will therefore delve deeper into the actual engagement activities developed with HE before examining the perceived barriers and facilitators to that engagement in the next section.

#### *4.1. Employers Engagement with HE at the Regional Level*

Skill problems include skill shortages, translated into hard-to-fill vacancies, and skill deficits or gaps which relate to the employers' perceptions of graduates' level of preparation for the world-of-work. These gaps were further typified under the categories of soft skills, work attitudes/maturity, and technical skills.

Regarding skill shortages and hard-to-fill vacancies, a non-negligible proportion of the employers (8 of 19) refer to difficulties in finding and retaining graduate employees in the region, giving rise to constraints in their activity. Most of these vacancies are found in technical areas such as Engineering (F6; F15; F18; F1; F11), ICT (F6; F10; F12), and Physics (F15). Three sets of factors are reported by the sampled employers to explain skill shortages. First, they blame HEIs for a skill shortage that causes an undersupply of graduates in these areas, despite the massification of HE. Second, the good labor market conditions, notably low level of unemployment and the greater bargaining power of graduates, which increase the labor costs: "*it's a region of full employment [* . . . *] in all areas of expertise it is now very difficult and it's the candidates who choose where they want to go and what they want to do*." (F16); "*I would say in engineering nowadays you are only unemployed if you choose to be (* . . . *) people are much more selective today*" (F15). Finally, the brain drain of young qualified people is a problem that affects the sampled firms. The demand for talented youngsters from large neighboring cities and the international labor market makes it more difficult for employers from the relatively small town where these firms are based to attract and retain a skilled workforce.

In terms of graduates' preparation for the world-of-work, there is widespread consensus that HE graduates are well endowed with technical skills, "*technical skills are excellent*" (F8), and "*nowadays students have a far higher skill level than when I finished university*" (F6). However, employers claim graduates are poorly prepared in soft skills and work attitudes (12/19). They report graduates often lack soft skills such as written and oral communication, problem-solving, and transversal skills in general: "*In terms of technical skills, they are better prepared but what they lack is some behavioral and social skills*" (F6); "*I feel that teamwork is an issue as is communication; the ability to communicate both orally and in writing is definitely a problem*". (F13); "*Nowadays, we give as much value to behavioral skills as we do to technical skills, something which did not happen a few years ago*" (F3).

The work attitudes, commitment, and behavioral skills of recent graduates are a major concern (15/19). These characteristics are highly valued by the firms and are widely regarded as instrumental for the use of other technical skills to the benefit of the organization (F4; F11; F3, F2). Thus, HEIs are often blamed for not sufficiently addressing soft skills and attitudes, alongside the technical skills (F6; F1): "*They [graduates] have difficulties in interaction and communication (* . . . *) but I don't see the universities addressing this issue of soft skills*" (F1).

The next question is the employers' willingness and/or ability to engage with HE to tackle skill problems. Table 2 displays the engagement activities of the sampled employers, as well as the barriers and facilitators of this engagement. As can be seen, HEIs serve as a recruitment channel to access the best candidates and engage in the training of graduates through internships. All firms develop at least one of these types of interaction and a non-negligible number (9/19) engages in three. Firms sometimes also participate in job fairs and events at the university (F16; F5).

**Table 2.** Firms' engagement activities with HE and barriers/facilitators to engagement.

The internship programs are regarded as an opportunity to provide some real-world training to students while allowing for candidates to be screened to create a pool of talent. The internships target both graduates (masters and PhD) and non-graduates and occur throughout the year or in the summer. Internships also result sometimes from students visiting the firm (F16). F4 regrets that some students have never had any contact with an organization and, consequently, are unaware of the world of work. These internship programs are often implemented through formal agreements between firms and HEIs and are reported as beneficial by the sampled employers:

"*That has been the way recently, we establish protocols with the universities in the region to receive interns and then recruitment comes from those visits (* . . . *) the truth is that they are very receptive to what we have to say just as we are very receptive to what they have to show us.*" (F16);

"*Yes, we work with different internship programs and we have our own program too. We like to receive those interns, especially because in nine months or so they will enter the labor market. And if we can have this contact before and the student can also get to know the reality of a company, it can facilitate their choice afterwards*" (F1);

"*We have some people coming here for internships. Some come to do their Master thesis. They usually have to face a problem and then come up with a solution. (* . . . *) It is good, it is a way of evaluating people and knowing if they are good enough to stay.*" (F18).

Activities such as teaching, collaboration in course design, or participating in HE governance bodies are scarcely reported by employers. However, they intend to engage more actively in these activities and some firms, namely medium and larger firms, are sometimes invited for this kind of collaboration. In such cases, these types of firm are consulted in the design and demanded an assessment of the course: "*Well, I know that one of our engineers here participates in one of these [governance] bodies (* . . . *) sometimes they ask to use some of our machines, they have visits here or we donate some equipment. I know they sometimes discuss some new degrees, pedagogic content, what makes more sense or not*" (F19). Nevertheless, employers call on the expertise of HE institutions if they are unable to properly prepare graduates: "*We have people with a lot of know-how that want to pass on that knowledge [to graduates] but they don't know how to do it (* . . . *) so what we did was a partnership with the* . . . *Business School*" (F17).

However, this is a random activity, often resulting from the employers' interpersonal relations with HE professionals, so it is far from a systematic and internalized process. Overall, the sampled employers indicate that there is room to deepen the engagement and they are willing to do so. Nevertheless, as we will see later, there are barriers that hamper closer ties.

The collaboration in R&D activities attracts wide attention and prevails in 8/19 firms. It should be noted that the firms in the sample are industry-based and need to develop new products and technologies, which entails close interaction with HEIs. At the same time, firms and HEIs have strong incentives and a long history of cooperation in R&D: "*Nowadays we have two R&D projects in cooperation as well as several in the 2020 [program] in the areas of innovation and markets, especially because we have opened up to cooperation with universities. Beforehand secrecy was the soul of business but not anymore* . . . " (F7). "*In R&D we have seven people, three of whom are doctorates (* . . . *) we will always have to resort to research centers because they have other skills that we don't have*." (F3). The engagement not only involves R&D but also post-graduate training and sometimes the recruitment of master or PhD graduates that participate in the development of products and technologies. A sectoral pattern emerges in the collaboration with HE in R&D in that firms from the agri-food and textile sectors seem to be more engaged in collaborative R&D than those from other sectors, namely metallurgy, machinery, and components for the automobile industry. One possible explanation is that the textile industry in the region has undergone a strong technological upgrade in recent decades that has been widely recognized, and therefore resorts more to this sort of collaboration. However, these results should be treated with caution because other industries, namely those related to the automobile industry, may be more prone to producing in-house R&D (or in collaboration with other firms in the sector).

In sum, the reported engagement activities show that employers are aware of the relevance of HEIs as skill suppliers and often contact them to acquire talented people. However, they are still far from participating systematically in the skill formation process, although some are trying to do so. As noted, the sampled employers are available and intend to extend their engagement, so it should be possible to make progress in the future. Currently, R&D continues to be the major activity of contact between employers and HEIs.

#### *4.2. Barriers and Facilitators to Employers' Engagement with HE*

The sampled employers reported a set of barriers and facilitators of engagement. More specifically, cultural differences that prevent fruitful communication between HEIs and firms are perceived as the major issue. HEIs are generally accused of being distant from firms (F16; F6; F8; F7; F18; F4; F11, F2) and from the world-of-work. "*The world is moving at one pace and universities are another*" (F6); "*I believe the universities are still distant (* . . . *) the students should be put in contact with firms sooner in their university trajectory as they are in programs abroad where the connections with firms start in the first year of college*" (F8). HEIs continue to be focused on academic activities and disregard the requirements of employers. According to these employers, it is the HEIs that fail in setting shared goals and a common language that would increase proximity and develop ties: "*We need a stronger connection with reality (* . . . *) and that does not happen. It does not happen on either side because firms try to get closer and then often lose their patience because of bureaucracies (* . . . *) and universities often use firms just for statistics so that they can say they have links with firms and present those numbers*" (F2). This detachment exacerbates the mismatch between the skills acquired at HE and those required by employers (F8; F18; F11; F2). Two employers (F3; F11) highlighted their willingness to collaborate with vocational schools. They noted the benefits of working with vocational training institutions rather than with universities or polytechnics, notably the flexibility to adapt the curriculum and pedagogical methods, and teachers' knowledge of skill requirements that help a better match of the skill supply.

Some employers recognize that some steps have been taken in recent years to overcome organizational barriers (F16; F15; F18; F19), and some HEIs try to be more responsive to firms' needs and engage more actively with them. However, for these firms, HEIs have not yet provided an adequate response to employers' apparent willingness to engage more actively with HE, and the costs of these engagement activities incurred by firms still exceed the potential benefits: "*The relations have improved, I'm not going to say they haven't, but they still need to be closer*" (F16); "*I believe relations are increasingly better and I see HEIs making strides to come and ask the firms, something which didn't happen before (* . . . *)* [HEIs] *are proactively trying to get closer to firms*" (F15); However, the trade-off between costs and benefits is at the heart of the discussion. "*We must also see what is the economic benefit for us (* . . . *) generally the end result is more of a burden and a loss for our activity*" (F19). Employers suggest engagement is a risky activity that has uncertain benefits.

The barriers are not limited to HEIs with some firms (F11; F2) referring that they refrain from establishing closer links to HE. Others acknowledge that their willingness to engage is often lessened by the above-mentioned barriers and they must take initiatives to improve the ties with HE. Faced with the trade-off between costs and benefits, the sampled employers propose solutions to increase effective engagement.

The sampled employers enumerate some facilitators: They believe that personal contacts, notably through alumni and teachers, are efficient ways of finding appropriate partners inside the HEIs and conducting research projects and/or finding talented candidates: "*We are not approached. Fortunately, we have a relationship (* . . . *) with universities and polytechnics, with teachers and alumni that allows us to implement our normal* [hiring] *processes. What we feel is that without these actions we would not be contacted either by universities or polytechnics*" (F11). However, F3 reports that "*Our relations with HEIs happen in two ways: either we proactively contact them because we have a specific need and then relations are established, or there is someone here at the firm that has a good relationship with someone at the university and then the collaboration follows through that different channel*." In other words, personal contacts appear to be a facilitator for closer ties between HEIs and the world of work.

Geographical proximity is an additional facilitator reported by the sampled employers. Regional universities easily create networks with local employers since they have a deeper knowledge of the region and employers' needs. Consequently, employers refer to local interactions rather than national and distant partners (F16; F6; F15; F7; F10; F1; F12; F3). On the one hand, geographical proximity eases face-to-face interactions and access to information about institutions and people able and willing to establish partnerships. On

the other hand, it helps create a pool of talents of young graduates in HEIs closer to their homes and searching for job opportunities in the local or regional labor market. Some employers speak about these advantages:

"*We work a lot with the [local] University (..) we give priority to the [local] University because of a partnership we established several years ago, because of geographic proximity (* . . . *) we have a lot of people here from that university. We have a good relationship with the presidency and the vice-presidency and great proximity also with the school of engineering" (F15). "We work with several institutions on account of proximity*, [local] *University, the Polytechnic* . . . [all in the Northern region of Portugal]" (F1). However, in addition to fruitful experiences, some underline the specificity of education programs and technological specificities of certain HEIs (F13; F18). Others are "*available to collaborate with any HEI as long as the attitude is appropriate*" (F4; F7; F8)

We now turn to the relationship between engagement activities and the barriers or facilitators reported by the sampled employers. One group is made up of firms that use HEIs as a recruitment channel but acknowledge that cultural barriers probably prevent other types of engagement (F8, F1, F2). Others follow the same strategy but take advantage of personal contacts and geographical proximity to overcome such barriers (F16, F6); use only of the proximity to overcome them (F7, F1, F4); or use only interpersonal contacts (F11). In other words, barriers are compensated by some facilitators, especially to allow firms to access talented graduates.

It is interesting to note that F19 is actively engaged with HEI and this was facilitated by personal contacts; however, he/she refers to skill shortages and admits that young graduates lack work attitudes and maturity. Furthermore, firms must also tackle cultural barriers to develop collaboration in R&D. This is given as the major factor inhibiting stronger university-business collaboration.

Finally, regarding the role of Famalicão Made IN, the local partnership, many employers recognize that it has already positively impacted R&D collaboration between HE and firms, access to funding, the visibility of firms, and the county and it has facilitated access to other local and national institutions and decision-makers. Some of the R&D partnerships and engagement activities have indeed resulted from post-graduate training work and knowledge interactions, and the county is currently trying to expand these projects. However, most employers are also aware that much work is required to address local skill problems and, more importantly, to align the supply of and demand for higher-level skills, as well as to overcome existing barriers between firms and HEIs. It should be noted that Famalicão Made IN has very recently taken steps to work with HE, and it can become an efficient broker to reduce cultural barriers in the future.

#### **5. Discussion and Conclusions**

This research contributes to our understanding of two key but often neglected questions in the literature: Are employers willing and able to actively engage with HEIs to become a viable solution for skill problems at the graduate level? What are the main barriers and facilitators of the engagement activities with HE experienced by firms?

Our results show that the sampled employers use HEIs as a recruitment channel and set aside active forms of engagement that would allow them to directly influence the supply of skills. These results are in line with the relatively sparse and fragmented literature on employer engagement with HE in education and training in Portugal, for example, [28,34,48]. However, despite the sampled employers' willingness to develop more active forms of engagement, some barriers remain.

Given the data displayed in Table 2, the efficiency of engagement activities must be questioned. We note that F19 laments skill shortages and a lack of graduates despite being engaged in active strategies. These examples confirm that there is no one-size-fitsall solution for skill problems [28]. Active engagement, through shared governance, for example, does not seem to be the answer either.

When examining the employers' perception of the barriers and facilitators of engagement, we found that a cultural mismatch between business strategy and academic language is the major barrier [19]. HE and the sampled firms seem unable to fully communicate through a common language and goals. This probably prevents active engagement and two employers (F3; F11) expressed their preference for vocational education institutions that show greater flexibility and responsiveness than HE. These firms noted structural barriers related to the strategy of the curriculum and pedagogical methods [18] that differentiate HE from other levels of education.

This raises the question of whether HE should change its goals and language so that it can cooperate closely with the world of work? Previous research has shown that the expectations of employers vary, for example, [28] with some employers preferring to hire ready-to-work candidates provided by HEIs, while others deliver training and ensure that newly hired workers are endowed with specific skills. In fact, the training policies of the employers in our sample involved preparing their workforce and using internships as the key to screening and training graduates.

Turning to the facilitators of engagement, the sampled employers emphasized the role of trust in developing sustainable collaboration. Some know which institutions have responses to their technical skills requirements [30]. Although HEIs' proactive approach to employers was noted [33], all employers are also aware of the costs of engagement [10] and acknowledged the trade-off between costs and benefits [13]. This leads us to the discussion of public investment in the engagement activities [36]. We agree that it involves multiple sources [37] and the stakeholders involved must acknowledge the relevance and the economic and social impacts of investing in engagement. Public intervention would probably be translated into regulations that guarantee equity among different types of HEI and different types of employers. The reform of Portuguese HE in 2007 strived to engage external stakeholders at non-executive governance bodies of HEIs. However, to date, their participation is still minimal to non-existent and is sometimes referred to as imaginary or non-interfering friends [52].

Our study found that the sampled employers reported geographical proximity as well as personal and social contacts; this later appeared as a major factor for successful cooperation. It is the alumni that sometimes open the door to HEIs and support collaboration [34]. So, when the sampled employers had suitable partners [15], they were able to access talented graduates. The multi-stakeholder partnership was also seen as a broker to help build bridges between HE and employers [25].

Geographical proximity interacts with this [13] as informal relations in a small region seem to be the perfect recipe for encouraging collaboration and communication. Finally, while the multi-stakeholder partnership attempts to build bridges between HE and the market [51], employers acknowledged that further work is required to improve the partnership's role as a broker to ease cooperation.

The reported facilitators raise trust which may encourage employers to become stakeholders that can help foster graduates' employability and create opportunities for the world of work to be embedded in HE. However, the development of employability skills is a shared responsibility [28] and HE has to fulfill multiple missions and respond to economic as well as social and cultural needs.

Although the findings achieved thus far are interesting, they should be interpreted with caution. This is a case study of one region using a sample of firms solely from industry sectors. Future research should therefore explore the skill problems of other sectors and preferably of other regions. This might be useful to ascertain the national and regional level differences of skill problems, as well as the solutions and the barriers/facilitators to tackle them. Policymakers, HEIs, and employers should understand that multiple solutions are required to reduce skill mismatch, shortages, and gaps, and all stakeholders are responsible for finding appropriate answers. However, each stakeholder has its own culture and language, and efforts should be taken to improve communication and interaction between them.

**Author Contributions:** Conceptualization, F.S. and P.V.; Data curation, E.A.; Methodology, F.S., P.V. and E.A.; Writing—Original draft, F.S. and P.V.; Writing—Review & editing, F.S., P.V. and E.A. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research was funded by Fundação para a Ciência e a Tecnologia: Project BRIGHET 'Bringing Together Higher Education, Training, and Job Quality' FCT-PTDC/SOC-SOC/30016/2017.

**Institutional Review Board Statement:** Not applicable.

**Informed Consent Statement:** Not applicable.

**Acknowledgments:** We would like to thank the employers, local stakeholders and members of Famalicão Made IN for sharing their invaluable time, experience and materials without which we would not have been able to conduct this research.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **References**

### *Article* **Teaching Sentiment in Emergency Online Learning—A Conceptual Model**

**Domingos Martinho 1,\* , Pedro Sobreiro 2 and Ricardo Vardasca 1**

**Abstract:** Due to the COVID-19 pandemic, higher education institutions with a face-to-face model have found themselves in the contingency of migrating to online learning. This study explores the perspective of all the lecturers at a Portuguese private higher education institution who were invited to participate, regardless of their research area, in this questionnaire. It aims to propose and test a conceptual model that combines attitudes, preferred activities, and technological experience with the sentiment about the impact of this experience on students' learning process, on their teaching activity, and on the strategy of higher education institutions. An online questionnaire was conducted to 65 lecturers engaging in emergency online lecturing. The obtained results showed that lecturers reveal a positive attitude towards online lecturing, tend to prefer activities in which they feel most comfortable in face-to-face lecturing, and consider having technological experience useful for online activities. Lecturers have a positive sentiment about the impact of online learning on students' learning, their faculty career, and the strategy of higher education institutions. The proposed conceptual model test shows that the model has well-fitting conditions. The results confirm the hypotheses formulated: namely, the predictive effect of attitude, preferred activities, and technological experience on sentiment. Faculty engagement in emergency online lecturing shows that the members are available to participate in the changing process, and the proposed conceptual model can be used to assess this readiness.

**Keywords:** COVID-19; emergency online learning; emergency online teaching; higher education; lecturers; online learning; Portugal; sentiment analysis

#### **1. Introduction**

The COVID-19 pandemic has affected higher education institutions (HEIs) in their activities in order to promote the protection of their lecturers, staff, and students in a public health emergency. The institutions had no alternative but to cancel all face-to-face lectures, including labs and other learning experiences, and to determine that lecturers completely switch the courses to emergency online learning, reducing contacts and thereby preventing the spread of the virus.

This teaching model that many call "emergency remote teaching" [1], includes the use of totally remote teaching solutions, mediated by the internet, to ensure activities that would otherwise be taught in a face-to-face form, returning to this format once the crisis or emergency is overcome [1]. The followed model seems similar to the online learning that has been stated by Anderson [2], referring to a teaching and learning type in which: (1) the student and the lecturer are at physical distance; (2) student–content, student–lecturer and student–student interactions are mediated by technology; and (3) some type of support is provided [2].

In the COVID-19 context, higher education lecturers were challenged by the need for the adoption of online learning practices, for which the majority were not prepared [3], and

**Citation:** Martinho, D.; Sobreiro, P.; Vardasca, R. Teaching Sentiment in Emergency Online Learning—A Conceptual Model. *Educ. Sci.* **2021**, *11*, 53. https://doi.org/10.3390/ educsci11020053

Academic Editor: Han Reichgelt Received: 13 January 2021 Accepted: 27 January 2021 Published: 30 January 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

there were no indications that they were interested in using it [4]. The faculty members had to prepare and teach their lectures from home, with all the practical and technical challenges that this entails, and often without adequate technical support [1]. In addition to the lack of required online specific pedagogical competences, it is generally agreed that in a normal situation, the challenge to effectively transfer what is taught in a face-to-face classroom to an online version remains a problem [3]. Most of these lecturers, who normally develop their activities face-to-face, do not reveal an interest in online learning (only about 30% to 35% consider this option) [4,5]. This position is caused by the lack of motivation and incentives resulting from various obstacles that can be summarized as technological readiness [6–8], absence of organizational incentive to compensate for extra work [9,10], and the prejudices related to the value of online teaching [5,11,12].

In a normal situation, the most relevant motivations for adopting online learning are related to the concern of reaching new audiences, diversifying the HEI's offer, and contributing to the management of organizational change and the positioning of the HEI´s offer in the context of online education [9,11–13].

In the emergency caused by COVID-19, lecturers needed, overnight, to use tools with which they felt comfortable [14]. Face-to-face lecturers thus needed to develop online teaching activities in order to avoid the collapse of the teaching and learning process. In this situation, lecturers adopted emergency remote teaching that. as stated by Hodges [1] (p. 6), "is a temporary shift of instructional delivery to an alternate delivery mode due to crisis circumstances". Emergency online teaching is different from all other situations in which online teaching and learning activities are planned by lecturers who have online teaching skills. For many of these lecturers with little or no experience in online teaching, the option was to transport the typical activities they developed for face-to-face teaching to the online environment and, gradually, introduce activities that would allow more meaningful learning [15]. Despite the skills and support limitations, lecturers have a positive sentiment about emergency online learning [16,17].

The present investigation focuses on the motivations of lecturers with no, or little, experience in online teaching. Without any other option, these lecturers were required to adopt emergency online teaching. In order to address this great challenge, lecturers changed their attitude towards online education, their favorite activities, and technological experience. This study aims to investigate whether these feelings and skills affect online teaching sentiment. It aims to understand how lecturers perceive the impact of this experience on students' learning, on their teaching activity, and in the development of HEI online learning strategy.

The document is organized into six sections: the present section, which introduces the research topic, the motivation, and the aim; the following section, which presents the conceptual model and hypothesis for the research; the methodology is then described, followed by the sections of the obtained results, its discussion, and final remarks in the conclusions.

#### **2. Conceptual Model and Hypotheses**

From the existing literature, several theories and models have emerged that have in common the objective of explaining the intention to use technologies through the relationship between latent, including external and outcome, variables [18,19] Although these models have been developed with the aim of explaining and predicting the acceptance of computer technologies in general, they have been adapted with a view for their application in more specific contexts, such as online teaching and learning [20,21].

Contrary to previous studies, this study is based on the migration from face-to-face to emergency online education. It was carried out without the lecturers involved having had any opportunity to carry out any type of training, and these had only minimal support. They were limited to providing access to the platforms and technologies used. For this study, a conceptual model is proposed that combines factors that can be measured when face-to-face lecturers have transferred their activities to emergency online learning, namely:

(1) online teaching attitude (OTA); (2) preferred online activities (POA); (3) technological experience (TEX); and (4) online teaching sentiment (OTS).

#### *2.1. Online Teaching Attitude (OTA)*

The attitude towards online teaching and learning is identical to that shown in other pre-pandemic studies [18–21]. It consists of appraising individuals' positive or negative feelings (evaluative affect) about the use of online education [9,18,19]. The following two hypotheses are proposed:

**Hypothesis 1 (H1).** *OTA positively affects OTS.*

**Hypothesis 2 (H2).** *OTA positively affects POA.*

#### *2.2. Preferred Online Activities (POA)*

The activities proposed by lecturers in emergency online learning, with which most did not have previous experience, ended up following those recommended in the existing literature. They choose to diversify the activities and the materials used, thus seeking to correspond to the different student learning profiles [22,23]. The activities preferred by lecturers when migrating activities to emergency online teaching can be compared with the concept of self-efficacy. According Jo et al. [24] (p. 50), "self-efficacy reports to lecturers' personal beliefs about their abilities and skills". It seems normal that lecturers prefer the activities in which they feel more qualified and competent. Thus, Hypothesis 3 (H3) is suggested: POA positively affects OTS.

#### *2.3. Technological Experience (TEX)*

Technological experience identifies the degree of technological readiness [25] of the lecturers from their perspective [26]. As mentioned by Abdullah and Ward [27], experience plays an important role in the adoption of online education and can be defined as "the amount and type of computer skills acquired by a person over time" [27] (p. 34). For Joo et al. [24], "it is important for lecturers to have enough time and opportunities to practice new technologies until they feel comfortable enough to use the technology and perceive that technology". In a context in which lecturers did not have that time, technological experience seems to be an important factor that can influence online teaching sentiment [28]. The following three hypotheses are proposed:

**Hypothesis 4 (H4).** *TEX positively affects OTS.*

**Hypothesis 5 (H5).** *TEX positively affects OTA.*

#### **Hypothesis 6 (H6).** *TEX positively affects POA.*

#### *2.4. Online Teaching Sentiment (OTS)*

According to Liu [29] (p.15), "sentiment is the underlying feeling, attitude, evaluation, or emotion associated with an opinion", which is represented by three aspects: the type, orientation, and intensity of the sentiment. In the context of this work, the lexicon-based approach that involves calculating the orientation of feeling from the semantic orientation of words or phrases was used. The orientation of the sentiment can be positive, neutral, or negative. Neutral means the absence of sentiment or no sentiment or opinion [29,30]. Sentiment intensity is an important aspect for the classification of the feeling associated with a sentence [31]. For example, "good is weaker than excellent, and dislike is weaker than detest" [29] (p. 16).

Sentiment analysis is studied in many different contexts, with machine learning and natural language processing being the most common techniques [32]. In the current research, sentiment analysis was based on processing natural language and extracting information that examine phrases and assign to each one of them a sentiment polarity (positive, negative, neutral) [29,33]. By this way, the opinions expressed by lecturers in relation to the impact of the online emergency teaching was assessed in three aspects: (1) impact on students' learning; (2) impact on their future teaching activity; (3) impact on the future HEI online learning strategy.

Based on the previous theoretical variables, the conceptual model with the relationships between all the factors that influence OTS is presented in Figure 1.

**Figure 1.** Conceptual model.

#### **3. Methodology**

#### *3.1. Participants*

The participants (*n* = 65) were lecturers form a Portuguese private HEI. This HEI has a total of 98 lecturers that were invited to participate in the questionnaire. The link to questionnaire was sent to everyone through e-mail message, along with an introduction about the research objectives.

#### *3.2. Data Colletion*

The data were collected through online surveys from April to May 2020. The aggregated response rate was 79%, and the final sample consisted of 66% of the reference population. From 98 potential respondents, 78 questionnaires were answered by respondents, of which 13 were rejected because of missing values.

#### *3.3. Lecturers' Personal Information/Demographic Data*

In the total of sample of lecturers, the percentage of females was 40%, while that of males was 60%. A total of 1.5% of lecturers were up to 29 years of age, 13.8% from 30 to 39 years of age, 46.2% between 40 to 49 years of age, 21.5% between 50 to 59 years of age, and 16.9% were 60 years of age or older. In terms of the academic qualifications of the lecturers, 23.1% of participants held bachelor's degrees, 33.8% held master's degrees, and 43.1% held a doctoral degree. The teaching experience shows that 18.5% had up to 4 years, 20% had from 5 to 9 years, 24.6% had 10 to 19 years of experience, and 36.9% disclosed 20 or more years of experience.

#### *3.4. Survey Instrument and Structure*

The questionnaire consisted of six sections. The first section intended to characterize the respondents. In the second section, respondents were asked about their attitude toward online teaching and learning with a 5-point Likert scale (1—lower; 2—sometimes lower; 3—no significant differences; 4—sometimes higher; 5—higher). The third section was to evaluate the degree of preference/satisfaction with the online activities. A 10-point end defined scale with ratings from null (1) to high (10) was chosen, in order to produce increased sensitivity of the measurement instrument [34]. In the fourth section, respondents were asked to self-assess their technology skills. A 4-point Likert scale was adopted (1—none; 2—up to 3 years; 3—from 3 to 6 years; 4—more than 6 years).

The fifth section of the questionnaire survey presents three open questions about the impact of emergency online teaching and learning in the present and in the future of (1) student´s learning, (2) teaching activities, and (3) online learning and teaching in HEI strategy. These questions are intended to collect data for sentiment analysis about online learning and teaching. Table 1 presents the constructs of each section and the sources which inspired them.

**Table 1.** Constructs and their inspiration sources.

(\*) Open questions.

#### *3.5. Pilot Study for the Questionnaire*

A pilot study was conducted to check the reliability of the questionnaire items. The sample size was set based on 20% of the aggregated sample size of this study (98 lecturers) and thus adhered strictly to the research criteria. Cronbach's alpha test was utilized for the computation of internal reliability [40] through IBM SPSS Statistics v26, in order to judge the outcomes of the pilot study. A value of 0.7 was taken to be an acceptable value for the reliability coefficient, considering the model for social science research [41–43]. The appropriate findings are shown in Table 2.

**Table 2.** Cronbach´s alpha value for pilot study.

#### *3.6. Sentiment Analysis*

The fifth section of the questionnaire presents three open questions about the impact of emergency online teaching and learning in the future of (1) student´s learning, (2) teaching activities, and (3) online learning and teaching in HEI strategy. These questions are intended to collect data for sentiment analysis towards online learning and teaching.

There are many applications and enhancements on sentiment analysis algorithms that have been proposed in the last few years [33]. For this work the OpLexicon 3.0 was used. It is a sentiment lexicon for the Portuguese language, built using multiple sources of information, and has four categories of words: verbs, adjectives, hashtag, and emoticons. The lexicon is constituted of around 32,000 polarized words classified by their morphological category and annotated with positive (1), negative (−1), and neutral (0) polarities [30,38].

The sentiment analysis was developed in R [44] following the following steps represented in Figure 2: (1) the words are extracted from each answer of the open questions in the questionnaire; (2) verification of whether the word is present in the OpLexicon and determination of the polarity; (3) the sum of the polarity of the word in the answer is determined; and the final step is (4) to convert the sum of polarity to a Likert scale.

**Figure 2.** Flow-chart representing the determination of the polarity of the open questions.

The conversion to a Likert scale was based in the following Algorithm 1, where each answer is processed after the determination of the cut points (median values) used to convert to a scale aligned with the other questions of the survey:

As an example, considering the opinion "I consider that my adaptation was made in a smooth way", the next step is the processing of each word: "I (1) consider (2) that (3) my (4) adaptation (5) was (6) made (7) in (8) a (9) smooth (10) way (11)". To determine the polarity of each word, OpLexicon 3.0 was used. In the example given, only the word "smooth" (10) returns value 1 (positive polarity) from OpLexicon; all the other words do not have an associated polarity, returning "word is not present in dataset". The algebraic sum of the returned values is 1. Consequently, this answer would get a polarity value of 1. After performing this step, an algorithm is developed following the "Likert calculation", calculating the median of the negative and positive words in each question: (1) negative values less or equal to the negative values median were assigned one, (2) negative values less than zero and greater than median were assigned two, (3) 0 (neutral) was assigned three, (4) positive values and less than positive median were assigned four, and positive

values greater than positive mean wrtr assigned five. The null values were replaced by 0 representing the absence of an answer.

#### *3.7. SPSS and SmartPLS 3*

The demographic data was evaluated with the aid of IBM SPSS Statistics v26. Smart-PLS 3 software was used with a graphical user-interface to estimate the PLS-SEM models [45]. This tool can cope with smaller sample size (<100), non-normal data, exploratory research for the same effect size and model complexity, and it can more easily specify formative constructs [46,47].

#### *3.8. Adjustment Quality for the SEM Model*

The following fit measures were considered to assess the adjustment quality of the model:

The values presented in Tables 3–5 show that the model has well-fitting conditions.

**Table 3.** Adjustment quality for the Structural Equation Modeling SEM model.

**Table 4.** Discriminate validity.

Diagonal values (in bold) are Composite reliability (CR).

#### **Table 5.** F-square.

Finally, evaluating the predictive validity or Stone–Geisser indicator for the accuracy of the adjusted model. Q2 > 0 implies the model has predictive relevance [46,52] (Table 6).

#### **Table 6.** Predictive validity (Q2 ).

SSO—sum of squares errors using mean for prediction; SSE—sum of squares prediction error.

#### **4. Results**

#### *4.1. Online Teaching Attitude*

The results showed that respondents have a positive attitude towards online teaching. The item "I have the same availability for online as for face-to-face teaching "(OTA3) has an average of 3.71, while the item "quality of online education in relation to face-to-face education" (OTA1) has 3.25, and the item "I like online education in the same way as face-to-face education" (OTA2) has an average of 3.14 (Table 7).

**Table 7.** Online teaching attitude.

#### *4.2. Preferred Online Activities*

Lecturers revealed greater preference for "online sessions" (POA4) with a mean of 8.48, "oral presentations" (POA3) with 7.66, and "written assignments" (POA2) with 7.34 (Table 8).

**Table 8.** Preferred online activities.

(\*) Standard-deviation.

#### *4.3. Technological Experience*

Respondents showed high experience in the use of "online meeting systems" (TEX1), with an average of 3.94. The remaining items evaluated obtained average values above 3.0 (Table 9).

**Table 9.** Technological experience.

(\*) Standard-deviation.

#### *4.4. Sentiment Analysis*

The results of sentiment analysis of the open questions allowed the identification of their sentiment value, as exemplified in Table 10, for impact on lecturers' careers.

**Table 10.** Example of qualitative sentiment for the impact on lecturer´s careers.

\* The answers related to the impact on lecturers' careers were translated to English to allow better comprehension.

The impact of online learning on students' learning has approximately 9 responses with a negative sentiment (14%), as well as a neutral sentiment with 14 answers (21%), and 30 responses with a positive sentiment (46%). The sentiment in relation to teaching activities

has 11 responses with a negative sentiment (17%), 11 answers of neutral sentiment (17%). and 31 positive sentiment responses (47%). In relation to the higher education institution, there are 3 answers with a negative sentiment (5%), 12 responses of neutral sentiment (18%), and 34 positive sentiment responses (52%).

The opinion in relation to the impact of online learning in the institution strategy is the one with a higher percentage of positive sentiment (52%), as opposed to 5% who expressed positive sentiment. The opinion in relation to teaching activities has the highest percentage of positive sentiment (17%) as well as neutral sentiment. The overall sentiment distribution is represented in Figure 3.

**Figure 3.** Frequency of the sentiment identified.

#### *4.5. PLS Analysis*

The path coefficients of the prediction model were positive in POA (0.390), and they were negative in OTA (−0.169) to the latent variable of OTS. TEX coefficients to the prediction model were positives to the latent variables of OTA and POA. These results show that TEX has direct and indirect (via OTA (0.160) and POA (0.427)) effects on OTS.

The model also presented OTS1 (student online learning) (0.772), OST2 (teaching career development) (0.866), and OTS3 (online learning in HEI) (0.867), which had positive path coefficients to OTS (Figure 4).

Specific indirect effects are show in the Table 11.

**Figure 4.** Partial least squares structure model (inner path coefficients and outer weights).

#### **5. Discussion**

A questionnaire was conducted with the participation of 66% (*n* = 65) of all lecturers (98) for a Portuguese private HEI who developed their activities in an emergency online teaching environment. The study examined their attitude toward online teaching, what online activities they most value, and investigated whether technological experience influences these attitude and preferences. The opinions of these lecturers in relation to emergency online teaching, namely their impact on students' learning, their professional development, and the development of HEI strategy was also examined. Finally, a conceptual model was proposed and tested to assess the effect of attitudes, activities, and technological experience on online teaching sentiment. In the following points, the results obtained in relation to the previous literature are discussed.

#### *5.1. Attitude toward Online Teaching*

The results showed that lecturers have a positive attitude towards emergency online teaching, showing an identical availability to face-to-face teaching. This conclusion coincides with other studies conducted in an emergency online teaching that show that lecturers report more on the advantages of distance education [53]. This is reinforced by the results obtained in the analysis of the impact of online teaching sentiment on teaching and students' learning.

Based on this conclusion, at least in an emergency situation, lecturers do not question the value of online teaching. Although this is not the same type of education, these conclusions are more positive than the results obtained in a normal situation when questioning face-to-face lectures about their availability and acceptance of online teaching [5,11].

#### *5.2. Preferred Activities*

The most preferred activities of lecturers ("online sessions", "oral presentations", and "written assignments") confirm the García-Peñalvo et al. study [15] and reveal that lecturers relied on the "tools" they dominated and only later did they begin to use resources more adjusted to online teaching and learning. This strategy is confirmed by Rapanta et al. [14], who state that many non-specialist online lecturers have chosen to focus on materials/resources that they would use anyway to teach the course content, regardless of whether they are face-to-face or online.

Despite the difficulties related to the emergency online teaching that cannot be compared with "normal" online teaching, some of the options found can be problematized. However, as concluded by Spoel et al. [54], there was the attempt to provide students with the basic ingredients for learning (online lectures, group activities, discussion forums, etc.) that reveal concern with diversification, thus seeking to correspond to the different student learning profiles [22,23].

This adaptability seems to confirm Anderson in that "an excellent e-teacher is an excellent teacher" [2] (p.360), possessing pedagogical skills that allow them to understand the teaching process, in order to be able to make the best use of the range of activities they have at their disposal.

#### *5.3. Technological Experience*

Pre-pandemic studies [7,8,10] show that technological readiness can be a factor that conditions the participation of lecturers in online teaching. Although these conclusions cannot be directly transposed to emergency online education, results show that the participants in this study had technological experience in some of the tools for the development of online activities.

#### *5.4. Sentiment Analysis*

Lecturers have a positive or neutral sentiment about the impact of emergency online learning on students' learning. These findings are similar to others, where it was concluded that lecturers expressed a favorable opinion about the students' academic performance during the COVID-19 pandemic outbreak [16,17]. The findings of this study are slightly more positive than the results reported by Tartavulea et al. [55], which concluded that emergency online teaching has an overall moderate positive impact on the educational process, albeit the overall effectiveness of the online educational experience is perceived to be lower than in the case of face-to-face teaching.

Likewise, lecturers expressed a neutral or positive sentiment regarding the impact of emergency online teaching on their professional activity. In addition to showing high availability for online teaching, lecturers do not refer to the eventual need for compensation for the required additional work caused by transposition of face-to-face to emergency online teaching, as studies about online teaching reveal [9].

Lecturers thus seem to prefer to take advantage of the professional development opportunity that the situation offers [4]. These conclusions reveal a positive stance that HEIs that intend to invest in online teaching strategies cannot miss. Studies carried out in a pandemic situation have not focused on this aspect, so it is not possible to make comparisons with similar situations. Despite this, there is pre-pandemic literature that shows that lecturers do not consider online teaching as having a positive impact on their careers [4,13].

The results verified in the sentiment analysis about the impact of emergency online teaching for the future development of the HEI are in line with other studies which were carried out outside the emergency context, and where the contribution to organizational change and positioning of the HEI offer are the aspects most frequently pointed out by lecturers on the adoption online teaching [9,11–13]. The extra time and effort invested by lecturers in emergency online teaching can explain the positive perception regarding the impact on HEI strategy [54].

#### *5.5. Conceptual Model*

The results of the conceptual model test show that the model has well-fitting conditions. In relation to each of the tested hypotheses it is concluded that five of the six hypotheses have been confirmed (Table 12). The obtained values show that the effect of POA and EXT on OTS, and TEX on POA are strong (>0.35), while the effects of OTA and TEX on POA are moderate (>0.15) [56].

**Table 12.** Hypothesis results.

#### *5.6. Limitations*

Some limitations of the present study must be highlighted. First, the study was carried out in an institution with 98 lecturers, of which 66% submitted valid responses (no missing data). The sample size (*n* = 65) represents the HEI population, but with all respondents belonging to a single HEI, the study does not allow generalizing the results for Portuguese HEIs.

Another limitation of the study is the fact that the results are based on the respondents' perceptions, which may cause a bias. Although it was clarified that the survey results would only be used for the purposes of the survey, respondents may be tempted to choose the "correct" answer or the more socially desired answer, thus being vulnerable to distortions [42].

The way in which the transition from face-to-face to the emergency online teaching was carried out may justify why lecturers expressed a greater degree of preference for lectures (online sessions). This preference, by itself, could indicate that they merely transposed the "bad" face-to-face practices to the online environment, namely the face-to-face expository sessions. However, despite this greater preference, there is a significant degree of adherence to other activities, namely oral presentations, written assignments (in group), discussion forums, and chat. The diversity and characteristics of these activities can enhance student–lecturer or student–student interaction, leaving good indications about teaching and learning process [2].

The conditions available were certainly not the same in all institutions, just as they are not the same in the face-to-face context. These differences may have affected, to a greater or lesser extent, the quality of the solutions adopted and should be considered as a moderating factor when extending the study to other HEIs.

#### **6. Conclusions**

After the emergency online teaching experiences related to COVID-19 pandemic situation, lecturers acquired an experience that will mark their teaching life forever. As the storm passes and face-to-face classes are resumed in a normal environment, HEIs can expect less resistance and more enthusiasm for online teaching from their lecturers [3,4]. So that this enthusiasm does not fade away, it will be necessary to support the training of lecturers by providing them with the skills and competences they require to act in the context of online education. Hybrid approaches integrating online teaching with faceto-face activities can represent a significant improvement when many studies reveal that online education constitutes a key factor for the development of HEIs [4].

This work only reflects the perspective of the lecturers. In parallel, another study is being carried out that will reflect the students' perspective and that will allow a comparison between the two perspectives to be established.

Further research, ideally expanding the sample size with participation of lecturers from different HEIs, is required to verify whether the proposed model continues to maintain theoretical validity. In the same way, this extension will allow confirmation of the findings. **Author Contributions:** Conceptualization, D.M. and P.S.; methodology, D.M.; software, D.M. and P.S.; validation, D.M., P.S. and R.V.; formal Analysis, D.M. and P.S.; investigation, D.M. and P.S.; resources, R.V.; data curation, D.M. and P.S.; writing–original draft preparation, D.M., P.S. and R.V.; supervision, D.M.; project administration, R.V. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research received no external funding.

**Institutional Review Board Statement:** The study was conducted according to the guidelines of the Declaration of Helsinki, and has obtained prior approval of the Ethic Committee of the ISLA Santarém (2020-002).

**Informed Consent Statement:** Informed consent was obtained from all subjects involved in the study.

**Data Availability Statement:** The data presented in this study are available on request from the corresponding author. The data are not publicly available.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **References**

### *Article* **Mechatronics: Experiential Learning and the Stimulation of Thinking Skills**

**Maki K. Habib 1,\* , Fusaomi Nagata 2 and Keigo Watanabe 3**

**Abstract:** The development of experiential learning methodologies is gaining attention, due to its contributions to enhancing education quality. It focuses on developing competencies, and build-up added values, such as creative and critical thinking skills, with the aim of improving the quality of learning. The interdisciplinary mechatronics field accommodates a coherent interactive concurrent design process that facilitates innovation and develops the desired skills by adopting experiential learning approaches. This educational learning process is motivated by implementation, assessment, and reflections. This requires synergizing cognition, perception, and behavior with experience sharing and evaluation. Furthermore, it is supported by knowledge accumulation. The learning process with active student's engagement (participation and investigation) is integrated with experimental systems that are developed to facilitate experiential learning supported by properly designed lectures, laboratory experiments, and integrated with course projects. This paper aims to enhance education, learning quality, and contribute to the learning process, while stimulating creative and critical thinking skills. The paper has adopted a student-centered learning approach and focuses on developing training tools to improve the hands-on experience and integrate it with project-based learning. The developed experimental systems have their learning indicators where students acquire knowledge and learn the target skills through involvement in the process. This is inspired by collaborative knowledge sharing, brainstorming, and interactive discussions. The learning outcomes from lectures and laboratory experiments are synergized with the project-based learning approach to yield the desired promising results and exhibit the value of learning. The effectiveness of the developed experimental systems along with the adopted project-based learning approach is demonstrated and evaluated during laboratory sessions supporting different courses at Sanyo-Onoda City University, Yamaguchi, Japan, and at the American University in Cairo.

**Keywords:** mechatronics; experiential learning; creative thinking; critical thinking; robotics; thinking skills; subsumption architecture; mobile robot; educational system; project-based learning

### **1. Introduction**

The current era of globalization and the high competition environment has set up a high demand for talent and intelligence requirements. The education quality represents the core elements of success in such competition. Mechatronics classified as a "unifying interdisciplinary and intelligent engineering science paradigm" [1], due to its capability to comprehend and synergize technologies with modern engineering sciences that exhibit quality performance, stimulate creativity and innovations, focus on the interaction of interdisciplinary knowledge, and building up thinking skills through the learning process. Furthermore, it is highly considered as a philosophy and a new way of dynamic work/practices that support new ways of learning to lead to new concurrent design methodologies and innovations [1,2]. Many studies have been conducted to understand

**Citation:** Habib, M.K.; Nagata, F.; Watanabe, K. Mechatronics: Experiential Learning and the Stimulation of Thinking Skills. *Educ. Sci.* **2021**, *11*, 46. https://doi.org/ 10.3390/educsci11020046

Academic Editor: Maria José Sousa Received: 31 December 2020 Accepted: 20 January 2021 Published: 26 January 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

the mechatronics education process and its comprehensive role to establish quality educational requirements and fulfilling future needs creatively. Essential parts of mechatronics education demand to complement theoretical knowledge with practical skills and experience at different levels of the educational process. Such relation aims to develop skills finding creative and innovative solutions, hands-on experience, and practical knowledge, decision-making capabilities at individual and teamwork levels. Hence, among the key features of mechatronics skill-building learning and education is the integration of problem/project-based learning within its structural requirements.

Mechatronics evolved as an engineering science discipline [3] that best reflects interdisciplinary concepts into practice, synergizing knowledge contributed by different disciplines in an innovative way. Mechatronics, with its effective educational process, is contributing to developing capable future creative engineers. Liliana and Florina [4] address the synergy that can be achieved between education, knowledge, and innovation through mechatronics education and its impact on future needs. Acar and Parkin [5] reviewed aspects of mechatronics education and training while comparing the effectiveness of generalist and specialist engineering education.

Researchers published work focused on developing technical tools and systems to enhance experience and skills through training. Akdogan et al. [6] introduced a study tool for teaching and training with a focus on stepper motor as a driving system and its associated techniques using a microcontroller and suitable interface. Camposaragna et al. [7] presented mechatronics educational activity at Bergamo University demonstrated using course projects aiming to teach mechatronics by designing Cartesian plotters. A mechatronics system was developed by Lee and Jung [8] to support learning by using intelligent control techniques. To illustrate the mechatronics intelligent system, an inverted pendulum system was designed and implemented using a neuro-fuzzy control method, based on the Tagaki-Sugeno model. Stockmans-Daou [9], a development system, was introduced, and it constitutes a framework along with automated tools that help to develop reconfigurable mechatronics platforms by combining LEGO Mindstorms systems with a focus on robotics education. Besides, Tokuyasu [10] used Mindstorms platforms and introduced mechatronics installations that provide functional blocks with different capabilities, motors, and sensors. Mindstorms with their sensors are widely considered as an effective toolkit for mechatronics education. Stark et al. [11] developed a low-cost personal laboratory setup supporting learning mechatronics control theory is featured by portability and accessibility. Furthermore, Yilmaz and Tuncalp [12] introduced a mixed learning model that is Web-based and integrates a range of perceptual techniques and methods for effective mechatronics education.

Besides the mentioned work, Sell et al. [13] introduced a methodology with its technical concept that helps students to use mechatronics platforms to support coursework and hands-on experiments through the Internet independent of physical place. The developed platform was demonstrated in Estonia and Germany, and the reflections of the experience were reported by Grover et al. [14]. For the purpose of enhancing microcontroller skills, an undergraduate course was designed for students with different backgrounds. This course was based on a low-cost Arduino platform. Pratumsuwan [15] presented a framework for the final year mechatronics project.

This paper aims to enhance education and learning quality and contributes to learning while stimulating creative and critical thinking skills. The paper has adopted a studentcentered learning approach and focused on developing training tools that contribute to improve the hands-on experience and integrate it with project-based learning. This enables students to apply the acquired knowledge in the project to develop new ideas and thinking skills. For this purpose, four educational and experimental systems are proposed to stimulate the learning process of undergraduate students. The developed experimental systems, along with the adopted project-based learning approach, are evaluated in actual laboratory sessions supporting different courses at Sanyo-Onoda City University, Yamaguchi, Japan, and at the American University in Cairo.

#### **2. Paper Contributions and Methodology**

The key contributions of this paper can be listed as follow:

The adopted methodology that has been used to achieve this aims to:

#### **3. Experiential Learning and Thinking Skills**

The trends guiding education for the 21st century are student-centered, experiential learning, technology-infused, and inquiry-based supported by compassion and empathy. Experiential learning is an active student-led learning process through direct engagement supported by experience, analysis, and reflections. It can also be described as an applied creativity, a learning approach that helps to identify the ability of skill and knowledge adaptation needs and applies the required changes to improve and maximize performance. Therefore, experiential learning combined with creative thinking represents the top important requirements featuring future careers.

Creativity is a collection of critical skills that facilitate divergently thinking and enables the brain to generate new concepts, new ideas, and new ways to solve problems. Creativity skill-building can be achieved through different efforts, such as readings, interactive discussion, brainstorming, exercises, experiments, team-based projects, etc. This helps students to assess real problems available in real life in new ways and enables them to find out the best solution, novel product design, and the way to adapt and adopt new technologies. Building creative thinking as a cognitive process enables innovative capabilities that enhance the learning process yielding innovation.

Engineering education supported by doing through experiments and project-based learning methodologies has direct contributions to motivate students' interest, improve their hands-on experience, and positively influence their learning capabilities. Moreover, it facilitates discussion and social interaction, encourage, inspire, and challenge students to improve their ability to apply the acquired knowledge. This approach enables students to use the added value knowledge to develop new skills to design and provide innovative solutions to problems in the real world. Such engagement supported by students' active efforts helps them to build up experience by doing and learning from it, leading to stimulate the brain, enhance thinking skills, and establish creative abilities to develop innovative solutions. In addition, a team-based project helps to create and enhance employability skills at the individual and group levels. Such an approach forms the foundation of experiential learning that aims to conduct hands-on experiments and projects, test and troubleshoots, gather and analyze data, evaluate outcomes, reflect/observe creative behaviors abstracted from the learning novel experience. The creative behaviors stimulate new ideas and solutions that contribute to the enhancement of the learning process and in developing new thinking skills. Furthermore, this has a direct impact on developing professional skills and contribute to achieving the quality of innovative learning, enabling the transformation of new concepts into reality with added value processes, functions, services, or products [16–23].

#### **4. Mechatronics and the Development of Creative Thinking Skills**

As an interdisciplinary field, mechatronics is evolved into a philosophy supporting new ways of thinking, interdisciplinary knowledge synergy, work and practices, skills, and innovations [2]. Mechatronics considers education as a social process with a focus to synergize knowledge, establish interactive communications, develop functional skills (through experiments and projects), stimulate motivation, facilitate new teaching techniques, and effectively contribute to the enhancement of student's independent learning skills with the stimulation of strongly couples critical and creative activities as individuals and in teams to bring complementary dimensions to the learning process. This should be supported by an effective exploratory learning environment that focuses on creativity while stimulating a student's motivation. In mechatronics education, the learning process is directly linked with functional skill development that is directly coupled with thinking and creativity. These skills enable thinkers to solve complex problems and work efficiently across different disciplines. In general, current human thinking is not that efficient, based on judgment more than ideas, design, knowledge, and experience. Thinking and creativity are skills that can be developed. These skills require learning it, practicing it, paying attention to it, and gaining confidence. Creative design is a constructive process that looks to develop and implement ideas with new values. Hence, it is essential that students and researchers (individuals and teams) develop their knowledge, experience, and accumulate the acquired creative thinking skills. Besides, engaging them to develop new innovative concurrent design capabilities.

Creative mechatronics as a concurrent and interactive design process helps to facilitate the learning process that leads to the generation of new ideas and solutions to fulfill the demand to maximize the level of creativity supported by creative thinking skills. Evidence of lateral thinking (horizontal) that is classified as deliberate creative skills aiming to understand existing challenges or opportunities, open new possibilities, maximize problemsolving agility for better or new solutions, and develop new ideas by trying to brainstorm new novel solutions, concepts, and perceptions, while understanding governing rules and constraints. In addition, evidence of vertical thinking involves logical thinking and reasoning, and it includes different functional principles in terms of accuracy, reliability, and geometric structure besides the application of general scientific design principles. In real problem-solving circumstances, a combination of lateral and vertical approaches should be considered to produce the best results. Moreover, practical design with experimental work is an integral part to stimulate thinking, while building up thinking skills. Integrating problem-solving approaches together with creative thinking practices into the concurrent design process of engineering and science projects/products demand not only to change the learning environment and the teaching methods, but it also requires to adopt a new holistic assessment, and evaluation methods [24,25]. Such learning environments contribute to establishing motivation, create curiosity and challenges, imagination, and success in understanding science, engineering, and technology [2].

#### **5. Mechatronics Experimental Educational Systems**

A set of experimental systems were developed in association with this paper. The developed experimental system is featured by the integration of the necessary knowledge and skills demanded in mechatronics as an interdisciplinary field. The developed experimental hardware and associated software alone are not making a major difference compared to the adopted educational learning process as a whole. Each experiment in the list of the laboratory work represents an integrated process that includes: Experiment set up and troubleshooting, running and results' evaluation, answering attached design problems according to the design requirements. This is going to challenge the gained skills throughout the experiment while stimulating students to develop new skills. Furthermore, it leads to improving writing and oral communication skills through the assigned presentation sessions.

#### *5.1. Mechatronics Experimental System I*

Figure 1 depicts the mechatronics experimental system I that consists of LEDs aboard showing LEDs as ON or OFF, a two wheels' mobile robot with position control. The developed mechatronics experimental system I aims to support hands-on experiments as a part of experimental sessions of the course "Experiments I for Mechanical Engineering" required for second-year students [18]. This course is a three credit-hour course, and it constitutes sixteen laboratory sessions divided into five modules associated with five topics. These are:

Each module constitutes three laboratory sessions. There are about 10 students per each laboratory session, which is divided into five groups. In general, two students per one group. The remaining laboratory session is dedicated to the group presentation on a selected module and for each group. The laboratory sessions are conducted weekly, and the duration of each is five hours. Accordingly, the laboratory module titled mechatronics experiment I is covered by three laboratory sessions, and that requires a total of 15 h (3 sessions × 5 h).

Figure 2 presents the board layout of the experimental LEDs. The board has 24 LEDs that arrayed into 8 LEDs per row, as depicted in Figure 3, showing the two output ports B and C, respectively. Lighting and timely flashing patterns can be set using hexadecimal code as required, while the flashing period can be adjusted using Window's timer interrupt or by external triggering. The experiment helps students to learn how to use input-output ports and to design LEDs' flashing using varieties of different flashing patterns by writing

proper computer programs to achieve the desired flashing according to the status of binary input received through sensors or by other means and checked through the input port. Five hours are allotted for this experiment covering hardware and programming skill requirements. Note that the input port A is used to receive the operational state of binary sensors. *Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 6 of 23

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 6 of 23

**Figure 1.** Presents the mechatronics experimental system I5.1.1 Input-Output ports and LEDs flashing experiment: Programing with a timer interrupt. **Figure 1.** Presents the mechatronics experimental system I5.1.1 Input-Output ports and LEDs flashing experiment: Programing with a timer interrupt. through the input port. Five hours are allotted for this experiment covering hardware and programming skill requirements. Note that the input port A is used to receive the operational state of binary sensors.

Figure 2 presents the board layout of the experimental LEDs. The board has 24 LEDs

**Figure 2.** The board layout of the experimental LEDs. **Figure 2.** The board layout of the experimental LEDs. *Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 7 of 23

The position control module to control a stepper motor is shown in Figure 4. A photo-interrupter is used and fixed together with the leadscrew that is mechanically coupled with the stepper motor. The sensor detects the position of the leadscrew movement through slit positions aligned under the base frame of the leadscrew. The pulse signals generated by the sensor during the movement are used to calculate the actual moved distance and compare it with the pulses sent to drive the stepping motor in steps equivalent to the total desired distance. This experiment helps students to program the motion and understand the effect of stepper motor resolution and the leadscrew resolution with their impact on the traveled distance and the position accuracy. With the implemented position control module and the leadscrew mechanism, each step of the used stepper motor has a resolution of 1.8 degrees/step, and this causes the leadscrew to move by 6.25 *µ*m, based on its thread structure. The desired number of pulses signal representing the desired distance is sent to the motor through port A, and the output port B is used to monitor the actual movement using the photo-coupler (output port B, see Figure 3). Table 1 shows the interpretation of the binary code displayed at port B to that corresponding motor motion action. Students can learn from the experiment how to generate pulses used to drive the motor, how to control the stepper motor, and track the current position of the moving leadscrew. Besides, students (after understanding the hardware structure) work to develop computer programs to control the leadscrew while counting the number of detected slits and find out the position accordingly. Five hours are allotted to conduct this

**Figure 2.** The board layout of the experimental LEDs. **Figure 3.** The implemented two 8 bits' output ports. **Figure 3.** The implemented two 8 bits' output ports.

experiment.

**Figure 4.** The position control module with leadscrew, stepper motor, and light sensor.

#### 5.1.1. Experiment on Position Control of a Stepper Motor photo-interrupter is used and fixed together with the leadscrew that is mechanically cou-

5.1.1. Experiment on Position Control of a Stepper Motor

**Figure 3.** The implemented two 8 bits' output ports.

7 6 5 4 3 2 1 0

7 6 5 4 3 2 1 0

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 7 of 23