A large number of organizations, covering a wide variety of industries, are ISO 9001 certified. Because quality standards and certifications are intended to unify and improve business practices as a whole, the following question arises: are companies that have more mature quality systems, and certified to ISO 9001 in particular, better equipped to manage risk? Researching companies who have quality management programs and how their processes have improved since implementing them may shed light on protecting a company's assets, operations, and its structure from adverse risk events. This research assists in confirming the following statement: A company's use of a TQM system, and particularly through ISO 9001 certification, ensures a high level of risk maturity as compared to that of companies that do not implement a TQM system and/or quality certifications.

In the next section, the literature is reviewed and research questions stated. This is followed by the research methodology, which includes a description of the data. Results are then presented. The findings of this research are subsequently discussed and conclusions drawn. This is followed by a discussion of areas for further study and limitations to the research.

## **2. Literature review**

A study involving supply chain risk starts with a classification of all potential risks. These typically include the following: supply risk, demand risk, process risk, technology risk, logistics risk, information risk, and environment risk [1, 2]. The current research focuses on supply and demand risks because these present supply chain managers with significant challenges due to the severity of the impact and difficulty of effective mitigation. Van Miegham [3] characterizes the loss of a key supplier as having a high effect for aggregate loss severity and a moderate probability of occurrence. Further, Chen et al. [4] report that demand risks have a direct negative effect on supply chain performance. The literature also recommends differing approaches to moderating the occurrence of supply or demand disruptions, such as firm innovativeness, process modularity, and interactive complexity reduction [5–7]. These all relate to the types of activities involved in a quality management strategy that seeks to simplify, standardize, and generally improve products and processes.

As noted, the occurrence of a supply chain risk event can be damaging to any organization no matter where they may be within the supply chain. Technically speaking, risk is defined as the (negative impact to objectives × likelihood of occurrence). Risk management contains four primary steps within its processes. These steps include the following:

**•** Risk identification,

Common certifications related to business include, but are not limited to: ISO 9001, 11000, 14001, and 22000. Most of these involve quality and safety, but none certify for supply chain risk management specifically. The closest standard to achieving this is the ISO 31000 risk management principles and guidelines. However, a certification is not available. Therefore, the rigor of a certification is absent from this standard. In order to be awarded an ISO certification, an organization must submit an application to ISO and undergo a rigorous six‐ stage evaluation process based on various criteria. Other certification bodies do exist that attempt to augment risk management activities in the supply chain. One standard that at least implies an organizational ability to manage risk is Customs‐Trade Partnership Against Terrorism (C‐TPAT). It is a voluntary program that focuses on improving the security of private companies' supply chains with respect to terrorism. However, it addresses only one particular type of risk event related to disruptions produced by actual or potential terrorist threats and does not address an organization's overall risk management maturity. Nonetheless, organiza‐ tional certification by ISO 9001 in particular may be able to signal risk management capabilities simply by virtue of the attention the standards bring to improving process management through the principles of total quality management (TQM). ISO 9001 was revised in 2000 to incorporate the principles of TQM into its certification criteria. Therefore, by extension, TQM maturity, in general, may also provide a signal of supply chain risk management maturity.

A large number of organizations, covering a wide variety of industries, are ISO 9001 certified. Because quality standards and certifications are intended to unify and improve business practices as a whole, the following question arises: are companies that have more mature quality systems, and certified to ISO 9001 in particular, better equipped to manage risk? Researching companies who have quality management programs and how their processes have improved since implementing them may shed light on protecting a company's assets, operations, and its structure from adverse risk events. This research assists in confirming the following statement: A company's use of a TQM system, and particularly through ISO 9001 certification, ensures a high level of risk maturity as compared to that of companies that do

In the next section, the literature is reviewed and research questions stated. This is followed by the research methodology, which includes a description of the data. Results are then presented. The findings of this research are subsequently discussed and conclusions drawn. This is followed by a discussion of areas for further study and limitations to the research.

A study involving supply chain risk starts with a classification of all potential risks. These typically include the following: supply risk, demand risk, process risk, technology risk, logistics risk, information risk, and environment risk [1, 2]. The current research focuses on supply and demand risks because these present supply chain managers with significant challenges due to the severity of the impact and difficulty of effective mitigation. Van Miegham [3] characterizes the loss of a key supplier as having a high effect for aggregate loss severity

not implement a TQM system and/or quality certifications.

136 Quality Control and Assurance - An Ancient Greek Term Re-Mastered

**2. Literature review**

Within these four steps are the proper responses to various types of risk. The first possible response is to mitigate risk. Mitigating entails performing an action to reduce the impact or likelihood of various risk events. The second possible response is to avoid risk. Avoiding risk entails completely ceasing the various activities that create said risk. The third response is to transfer risk. Transferring risk involves shifting risk to other operational areas of the supply chain that are better equipped to handle risk events. The fourth and final way to manage risk is to accept risk. Risk may be accepted if its consequences do not outweigh the benefits of surrounding the risk that is created [8]. Three common methods of assessing risk are effective, but not unified in their approach.

The first method is the Delphi method. The Delphi method was originally developed in 500 B.C. by Greek prophets [9]. The prophets would hear various people's complaints, develop a response, and allow the people to formulate a revised complaint. This method was revised by the RAND Corporation in the 1950s and followed the Greek's original method. The RAND Corporation's method consisted of surveys, followed by a response, followed by revised surveys based on initial results. This process gave participants a chance to re‐assess criteria and re‐evaluate based on the responses, which provided greater insight to their issues at hand. The main drawback to this method is it is time consuming, because of the analysis of the initial surveys followed by the revisions of the second round of surveys.

The second popular method is Monte Carlo Simulation. This method was developed in the Monte Carlo casinos to gauge risk brought upon by the various gambler's chances of winning. This method focuses on uncertain risk and is assessed by model construction and analysis through computer simulation. The negative aspect of this method occurs, because it requires extensive mathematical prowess and requires significant amount of education to be proficient.

The third method is decision tree analysis. Decision tree analysis utilizes graphical methods to draw correlations to common risks. This method is effective, because it creates a visual image of how various risks are linked, but suffers because of its simplicity [10].

Normally, since avoiding risk is so vital, there should be a series of guidelines in place to facilitate conducting supply chain risk management (SCRM) while supporting total quality management (TQM). Performed literature review indicates that there is no unified series of standards of avoiding risk. Even though most companies have their own emergency prepar‐ edness plans in place, a majority of company executives do not review or approve them and only 42% conduct emergency preparedness practices on a regular basis [11]. Companies must learn how to handle diverse amounts of risk such as natural disasters, political unrest, or acts of terrorism. Prior to September 11, 2001, preparedness levels addressing terrorism did not exist.

The World Economic Forum (WEF) has reported that although supply chain risk is an important issue, it is widely mismanaged [12]. Consistent mismanagement of risk across multiple industries might have a ripple effect on global risk which tends to amplify the disruptive impacts of a local risk event with resulting impacts far beyond the corporate sector. The believed cause is companies' inability to detect these ripples (such as the effects of catastrophes and pandemics on the other side of the world) before they become waves that disrupt their supply chains [13]. Many of these preparedness plans have been found to be inefficient because of lack of communication and collaboration, such as Alabama and Louisi‐ ana's responses to Hurricane Katrina [14].

Along with natural disasters and man‐made risk, there is organizational and network risk. Organizational risks include inventory risk, process/operational risk, quality risk, and management risk, while network risks result from interactions between organizations within the supply chain. Agility, flexibility, contingency planning, and preparedness are preferred generic strategies for managing such risks in general [15]. It has been found that larger companies in the private sector are better equipped to individually handle disaster than smaller companies in the public sector [16–18]. These small companies simply do not have the resources to develop a proactive approach, so instead they take a more defensive (reactive) approach constituting risk elimination aspects [19]. Performed research indicates a gap between the risk management policies that large companies are capable of implementing and what their smaller counterparts are able to produce. Many have tried to develop supply chain risk management policies linking risk identification, risk assessment, and risk mitigation to risk performance [20]. Academics along with industry leaders see a need to replace traditional and varying risk management techniques for ones that are better designed to handle extreme complexities, unpredictable events, and threats. They have sought to discover a link between vulnerable factors and controllable factors. The Supply Chain Resilience Assessment and Management (SCRAM) framework was developed, but only served to suit the needs of a select few companies [21]. Companies must also have a framework in place to address general risk between inter‐organizational partnering. Once two or more companies become partners, they assume the other's risk in some or a large capacity, as previously mentioned [22]. According to Zhao et al. [23], "…*supplier, internal, and customer integration are the most important drivers for schedule attainment, competitive performance, and customer satisfaction, respectively"* (p. 115). They find that supply chain risks are negatively related to supply chain integration. Although global integration is crucial and necessary to be competitive, it inherently carries an increased amount of risk.

Performed literature review shows that multiple organizations and companies have tried to institute best practices for mitigating risk. However, there is not one general set standard for how to approach the subject of avoiding or correcting risk in the supply chain. The reason for this variety in practices is possibly because of the variety of industries coupled with the varying ways in which risk can present itself. The food industry will not have the same problems as a metal manufacturing industry. Even within the same industry, companies might not have the same problems. A seafood distributor will have different issues to assess than a fruit producer. These varying sources and types of risk have made it difficult to create guidelines in order to steer companies in the right direction to manage risk in their supply chain. Also, since there is a lack of general guidelines concerning risk management, every industry (even every compa‐ ny) has taken it upon themselves to develop their own guidelines.

The International Standard for Organization (ISO) seeks to remedy the varying levels of supply chain risk preparedness and quality by allowing companies to become universally certified in various ISO certifications to promote a unity of practice.

The steps for ISO certification are as follows:

Normally, since avoiding risk is so vital, there should be a series of guidelines in place to facilitate conducting supply chain risk management (SCRM) while supporting total quality management (TQM). Performed literature review indicates that there is no unified series of standards of avoiding risk. Even though most companies have their own emergency prepar‐ edness plans in place, a majority of company executives do not review or approve them and only 42% conduct emergency preparedness practices on a regular basis [11]. Companies must learn how to handle diverse amounts of risk such as natural disasters, political unrest, or acts of terrorism. Prior to September 11, 2001, preparedness levels addressing terrorism did not

The World Economic Forum (WEF) has reported that although supply chain risk is an important issue, it is widely mismanaged [12]. Consistent mismanagement of risk across multiple industries might have a ripple effect on global risk which tends to amplify the disruptive impacts of a local risk event with resulting impacts far beyond the corporate sector. The believed cause is companies' inability to detect these ripples (such as the effects of catastrophes and pandemics on the other side of the world) before they become waves that disrupt their supply chains [13]. Many of these preparedness plans have been found to be inefficient because of lack of communication and collaboration, such as Alabama and Louisi‐

Along with natural disasters and man‐made risk, there is organizational and network risk. Organizational risks include inventory risk, process/operational risk, quality risk, and management risk, while network risks result from interactions between organizations within the supply chain. Agility, flexibility, contingency planning, and preparedness are preferred generic strategies for managing such risks in general [15]. It has been found that larger companies in the private sector are better equipped to individually handle disaster than smaller companies in the public sector [16–18]. These small companies simply do not have the resources to develop a proactive approach, so instead they take a more defensive (reactive) approach constituting risk elimination aspects [19]. Performed research indicates a gap between the risk management policies that large companies are capable of implementing and what their smaller counterparts are able to produce. Many have tried to develop supply chain risk management policies linking risk identification, risk assessment, and risk mitigation to risk performance [20]. Academics along with industry leaders see a need to replace traditional and varying risk management techniques for ones that are better designed to handle extreme complexities, unpredictable events, and threats. They have sought to discover a link between vulnerable factors and controllable factors. The Supply Chain Resilience Assessment and Management (SCRAM) framework was developed, but only served to suit the needs of a select few companies [21]. Companies must also have a framework in place to address general risk between inter‐organizational partnering. Once two or more companies become partners, they assume the other's risk in some or a large capacity, as previously mentioned [22]. According to Zhao et al. [23], "…*supplier, internal, and customer integration are the most important drivers for schedule attainment, competitive performance, and customer satisfaction, respectively"* (p. 115). They find that supply chain risks are negatively related to supply chain integration. Although global

exist.

ana's responses to Hurricane Katrina [14].

138 Quality Control and Assurance - An Ancient Greek Term Re-Mastered

Stage 1: Proposal stage—development of initial proposal of operational standards

Stage 2: Preparatory stage—preparing proposal for submission

Stage 3: Committee stage—committee review of proposal

Stage 4: Enquiry stage—questioning into company operations post‐review

Stage 5: Approval stage—approval for ISO certification

Stage 6: Publication stage—ISO certification publicized and legitimized

Certifications, such as ISO 9001, and guidelines, such as ISO 31000, are best suited to prepare companies to handle risk (www.iso9001.com, 1) (www.iso.org, 1). Specifically, ISO 31000 (containing ISO Guide 73:2009 and ISO/IEC 31010:2009) seeks to establish guidelines for risk. ISO 31000 contains within it risk management principles, a framework for risk management, and a process for managing risk. It purports to be applicable to any organization regardless of size or sector and claims to increase the likelihood of improving the identification of oppor‐ tunities and threats, thereby allowing an organization to more effectively allocate and use resources for risk treatment. ISO 31000 does not allow for certification but can aid with internal and external audits. These guidelines allow preparation for strategic, operational, and management risk and provide insight into the philosophy and practices that an organization might adopt in building an effective risk management system.

The ISO 9001 standards are also closely related to the approach used in quality improvement systems. In fact, the literature reports that there is a relationship between TQM and ISO 9001. TQM principles were incorporated into the ISO 9001 standard in the year 2000, so a significant relationship is likely to exist [24]. Studies have since shown a relationship between ISO 9001 and TQM. Psomas et al. [25] found that ISO 9001 certified companies achieve significant quality improvement through the implementation of the core process management practices charac‐ teristic of TQM. Sampaio et al. [26] found that most researchers in this area reason that ISO 9001 and TQM are very similar and should both be implemented in the organization together [27–29]. Others suggest that the structure of ISO 9001 can actually aid in the implementation of TQM practices [30]. Therefore, one would expect TQM principles and practices to be present in organizations that have achieved ISO 9001 certification. A few supply chain risk and quality studies do exist. Chapman et al. [31] studied delivery lead time variability and quality management. Tse et al. [32] explored quality and safety problems in the supply chain and introduced a supply chain risk management framework to reduce quality risk. Therefore, the relationship of supply chain risk management to quality management practices has not been widely studied. Based on this review of the literature, the following statements will be assessed empirically:

## **3. Data and research methodology**

Data for this study came from a list of approximately 3000 United States–based organizations, compiled from the 2014 IAAR Directory, ISM‐CV, and CSCMP member lists. These organiza‐ tions cover varying locations throughout the United States spanning industries such as medical, manufacturing, government, etc. The approach used in this study is a survey questionnaire, conducted via e‐mail, that includes demographic questions and questions specific to quality and risk management practices to gauge the relationship of quality man‐ agement practices and risk management practices of organizations. The answers to the survey questions are then used to compare the level of risk management between organizations who more effectively use quality improvement practices and those who do not.

The survey was distributed to potential respondents using Qualtrics survey software. The survey components contained questions measuring the respondent's position within the organization, their familiarity with the organization's quality management and risk manage‐ ment practices, the organization's size, industry and position within the supply chain, where and how the organization manages risk, the company's quality management practices, and their ISO 9001 certification status. The answers to most of these questionnaire statements were then ranked with answers on a 1–5 scale (1 being the worst and 5 being the best). These responses were analyzed using SPSS software. Selected items from the survey questionnaire are provided in the Appendix. These statements include the number of responses in each category, as appropriate.

The measurement of quality management practice within organizations was taken from the work of Dellana and Kros [33]. They developed a set of 13 statements that cover the main quality issues in an organization that could be found at any point in the supply chain and also in any type of industry. In their study, these 13 statements were split into two distinct con‐ structs. Based on data from 565 respondents, 8 of the statements were assigned to the construct of internal‐downstream quality (i.e., customer‐related), while the remaining 5 statements were grouped under the construct of external‐upstream quality (i.e., supplier‐related). In their investigation, the internal‐downstream construct was found to be associated with industry class and a measure of supply chain position, while the external‐upstream construct was not. Therefore, in the current research, the measure of quality maturity is limited to the eight statements from Dellana's and Kros's previous work that made up the internal‐downstream construct (ref. question 9 of the Appendix). The measurement of risk management practice in organizations was more challenging. No generally accepted measure of risk management maturity was found in the literature. Therefore, a set of statements was inspired and derived from a number of sources, including the ISO 31000 guidelines [34–42]. This resulted in the 15 statements listed in the questionnaire for question 7 of the Appendix.

## **4. Results**

and TQM. Psomas et al. [25] found that ISO 9001 certified companies achieve significant quality improvement through the implementation of the core process management practices charac‐ teristic of TQM. Sampaio et al. [26] found that most researchers in this area reason that ISO 9001 and TQM are very similar and should both be implemented in the organization together [27–29]. Others suggest that the structure of ISO 9001 can actually aid in the implementation of TQM practices [30]. Therefore, one would expect TQM principles and practices to be present in organizations that have achieved ISO 9001 certification. A few supply chain risk and quality studies do exist. Chapman et al. [31] studied delivery lead time variability and quality management. Tse et al. [32] explored quality and safety problems in the supply chain and introduced a supply chain risk management framework to reduce quality risk. Therefore, the relationship of supply chain risk management to quality management practices has not been widely studied. Based on this review of the literature, the following statements will be assessed

**1.** Organizations that are ISO 9001 certified have more mature risk management systems

**2.** Organizations that have more mature risk management systems also demonstrate more

Data for this study came from a list of approximately 3000 United States–based organizations, compiled from the 2014 IAAR Directory, ISM‐CV, and CSCMP member lists. These organiza‐ tions cover varying locations throughout the United States spanning industries such as medical, manufacturing, government, etc. The approach used in this study is a survey questionnaire, conducted via e‐mail, that includes demographic questions and questions specific to quality and risk management practices to gauge the relationship of quality man‐ agement practices and risk management practices of organizations. The answers to the survey questions are then used to compare the level of risk management between organizations who

The survey was distributed to potential respondents using Qualtrics survey software. The survey components contained questions measuring the respondent's position within the organization, their familiarity with the organization's quality management and risk manage‐ ment practices, the organization's size, industry and position within the supply chain, where and how the organization manages risk, the company's quality management practices, and their ISO 9001 certification status. The answers to most of these questionnaire statements were then ranked with answers on a 1–5 scale (1 being the worst and 5 being the best). These responses were analyzed using SPSS software. Selected items from the survey questionnaire are provided in the Appendix. These statements include the number of responses in each

The measurement of quality management practice within organizations was taken from the work of Dellana and Kros [33]. They developed a set of 13 statements that cover the main

more effectively use quality improvement practices and those who do not.

than organizations that are not ISO 9001 certified.

mature quality management systems.

140 Quality Control and Assurance - An Ancient Greek Term Re-Mastered

**3. Data and research methodology**

category, as appropriate.

empirically:

The results of the survey based on SPSS statistical software analysis are as follows. A total of 500 of the approximately 3000 potential respondents reached an active e‐mail account. Of these 500, responses to the questionnaire were received from 40 individuals. Of these 40 responses, 18 respondents indicated their organization as ISO 9001 certified, while 20 responded that their organization was not ISO 9001 certified (two did not respond to the ISO certification question and therefore these two questionnaires were unusable). Analysis of nonresponse bias was performed by comparing the mean score for TQM of early versus late responders. There was no significant difference in the mean score between the two groups (t = 1.40, p = 0.18). However, given the small sample size, a comparison of means is by no means conclusive. Therefore, this study and its findings are considered preliminary and exploratory in nature. It should also be noted that this survey was conducted prior to the issuing of the ISO 9001:2015 standards, which incorporate a focus on risk management. The standards in force at the time of this research were ISO 9001:2008.

A general consideration of the survey responses related to demographics of the individual and organization follows. The job title and level of the respondents was generally varied. However, most of the respondents were at the manager level and above (79%). Most were somewhat to very familiar with their organization's supply chain risk management process (78%), with 38% reporting being very familiar. The overwhelming majority were somewhat to very familiar with their organization's quality management practices (94%), with 53% reporting being very familiar. The organizational size, based on number of employees, also varied quite a lot. Grouping into three categories of small, medium, and large yields relative percentages of 42, 29, and 29%, respectively. About 40% of respondent organizations have a department dedi‐ cated to supply chain risk management, while about 32% have each department responsible for assessing supply chain risk related to their particular function. Approximately, 35% are involved in the manufacturing part of the supply chain, while about 38% are involved in distribution. The rest appear to be various services that relate to differing parts of the supply chain. The organizations fall mostly into the general industry categories related to manufac‐ turing (34%) and services (34%). This is followed by transportation/distribution at about 21%. Finally, 47% of respondents reported their organizations are ISO 9001 certified while 53% reported not having ISO 9001 certification.

A factor analysis using principal component analysis was performed on the eight TQM‐related questions to assess whether or not they were consistent with prior research. However, it should be noted that, for the purpose of factor analysis, the present sample did not meet the require‐ ment that minimum sample size be at least 10 times the number of variables per Nunnally [43] (*n* = 40, <10 × 8 = 80). That said, these 8 statements were already shown in Dellana and Kros's previous study to have met the requirements for factor analysis.

It was found that the eight TQM‐related statements all loaded strongly on a single factor. Metrics related to factor analysis were run. The KMO measure of sampling adequacy (index  = 0.863) and Bartlett's test of sphericity (p = 0.0000) were found to be acceptable. Therefore, the results presented herein were very consistent with prior work, which gave support for this research. The model met underlying assumptions (except sample size issues). The reliability of all eight questions together also was very good (Cronbach alpha reliability = 0.93). The TQM maturity variable value used in SPSS analysis was the average score for the eight statements out of a possible score of 5.0. Therefore, a higher average score indicates greater TQM maturity.

**Table 1.** Results of factor analysis for supply chain risk management (SCRM) measurement.

Because the result was so positive for the TQM measure, factor analysis was also conducted on the supply chain risk management (SCRM) statements using principal components analysis in SPSS. Once again, the sample size requirement of Nunnally was not met in this case, so the analysis is preliminary. The KMO measure of sampling adequacy (index = 0.783) and Bartlett's test of sphericity (p = 0.0000) were found to be acceptable. Two factors were extracted using a varimax rotation and Kaiser normalization. The reliability of the statements in the two SCRM factors held very well with Cronbach's alpha measures of 0.88 and 0.86 for Factor 1 and Factor 2, respectively. The two SCRM factor sets are described in **Table 1** and are synonymous with the survey statement designations. Factor 1 seems to describe more so "What" is managed during risk assessment, while Factor 2 seems to describe "How" this risk is assessed and corrected. An overall measure of SCRM combines these two sets of statements for a total of nine statements. In all cases of the risk maturity variables (i.e., SCRM, SCRM1 and SCRM2) the value used in SPSS analysis was the average score for the related statements out of a possible score of 5.0. Therefore, a higher average score indicates greater SCRM maturity.

#### **4.1. Research question 1**

distribution. The rest appear to be various services that relate to differing parts of the supply chain. The organizations fall mostly into the general industry categories related to manufac‐ turing (34%) and services (34%). This is followed by transportation/distribution at about 21%. Finally, 47% of respondents reported their organizations are ISO 9001 certified while 53%

A factor analysis using principal component analysis was performed on the eight TQM‐related questions to assess whether or not they were consistent with prior research. However, it should be noted that, for the purpose of factor analysis, the present sample did not meet the require‐ ment that minimum sample size be at least 10 times the number of variables per Nunnally [43] (*n* = 40, <10 × 8 = 80). That said, these 8 statements were already shown in Dellana and Kros's

It was found that the eight TQM‐related statements all loaded strongly on a single factor. Metrics related to factor analysis were run. The KMO measure of sampling adequacy (index  = 0.863) and Bartlett's test of sphericity (p = 0.0000) were found to be acceptable. Therefore, the results presented herein were very consistent with prior work, which gave support for this research. The model met underlying assumptions (except sample size issues). The reliability of all eight questions together also was very good (Cronbach alpha reliability = 0.93). The TQM maturity variable value used in SPSS analysis was the average score for the eight statements out of a possible score of 5.0. Therefore, a higher average score indicates greater TQM maturity.

> **SCRM Factor 2 statements** (Questionnaire statements b, d, e, f)

annually

b. Risk management is an ad hoc process for us that

d. Our risk management assessment is based on

e. Our risk management assessment occurs at least

f. Risk assessment is a quantitative process for us

occurs informally on an as‐needed basis

probability analysis of relevant risks

reported not having ISO 9001 certification.

142 Quality Control and Assurance - An Ancient Greek Term Re-Mastered

**SCRM Factor 1 statements**

education, and treatment

potential supply chain risks

potential supply chain risks

management process (e.g., ISO 31000)

organization

(Questionnaire statements i, k, l, m, n)

i. We follow the four step process of risk identification, analysis,

k. We prioritize risk events based on severity of impact to our

l. We involve our suppliers in identification and mitigation of

m. We encourage our suppliers to use a structured risk

n. We work with our customers to identify and mitigate

**Table 1.** Results of factor analysis for supply chain risk management (SCRM) measurement.

Because the result was so positive for the TQM measure, factor analysis was also conducted on the supply chain risk management (SCRM) statements using principal components analysis in SPSS. Once again, the sample size requirement of Nunnally was not met in this case, so the analysis is preliminary. The KMO measure of sampling adequacy (index = 0.783) and Bartlett's test of sphericity (p = 0.0000) were found to be acceptable. Two factors were extracted using a varimax rotation and Kaiser normalization. The reliability of the statements in the two SCRM

previous study to have met the requirements for factor analysis.

The first research question explores whether organizations that are ISO 9001 certified have more mature risk management systems than organizations that are not ISO 9001 certified.

The relationship of ISO certification with SCRM was measured with the dependent variable entered as the respective risk management variables (i.e., SCRM1, SCRM2, and SCRM) using a univariate linear regression in SPSS with ISO 9001 certification as an independent variable. In all cases, the number of employees, supply chain position, and TQM score were also entered as control variables. Because of low frequencies by category, some data consolidation was performed. The number of employees was collapsed into small, medium, and large groups. Supply chain position was generally sorted into manufacturing and distribution and other.

The results of the regression analysis are given in **Table 2**. Backward removal of variables in SPSS resulted in the ISO 9001 reaching a significance of at best p = 0.064. This was for the case of the SCRM overall. SCRM2 was second best with a significance of p = 0.079. Neither of these make the generally accepted threshold of 0.05 significance level and were, therefore, not included in the final SPSS models. The models were also run with all control variables excluded to determine whether this would make a difference to the result. The outcome was similar to the model that included control variables, with SCRM1 at a significance of p = 0.806, SCRM2 at a significance of p = 0.061, and SCRM at a significance p = 0.248. Therefore, the evidence is weak for a conclusion that ISO 9001 certified organizations have more mature risk management systems than non‐ISO 9001 certified organizations.

A breakdown by industry type had been a goal of this research, but the sample size was too small to accommodate a rigorous statistical analysis. However, it is worth noting that the split of ISO and non‐ISO certified organizations was quite different between organizations classified as manufacturing (9 certified, 4 noncertified) and those classified as service (2 certified and 11 noncertified). Distribution‐related organizations were more evenly split (five certified and three noncertified). **Table 3** shows a breakdown of the scores for SCRM and TQM by major industry type. The scores for SCRM seemed to favor the service organization over manufac‐ turing, especially for SCRM2. Although ISO 9001 does not clearly differentiate regarding SCRM maturity, there is at least the implication in this research that industry type may be a differentiator. A similar analysis could be conducted by supply chain position. It might be expected that organizations further downstream in the supply chain would exhibit greater SCRM maturity than those upstream given the lengthening of the supply chain heading downstream, which increases the chance of risk event occurrence that disrupts the supply chain.

**Table 2.** Linear regression analysis results for SCRM variables versus ISO 9001.

**Table 3.** Mean SCRM and TQM scores by general industry type.

#### **4.2. Research question 2**

The second research question explores whether organizations that have more mature supply chain risk management systems also demonstrate more mature quality management systems. The simple linear regression analysis of **Table 2** incorporates the TQM score as an independent variable in the analysis of relationships with SCRM variables. TQM was found to be strongly significantly related to SCRM1 (p < 0.000) and SCRM overall (p = 0.002). Both of these met the generally accepted threshold of 0.05 significance level and were, therefore, included in the final SPSS models. SCRM Factor 1 statements tend to describe the actual risk management process details, outlining the process steps and who is involved. SCRM Factor 2 statements are more general and have more to do with the timing and nature of the process. The models were also run for SCRM variables with TQM as the only independent variable in each case to determine whether this would make a difference to the result. The outcome was similar to the model that included control variables with SCRM1 at a significance of p < 0.000, SCRM2 at a significance of p = 0.248, and SCRM at a significance p = 0.003. This suggests that the presence of a mature TQM program may signal a more mature supply chain risk management system based on the actual process scope and steps.

Analysis was also run on the correlation between TQM and SCRM1 specifically to see where there were strong relationships between specific quality management statements and supply chain risk management maturity statements. SCRM1 statements i, k, and l were all positively correlated to TQM statements a, b, c, d, e, g, and h (see **Table 1** for the specific questions). SCRM1 statement m was positively correlated to TQM statements a, b, c, g, and h. SCRM1 statement n, b, g, h was positively correlated to TQM statements b, g, and h. Therefore, statement f, "Quality metrics or standards are kept," was not correlated to any SCRM1 statements. This analysis suggests that TQM is strongly related to most of the SCRM1 state‐ ments, with only a few weak relationships. Statement 6 stands out as unrelated to SCRM1, while statement n of SCRM1, "We work with our customers to identify and mitigate potential supply chain risks", stands out as only weakly related to TQM.

## **5. Conclusions**

downstream, which increases the chance of risk event occurrence that disrupts the supply

0.209 0.630 0.117 −0.070

0.117 0.280 0.310 −0.059

0.154 0.547 0.304 −0.119

**Score category Manufacturing (n = 13) Service (n = 13) Distribution (n = 8)**

The second research question explores whether organizations that have more mature supply chain risk management systems also demonstrate more mature quality management systems. The simple linear regression analysis of **Table 2** incorporates the TQM score as an independent variable in the analysis of relationships with SCRM variables. TQM was found to be strongly significantly related to SCRM1 (p < 0.000) and SCRM overall (p = 0.002). Both of these met the

SCRM 3.13 **3.71** 3.56 SCRM1 3.32 **3.60** 3.58 SCRM2 2.84 **3.84** 3.53 TQM 4.02 4.05 **4.32**

**Table 2.** Linear regression analysis results for SCRM variables versus ISO 9001.

**Table 3.** Mean SCRM and TQM scores by general industry type.

1.519 4.516 0.743 −0.467

0.625 1.653 1.814 −0.324

0.903 3.469 1.928 −0.714 0.139 **0.000** 0.464 0.644

0.537 0.109 0.079 0.749

0.374 **0.002** 0.064 0.481

**Variable Beta t Significance**

144 Quality Control and Assurance - An Ancient Greek Term Re-Mastered

chain.

SCRM1

SCRM2

SCRM

**•** SC position **•** TQM **•** ISO 9001

**•** SC position **• TQM •** ISO 9001

**•** SC position **• TQM •** ISO 9001

**•** Number employees

**•** Number employees

**•** Number employees

**4.2. Research question 2**

These results suggest that ISO 9001 is not strongly related to supply chain risk management maturity. There was not a strong relationship between ISO certification and company's preparedness level of SCRM and TQM. However, TQM is clearly related to SCRM1, no matter the variables that are included. The more mature a company's TQM practices, the more mature the company is in terms of SCRM as identified by the SCRM Factor 1 (which describes the actual risk management process details, outlining the process steps and who is involved). Most of the statements comprising this factor were positively correlated to the TQM statements (denoting total quality management practices). In particular, these included the following SCRM1 statements:

The statement related to customers (i.e., "We work with our customers to identify and mitigate potential supply chain risks") was only weakly associated to TQM.

The lack of significance between ISO9001:2008 and the two factors of SCRM, based on sur‐ vey results and analysis, suggests that the standards for ISO 9001:2008 and 31000 do not pro‐ vide a significant advantage to risk assessment and management.

Though they provide positive frameworks for overall company structure and project devel‐ opment, ISO 9001:2008 was not equipped to provide a framework for risk management that eclipses alternative methods. It should be noted that ISO 31000 is a new addition and has not yet been fully developed. Before the implementation of ISO 31000, risk was not explicitly addressed under ISO standards.

## **6. Limitations and further research**

The number of respondents in this study was relatively small and necessarily reduces the power of the statistical tests in this study. The low response rate also introduces the potential for nonresponse bias. Therefore, the conclusions should be treated with caution. However, the results suggest that a higher response rate would indicate a similar pattern of results based on prior studies, in particular related to the TQM maturity metric.

To further aid such a study, case studies may be performed to monitor the daily processes of companies who are and are not ISO certified. This would provide more of an in depth analy‐ sis of the effects of ISO certification on risk management. E‐mail provided certain limitations to accessibility, because of company filtering software and lack of personal interaction. Joint participation with the ISO organization into the effectiveness of their certifications would al‐ so lead to a better understanding and greater ease of access.

However, the results of this initial survey and analysis suggest that ISO does not play a sig‐ nificant role in risk assessment and correction. Though, the more mature a company is able to be in terms of their total quality management procedures, the better equipped it will be to handle supply chain risk.

Further study should be conducted to seek a larger sample size in order to assess the relia‐ bility of the results in this study. It would also help shed light on differences by industry type and supply chain position. However, this will necessarily involve the newer standards for ISO 9001:2015. Therefore, it is unlikely that this study can be replicated. It would, none‐ theless, still be of interest to assess the degree to which the new standards have impacted on organizational effectiveness in managing risk in the supply chain. Since the new standards now focus attention specifically on risk management, it should signal to customers that ISO 9001:2015 certification is a good supplier prequalifier when it comes to supply chain risk management and not just quality. Or at least that would be the expectation. Further study is needed to confirm this.

**Appendix: Selected survey questions** (number responding given in parentheses)

	- **•** CEO/President (7)
	- **•** Vice President (8)
	- **•** Director (4)

The statement related to customers (i.e., "We work with our customers to identify and mitigate

The lack of significance between ISO9001:2008 and the two factors of SCRM, based on sur‐ vey results and analysis, suggests that the standards for ISO 9001:2008 and 31000 do not pro‐

Though they provide positive frameworks for overall company structure and project devel‐ opment, ISO 9001:2008 was not equipped to provide a framework for risk management that eclipses alternative methods. It should be noted that ISO 31000 is a new addition and has not yet been fully developed. Before the implementation of ISO 31000, risk was not explicitly

The number of respondents in this study was relatively small and necessarily reduces the power of the statistical tests in this study. The low response rate also introduces the potential for nonresponse bias. Therefore, the conclusions should be treated with caution. However, the results suggest that a higher response rate would indicate a similar pattern of results based on

To further aid such a study, case studies may be performed to monitor the daily processes of companies who are and are not ISO certified. This would provide more of an in depth analy‐ sis of the effects of ISO certification on risk management. E‐mail provided certain limitations to accessibility, because of company filtering software and lack of personal interaction. Joint participation with the ISO organization into the effectiveness of their certifications would al‐

However, the results of this initial survey and analysis suggest that ISO does not play a sig‐ nificant role in risk assessment and correction. Though, the more mature a company is able to be in terms of their total quality management procedures, the better equipped it will be to

Further study should be conducted to seek a larger sample size in order to assess the relia‐ bility of the results in this study. It would also help shed light on differences by industry type and supply chain position. However, this will necessarily involve the newer standards for ISO 9001:2015. Therefore, it is unlikely that this study can be replicated. It would, none‐ theless, still be of interest to assess the degree to which the new standards have impacted on organizational effectiveness in managing risk in the supply chain. Since the new standards now focus attention specifically on risk management, it should signal to customers that ISO 9001:2015 certification is a good supplier prequalifier when it comes to supply chain risk management and not just quality. Or at least that would be the expectation. Further study is

potential supply chain risks") was only weakly associated to TQM.

146 Quality Control and Assurance - An Ancient Greek Term Re-Mastered

vide a significant advantage to risk assessment and management.

prior studies, in particular related to the TQM maturity metric.

so lead to a better understanding and greater ease of access.

addressed under ISO standards.

handle supply chain risk.

needed to confirm this.

**6. Limitations and further research**

	- **•** 20 or fewer (5)
	- **•** 21 to 100 (11)
	- **•** 101 to 500 (6)
	- **•** 501 to 5000 (5)
	- **•** 5001 to 10,000 (3)
	- **•** More than 10,000 (8)
	- **•** Not sure
	- **a.** Aerospace manufacturing (2)
	- **b.** Automotive manufacturing (1)
	- **c.** Chemicals/plastics manufacturing (2)
	- **d.** Computer equipment manufacturing (0)
	- **e.** Electronics/electrical manufacturing (1)
	- **f.** Fabricated metal products manufacturing (2)
	- **g.** Food products manufacturing (0)
	- **h.** Government/public administration (1)
	- **i.** Machinery manufacturing (Industrial/Commercial) (4)
	- **j.** Pharmaceuticals manufacturing (1)
	- **k.** Retail trade/merchandising (0)
	- **•** We outsource our supply chain risk analysis to a consulting firm. (1)
	- **•** We have a dedicated Risk Management Department (i.e., an internal consultant) that performs supply chain risk analysis for the organization. (9)
	- **•** Each department is responsible for assessing supply chain risk related to their particular function. (12)
	- **•** We have a specific department (e.g., procurement) that is charged with managing supply chain risk. (Please specify department:) (6)
	- **•** Other. Specify: (9)
	- **•** Raw/basic material manufacture (1)
	- **•** Component/sub‐assembly manufacture (1)
	- **•** Final product manufacture (11)
	- **•** Warehousing/distribution (14)
	- **•** Retail (0)
	- **•** Other. Specify: (10)
	- **•** Very unfamiliar (7)
	- **•** Unfamiliar (1)
	- **•** Somewhat familiar (7)
	- **•** Familiar (8)
	- **•** Very familiar (14)

**7.** The following statements relate to your organization's supply chain risk management system. Please indicate the degree to which you agree or disagree with each statement in relation to your organization.

(1 = Strongly disagree, 2 = Disagree, 3 = Neither agree nor disagree, 4 = Agree, 5 = Strongly agree)

\*Coded in reverse in the analysis.

	- **•** Very unfamiliar (1)
	- **•** Unfamiliar (1)

**l.** Services: Educational services (1)

148 Quality Control and Assurance - An Ancient Greek Term Re-Mastered

**p.** Warehousing/distribution (7)

**n.** Transportation (1)

**q.** Wholesale trade (2)

**r.** Other. Specify: (1)

function. (12)

**•** Other. Specify: (9)

the supply chain?

**•** Retail (0)

**•** Other. Specify: (10)

**•** Very unfamiliar (7)

**•** Somewhat familiar (7)

**•** Unfamiliar (1)

**•** Familiar (8)

**•** Very familiar (14)

**•** Raw/basic material manufacture (1)

**•** Final product manufacture (11)

**•** Warehousing/distribution (14)

**•** Component/sub‐assembly manufacture (1)

**o.** Utilities (1)

**m.** Services: Professional/scientific/technical (12)

**4.** How does your organization conduct supply chain risk analysis? (select one)

**•** We outsource our supply chain risk analysis to a consulting firm. (1)

performs supply chain risk analysis for the organization. (9)

supply chain risk. (Please specify department:) (6)

**•** We have a dedicated Risk Management Department (i.e., an internal consultant) that

**•** Each department is responsible for assessing supply chain risk related to their particular

**•** We have a specific department (e.g., procurement) that is charged with managing

**5.** Which of the following positions best describes your organization's business function in

**6.** How familiar are you with your organization's supply chain risk management process?

**9.** The following statements relate to your organization's quality management system. Please indicate the degree

to which you agree or disagree with each statement in relation to your organization.

(1 = Strongly disagree, 2 = Disagree, 3 = Neither agree nor disagree, 4 = Agree, 5 = Strongly agree)

	- **•** Yes (18)
	- **•** No (20)

## **Acknowledgements**

This chapter is built on prior research from "The Relationship of Organizational Quality Management Practice with Supply Chain Risk Management Program Maturity", Thesis for East Carolina University Honors College

## **Author details**

Tyler Florio

Address all correspondence to<EMAIL_ADDRESS>
College of Business, East Carolina University, Greenville, USA

## **References**

**9.** The following statements relate to your organization's quality management system. Please

to which you agree or disagree with each statement in relation to your organization.

(1 = Strongly disagree, 2 = Disagree, 3 = Neither agree nor disagree, 4 = Agree, 5 = Strongly

**a.** Customer satisfaction/service data for key indicators are actively accumulated and

**c.** Customer complaints are tracked and managed through a formal complaint system

**e.** Employee use of quality‐related problem solving techniques is actively supported

This chapter is built on prior research from "The Relationship of Organizational Quality Management Practice with Supply Chain Risk Management Program Maturity", Thesis for

**b.** Customer data are used to make internal quality improvements.

indicate the degree

analyzed.

with feedback.

**d.** Customer service standards are kept.

150 Quality Control and Assurance - An Ancient Greek Term Re-Mastered

(ex. DMAIC, control charting, etc.).

**f.** Quality metrics or standards are kept.

**g.** Team‐based quality improvement is actively supported.

**10.** Is your organization certified for ISO 9001 or equivalent?

**h.** Top management has a demonstrated commitment to quality.

agree)

**•** Yes (18)

**•** No (20)

**Author details**

Tyler Florio

**Acknowledgements**

East Carolina University Honors College

Address all correspondence to<EMAIL_ADDRESS>
College of Business, East Carolina University, Greenville, USA

[27] Escanciano C., Fernandez E., Vasquez C., Influence of ISO 9000 certification on the progress of Spanish industry towards TQM. International Journal of Quality & Reliability Management. 2011; 18(5):481–494.

[14] Chacon N., Doherty S., Hayashi C., Green R., New models for addressing supply chain and transport risk: an initiative of the risk response network in collaboration with Accenture. World Economic Forum: Cologny/Geneva Switzerland;

[15] Ghadge A., Dani S., Kalawsky R., Supply chain risk management: present and future scope. International Journal of Logistics Management. 2012; 23(3):313–339.

[16] Kumar S. Managing risks in a relief supply chain in the wake of an adverse event. OR

[17] Aguiar Y.M., Assessing the CARVER+S risk management model of terrorism prepar‐ edness in business continuity planning. Northcentral University, USA: PhD disserta‐

[18] Skipper J.B., Hanna J.B., Gibson B.J., Alabama power response to Katrina: managing a severe service supply chain disruption. Arden: Jordan Whitney Enterprises Inc; 2010.

[19] Ellegaard C., Supply risk management in a small company perspective. Supply Chain

[20] Kern D., Moser R., Hartmann E., Moder M., Supply risk management: model devel‐ opment and empirical analysis. International Journal of Physical Distribution &

[21] Pettit T.J., Supply chain resilience: development of a conceptual framework, an assessment tool and an implementation process. The Ohio State University, USA: PhD

[22] Finch P., Supply chain risk management. Supply Chain Management. 2004; 9(2):183–

[23] Zhao L., Huo B., Sun L., Zhao X., The impact of supply chain risk on supply chain integration and company performance. Supply Chain Management: An International

[24] Fotopoulos C., Psomas E., The use of quality management tools and techniques in ISO 9001:2000 certified companies: the Greek case. International Journal of Productivity and

[25] Psomas E.L., Fotopoulos C.V., Kafetzopoulos D.P., Core process management practices, quality tools and quality improvement in ISO 9001 certified manufacturing companies.

[26] Sampaio P., Saraiva P., Rodrigues A.G., ISO 9001 certification research: questions, answers and approaches. International Journal of Quality & Reliability Management.

2012.

tion; 2011.

dissertation; 2008.

2009; 26(1):38–58.

Journal. 2013; 18(2):115–131.

196.

Insight. 2011; 24(2):131–157.

152 Quality Control and Assurance - An Ancient Greek Term Re-Mastered

Management. 2008; 13(6):425–434.

Logistics Management. 2012; 42(1):60–82.

Performance Management. 2009; 58(6):564–580.

Business Process Management Journal. 2011; 17(3):437–460.

#### **ALAMEDA Ecosystem: Centering Efforts in Software Testing Development ALAMEDA Ecosystem: Centering Efforts in Software Testing Development**

José González Enríquez, Julián Alberto García-García, Francisco José Domínguez-Mayo and María José Escalona Cuaresma José González Enríquez, Julián Alberto García-García, Francisco José Domínguez-Mayo and María José Escalona Cuaresma

Additional information is available at the end of the chapter Additional information is available at the end of the chapter

http://dx.doi.org/10.5772/66043

#### **Abstract**

[41] Thompson A. Risk Management Framework. Government of South Australia, Ade‐

[42] COSO., Strengthening enterprise risk management for strategic advantage. Committee of Sponsoring Organizations of the Treadway Commission; 2009 [cited 2016 Aug 21].

laide: Department of Communities and Social Inclusion; 2009.

[43] Nunnally J.C., Psychometric theory. 2nd ed. New York: McGraw‐Hill; 1978.

Available from: www.coso.org.

154 Quality Control and Assurance - An Ancient Greek Term Re-Mastered

One of the most important and critical aspects to improve the quality assurance in software is to improve the testing process by utilizing techniques and tools, which will enhance the software testing process, making it more effective and efficient. This chapter presents ALAMEDA ecosystem, a software package that centers its efforts in software testing development and is a result from a real-world project. ALAMEDA provides support to lifecycles focused on the generation, implementation, and testing organization from the earliest stages of software development. In addition, the ecosystem provides an environment of rating the degree of compliance of organizations with the International Standard for Testing ISO/IEC-29119. It is proposed as a tool to use during the various iterations that may occur in an agile software development process.

**Keywords:** testing development, model-driven testing, quality assurance

## **1. Introduction**

The area of Information Technology and Communications (ICT) is an essential element of innovation in other business sectors such as industrial, logistics, health, etc., where the effectiveness and efficiency in software development is critical for a fundamental and safe operation.

© 2017 The Author(s). Licensee InTech. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. © 2017 The Author(s). Licensee InTech. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

The process of software development involves a series of activities in which the chances of a human error is high (mistakes can happen in the beginning of the process, in which the objectives can be inadequately specified, as well as during later steps). Therefore, it is important to research and define techniques and methods (as well as supporting tools) in order to detect and solve any area of special attention as soon as possible. In this sense, over recent decades, a number of studies have analyzed the cost of resolving software defects. The cost increases with the time of the defect in the system [1]. As such, the cost of rectifying an error is lower, the earlier the problem is detected.

The budget of software projects is critical and limited in many cases [2]. For this reason, both research institutions and enterprises look for solutions to improve quality assurance of the software development process [3]. The finding of methods and tools to improve the quality, reduce costs and increase the guarantee of results becomes an essential aim for enterprises and development teams [4].

One of the most important and critical aspects to improve the quality assurance in software is to improve the testing process by utilizing techniques and tools, which will enhance the software-testing process, making it more robust. It is a systematic method to determine interrelated factors and parameters affecting a process and its desired output [5]. It is usually performed for one of two reasons: to detect defects and to estimate reliability. The key to software testing is trying to find the modes of failure—something that requires testing the code on all possible inputs and depending on the environment it can be implemented at any time during the development process [6].

This chapter presents a testing solution (named ALAMEDA) to innovate in all these aspects related to software testing. ALAMEDA offers to project managers and developers a systematic and tool-based approach for managing (i.e., defining, controlling, measure, etc.) test cases. This systematic and automatic management allows the detection and correction of software errors before they become costly interruptions in the production line.

The ALAMEDA ecosystem aims to provide a set of applications which allows software development companies to ensure the management, automation, and integration of test cases from an early stage of the develop lifecycle. For this purpose, ALAMEDA integrates the wellknown Model-Driven Engineering (MDE) [7, 8] paradigm on software testing and its automation of software and testing development by use of the NDT (Navigational Development Techniques) methodology [9] and NDT-Suite [10].

MDE has been used as a direction vector of the ALAMEDA ecosystem because it is one of most entrenched paradigms within the software engineering area. As such, on the one hand it will help to define the approach and on the other hand, it will manage the conceptual complexity the process engineering area entails. In addition, MDE has been also successfully applied to real software environments and it has shown a considerable area of impact on reducing timeto-market and improving software product quality. For instance, in the business process management [11], healthcare, software product lines [12], or web engineering among others.

The remainder of this chapter is organized as follows: Section 2 presents the theoretical foundations of ALAMEDA. Section 3 presents the ALAMEDA ecosystem for software testing. Finally, Sections 4 and 5 briefly discuss some related work and highlight conclusions.

## **2. Theoretical foundations of ALAMEDA**

The process of software development involves a series of activities in which the chances of a human error is high (mistakes can happen in the beginning of the process, in which the objectives can be inadequately specified, as well as during later steps). Therefore, it is important to research and define techniques and methods (as well as supporting tools) in order to detect and solve any area of special attention as soon as possible. In this sense, over recent decades, a number of studies have analyzed the cost of resolving software defects. The cost increases with the time of the defect in the system [1]. As such, the cost of rectifying an error is lower,

The budget of software projects is critical and limited in many cases [2]. For this reason, both research institutions and enterprises look for solutions to improve quality assurance of the software development process [3]. The finding of methods and tools to improve the quality, reduce costs and increase the guarantee of results becomes an essential aim for enterprises and

One of the most important and critical aspects to improve the quality assurance in software is to improve the testing process by utilizing techniques and tools, which will enhance the software-testing process, making it more robust. It is a systematic method to determine interrelated factors and parameters affecting a process and its desired output [5]. It is usually performed for one of two reasons: to detect defects and to estimate reliability. The key to software testing is trying to find the modes of failure—something that requires testing the code on all possible inputs and depending on the environment it can be implemented at any time

This chapter presents a testing solution (named ALAMEDA) to innovate in all these aspects related to software testing. ALAMEDA offers to project managers and developers a systematic and tool-based approach for managing (i.e., defining, controlling, measure, etc.) test cases. This systematic and automatic management allows the detection and correction of software errors

The ALAMEDA ecosystem aims to provide a set of applications which allows software development companies to ensure the management, automation, and integration of test cases from an early stage of the develop lifecycle. For this purpose, ALAMEDA integrates the wellknown Model-Driven Engineering (MDE) [7, 8] paradigm on software testing and its automation of software and testing development by use of the NDT (Navigational Development

MDE has been used as a direction vector of the ALAMEDA ecosystem because it is one of most entrenched paradigms within the software engineering area. As such, on the one hand it will help to define the approach and on the other hand, it will manage the conceptual complexity the process engineering area entails. In addition, MDE has been also successfully applied to real software environments and it has shown a considerable area of impact on reducing timeto-market and improving software product quality. For instance, in the business process management [11], healthcare, software product lines [12], or web engineering among others.

the earlier the problem is detected.

156 Quality Control and Assurance - An Ancient Greek Term Re-Mastered

during the development process [6].

before they become costly interruptions in the production line.

Techniques) methodology [9] and NDT-Suite [10].

development teams [4].

The model-driven software development has become one of the most important paradigm in computer engineering. Indeed, it has shown a considerable impact on reducing time-to-market and has improved overall quality.

However, the development of high-quality software solutions does not only require systematic processes of software development, but it also requires systematic processes of software testing. At present, this latter aspect has not been attracting attention within the software industry.

It is a fact that if software companies do no invest resources and efforts in the process of software testing, it is quite likely that there will be a waste of money in the short term because of a large volume of errors. Therefore, if a systematic process is not used in tasks of testing, software developers will spend most of the time to check quality and application functionality rather than developing new projects [13].

The ALAMEDA ecosystem aims to innovate in the systematic and automatic application of the process of software testing in the real environment. For this purpose, the ecosystem presented herein takes into account the new standard for software testing (ISO/IEC/IEEE 29119) and a well-validated methodology named NDT. The latter proposes techniques and mechanisms to generate test cases from the early stages of the software lifecycle. These theoretical foundations are explained in the next sections.

#### **2.1. ISO/IEC/IEEE 29119 software testing**

The International Software Testing Standard (ISO/IEC/IEEE 29119) [14] is a standard created between 2013 and 2015. It was created in order to unify all knowledge on software testing and establish basic knowledge concerning this discipline.

On the one hand, this standard provides useful methodological and conceptual guidelines from the perspective of professional or software providers. In addition, it provides (offers) a common terminology, certifications, well-defined techniques, professional qualification, continuous improvement of the process of software testing, inter-operability, and consistency, among others.

On the other hand, and from the perspective of customers, ISO/IEC/IEEE 29119 provides customer confidence on their software provider, an industry benchmark for good practice, and a contractual relationship between the customer and the software provider, among others.

ISO/IEC/IEEE 29119 focuses on a process model of three levels based on the risk factor in the software-testing phase. This model provides guidelines on strategies and policies so as to manage the process of software testing (i.e., defining testing plans, monitoring and controlling software tests, establishing the test environment, executing tests, etc.). All these aspects have been described in detail in five volumes, as follows:

#### **2.2. Model-driven engineering on software testing**

The model-driven engineering paradigm [15] came up in order to tackle the complexity of platforms and the inability of third generation languages to relieve this complexity. It effectively expresses the domain concepts of the problem. This new paradigm, apart from raising the level of abstraction [16], intends to increase automation during the life cycle of software development.

MDE works, as the primary form of expression, with definitions of models (which can be defined, e.g., by means of Unified Modeling Language (UML) [17]) and transformations (which can be defined, e.g., by means of Query/View/Transformation (QVT) [18]) among these models which entail the production of other models. Both elements are essentials to apply MDE. In addition, it is important to take into account the concepts of metamodel and transformation rules. On the one hand, a metamodel is composed of a set of basic elements, its relationships, and its semantic constraints to build well-defined models. On the other hand, a transformation between two models defines a set of relationships between elements of metamodel A (source) and elements of metamodel B (target) [19].

Although MDE offers a theoretical framework, it has not been standardized exactly in these terms in order to facilitate the application of MDE in real projects. OMG presents MDA, which stands for model-driven architecture [20], as a platform to support the MDE paradigm. MDA proposes to base the software development on models which make transformations in order to generate a code or another model with characteristics of a particular technology (or lowest level of abstraction). As transformations go on, it may be noted that the models become more concrete and the abstract model changes into another one compatible with a particular technology or platform. MDA is based on four types of levels or models. These models are (of highest to lowest level of abstraction):

manage the process of software testing (i.e., defining testing plans, monitoring and controlling software tests, establishing the test environment, executing tests, etc.). All these aspects have

**•** ISO/IEC 29119-1: Concepts & Definitions. It introduces the vocabulary on which all standards in the 29119 series are built. It also provides examples of the application of each concept

**•** ISO/IEC 29119-2: Test Processes. It defines a generic process model for software testing that can be used to govern, manage, and implement software testing in any organisation, project, or testing activity. This model is based on three layers: (i) organizational test specifications (e.g., organizational test policy, organizational test strategy); (ii) test management; and (iii)

**•** ISO/IEC 29119-3: Test Documentation. It defines templates for test documentation that cover

**•** ISO/IEC 29119-4: Test Techniques. It defines techniques and methods to design test cases. **•** ISO/IEC 29119-5: Keyword Driven Testing. It defines guidelines for supporting keyworddriven testing which is a way of describing test cases by using a pre-defined set of keywords. These keywords are names which are associated with a set of actions that are required to perform a specific step in a test case. By using keywords to describe test steps instead of natural language, test cases can be made easier to understand, to maintain, and to automate.

The model-driven engineering paradigm [15] came up in order to tackle the complexity of platforms and the inability of third generation languages to relieve this complexity. It effectively expresses the domain concepts of the problem. This new paradigm, apart from raising the level of abstraction [16], intends to increase automation during the life cycle of

MDE works, as the primary form of expression, with definitions of models (which can be defined, e.g., by means of Unified Modeling Language (UML) [17]) and transformations (which can be defined, e.g., by means of Query/View/Transformation (QVT) [18]) among these models which entail the production of other models. Both elements are essentials to apply MDE. In addition, it is important to take into account the concepts of metamodel and transformation rules. On the one hand, a metamodel is composed of a set of basic elements, its relationships, and its semantic constraints to build well-defined models. On the other hand, a transformation between two models defines a set of relationships between elements of metamodel A (source)

Although MDE offers a theoretical framework, it has not been standardized exactly in these terms in order to facilitate the application of MDE in real projects. OMG presents MDA, which stands for model-driven architecture [20], as a platform to support the MDE paradigm. MDA proposes to base the software development on models which make transformations in order to generate a code or another model with characteristics of a particular technology (or lowest

been described in detail in five volumes, as follows:

158 Quality Control and Assurance - An Ancient Greek Term Re-Mastered

in practice.

dynamic testing.

software development.

the entire software testing life cycle.

**2.2. Model-driven engineering on software testing**

and elements of metamodel B (target) [19].

MDE in general (or MDA in particular) has been successfully applied in different environments and research areas [21], software testing is one of them. In fact, this particular case is named as model-driven testing (MDT).

MDT is becoming more standardized in order to approach the automation of software testing, thanks to different research works and proposals [22–24]. This approach can significantly reduce the most painstaking cycle of all software development efforts—testing. Testing currently comprises between 30 and 70% of all software development projects, which calls for a significant amount [25] of resources that must be properly managed. Therefore, MDT and supporting tools enable software developers and testers to become far more productive and reduce the time-to-market, while maintaining high standards of software quality.

**Figure 1.** General scheme for generation of test cases integrated in ALAMEDA.

**Figure 1** shows the vision of MDT from the perspective of MDA. As this suggests, models belonging to the PIM level can be transformed into models belonging to the PSM level, getting the code from the systems derived from PSM through another transformation step. This is the general philosophy of work proposed by MDA, but if this one is applied to software testing, some concepts change. In this case, the process of testing starts with models of System's Test which belong to the PIM level; or also named platform independent test (PIT). The quality team will elaborate on these tests to provide additional information. Next the PSM, which contains specific information about the implementation of the system, can be translated into the Platform Specific Test (PST), using the previous information from PIT and the knowledge of the quality team. Finally, these tests are morphed into codes for testing the system, ensuring the quality of the test and covering all the project-specific requirements.

All these theoretical foundations of MDT have been systematically applied and adapted within the ALAMEDA ecosystem. For this purpose, ALAMEDA implements a methodological framework to generate case tests from functional requirements (FR) as well as managing and controlling these case tests. This framework is based in NDT, a well-contrasted methodology. The foundations of NDT and how this one has been integrated into ALAMEDA are detailed in Section 2.3.

#### **2.3. The NDT methodology from the perspective of software testing**

NDT is a web methodology and is covered by the MDE paradigm. At present, NDT supports classical and agile software development lifecycles. For this purpose, NDT defines a set of metamodels for each phase of the software lifecycle (viability study, requirements, analysis, design, implementation, testing and maintenance) and it establishes a set of transformation rules (based on QVT) to generate models from other ones systematically. This implies a lower cost for the software development because of the automation of the process.

The application of MDE-based methodologies (such as NDT) and particularly, the application of transformations among models may become monotonous and very expensive if there are no supporting tools. These tools automate the process in order to get all the potential of MDE which provides a practical and useful environment to the enterprises. This aspect is one of the virtues of NDT by which this methodology has been applied successfully in many real projects. In this sense, NDT defines a set of supporting tools grouped in NDT-Suite [26]. The main tools in NDT-Suite are: (i) NDT-Profile, which defines UML profiles for each NDT metamodel; (ii) NDT-Quality [27], which measures the quality of use of NDT and checks semantic constraints of NDT; and (iii) NDT-driver [28], which allows running each QVT transformations between models in an automatic manner (for instance, this tool generates test cases from the specification of FRs of a project).

From the perspective of the process of software testing, ALAMEDA uses mechanisms defined by NDT to generate test cases from functional requirements defined in the first phase of the software lifecycle (i.e., the requirement phase). ALAMEDA also proposes how functional testing can be deeply improved by means of *early testing*. This paper aims to briefly present how this mechanism of generation of test cases works [29, 30].

As mentioned above, ALAMEDA implements the concept of *early testing* which means that test cases can be generated from early stages of the software lifecycle. It does not need to define test cases when the system is developed (classic software development lifecycle). In this sense, the generation of test cases of ALAMEDA is based on a four-step process (**Figure 2**):

**Figure 2.** Test generation of ALAMEDA ecosystem.

**Figure 1** shows the vision of MDT from the perspective of MDA. As this suggests, models belonging to the PIM level can be transformed into models belonging to the PSM level, getting the code from the systems derived from PSM through another transformation step. This is the general philosophy of work proposed by MDA, but if this one is applied to software testing, some concepts change. In this case, the process of testing starts with models of System's Test which belong to the PIM level; or also named platform independent test (PIT). The quality team will elaborate on these tests to provide additional information. Next the PSM, which contains specific information about the implementation of the system, can be translated into the Platform Specific Test (PST), using the previous information from PIT and the knowledge of the quality team. Finally, these tests are morphed into codes for testing the system, ensuring

All these theoretical foundations of MDT have been systematically applied and adapted within the ALAMEDA ecosystem. For this purpose, ALAMEDA implements a methodological framework to generate case tests from functional requirements (FR) as well as managing and controlling these case tests. This framework is based in NDT, a well-contrasted methodology. The foundations of NDT and how this one has been integrated into ALAMEDA are detailed

NDT is a web methodology and is covered by the MDE paradigm. At present, NDT supports classical and agile software development lifecycles. For this purpose, NDT defines a set of metamodels for each phase of the software lifecycle (viability study, requirements, analysis, design, implementation, testing and maintenance) and it establishes a set of transformation rules (based on QVT) to generate models from other ones systematically. This implies a lower

The application of MDE-based methodologies (such as NDT) and particularly, the application of transformations among models may become monotonous and very expensive if there are no supporting tools. These tools automate the process in order to get all the potential of MDE which provides a practical and useful environment to the enterprises. This aspect is one of the virtues of NDT by which this methodology has been applied successfully in many real projects. In this sense, NDT defines a set of supporting tools grouped in NDT-Suite [26]. The main tools in NDT-Suite are: (i) NDT-Profile, which defines UML profiles for each NDT metamodel; (ii) NDT-Quality [27], which measures the quality of use of NDT and checks semantic constraints of NDT; and (iii) NDT-driver [28], which allows running each QVT transformations between models in an automatic manner (for instance, this tool generates test cases from the specifica-

From the perspective of the process of software testing, ALAMEDA uses mechanisms defined by NDT to generate test cases from functional requirements defined in the first phase of the software lifecycle (i.e., the requirement phase). ALAMEDA also proposes how functional testing can be deeply improved by means of *early testing*. This paper aims to briefly present

the quality of the test and covering all the project-specific requirements.

160 Quality Control and Assurance - An Ancient Greek Term Re-Mastered

**2.3. The NDT methodology from the perspective of software testing**

cost for the software development because of the automation of the process.

how this mechanism of generation of test cases works [29, 30].

in Section 2.3.

tion of FRs of a project).

## **3. ALAMEDA ecosystem**

The ALAMEDA ecosystem for software testing consists of:

**•** a methodological framework based on the ISO 29119 International Testing Standard and

**•** a technological support that allows carrying out of the above methodological framework and is composed of a web platform and a desktop platform. It is worth mentioning at this point that both allow the application of MDT and the automation of development tests by the use of NDT-Suite, respectively. In addition to this, they incorporate a set of tools to automate the dynamic tests and perform the management and quality of their software in an integrated way.

Concretely, the ALAMEDA ecosystem offers a platform as a service (PaaS) ready-to-use, a quick and a progressive methodological implementation and adaptation. It is also flexible and easily scalable and offers an alignment with the ISO 29119 International Testing Standard.

#### **3.1. Methodological framework**

In this section, the different actors and activities involved in the ecosystem are shown. The different cases performed by an actor are also described.

**Figure 3** shows the three main types of actors who inherit different subtypes.

**Figure 3.** Actors of ALAMEDA ecosystem.

	- **◦ AC-02. Director**: Actor who represents a registered user of director or head of the corporation type to ensure product quality software. This actor is divided into two subtypes:
		- **▪ AC-03. Test manager**. Actor who must manage a testing a project and initiating the necessary meetings to ensure its success.
		- **▪ AC-04. Technical manager**. Actor who represents the person with overall responsibility for the management or conduct of the projects concerned.

**◦ AC-05. Tester**. Actor representing the user or users responsible for testing the system. In turn, this actor can be defined as a specific type:

**•** a technological support that allows carrying out of the above methodological framework and is composed of a web platform and a desktop platform. It is worth mentioning at this point that both allow the application of MDT and the automation of development tests by the use of NDT-Suite, respectively. In addition to this, they incorporate a set of tools to automate the dynamic tests and perform the management and quality of their software in

Concretely, the ALAMEDA ecosystem offers a platform as a service (PaaS) ready-to-use, a quick and a progressive methodological implementation and adaptation. It is also flexible and easily scalable and offers an alignment with the ISO 29119 International Testing Standard.

In this section, the different actors and activities involved in the ecosystem are shown. The

**• AC-01. Registered user**: Actor who represents a team member and is also a registered user

**◦ AC-02. Director**: Actor who represents a registered user of director or head of the corporation type to ensure product quality software. This actor is divided into two

**▪ AC-03. Test manager**. Actor who must manage a testing a project and initiating the

**▪ AC-04. Technical manager**. Actor who represents the person with overall responsibil-

ity for the management or conduct of the projects concerned.

of ALAMEDA technological tools. This actor, is nested in:

necessary meetings to ensure its success.

**Figure 3** shows the three main types of actors who inherit different subtypes.

an integrated way.

**3.1. Methodological framework**

**Figure 3.** Actors of ALAMEDA ecosystem.

subtypes:

different cases performed by an actor are also described.

162 Quality Control and Assurance - An Ancient Greek Term Re-Mastered

There are 13 functionalities that define the methodological framework of ALAMEDA ecosystem; those are:

It is important to note that, to perform this step, the "FR-10. Test execution" must be performed by the testers.

important to note that, to carry out this scenario, the "FR-10. Test execution" must be accomplished.

#### **3.2. Software tools ecosystem for testing**

It is important to note that, to perform this step, the "FR-10. Test execution" must be

**• FR-06. Test plan monitoring**. In this scenario, two actors are involved: the test manager and the technical manager. The first thing that the test manager has to do is to manage different tasks for configuring the environment and the test plan and later, to set up the environment for monitoring the test plan. During the process of monitoring, several decisions may be taken. The major duties on the part of the test manager are: (i) report on the progress of the test plan, (ii) control the test plan and report control actions to the technical director. On the side of the technical manager has to: (i) access the ALAMEDA web platform for gathering

**• FR-07. Test plan creation and maintenance**. In this scenario, the actor "tester" in the role of "tester designer" or an "advanced user" is involved. The first action in this scenario is to understand the context of the project. Once understood, the functional requirements of the project must be checked with through EA and NDT-Quality tools. Then, it is necessary to generate the requirements and acceptance tests documented in the TestLink using the ALAMEDA web platform, which is based on the functional requirements defined previously. The next activities that the user must take are to identify and analyze risks and the treatment that will be applied, determine how long the test plan and its programming will

**• FR-08. Design and implement tests**. In this scenario, the actor "tester" in the role of a "tester designer" or an "advanced user" is involved. The first action of this scenario is to generate the test cases using the ALAMEDA web platform. The following actions to be performed: identify feature sets, derive test conditions, derive test coverage items, derive test cases, assemble test sets and finally, derive test procedures. It is important to note that, to perform

**• FR-09. Test environment set-up and maintenance process**. In this scenario, the actor "tester" is involved. For performing this scenario, the test plan and the specification test procedure documents generated in previous steps are necessary. The first action of this scenario is to plan and design the test environment. Then, it has to be properly configured and set up. Finally, it is adamant to prepare data that will be used for the tests. It is important to note that, to perform this step, the "FR-08. Design and implement tests" must be per-

**• FR-10. Test execution**. In this scenario, the actor "tester" is involved. Through the web platform of ALAMEDA ecosystem, the tester will: (i) carry-out the acceptance tests, (ii) check the code's quality of the project, and (iii) perform the functional and stress tests. From the web platform, the user will register the results for comparing them with older versions. It is important to note that, to perform this step, the "FR-09. Test environment set-up and

**• FR-11. Test incident reporting**. In this scenario, the actor "tester" is involved. The first action of this scenario is to analyze the results of the test execution from the web platform. If there are issues, the incident report must be created or updated (where appropriate). It is

this step, the "FR-07. Test plan creation and maintenance " must be performed.

information about control actions and progress of the test plan.

performed by the testers.

164 Quality Control and Assurance - An Ancient Greek Term Re-Mastered

be and finally, record it.

formed.

maintenance process" must be executed.

The software tools that compose the ALAMEDA ecosystem are:

**Table 1** shows how the activities of the methodological framework described before and these tools are implemented in ALAMEDA.

**Table 1.** Description of the implementation of the methodological framework.

#### **3.3. ALAMEDA from a user's perspective**

In this section, functionalities that can be made through the desktop and the web platform of the ALAMEDA ecosystem are described.

#### *3.3.1. Desktop platform*

**Activities Technological support Description of technological support**

report

requirements.

requirements.

Jenkins

JMeter)

Access to the web platform control panel module where user

Access to the web platform to inform about the test execution

Access to the web platform where the indicators related to the

Access to the Enterprise Architect and to the web platform where user will generate tests depending on the defined

Access to Testlink for editing and visualizing test plans.

Access to the Enterprise Architect and to the web platform where user will generate tests depending on the defined

Access to Testlink for editing and visualizing test plans.

Access to the web platform where the indicators related to the

state of the executions or build/s can be displayed.

Access to the web platform where user sill:

Access to the web platform where user sill: - manage the tools configuration. - manage the ISO versions

Access to the web platform where user will: - manage the configuration of ALAMEDA tools

Access to the web platform where user will: - execute load and performance tests Access to Jenkins where user will:

Access to Jenkins where user will: - configure the plugins installed in Jenkins - configure the automated tasks with Jenkins

Access to Testlink where user will:

Access to JMeter where user will: - execute load and performance tests.

will check the status of SonarQuBe executions

state of the executions or build/s can be displayed

Web platform of ALAMEDA ecosystem

166 Quality Control and Assurance - An Ancient Greek Term Re-Mastered

Web platform of ALAMEDA ecosystem

Web platform of ALAMEDA ecosystem - Tools to automate dynamic testing and quality Software

**Table 1.** Description of the implementation of the methodological framework.

FR-04. Software quality management

FR-06. Test plan monitoring

FR-07. Test plan creation and maintenance

FR-08. Design and Implement Tests

FR-09. Test Environment Set-Up & Maintenance Process

FR-10. Test execution

FR-11. Test incident reporting

FR-12. Users and services management

FR-13. Tools set-up management

FR-05. Test execution report

> The desktop platform of ALAMEDA ecosystem consists of an assistant for the creation of functional requirements as a preliminary step to the process of generating automated acceptance tests associated with them.

> It is presented as a software layer above deployed EA assisting in the construction of very specific components: actors and system functional requirements. These components are included in EA by previous installation of NDT-Suite and correspond to the basic components to build and validate when creating a specification's requirements model.

The specification's requirements model must follow the structure shown in **Figure 4**.

**Figure 4.** Specification's requirements model.

The construction of the models of functional requirements is based on two operations:

Following the previous structure, steps that are necessary to perform are:

**•** System actor creation/modification. The elements that represent the actors of the system are stereotyped as "AC" and can be found in the EA toolbox provided by NDT-Suite called <NDT System Requirements> DRS Actors. There are two ways of creating these elements:

	- **◦** Extends or include: only applied between RF elements.
	- **◦** Use case: only applied between AC and RF elements.

#### *3.3.2. Web platform*

The web platform of the ALAMEDA ecosystem brings together the bulk of the functionality of ALAMEDA. In this section, the applicability of each module of this platform is addressed.

	- **◦** New user: it allows registering new users.
	- **◦** Edit user: it allows to edit both basic information about users and performs configuration management tools that are assigned to users. Removing users is also allowed.
	- **◦** New self-assessment ISO/IEC-29119: it allows the installation of a new version of the ISO/ IEC-29119 standard. A new version carries a glossary of terms associated with that role and an associated self-assessment questionnaire.
	- **◦** ISO/IEC 29119 management: it allows the management of the versions of the ISO/ IEC-29119 standard loaded in the system, which includes removing and updating previous versions.
	- **◦** Glossary of terms. The glossary of terms shows the collection of terms that defines the norm in the first part for standardizing the terminology used in the field of Testing. In the glossary of terms, each term is shown along with the associated definition.
	- **◦** Self-assessment. The self-assessment consists of a series of questions about compliance with the rule that the user must answer based on what is known in reference to his/her

organization. The format shown in the questions are: "Yes", "No" or "Partly", so that each answer carries a score. The questionnaire is broken down into the parts of the standard and within each part, every field and block is referred to the corresponding question. There is an option for storing the state of the questionnaire and complete it later.

**◦** Drag the item from the toolbox to the corresponding diagram.

**◦** Extends or include: only applied between RF elements.

**◦** Use case: only applied between AC and RF elements.

**◦** New user: it allows registering new users.

and an associated self-assessment questionnaire.

to access to the other sections of the module. These sections are:

option and selecting the actor item.

168 Quality Control and Assurance - An Ancient Greek Term Re-Mastered

*3.3.2. Web platform*

previous versions.

**◦** Select the directory and include the item making right click, choosing the "Add Element"

**•** Functional requirements creation/modification. The elements that represent the functional requirements of the system are stereotyped as "FR" and can be found in EA toolbox provided by NDT-Suite called <NDT System Requirements> DRS Functional Requirements. FRs are the most complex elements of the model; they do not just present a definition of a particular value for a specific property, but rather have collections of properties (pre- and post-conditions) and may be specified based on scenarios (sequence of steps) or activity diagrams. The way the RFs are created are in the same way with the actors one. There are different types of relations between the RF and depend on the side of the connection.

The web platform of the ALAMEDA ecosystem brings together the bulk of the functionality of ALAMEDA. In this section, the applicability of each module of this platform is addressed. **•** Platform management: for access to the management module it is necessary to access as a manager. The administration panel of the platform (**Figure 5**) displays the following options:

**◦** Edit user: it allows to edit both basic information about users and performs configuration

**◦** New self-assessment ISO/IEC-29119: it allows the installation of a new version of the ISO/ IEC-29119 standard. A new version carries a glossary of terms associated with that role

**◦** ISO/IEC 29119 management: it allows the management of the versions of the ISO/ IEC-29119 standard loaded in the system, which includes removing and updating

**•** Compliance with ISO/IEC-29119: the module in accordance with ISO/IEC-29119 standard (**Figure 6**) has four sections; it depends on an existing version of the standard installed on the platform. Therefore, if there is no version installed on the platform, it will not be possible

**◦** Glossary of terms. The glossary of terms shows the collection of terms that defines the norm in the first part for standardizing the terminology used in the field of Testing. In the

**◦** Self-assessment. The self-assessment consists of a series of questions about compliance with the rule that the user must answer based on what is known in reference to his/her

glossary of terms, each term is shown along with the associated definition.

management tools that are assigned to users. Removing users is also allowed.

	- **◦** Tests generation. The test generation process receives as input a specification requirements model in EA format and it returns a test plan generated in the TestLink tool with a build that allows its execution in this tool. The build process begins with a validation phase input model. If errors are found in the model definition, the test generation is not addressed. Instead, a list with the errors found during the process is provided.
	- **◦** Generated tests display. The result of the test generation process, regardless of whether they have been generated with errors or not during the validation phase, are displayed in the result viewer.
	- **◦** Tasks generation. The process consists of a scanning of the test cases executed in TestLink to generate tasks associated to the Redmine management tool.
	- **◦** Tasks monitoring and visualization. The result viewer is similar to the one shown in the test generation module with the difference that the tool deployed is Redmine. The available project tree corresponds to the generated tests. From the generated tests, the viewer can carry out the task monitoring.
	- **◦** Static code analysis. The tool used to perform static code analysis using the web platform is SonaQube. The access form to the execution of static code analysis requests two inputs to perform the test: the project to analyze and its version. The use of compressed projects files as rar or zip format is allowed.

**Figure 5.** Management platform.

**Figure 6.** Compliance with ISO/IEC-29119.

**◦** measure the quality of the code source of software products through static tests (Sonar-

**◦** execute acceptance tests previously generated in the test execution module (Testlink).

**•** Furthermore, as in the test generation module, a result viewer to monitor and track all the

**◦** Static code analysis. The tool used to perform static code analysis using the web platform is SonaQube. The access form to the execution of static code analysis requests two inputs to perform the test: the project to analyze and its version. The use of compressed projects

**•** Control panel. This module allows the monitoring of the elements generated in all the

**◦** perform dynamic tests in order to monitor the web nature software product measurement

Qube tool).

(JMeter).

conducted activities is provided.

files as rar or zip format is allowed.

170 Quality Control and Assurance - An Ancient Greek Term Re-Mastered

different modules of the web platform.

**Figure 5.** Management platform.

**Figure 7.** Test generation.

**Figure 8.** Software and testing analysis.

## **4. Conclusions and future work**

This chapter presents the ALAMEDA ecosystem for software testing. Over the last decade, a number of studies, as well as technical and theoretical proposals have emerged to improve the process of software testing. However, the software industry has often carried out this process in a manner that was not standardized and hence, proved difficult to imply to systematization, management and continuous improvement of this process.

The ALAMEDA platform has been developed so as to cover the observed gap in the industry. In this sense, ALAMEDA provides a methodological framework based on the ISO/IEC 29119 standard and the application of the MDE paradigm of software testing (known as MDT).

Regarding the ISO/IEC 29119, this one is a new standard which implies a certain novelty of the software industry. ALAMEDA integrates the main foundations of this standard within its ecosystem in order to facilitate the adoption of this standard by industry. In addition, ALA-MEDA provides mechanisms to measure the degree of compliance to the ISO/IEC 29119.Thus, any software company can measure its evolution to meet this standard and carry out a process of continuous improvement in its software development processes.

Regarding the implementation of MDT, ALAMEDA has included the concept of early testing, thanks to it for integrating the NDT methodology within its ecosystem. Thus, it is possible to obtain test cases from the functional requirements and manage these tests effectively. Moreover, this mechanism provides traceability between tests and functional requirements allowing always to know which functionality is being tested.

Finally, and taking into account the features of ALAMEDA, it has not been possible to find a technological solution to cater for all needs associated with the process of software testing from the early stages and complying with the international standard ISO/IEC 29119. In this sense, ALAMEDA improves the state-of-the-art within the management of software quality assurance because it provides an innovative technological solution which is based on well-validated theoretical foundations.

## **Acknowledgements**

This research has been supported by the MeGUS project (TIN2013-46928-C3-3-R), by the SoftPLM Network (TIN2015-71938-REDT) of the Spanish Ministry of Economy and Competitiveness and Fujitsu Laboratories of Europe (FLE). In addition, we would like to acknowledge the leadership of Servinform during the development of the ALAMEDA ecosystem. Our research group has collaborated in the conceptual design of ALAMEDA as well as in analysis, design, and testing tasks of the platform.

## **Author details**

**Figure 8.** Software and testing analysis.

**4. Conclusions and future work**

172 Quality Control and Assurance - An Ancient Greek Term Re-Mastered

management and continuous improvement of this process.

of continuous improvement in its software development processes.

always to know which functionality is being tested.

theoretical foundations.

This chapter presents the ALAMEDA ecosystem for software testing. Over the last decade, a number of studies, as well as technical and theoretical proposals have emerged to improve the process of software testing. However, the software industry has often carried out this process in a manner that was not standardized and hence, proved difficult to imply to systematization,

The ALAMEDA platform has been developed so as to cover the observed gap in the industry. In this sense, ALAMEDA provides a methodological framework based on the ISO/IEC 29119 standard and the application of the MDE paradigm of software testing (known as MDT).

Regarding the ISO/IEC 29119, this one is a new standard which implies a certain novelty of the software industry. ALAMEDA integrates the main foundations of this standard within its ecosystem in order to facilitate the adoption of this standard by industry. In addition, ALA-MEDA provides mechanisms to measure the degree of compliance to the ISO/IEC 29119.Thus, any software company can measure its evolution to meet this standard and carry out a process

Regarding the implementation of MDT, ALAMEDA has included the concept of early testing, thanks to it for integrating the NDT methodology within its ecosystem. Thus, it is possible to obtain test cases from the functional requirements and manage these tests effectively. Moreover, this mechanism provides traceability between tests and functional requirements allowing

Finally, and taking into account the features of ALAMEDA, it has not been possible to find a technological solution to cater for all needs associated with the process of software testing from the early stages and complying with the international standard ISO/IEC 29119. In this sense, ALAMEDA improves the state-of-the-art within the management of software quality assurance because it provides an innovative technological solution which is based on well-validated José González Enríquez\* , Julián Alberto García-García, Francisco José Domínguez-Mayo and María José Escalona Cuaresma

\*Address all correspondence to<EMAIL_ADDRESS>
Computer Languages and Systems Department, University of Seville, Seville, Spain

## **References**

[24] Dai ZR. (2004). Model-Driven Testing with UML 2.0. In Proceedings of the 2nd European Workshop on Model Driven Architecture, Computing Laboratory, University of Kent, Canterbury, UK, 2004, pp. 179–187

[9] Miller J & Mukerji J. (2003). Model Driven Architecture 1.0.1 Guide. Object Management

[10] Escalona MJ & Aragón G. (2008). NDT. A Model-Driven Approach for Web Require-

[11] García-García JA, Ortega MA, García-Borgoñon L & Escalona MJ. (2012). NDT-Suite: A Model-Based Suite for the Application of NDT. In Web Engineering. Springer, Berlin

[12] García-García JA, Escalona MJ, Martinez-Garcia A, Parra C & Wojdyeski T. (2015). Clinical Process Management: A Model-Driven & Tool-Based Proposal. In Information Systems Development: Transforming Healthcare through Information Systems.

[13] Tassey G. (2002). The Economic Impacts of Inadequate Infrastructure for Software Testing. National Institute of Standards and Technology, RTI Project, 7007(011).

[14] López AS, Valle DC, Escalona MJ, Lee V & Goto M. (2015). Patient Lifecycle Management: An Approach for Clinical Processes. In Bioinformatics and Biomedical Engineer-

[15] Kent S. (2002, May). Model Driven Engineering. In International Conference on

[16] ISO: ISO/IEC/IEEE 29119 Software Testing. http://www.softwaretestingstandard.org/

[17] Fondemenet F & Silaghi R. (2004). Defining Model Driven Engineering Process. 3rd Workshop in Software Model Engineering (WISME2004). October 11–15, Lisbon,

[18] OMG. (2005). UML2.0, Unified Modelling Language. Object Management Group,

[19] OMG. (2008). Documents Associated with Meta Object Facility (MOF) 2.0 Query/View/ Transformation. Object Management Group. http://www.omg.org/spec/QVT/1.0/.

[20] Thiry L & Thirion B. (2009). Functional Metamodels for Systems and Software. Journal of Systems and Software. 82(7):1125–1136. DOI: http://dx.doi.org/10.1016/j.jss.

[21] Whittle J, Hutchinson J & Rouncefield M. (2014). The State of Practice in Model-Driven

[22] OMG. (2003). MDA Guide of OMG. Version 1.0.1, http://www.omg.org/docs/omg/

[23] Object Management Group (OMG). http://utp.omg.org/. Last accessed: July, 2016

Integrated Formal Methods. Springer, Berlin Heidelberg. pp. 286–298.

ments. IEEE Transactions on Software Engineering. 34:377–394.

ing. Springer International Publishing. pp. 694–700.

Group, Inc.

Heidelberg. pp. 469–472.

174 Quality Control and Assurance - An Ancient Greek Term Re-Mastered

(2013). Accessed 10 May 2016.

Portugal. 2004.

2009.01.042. 2009.

03-06-01.pdf

Engineering. IEEE Software. 31(3):79–85.

2008.

formal/2005-07-05, 2005.

## **Improving Quality Assurance in Multidisciplinary Engineering Environments with Semantic Technologies Improving Quality Assurance in Multidisciplinary Engineering Environments with Semantic Technologies**

Dietmar Winkler, Marta Sabou and Stefan Biffl Dietmar Winkler, Marta Sabou and Stefan Biffl

Additional information is available at the end of the chapter Additional information is available at the end of the chapter

http://dx.doi.org/10.5772/66222

#### **Abstract**

In multidisciplinary engineering (MDE) projects, for example, automation systems or manufacturing systems, stakeholders from various disciplines, for example, electrics, mechanics and software, have to collaborate. In industry practice, engineers apply individual and highly specialized tools with strong limitation regarding defect detection in early engineering phases. Experts typically execute reviews with limited tool support which make engineering projects defective and risky. Semantic Web Technologies (SWTs) can help to bridge the gap between heterogeneous sources as foundation for efficient and effective defect detection. Main questions focus on (a) how to bridge gaps between loosely coupled tools and incompatible data models and (b) how SWTs can help to support efficient and effective defect detection in context of engineering process improvement. This chapter describes success-critical requirements for defect detection in MDE and shows how SWTs can provide the foundation for early and efficient defect detection with an adapted review approach. The proposed defect detection framework (DDF) suggests different levels of SWT contributions as a roadmap for engineering process improvement. Two selected industry-related real-life cases show different levels of SWT involvement. Although SWTs have been successfully applied in real-life use cases, SWT applications can be risky if applied without good understanding of success factors and limitations.

**Keywords:** Semantic Web Technologies, automation systems engineering, multidisciplinary engineering, quality assurance, defect detection, engineering process improvement

© 2017 The Author(s). Licensee InTech. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. © 2017 The Author(s). Licensee InTech. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

## **1. Introduction**

In multidisciplinary engineering (MDE) environments, engineers coming from different disciplines have to collaborate and exchange data. For instance, in automation systems engineering (ASE) projects, mechanical, electrical, and software engineering disciplines work together to build manufacturing plants, power plants, or steel mills [1]. However, engineers typically use isolated tools with data models that provide only limited capabilities regarding data exchange and synchronization of engineering plans across disciplines and organizations [2]. Such loosely coupled tools and incompatible data models hinder efficient engineering processes, project management, quality assurance, and engineering process improvement [3]. Due to limited collaboration capabilities, engineering projects become risky and error-prone.

However, the collaboration of different engineering disciplines requires efficient mechanisms for quality assurance and defect detection [4]. In the context of this paper, defects represent errors in engineering plans and design documents, deviations and inconsistencies between engineering plans. Established tool suites, that is, all-in-one solutions, such as EPlan Engineering Configuration (EEC)1 or COMOS2 , with integrated data models typically include basic quality assurance mechanisms, such as syntax checks or basic consistency checks to identify intra-disciplinary defects (i.e., defects within one engineering artifact or between artifacts within one discipline). It is worth noting that there is no strong support for identifying interdisciplinary defects, that is, defects that affect two or more disciplines. Thus, in tool networks, where no integrated data models are available, further research is required to enable effective and efficient defect detection. Performed research revealed that there is very limited support for defect detection in heterogeneous environments. The latter include but are not limited to identifying inconsistencies between variables of mechanical and software (control) engineering artifacts because of the technical heterogeneity of tools and the semantic heterogeneity of data models [5].

The application of reviews [6] (or software inspection) in software engineering supports early defect detection [6], typically executed by human experts in a paper-based way, that is, without or with limited tool support. Thus, defect detection requires a high effort and includes risks to oversee important defects, even in homogenous engineering artifacts such as design documents or software code. In distributed and heterogeneous engineering environments, reviews require additional knowledge of human experts, who are sufficiently familiar with at least two related disciplines and require high cognitive skills. Again, review tasks are often executed manually by these experts and require as such considerable effort. It is worth mentioning that preliminary research indicates that they often miss important defects [7], especially if more than one engineering discipline is involved. Thus, an important issue is the heterogeneity of data models, embodied within applied tools.

Semantic Web Technologies (SWTs) can provide concepts for closing the gap between data models for data management, that is, the identification of data entities and knowledge, data

<sup>1</sup> EPlan: www.eplan.de/.

<sup>2</sup> COMOS: w3.siemens.com/mcms/plant-engineering-software/de/Seiten/Default.aspx.

organization, storage, and querying for project analysis purposes [8]. Thus, the application of SWTs can provide the foundation for effective and efficient defect detection mechanisms. However, even without using SWTs, systematic defect detection approaches, such as (software) reviews or inspection [9], can support defect detection processes, by applying guidelines and reading techniques for defect detection support. Thus, the main research challenges focus on (a) providing expert support for identifying defects in an interdisciplinary context more effectively and efficiently and (b) improving defect detection in organizations with or without SWTs. A defect detection framework (DDF) aims at providing an approach to provide the degree of SWT contributions from solution approaches without SWT (lowest level) to a fully supported SWT solution (highest level). For every level, this chapter describes a prototype solution of a real-life use case, developed together with industry and research experts for evaluation purposes.

Thus, the main goals of this chapter focus on (a) providing a defect detection framework (DDF) to support the introduction of SWTs in an organization on various levels of granularity and (b) to provide lessons learned from industry prototypes in context of SWT applications for defect detection in industrial and research real-life use cases. Further, the chapter demonstrates different levels of SWT capabilities in industry use cases and report on lessons learned on the strengths and limitations of these approaches.

The remainder of this paper is structured as follows: Section 2 presents related work on multidisciplinary engineering (MDE), defect detection, and Semantic Web Technologies. Section 3 motivates research issues, and Section 4 presents the solution approach, that is, requirements and capabilities for defect detection in context of MDE and SWT and the defect detection framework with and without SWTs. Section 5 discusses industry (real-life) use cases and prototype implementations regarding defect detection capabilities, and Section 6 discusses qualitative assessment results and lessons learned on the limitations of individual levels of semantic integration.

## **2. Related work**

**1. Introduction**

178 Quality Control and Assurance - An Ancient Greek Term Re-Mastered

neering Configuration (EEC)1

geneity of data models [5].

1

2

EPlan: www.eplan.de/.

data models, embodied within applied tools.

COMOS: w3.siemens.com/mcms/plant-engineering-software/de/Seiten/Default.aspx.

In multidisciplinary engineering (MDE) environments, engineers coming from different disciplines have to collaborate and exchange data. For instance, in automation systems engineering (ASE) projects, mechanical, electrical, and software engineering disciplines work together to build manufacturing plants, power plants, or steel mills [1]. However, engineers typically use isolated tools with data models that provide only limited capabilities regarding data exchange and synchronization of engineering plans across disciplines and organizations [2]. Such loosely coupled tools and incompatible data models hinder efficient engineering processes, project management, quality assurance, and engineering process improvement [3]. Due to limited collaboration capabilities, engineering projects become risky and error-prone. However, the collaboration of different engineering disciplines requires efficient mechanisms for quality assurance and defect detection [4]. In the context of this paper, defects represent errors in engineering plans and design documents, deviations and inconsistencies between engineering plans. Established tool suites, that is, all-in-one solutions, such as EPlan Engi-

quality assurance mechanisms, such as syntax checks or basic consistency checks to identify intra-disciplinary defects (i.e., defects within one engineering artifact or between artifacts within one discipline). It is worth noting that there is no strong support for identifying interdisciplinary defects, that is, defects that affect two or more disciplines. Thus, in tool networks, where no integrated data models are available, further research is required to enable effective and efficient defect detection. Performed research revealed that there is very limited support for defect detection in heterogeneous environments. The latter include but are not limited to identifying inconsistencies between variables of mechanical and software (control) engineering artifacts because of the technical heterogeneity of tools and the semantic hetero-

The application of reviews [6] (or software inspection) in software engineering supports early defect detection [6], typically executed by human experts in a paper-based way, that is, without or with limited tool support. Thus, defect detection requires a high effort and includes risks to oversee important defects, even in homogenous engineering artifacts such as design documents or software code. In distributed and heterogeneous engineering environments, reviews require additional knowledge of human experts, who are sufficiently familiar with at least two related disciplines and require high cognitive skills. Again, review tasks are often executed manually by these experts and require as such considerable effort. It is worth mentioning that preliminary research indicates that they often miss important defects [7], especially if more than one engineering discipline is involved. Thus, an important issue is the heterogeneity of

Semantic Web Technologies (SWTs) can provide concepts for closing the gap between data models for data management, that is, the identification of data entities and knowledge, data

, with integrated data models typically include basic

or COMOS2

This section summarizes related work on multidisciplinary engineering (MDE) environments, defect detection, and Semantic Web Technologies as foundation for defect detection tool prototypes in context of the defect detection framework (DDF) with and without SWTs.

#### **2.1. Multidisciplinary engineering environments and automation systems engineering**

In *multidisciplinary engineering (MDE)* environments for *automation systems engineering (ASE)*, various stakeholders coming from different disciplines have to collaborate along the automation systems life cycle [1, 10]. Typical examples of such MDE environments for ASE include hydro power plant engineering, steel mill engineering, or industrial production systems. Common to all application areas is the contribution of heterogeneous engineering disciplines, that is, mechanical, electrical, and software engineering, with specialized engineering tools and related specific and heterogeneous data models. However, these expert tools typically have limitations regarding seamless data exchange and collaboration support of engineers [1, 2, 11], a challenge for effective and efficient defect detection. Defects are typically considered as deviations between engineering plans of heterogeneous disciplines, for example, a mismatch of variable definitions and their usage, missing components, or wrong components. Yet, these deviations could be real defects or required and intended changes, requested by an engineer of one specific discipline. For example, the exchange of a hardware sensor from analogue to a digital (required by the electrical engineer) might result in different numbers of pins that need to be addressed by the software engineer in the control software. This change needs to be identified and analyzed early in the engineering process. Changes that arise late in the engineering project, during the commissioning phase, typically require significantly higher efforts for rework [12], during the commissioning phase.

**Figure 1** presents a typical sequential engineering process approach in the ASE domain with parallel engineering activities along the project course. Furthermore, selected and important engineering artifacts of leading engineering disciplines and isolated and distributed quality assurance activities for individual process steps are included in this sample process. In common industry projects, engineers often follow such a simplified sequential engineering process [1, 4], that is, system design, implementation, test and commissioning, and operation (**Figure 1**). Changes and defects can have a critical impact on products, engineering projects, and processes, as they are likely to propagate from early to later stages. Therefore, a need arises for early support of defect detection mechanisms in MDE for ASE projects.

**Figure 1.** Typical sequential engineering process approach with parallel engineering activities [13].

Synchronization and data exchange in an engineering tool chain are typically executed manually or semi-automatically by applying local supporting tools (e.g., based on local database solutions or spread sheet solutions) [13]. However, these approaches incur high maintenance effort and assume the availability of experts, who are responsible for operating and maintaining these tools (often as add-on activities to their primary work tasks). Thus, there is a need for mechanisms that provide capabilities for mapping heterogeneous data models and for supporting defect detection processes across tools, domains, and data models. To overcome semantically heterogeneous data models, a common data exchange format could bridge the semantic gaps between engineering disciplines. The data format needs to be defined by consideration of all related tools; data to be exchanged need to be mapped to pass information and changes from one discipline (and data model) to the other. To overcome individual effort for data format definition and mapping, a standardized data exchange format is reasonable. *AutomationML* [14–16] is such an emerging data exchange standard that supports efficient synchronization of data produced by different engineering disciplines. However, *AutomationML* represents a data exchange standard and needs additional components that support data exchange, synchronization, and defect detection. The *AML.hub*<sup>3</sup> [17] is a platform for providing efficient data exchange/synchronization based on *AutomationML*. Although the *AML.hub* provides mechanisms for data exchange (e.g., mapping and merging), there is currently no support for quality assurance and defect detection.

#### **2.2. Defect detection in software and systems engineering**