11institutetext: Musengamana Jean de Dieu 22institutetext: Peng Liang (✉)
33institutetext: School of Computer Science, Wuhan University, Wuhan, China
33email<EMAIL_ADDRESS><EMAIL_ADDRESS>44institutetext: Mojtaba Shahin
55institutetext: School of Computing Technologies, RMIT University, Melbourne,
Australia
55email<EMAIL_ADDRESS>66institutetext: Chen Yang
77institutetext: School of Artificial Intelligence, Shenzhen Polytechnic,
Shenzhen, China
State Key Laboratory for Novel Software Technology, Nanjing University,
Nanjing, China
77email<EMAIL_ADDRESS>88institutetext: Zengyang Li 99institutetext:
School of Computer Science & Hubei Provincial Key Laboratory of Artificial
Intelligence and Smart Learning, Central China Normal University, Wuhan, China
99email<EMAIL_ADDRESS>
# Mining Architectural Information: A Systematic Mapping Study

Musengamana Jean de Dieu  Peng Liang  Mojtaba Shahin  Chen Yang  Zengyang Li

(Received: date / Accepted: date)

###### Abstract

Mining Software Repositories (MSR) has become an essential activity in
software development. Mining architectural information (e.g., architectural
models and views) to support architecting activities, such as architecture
recovery and understanding, has received significant attention in recent
years. However, there is a lack of clarity on what literature on mining
architectural information is available. Consequently, this may create
difficulty for practitioners to understand and adopt the state-of-the-art
research results, such as what approaches should be adopted to mine what
architectural information in order to support architecting activities. It also
hinders researchers from being aware of the challenges and remedies for the
identified research gaps. We aim to identify, analyze, and synthesize the
literature on mining architectural information in software repositories in
terms of architectural information and sources mined, architecting activities
supported, approaches and tools used, and challenges faced. A Systematic
Mapping Study (SMS) has been conducted on the literature published between
January 2006 and December 2022. Of the 87 primary studies finally selected, 8
categories of architectural information have been mined, among which
architectural description is the most mined architectural information; 12
architecting activities can be supported by the mined architectural
information, among which architecture understanding is the most supported
activity; 89 approaches and 54 tools were proposed and employed in mining
architectural information; and 4 types of challenges in mining architectural
information were identified. This SMS provides researchers with promising
future directions and help practitioners be aware of what approaches and tools
can be used to mine what architectural information from what sources to
support various architecting activities.

###### Keywords:

Mining Architectural Information Software Repositories Architecting Activity
Software Development Systematic Mapping Study

##  1 Introduction

Software repositories contain a wealth of valuable information (e.g., source
code, bug reports) related to software development. This information can
assist software engineers (e.g., architects and developers) to comprehend
various aspects of a software system. Architectural information is one of the
most important types of information in software development, and it is not
only used in the early stages (e.g., architecture design) of the development,
but also in the later stages of software development life cycle, like
maintenance and evolution (Li et al., 2013). Architectural information, such
as benefits and drawbacks of certain architectural solutions (e.g., patterns
and tactics) in specific application domains, can help architects and related
stakeholders to conduct architecture evaluation on candidate architectural
solutions during the architecting process (Hofmeister et al., 2007). However,
architectural information is scattered in various sources of software
repositories (e.g., Q&A sites (Soliman et al., 2016), technical blogs and
tutorials (Soliman et al., 2021c), issue tracking systems (Soliman et al.,
2021a), developer mailing lists (Ding et al., 2015b), and chat messages
(Borrego et al., 2019)), and is often described in a combination of textual
and graphical representation (Malavolta et al., 2013). This constitutes a
large volume of architectural information that is sometimes tacit and not
documented at all in those sources (Ding et al., 2014). Therefore, the
intricate information of a system, especially in a large and complex system
easily evaporates if architectural information is non-documented. The
consequences would be incurring design and implementation issues (Capilla et
al., 2016). With the increase in the complexity and size of systems,
architectural information management becomes even more challenging, and
software engineers need to find the right information and recover it
efficiently. However, searching and finding or recovering relevant
architectural information from artifacts (e.g., code, requirements document)
is a challenging task for software engineers (Jansen et al., 2009). Therefore,
approaches and tools for searching and mining architectural information from
development artifacts are much needed to assist the development process (e.g.,
architecting process).

The Mining Software Repositories (MSR) field analyzes the rich data available
in software repositories to uncover interesting and actionable information
about software systems (Hassan, 2008). Mining software repositories for
Software Architecture (SA) has been the subject of the software architecture
research community in recent years (Soliman et al., 2021b). SA researchers and
practitioners have developed approaches and tools to ease the mining of
architectural information from various sources of software repositories in
order to support architecting activities (Hofmeister et al., 2007; Li et al.,
2013). Some of the supported architecting activities include architecture
analysis (Velasco-Elizondo et al., 2016), architecture understanding (do
Nascimento Vale and Maia, 2015), and architecture recovery (Shahbazian et al.,
2018). Due to the increasing importance of mining architectural information, a
growing number of researchers have explored different aspects of this topic.
Despite this, there is a lack of deep understanding of current literature on
mining architectural information, which hinders researchers from being aware
of the potential challenges and remedies for the identified research gaps. It
also creates difficulty for practitioners to understand and adopt the state-
of-the-art research results, such as what approaches should be adopted to mine
what architectural information in order to support architecting process. Thus,
this motivated us to carry out a literature review of the reported research on
mining architectural information from software repositories to support
architecting activities.

There are several methods for conducting literature reviews, such as SMS and
Systematic Literature Review (SLR). Both SMS and SLR are typically utilized to
survey the literature on a specific topic. An SLR focuses on investigating,
evaluating, and interpreting the available studies related to specific
research questions towards a topic or phenomenon (Kitchenham et al., 2007),
while an SMS provides an overview of a research area to systematically
identify and evaluate the evidence in literature. One of the main differences
between an SMS and an SLR is that an SMS aims to discover the research trends
and covers a broad topic in the literature, while an SLR usually has a
relatively narrow and in-depth research scope and focuses on specific research
questions. The architectural information has an impact on different levels of
software systems and affects various aspects in software development (e.g.,
development activities). Hence, to establish an overview of this topic area in
the context of software development, we decided to conduct an SMS rather than
an SLR on the studied topic (i.e., mining architectural information in
software development). More specifically, we conducted an SMS (Petersen et
al., 2015) to investigate the architectural information and their sources
mined, the approaches and tools used, the architecting activities supported,
and the challenges and issues faced. This paper make the following
contributions:

(1) We presented a comprehensive analysis of 87 primary studies published
between 2006 and 2022 on mining architectural information in software
development.

(2) We provided a classification of mined architectural information to support
the development.

(3) We summarized the sources used in mining architectural information.

(4) We investigated the architecting activities that can be supported by the
mined architectural information.

(5) We collected the approaches and tools that have been used in mining
architectural information.

6) We discuss distinct challenges on mining architectural information and
outline key future directions for research on mining architectural information
in software development.

The remainder of this paper is organized as follows: Section 2 introduces the
context of this SMS. Section 3 elaborates on the design and research questions
of this SMS. Section 4 presents the results of each research question. Section
5 discusses the results of the research questions and their implications for
researchers and practitioners. Section 6 examines the threats to validity.
Section 7 presents the related work. Finally, Section 8 concludes this SMS
with potential areas of future research.

##  2 Research Context

To clarify the scope of this SMS, two fundamental concepts need to be
explained, “mining software repositories” and “architectural information”.

###  2.1 Mining Software Repositories

The studies on mining software repositories collect and analyze the rich data
available in software repositories to uncover interesting and actionable
information about software projects (Hassan, 2008). VCS (e.g., GitHub), Q&A
sites (e.g., Stack Overflow), and issue tracking systems (e.g., Jira), among
others, are examples of sources of software repositories that are commonly
used for mining development related information. These sources are useful for
both software practitioners and researchers. For example, practitioners (e.g.,
architects) can share and learn architectural tactics and quality attributes
knowledge from their peers in Stack Overflow (Bi et al., 2021b). Researchers
can benefit from the available data (e.g., source code, commit messages) in
software repositories to conduct their research on software development, such
as code clone detection (Nafi et al., 2019) and dependency analysis (Bavota et
al., 2014). To achieve this, researchers are required to select software
repositories and data sources that fit their research needs, extract data from
these repositories, and analyze the data to obtain evidence for answering
their research questions. In this SMS, we gather the literature on mining
architectural information from software repositories to support the
development. Specifically, we search the studies that develop approaches and
tools for mining architectural information, aiming to support architecting
activities (e.g., mining source code to support architecture maintenance and
evolution (Kazman et al., 2015)).

###  2.2 Architectural Information

Architectural information states a high-level abstraction of software systems,
and it is one of the most important types of information in software
development (Bass et al., 2012). There are various types of architectural
information, such as architectural solutions and architectural technical debt,
that are intensively explored during the development. In this section, we
introduce the concept of architectural information through these two concrete
types of architectural information.

Architectural solutions are the fundamental building blocks in modern software
design and they are used to address architecture concerns. Architectural
patterns (e.g., Model–View–Controller), tactics (e.g., resource pooling), and
frameworks (e.g., Windows Presentation Foundation (WPF)) are typical
architectural solutions. On the other hand, Architectural technical debt is
incurred by architecture decisions that unconsciously or consciously
compromise system-wide quality attributes, particularly maintainability and
evolvability (Li et al., 2014). Typical architectural technical debt includes
violations of architecture design principles or rules (Li et al., 2015).
Architectural technical debt needs to be identified and removed since it is
harmful to the system’s long-term health (Li et al., 2015).

Architectural information is often described in different formats, such as
textual and graphical representation (Malavolta et al., 2013), and this
information is recorded in various artifacts, such as code (Granchelli et al.,
2017), architecture documents (Ding et al., 2014), that are contained in
diverse sources of software repositories. However, it is very challenging for
architects and developers to mine and reuse architectural information that is
contained in those artifacts due to, for instance, unstructured representation
of this information in texts or graphs. As mentioned in Section 1, mining
repositories for SA has been the subject of the architecture research
community in recent years (Soliman et al., 2021b). Researchers and
practitioners have developed many approaches and tools to facilitate the
mining of architectural information from various artifacts that are contained
in software repositories in order to support the development.

##  3 Mapping Study Design

###  3.1 Goal and Research Questions

The goal of this SMS is to analyze the primary studies on mining architectural
information for the purpose of understanding with respect to the sources of
architectural information, types of mined architectural information,
architecting activities supported, approaches and tools employed, and
challenges from the point of view of researchers and practitioners in the
context of software development. In order to get a comprehensive overview of
mining architectural information in software development, we further
decomposed the goal of this SMS into five Research Questions (RQs) as listed
in Table 1.

Table 1: Research questions and their rationale

Research Question |  Rationale
---|---
RQ1. What architectural information is mined in software development? |  SA researchers have mined different types of architectural information, such as architectural description (Chaabane et al., 2017) and system requirements (e.g., quality attributes (Bi et al., 2021b)). This RQ intends to explore the types of mined architectural information to support software development. The answer of this RQ can (i) help practitioners know and utilize the types of architectural information that have been mined to support the development and (ii) help researchers be aware of the types of architectural information that have and have not been mined so that they can explore how to use the mined architectural information and/or mine the architectural information that is not yet mined.
RQ2. What sources are used to mine architectural information? |  Architectural information is scattered in various sources of software repositories, such as Q&A sites (e.g., Stack Overflow) (Soliman et al., 2016), issue tracking systems (e.g., Jira) (Bhat et al., 2017), and technical blogs and tutorials (Soliman et al., 2021c). Thus, there is no single source of architectural information that contains all the required architectural information. The selection of a source is determined by the type of information to be mined as well as the purpose of mining architectural information. For example, information related to architectural issues or debt may require selecting issue tracking systems, such as Bugzilla or Jira repositories. Therefore, researchers have mined various sources for the sake of certain architectural information to support the development. The answer of this RQ provides insights into specific sources researchers utilize when mining architectural information, and such insights can (i) motivate other researchers to further investigate architectural information in those sources and (ii) help practitioners know where they can search architectural information to address their architectural design concerns.
RQ3. What architecting activities can be supported by the mined architectural information? |  There are various architecting activities (e.g., architecture analysis, architecture synthesis, architecture evaluation (Hofmeister et al., 2007)) that are performed during the architecture life cycle. Each architecting activity may require specific architectural information to be conducted effectively. The answer of this RQ can help practitioners know which architecting activities can be supported by the mined architectural information. For example, during architecture evaluation, architectural information, such as the benefits and drawbacks of certain architectural solutions (e.g., architectural patterns and tactics) can help architects evaluate the architectural solutions (e.g., choosing suitable architectural patterns according to the positively and negatively affected quality attributes) in specific application domains.
RQ4. What approaches and tools are used to mine architectural information? |  There might approaches and tools that have been proposed and used to mine architectural information. The answer of this RQ can provide practitioners with an overview of the readily available approaches and tools that they can utilize to mine architectural information from software repositories, and to some extent inspire researchers to develop new and dedicated architectural information mining approaches and tools.
RQ5. What are the challenges in mining architectural information? |  Mining architectural information in software repositories has many challenges. With this RQ, we want to identify and report these challenges as potential future research directions in this area.

###  3.2 Mapping Study Execution

We designed this SMS according to the guidelines for systematic mapping
studies proposed by Petersen et al. (2015). Figure 1 illustrates the five
phases (i.e., automatic search, study screening, snowballing, data extraction,
and data synthesis) that we followed to execute this mapping study.

Figure 1: Mapping study execution

####  3.2.1 Phase 1: Automatic Search

We performed the automatic search in seven electronic databases (see Table 2)
with three steps (i.e., electronic databases selection, choosing the search
scope, and defining the search string).

(i) Electronic databases selection: The selected electronic databases are
shown in Table 2 and they were selected based on the guidelines provided by
Chen et al. (2010). These databases are regarded as the most common and
popular databases in the software engineering field, and they are considered
appropriate to search for relevant studies (Chen et al., 2010). Google Scholar
and Scopus were not included in this SMS because through a pilot search we
observed that both Google Scholar and Scopus produced a significant number of
irrelevant studies and the results from Google Scholar and Scopus were
overlapped with the studies returned from other databases.

(ii) Choosing the search scope: We set the starting date to January 2006. The
starting year is justified by considering the milestone paper about the golden
age of software architecture in research and practice that was published in
2006 by Shaw and Clements (2006). The ending search period is settled as
December 2022 when we started this SMS.

(iii) Defining the search string: To define the search string used in this
SMS, we followed the following steps: (1) We extracted the major terms (e.g.,
“architect*”, “mining software repositor*”, “Stack Overflow”) based on the
research topics and research questions in the existing relevant studies (e.g.,
Bi et al. (2021b); Malavolta et al. (2021)) that were already known to us. (2)
We generated a list of synonyms (i.e., topic-related terms) for each extracted
major term. For example, the synonym terms for “architect*” term are “design*”
and “structur*”. (3) To compose the search string, we combined each extracted
major term and its synonyms with the Boolean operator OR, and (4) we linked
the major terms with the Boolean operator AND.

Note that, before the formal search and selection, we conducted a pilot search
and selection in order to decide the search terms used in the formal search
and selection. First, the pilot search can help us settle the appropriate
search terms for the formal search (Petersen et al., 2015). Two search strings
were used in this pilot search, i.e., (“architecture” OR “design” OR
“structure”) AND (“mining software repository” OR “repository mining”) and
(“architecting” OR “designing” OR “structuring”) AND (“mining software
repository” OR “repository mining”). We noticed that the search results by
different search terms are complementary to each other, for example, using the
term “architecture” in IEEE Explore did not return all relevant studies that
were retrieved by using the terms “architecting”, “designing”, and
“structuring”. Therefore, we decided to include all these terms in the final
construction of the search string to avoid missing potentially relevant
studies. Moreover, it should be noted that we did not include the terms
related to specific types of architectural information, such as architecture
decision, quality attribute requirements, architecture smell, in the search
due to two reasons: (1) part of the goal of this SMS is to identify the types
of mined architectural information in software development (i.e., RQ1),
therefore, we cannot have a relatively comprehensive list of those types
before conducting this SMS; (2) including the terms related to the specific
types of architectural information in the search query may lead to the
situation that the search results are biased to these types. Furthermore, some
electronic databases have their own limitations on the number of search terms
and operators. For example, IEEE Explore does not allow more than seven
wildcards (*) and ScienceDirect does not support more than eight Boolean
connectors. Hence, the final search string was adjusted according to the
restrictions and settings of each database. Overall, the final search string
used in the formal search was defined as:

(“mining software repositor*” OR “repositor* mining”) AND (architect* OR
design* OR structur*) AND (“Stack Overflow” OR StackOverflow OR GitHub OR
“open source software” OR “open source communit*” OR “online developer
communit*” OR “online communit*” OR “online developer forum*” OR “online
forum*” OR “question and answer site*” OR “question and answer website*” OR
“Q&A site*” OR “Q&A website*” OR “mailing list*” OR gitter OR slack OR chat OR
“issue tracker” OR “issue tracking” OR “issue management”)

Table 2: Electronic databases used for the automatic search

Database |  URL |  Search Scope in Database
---|---|---
ACM Digital Library |  https://dl.acm.org/ |  Study title, abstract
IEEE Explore |  https://ieeexplore.ieee.org/ |  Study title, keywords, abstract
Science Direct |  https://www.sciencedirect.com/ |  Study title, keywords, abstract
EI Compendex |  https://www.engineeringvillage.com/ |  Study title, abstract
Springer Link |  https://link.springer.com/ |  Study title, abstract
Wiley InterScience |  https://onlinelibrary.wiley.com/ |  Study title, abstract
ISI Web of Science |  https://login.webofknowledge.com/ |  Study title, keywords, abstract

####  3.2.2 Phase 2: Study Screening

We defined the inclusion and exclusion criteria (see Table 3) to screen the
retrieved studies from each database (see Table 2). Before the formal study
screening (manual inspection), to reach an agreement about the inclusion and
exclusion criteria (see Table 3), a pilot study screening was performed
whereby the first two authors randomly selected 100 primary studies from the
results of automatic search (see Figure 2 in Section 4) and checked them
independently. Specifically, this study screening process involved the
following rounds and the inclusion and exclusion criteria elaborated in Table
3 were applied in each round: (1) In the first round of study screening, the
first two authors independently screened the 100 studies by reading the
titles. Any uncertain studies (i.e., they could not be decided by the titles)
were temporarily included and kept in the second round. (2) In the second
round of study screening, the first two authors independently screened the
studies left in the first round screening by reading their abstracts. Those
studies were retained for the third round for which the first two authors had
not made any decision. (3) In the third round of study, the first two authors
independently screened studies by reading the full text of the studies left in
the second round. Moreover, the first two authors held a meeting to compare
the pilot study screening results and then discuss their disagreements (if
any) in an effort to reconcile them and arrive at a final version in which as
many discrepancies as possible have been resolved. To measure the inter-rater
agreement between the first two authors, we calculated the Cohen’s Kappa
coefficient (Cohen, 1960) and got an agreement of 0.935. During the formal
study screening process, we followed the same rounds of study screening, which
were used during the pilot study screening process, and they are described
below. The selected number of studies in each round of the study screening
process is provided in Section 3.3.

(i) First round of study screening (i.e., reading titles): Through the use of
the inclusion and exclusion criteria elaborated in Table 3, the first author
screened the studies by reading the titles of the remaining studies from the
pilot study screening in order to select potential primary studies. Moreover,
he recorded the key terms that led to the inclusion/exclusion of a specific
study, and those terms were further used for discussion during the consensus
meetings and reassessment process. In order to mitigate potential personal
bias during study screening, the results of the first round were checked and
validated by the second author. In some cases, where the relevance of a study
was still unclear from the title, the first and second authors decided to
temporarily include it to the second round of study screening.

(ii) Second round of study screening (i.e., reading abstracts): In the second
round of study screening, the first author read the abstracts of the retained
studies from the first round, and he screened the studies based on the
inclusion and exclusion criteria (in Table 3). He followed the same procedure
(i.e., recording the key terms that led to inclusion/exclusion of a specific
study) that was employed in the previous round study screening. Similar to the
first round screening, to mitigate potential bias, the second author checked
and validated the results from the second round. Moreover, if any
disagreements (i.e., whether a study could be included or not) between the two
authors arose, such conflicts were discussed and resolved among the first two
authors. Those studies were kept for the final round which were hard to decide
based on their abstracts.

(iii) Third round of study screening (i.e., reading full texts): The first
author read the full text of each study that was retained from the second
round, and he screened the studies based on the selection criteria (in Table
3). He followed the same procedure (i.e., recording the key terms that led to
inclusion/exclusion of a specific study, and validating the study screening
results with the second author) that was used in the previous rounds. Note
that, we got controversies on 11 primary studies from the study screening
results (i.e., whether a study should be excluded or included in the final
round of study screening). Such controversies between the first two authors
were discussed in meetings involving the third author till a consensus was
reached.

Table 3: Inclusion (I) and Exclusion (E) criteria for selecting the primary
studies

Inclusion criterion
---
I1. A study that focuses on mining architectural information in software
development.
Exclusion criteria
E1. A study focuses on architectural information without mining it.
E2. A study focuses on mining other types of information instead of
architectural information.
E3. A study not written in English is excluded.
E4. If two studies publish the same work in different venues (e.g., conference
and journal), the less mature one is excluded.
E5. A study that is grey literature (e.g., technical report) is excluded.

####  3.2.3 Phase 3: Snowballing

In order to minimize the threat of missing relevant studies that were possibly
missed out during the automatic search, we conducted snowballing by following
the approach proposed by Wohlin (2014). We adopted the forward (i.e.,
collecting those studies citing the selected studies) and backward (i.e.,
using the references of the selected studies) snowballing, which is also used
in many other SMSs, such as (Waseem et al., 2020; Tian et al., 2021)).
Specifically, we conducted the snowballing process iteratively. We checked the
references lists of the studies selected from the final round of study
screening in automatic database search (in Section 3.2.2) and those studies
citing the selected studies, then the newly selected studies of the last
iteration were checked in the subsequent iterations. The iterative process was
completed when there were no newly selected studies. Each iteration followed
the three rounds of study screen process described in Section 3.2.2, i.e.,
based on the titles, abstracts, and full texts.

###  3.3 Study Search and Screening Results

In this section, we present the results of studies search (both automatic
search and snowballing) and the results of study screening. Figure 3.3
illustrates the number of studies that we got from the automatic search, each
round of study screening, and the snowballing. The automatic search in the
seven electronic databases resulted in 23,974 potentially relevant studies.
2,547 studies were shortlisted after the first round of study screening (i.e.,
by reading titles), 318 primary studies were kept after the second round of
study screening (i.e., by reading abstracts), and 73 primary studies were
selected after the third round of study screening (i.e., by reading full
texts). As elaborated in the previsions paragraph, we conducted the
snowballing process (i.e., forward and backward snowballing), and 3,125
references (of the 73 studies that we got from the third round of study
screening) were returned. These 3,125 references were further screened based
on the titles (204 studies retained), abstracts (37 studies retained), and
full texts (14 studies retained). Finally, 87 (i.e., 73+14) primary studies
were included in this SMS. The details of the selected 87 primary studies are
provided in Appendix A.

Figure 2: Results of study search and screening

####  3.3.1 Phase 4: Data Extraction

To provide the demographic information of the selected studies and answer the
five RQs of this SMS (see Table 1), we carefully read all the primary studies
selected in this SMS (see Figure 1) and extracted the required data items as
listed in Table 4. Before the formal data extraction, the first three authors
discussed the meaning of each data item and the way about how to extract data.
To ensure an unambiguous understanding of the data items, the first author
conducted a pilot data extraction. Specifically, the first author randomly
selected 15 primary studies (from 87 selected studies, see Figure 2) and
extracted the data according to the data items in Table 4. When such studies
were unclear and the first author got confused while extracting the data
(e.g., if such a selected study did not clearly describe such a data item
defined in Table 4), physical meetings with the second author were scheduled
to solve such confusion. The process continued till all the 15 studies were
manually checked. To suppress the effect of subjective bias, the pilot data
extraction results were reviewed and validated by other two authors (i.e., the
second and third authors) of this study so that three authors could get a
consensus on the understanding of the extracted data items. Likewise, in the
formal data extraction process, the data extraction was performed by the first
author and validated by the second and third authors, and any divergences and
ambiguities of the results in the extracted data were discussed together for
reaching an agreement. In this way, we can ensure that the extracted data in
the formal data extraction process are valid. The description of the data
items and their relevant RQs are presented in Table 4. Specifically, nine data
items have been defined to be extracted from the selected studies. The three
data items (i.e., D1-D3) are used to extract the demographic details of the
selected studies, and the remaining data items (i.e., D4-D9) are used to
answer the five RQs (see Table 1). The data extraction was subsequently
followed by data synthesis, and these two processes were conducted and
recorded with the aid of MAXQDA (a qualitative data analysis
tool)111https://www.maxqda.com/.

Table 4: Data items extracted from the selected studies with the relevant RQs

# |  Data item |  Description |  Data analysis approach |  Relevant RQ
---|---|---|---|---
D1 |  Publication year |  The publication year of the study. |  Descriptive statistics |  Overview
D2 |  Publication venue |  The name of the venue where the study is published. |  Descriptive statistics |  Overview
D3 |  Publication type |  The type of the study (i.e., journal, conference, or workshop). |  Descriptive statistics |  Overview
D4 |  Architectural information |  The type of mined architectural information. |  Open coding & constant comparison, predefined classifications, and descriptive statistics |  RQ1
D5 |  Source |  The source used to mine architectural information. |  Open coding & constant comparison and descriptive statistics |  RQ2
D6 |  Architecting activity |  The architecting activity that a study claims to support. |  Open coding & constant comparison, predefined classifications, and descriptive statistics |  RQ3
D7 |  Approach |  The approach used to mine architectural information. |  Open coding & constant comparison and descriptive statistics |  RQ4
D8 |  Tool |  The tool used to mine architectural information. |  Open coding & constant comparison and descriptive statistics |  RQ4
D9 |  Challenge |  The challenge faced when mining architectural information. |  Open coding & constant comparison and descriptive statistics |  RQ5

####  3.3.2 Phase 5: Data Synthesis

During this phase, we synthesized the extracted data that we gathered from the
previous phase (i.e., Phase 4: Data extraction) in order to answer the five
RQs (see Table 1). We used open coding & constant comparison to analyze
several data items and answer certain RQs of this SMS (see Table 4). Open
coding & constant comparison are two widely used techniques in software
engineering (Seaman, 1999) during qualitative data analysis. Open coding
generates codes for incidents that can be further classified into concepts and
categories (Seaman, 1999). Constant comparison is a continuous process for
verifying the generated concepts and categories. Both concepts and categories
evolve and saturate until they fit the data (Seaman, 1999). Specifically, in
this SMS, open coding & constant comparison were employed for analyzing the
data item D5 to answer RQ2 (sources), the data items D7 and D8 to answer RQ4
(approaches and tools), and the data item D9 to answer RQ5 (challenges). In
addition, descriptive statistics (Wohlin et al., 2003) was also used for
analyzing the data item D5 to answer RQ2 (sources), the data items D7 and D8
to answer RQ4 (approaches and tools), and the data item D9 to answer RQ5
(challenges). Descriptive statistics can provide quantitative summaries based
on the initial description of the extracted data. Moreover, we used a hybrid
approach which is a combination of open coding & constant comparison,
predefined classifications (i.e., the conceptual model for architectural
description in the ISO 42010:2011 standard (ISO/IEC/IEEE, 2011), the
categories of architecture decisions by Kruchten (2004), and the categories of
architectural changes by Williams and Carver (2010)), and descriptive
statistics for analyzing the data item D4 to answer RQ1 (mined architectural
information). On another hand, a hybrid approach which is a combination of
open coding & constant comparison, predefined classifications of architecting
activities by Hofmeister et al. (2007), Tang et al. (2010), and Li et al.
(2013), and descriptive statistics was employed for analyzing the data item D6
to answer RQ3 (architecting activities).

As mentioned above, we utilized a qualitative data analysis tool MAXQDA to
support the analysis process. MAXQDA assists human annotators to label text
segments within their contexts and assign them to categories. Specifically,
before the formal data analysis (manual labeling), to reach an agreement about
the data items that we gathered from the previous phase (i.e., Phase 4: Data
extraction), we first performed a pilot data analysis. Specifically, this
analysis process involved the following steps: (1) The first and second
authors independently checked and read a random sample of 5 primary studies
(from 87 selected studies, see Figure 2). (2) The first two authors
independently labeled the extracted data with codes that succinctly summarize
the data items (see Table 4) for answering RQs. (3) The first two authors
independently grouped all the codes into higher-level concepts and turned them
into categories or subcategories. The grouping process was iterative, in which
each author continuously went back and forth between the code, concepts,
categories, and extracted data items to revise and refine both the code,
concepts, and categories. In order to improve the reliability of the pilot
data analysis results, the first two authors held a meeting and followed a
negotiated agreement approach (Campbell et al., 2013) to compare the coding
results, then discussed their disagreements, confusions, and uncertain
judgments on the data encoding results in an effort to reconcile them and
arrive at a final version of the pilot data analysis results in which all the
discrepancies have been resolved. For example, when answering RQ1 (mined
architectural information), the first author was checking this study (i.e.,
[S2]) in order to label the mined architectural information, and he came
across with these sentences: “(…) Continuously means that they are analyzed in
response to every modification. When analyzing the implementation, the system
tries to extract as much architectural information from implementation
artifacts as possible (…)”. In this case, the first author got confused on
which label should be assigned to those sentences as the mined architectural
information. In a meeting with the second author, they discussed and agreed to
label it as “general architectural information”. The first author carried on
with the formal data analysis and followed the same steps used during the
pilot data analysis. In the following paragraph, we provide the details of the
formal data analysis process.

The first author fully read the full text of each selected study from the
remaining primary studies. Subsequently, he summarized the main ideas stated
in several sentences in the selected study. He continued to encode the
summarized ideas (from each selected study) to generate codes, concepts, and
categories and subcategories. For example, when answering RQ1 (mined
architectural information), he referred to these sentences in the selected
study [S27] that states: “(…) Implementation of a tactic often spreads across
more than one source file. Consequently, domain topics that motivate an
architectural tactic are often not fully presented in the tactical file itself
but also appear in neighboring files (i.e., files which use tactical files or
are used by tactical files). Therefore, we need to identify the whole context
in which a tactic is implemented. The context analysis is comprised of two
steps. First, we identify complete tactic instances whose implementation
spreads across multiple tactical files. Second, we consider the full technical
context of an implemented tactic instance (…) To discover this full context in
which a tactic instance is implemented, we use Algorithm 2 to discover the
full context in which each tactic instance is implemented (…)”. In this case,
he summarized and encoded those sentences as the code “identify the whole
context in which a tactic is implemented”. Afterwards, the first author
grouped this code into a higher-level concept (i.e., “tactic and context”).
Then, he applied constant comparison to compare the concepts identified in one
summarized idea with the concepts that emerged from other summarized ideas to
identify the concepts which have similar semantic meanings. He proceeded to
group similar concepts into main categories and subcategories. For example,
the concept “tactic and context” was merged into the subcategory
“architectural tactic and context relationship” that was further categorized
into the category “design relationship” (see Table
LABEL:minedArchitecturalInfo). Moreover, as stated before in this section, we
used predefined categorization of architectural information to generate
categories and subcategories of mined architecture information for answering
RQ1. Among others, we utilized an existing category of architectural design
decisions by Kruchten (2004) (i.e., structural, behavioral, and ban decisions)
to classify the mined architectural decisions. To mitigate the personal bias
during the formal data analysis, the second and third authors of this SMS
reviewed and validated the generated codes, concepts, categories, and
subcategories. The results of this SMS are provided in Section 4.

##  4 Results

We first present the overview of the selected studies in Section 4.1. The
results of five RQs are reported in Section 4.2 to Section 4.6.

###  4.1 Overview of the Selected Studies

In this section, we present an overview of the selected primary studies from
two perspectives (i.e., publication years and venues).

(a) Distribution of the selected studies over the years

To better understand the publication years of the selected studies, we
summarized the publication number of these studies per year (from 2006 till
2022) and analyzed the growth of publication number as time goes by. Figure 3
shows the distribution of the 87 selected studies over the past sixteen years
(i.e., 2006–2022). It is clear that there are very few studies (i.e., 11
studies) published from 2006 to 2010 on mining architectural information.
However, we can see a steady amount of attention on mining architectural
information from 2011 to 2022 (2-13 studies were published per year). Overall,
we found evidence of a fair increasing interest in the software architecture
community on mining architectural information.

Figure 3: Distribution of the selected studies over the years

(b) Distribution of the selected studies in publication venues

The selected studies were published in 53 venues. We only list the venues that
publish more than one selected studies (see Table LABEL:PublicationVenues in
Appendix B). Note that, merged conferences were only counted as one
conference, for example, Working International Conference on Software
Architecture (WICSA) and International Conference on Software Architecture
(ICSA) were counted as one conference. As shown in Table
LABEL:PublicationVenues, some venues tend to publish more studies on mining
architectural information. For instance, the dedicated conferences on software
architecture record the most publications (i.e., 18), which are International
Conference on Software Architecture (ICSA) and European Conference on Software
Architecture (ECSA) with 13 and 5 publications, respectively. The conference
with the most number of studies (i.e., 13 studies) is ICSA. In terms of
journals, both the Journal of Systems and Software and IEEE Transactions on
Software Engineering index the most studies, each with 6 studies (see Table
LABEL:PublicationVenues in Appendix). As shown in Figure 4, the 87 studies
cover three types of publication venues, i.e., conference, journal, and
workshop. Most of the selected studies were published in conferences (i.e.,
58.6%, 51 out of 87) compared with journals (i.e., 33.3%, 29 out of 87) and
workshops (i.e., 8.0%, 7 out of 87), which indicates that conferences are the
popular venues to disseminate the work on mining architectural information.

Figure 4: Publication venues based on the distribution of the selected studies

###  4.2 Results of RQ1: Mined Architectural Information

As described in Section 3.3.2, a hybrid approach (which is a combination of
open coding & constant comparison, predefined classifications, and descriptive
statistics) was used to analyze the extracted data item D4 (see Table 4) and
identify the mined architectural information from software repositories. Our
data synthesis generated 8 main categories and 30 subcategories of mined
architectural information (see Table LABEL:minedArchitecturalInfo), in which
architectural description (59.8%, 52 out of 87 studies), architectural
decision (24.1%, 21 out of 87 studies), architectural solution (18.4%, 16 out
of 87 studies), and system requirement (18.4%, 16 out of 87 studies) are the
top four most mined architectural information to support the development
process. Note that, one study may mine more than one type of architectural
information, therefore, the total number of studies in Table
LABEL:minedArchitecturalInfo is larger than 87.

Table 5: Categories and subcategories of mined architectural information Category | Subcategory | Studies | Count
---|---|---|---
Architectural description (59.8%, 52) |  Architectural model |  [S11] [S12] [S29] [S30] [S36] [S37] [S39] [S46] [S47] [S51] [S54] [S64] [S65] [S67] [S68][S73] [S75] [S80] [S83] [S86] | 20
|  Architectural view |  [S8] [S18] [S35] [S37] [S39] [S46] [S47] [S52] [S54] [S59] [S64] [S65] [S68] | 13
|  Architectural rationale |  [S1] [S11] [S14] [S24] [S25] [S26] [S38] [S47] | 8
|  Architectural concern |  [S6] [S7] [S8] [S9] [S11] [S66] | 6
|  System of interest |  [S11] [S26] [S29] [S39] [S51] | 5
Architectural decision (24.1%, 21) |  Structural decision |  [S19] [S22] | 2
|  Behavioral decision |  [S19] [S22] | 2
|  Ban or non-existence decision |  [S19] [S22] | 2
|  General architectural decision |  [S1] [S8] [S10] [S14] [S18] [S24] [S25] [S26] [S33] [S35] [S38] [S43] [S47] [S85] [S86] | 15
Architectural solution (18.4%, 16) |  Architectural tactic |  [S1] [S6] [S7] [S13] [S15] [S26] [S9] [S24] [S56] [S57] | 10
|  Architectural pattern |  [S1] [S35] [S24] [S63] [S82] | 5
|  Framework for addressing scalability |  [S26] | 1
System requirement (18.4%, 16) |  Quality attribute |  [S1] [S8] [S11] [S24] [S26] [S42] [S66] [S76] [S83] | 9
|  Functional requirement |  [S1] [S43] [S45] [S55] [S76] [S83] | 6
|  General system requirement |  [S25] | 1
Architectural change (16.1%, 14) |  Perfective change |  [S40] [S41] | 2
|  Corrective change |  [S40] [S41] | 2
|  Adaptive change |  [S40] [S41] | 2
|  Preventative change |  [S40] [S41] | 2
|  General architectural change |  [S1] [S18] [S35] [S46] [S47] [S77] | 6
Design relationship (14.9%, 13) |  Component-Component relationship |  [S12] [S20] [S59] [S67] [S74] | 5
|  Architectural tactic-Context relationship |  [S6] [S27] | 2
|  Architectural pattern-Architectural pattern relationship |  [S63] [S78] | 2
|  Architectural pattern-Context relationship |  [S87] | 1
|  Architectural pattern-Quality attribute relationship |  [S31] | 1
|  Architectural tactic-Quality attribute relationship |  [S6] | 1
|  Design pattern-Architectural tactic relationship |  [S21] | 1
Architectural technical debt (11.5%, 10) |  Architectural compliance issue |  [S30] [S34] [S36] [S73] | 4
|  Architectural anti-pattern |  [S37] [S64] [S72] [S84] | 4
|  Architectural smell |  [S29] [S71] | 2
General architectural information (21.8%, 19) |  |  [S2] [S3] [S5] [S16] [S17] [S23] [S28] [S32] [S44] [S48] [S49] [S50] [S53] [S60] [S61] [S69] [S70] [S79] [S81] | 19

Architectural description is one of the important sources used for
communicating and sharing architectural information about the high-level
design of a software system (Clements et al., 2003). For example, the
information in architectural description, such as architectural views, which
describe systems in multiple views (e.g., logical view, process view,
development view), enable the architecture to be communicated and understood
by different stakeholders (e.g., architects, developers, and project managers)
(Kruchten, 1995). However, this information is not always well described or
documented during the development process (Ding et al., 2014). Hence, due to
the lack of architectural description, software engineers (e.g., developers)
may modify the related artifacts (e.g., source code) without fully
understanding the underlying architecture (Perry and Wolf, 1992), which
results in gradual degradation of architectural quality (Herold et al., 2016).
Even though this information is not explicitly documented, they are implicitly
captured in different artifacts, code, requirement, and architecture
documents. 59.8% of the selected studies (52 out of 87) proposed approaches
and tools for mining or recovering architectural descriptions of software
systems from several artifacts (e.g., code [S29][S39][S46][S51]) that are
contained in various software repositories, such as VCS (e.g., GitHub), to
assist development process. We further classified the mined architectural
information in the architectural description category by using the conceptual
model for architectural description in the ISO 42010:2011 standard
(ISO/IEC/IEEE, 2011), which defines a set of architectural elements that make
up the architectural description model (ISO/IEC/IEEE, 2011). These
architecture elements have been used to identify and categorize architecture
elements in SA documents of OSS (Ding et al., 2014), and they also have been
utilized to categorize architectural information communicated during OSS
development (Bi et al., 2021a). As shown in Table
LABEL:minedArchitecturalInfo, we collected five subcategories of mined
architectural information (i.e., architectural elements) in the architectural
description category.

  * •

Architectural model is usually employed to represent a whole architecture or a
part of it (ISO/IEC/IEEE, 2011). Among other usages, an effective
architectural model can be used as the basis for design time analysis to
determine whether the software system will meet desired quality attributes
(Garlan and Schmerl, 2004). However, creating an effective architectural model
of a system, specifically, an effective architectural model of a large and
complex system, and keeping that architectural model consistent with
architectural changes that may occur as the system evolves, is one of the
serious problems that confront today’s software developers (Szvetits and Zdun,
2016). Architectural model is the most mined (20 out of 52 studies)
architectural information in architectural description category. For example,
Granchelli et al. [S39] proposed a semi-automatic approach based on Model-
Driven Engineering and Domain-Specific Language for automatically mining and
recovering architectural models of microservice-based systems from source
code. The proposed approach can also allow an architect to manually refine the
initial extracted architecture model into refined architectural models that
are suitable for his/her needs, such as for performing architecture change
impact analysis.

  * •

Architectural view expresses the architecture of the system in accordance with
an architecture viewpoint which addresses one or more of the concerns held by
the system’s stakeholders (ISO/IEC/IEEE, 2011). Architectural view is reported
as the second most common (13 out of 52 studies) mined architectural
information in the architectural description category. For instance, an
architecture recovery approach and a prototype tool were developed in [S46] to
help software engineers (e.g., architects) automatically extract and recover
various architectural views, including system-level and component-level
architectural views from source code. The mined architectural views can assist
in determining, for example, when, how, and to what extent system-level or
component-level architectural views evolve during software development and
evolution.

  * •

Architectural rationale records explanation, justification, and reasoning
about architectural design decisions that are made in architecture design
(ISO/IEC/IEEE, 2011). The results in Table LABEL:minedArchitecturalInfo reveal
that architectural rationale is the third common (8 out of 52 studies) mined
architecture information in the architectural description category. For
example, in [S24], Lopez et al. presented an ontology-based approach named
TREx (Toeska Rationale Extraction) for extracting, representing, and exploring
architectural rationale information from text documents (such as email, wiki,
and meeting notes). Specifically, this approach consists of three components:
(1) pattern-based information extraction to recover rationale, (2) ontology-
based representation of rationale and architectural concepts, and (3) facet-
based interactive exploration of rationale. Initial results from applying TREx
suggest that architectural rationale (such as reasons behind architectural
decisions) can be semi-automatically extracted and mined from a project’s
unstructured text documents. The mined architectural rationale provides
several benefits in software development, such as rationale reuse, rationale
auditing, and architects training.

  * •

Architectural concern denotes the interest pertaining to the system
development and is related to one or more stakeholders (ISO/IEC/IEEE, 2011).
For instance, in [S66], Gokyer et al. proposed an approach based on NLP and ML
techniques to automatically mine architectural concerns from non-functional
requirements expressed in plain text. The mined architectural information
(i.e., architectural concerns) can guide architects in making design decisions
effectively.

  * •

System of interest refers to the system whose architecture is under
consideration during the development (ISO/IEC/IEEE, 2011). In [S11], Bi et al.
mined architectural information (including system of interest) that is
communicated during the development of Open Source Software (OSS). The mined
architectural information (e.g., system of interest) can be used to enrich
architecture documentation, especially the documentation of architectural
design decisions, in OSS development.

Architectural decision: Designing the architecture of a software system
typically involves making numerous architecture decisions and each decision
could positively or negatively affect the functional and non-functional
properties of the system (Shahin et al., 2009). Recovering and understanding
these design decisions can help inform future architectural decisions and
implementation choices, and can avoid introducing architectural inefficiencies
(Shahbazian et al., 2018) or architectural decay later (Hassaine et al.,
2012). 24.1% of the selected studies (21 out of 87) proposed approaches and
tools for mining architectural decisions from various sources, such as issue
tracking systems (e.g., Jira) and VCS (e.g., GitHub), to support the
development. We further categorized the mined architectural decisions into
three subcategories by following an existing classification of design
decisions by (Kruchten, 2004) (see Table LABEL:minedArchitecturalInfo). The
mined architectural decisions in this category can help architects and
developers utilize these decisions in similar development contexts.

  * •

Structural decision refers to design decisions that lead to the creation of
subsystems, layers, partitions, and components in the logical view of
architecture (Kruchten, 2004). For example, Bhat et al. [S22] employed an ML-
based approach to mine and classify architectural decisions from issue
tracking systems (e.g., Jira) into three subcategories, including structural
decisions.

  * •

Behavioral decision is more related to how the architectural elements interact
with each other to provide functionality or to satisfy specific non-functional
requirements (e.g., quality attributes) (Kruchten, 2004). For example, Bhat et
al. [S19] developed an approach with a prototype tool to automatically curate
design decision knowledge for architectural decision recommendations. One of
the components (i.e., Document Classifier) of the tool could categorize the
design decisions described in issue tracking systems (e.g., Jira) into three
categories, including behavioral decisions.

  * •

Ban or non-existence decision refers to design decisions that result in the
removal of an architectural artifact or interaction between architectural
artifacts (Kruchten, 2004). In [S22], Bhat et al. developed an ML-based
approach to mine and classify architectural decisions from issue tracking
systems into three subcategories, including ban or non-existence decisions.

  * •

General architectural decision refers to a study that has a generic
description about architectural decisions (e.g., design decisions in [S47]).
In other words, this study does not explicitly specify a concrete category or
subcategory of mined architectural decisions that belongs to one of the
abovementioned categories or subcategories.

Architectural solution: Architectural solutions are the fundamental building
blocks in architecture design, and they are used to address architecture
design concerns (Bass et al., 2012). 18.4% (16 out of 87) studies researched
on mining architectural solutions to support the development. We further
categorized the mined architectural solutions into three subcategories (see
Table LABEL:minedArchitecturalInfo).

  * •

Architectural tactic is a design solution used to satisfy a specific quality
attribute of a system (e.g., reliability, performance) (Bass et al., 2012). As
shown in Table LABEL:minedArchitecturalInfo, 10 out of 87 studies focused on
mining architectural tactics from software repositories. For example,
Chinnappan et al. [S9] employed a manual approach (i.e., qualitative data
analysis) to mine energy-aware architectural tactics for Robotic Operating
System (ROS) based software systems from GitHub and Stack Overflow. To do so,
they carried out a multi-phase study that resulted in seven energy-awareness
architectural tactics (such as energy savings mode and stop current task &
recharge). To foster the applicability of the identified tactics even beyond
the ROS community, they described these tactics in a generic implementation-
independent manner by utilizing diagrams inspired by the UML component and
sequence diagram notations. These energy-aware architectural tactics can serve
as guidance for roboticists, as well as other developers interested in
architecting and implementing energy-aware software.

  * •

Architectural pattern: In common architectural pattern documents or
collections (such as architecture pattern books), architecture patterns are
documented with templates that consist of multiple attributes, such as intent,
structure, and context (Buschmann et al., 1996a). To adapt to modern
development, developers are required to consider specific attributes, for
example, “known uses” which exemplifies the use scenarios of architectural
patterns in practice (Liu et al., 2020). However, it is not easy to update the
contents of these attributes manually due to the complexity of and rapidly
evolving architectural technologies (Barua et al., 2014). To mitigate this
problem, Liu et al. [S63] proposed a semi-automatic approach for mining
architecture/design pattern use scenarios and related pattern pairs from Stack
Overflow. This mined architectural information can aid developers to acquaint
with the usage and relatedness of architecture/design patterns in today’s
modern software development.

  * •

Framework for addressing scalability: Software frameworks provide developers
with powerful tools to develop more flexible and less error-prone applications
in a more effective way. Software frameworks often help expedite the
development process by providing necessary functionality “out of the box”.
Providing frameworks for reusability, scalability, and separation of concerns
is key to software development (Schmidt and Buschmann, 2003). For example,
Márquez et al. [S26] proposed a semi-automatic approach for mining frameworks
in OSS for addressing scalability concerns in microservice applications, and
these frameworks can assist developers in designing scalable microservice
applications.

System requirement: Analyzing requirements to relate and map them to their
corresponding architectural elements (e.g., architectural components) has
become one of the major challenges faced by architects during the development
process (Casamayor et al., 2012a). The failure of a high percentage of
software projects is often caused by, for example, the lack of proper
requirements analysis, incomplete requirement specifications, and changing
requirements, among others (Hull et al., 2005). Thus, 18.4% of the studies (16
out of 87) focus on mining the requirements of systems from repositories for
assisting architecting process. We further categorized the mined architectural
information in this category into three subcategories (see Table
LABEL:minedArchitecturalInfo).

  * •

Quality attribute: While all systems have quality attributes or Non-Functional
Requirements (NFRs), they may not be explicitly stated in requirements
specification. Furthermore, as some NFRs represent significant properties of
the systems, specifically, properties of fault-tolerant systems, those NFRs
require appropriate analysis effort to ensure they are met during the
architecture design (Chung et al., 2012). When the specified NFRs are not met,
the system incurs a costly re-work. Quality attribute requirement is the most
(9 out of 87 studies) mined architectural information in the system
requirement category to support the development (see Table
LABEL:minedArchitecturalInfo). For instance, in [S66], Gokyer et al. proposed
an approach based on NLP and ML techniques for automatically mining NFRs
expressed in plain text and mapping these NFRs to architectural elements, such
as architectural components, in the problem domain. The mined architectural
information can guide architects in making architectural decisions
effectively.

  * •

Functional requirement: Essentially, a system’s utility is determined by both
its functionality and non-functional characteristics (Chung et al., 2012).
Even though both functional and non-functional characteristics must be taken
into consideration in the development of a quality software system, there has
been a lopsided emphasis on the functionality of the system (Chung et al.,
2012). 6 out of 87 studies paid attention to mining functional requirements to
support the development (see Table LABEL:minedArchitecturalInfo). For example,
Casamayor et al. [S43] presented an approach based on NLP and ML techniques
for semi-automatically mining and analyzing functional requirements (from the
textual description of requirements) that will become the responsibilities of
certain architectural components in the system, in order to help bridge the
gap between requirements analysis and architectural design.

  * •

General system requirement refers to the study that has a generic description
about system requirements (e.g., [S25]). In other words, this study does not
explicitly specify a concrete category or subcategory of mined system
requirements that belongs to one of the abovementioned categories or
subcategories.

Architectural change: Software changes are inevitable. There are many reasons
for software changes, such as repairing defects and evolving user requirements
(Williams and Carver, 2010). When changes affect the architecture, software
engineers need a comprehensive understanding of the causes of architecture
changes and their impact (Williams and Carver, 2010). This understanding is
important because, as changes may affect many aspects of a system and
introduce complexities in architecture, which will likely introduce a
significant number of architectural issues. 16.1% (14 out of 87) studies mined
and analyzed architectural change information from different perspectives
(e.g., analyzing the causes of architectural changes (Williams and Carver,
2010)) to support the development. Similar to these studies (i.e., Ding et al.
(2015a); Mondal et al. (2019)), we adopted a predefined classifications of
architectural changes introduced by Williams and Carver (Williams and Carver,
2010) to categorize the mined architectural change information into four
subcategories (see Table LABEL:minedArchitecturalInfo).

  * •

Perfective change refers to architectural changes that are resulted from new
or changed requirements. This subcategory of architectural changes improves
the system to better meet user needs (Williams and Carver, 2010). For example,
Mondal et al. [S40] proposed an ML-based approach for mining information
related to architectural changes from source code and commit messages, and
labeled them into four subcategories, including perfective changes. This mined
architectural information can help developers better analyze and characterize
the causes and impact of architectural changes prior to their implementations.

  * •

Corrective change denotes architectural changes that occur in response to
defects in software systems (Williams and Carver, 2010). Mondal et al. [S41]
proposed an approach for automatically classifying architectural changes into
four subcategories (including corrective changes) from commit messages, to
reduce the manual effort when performing architectural change analysis (such
as analyzing architecture changes for detecting architectural anti-patterns).

  * •

Adaptive change refers to architectural changes that happen when moving to a
new environment or platform or accommodating new standards (Williams and
Carver, 2010). In [S40], Mondal et al. proposed a text classifier called
ArchiNet that classfies architectural changes into four subcategories
(including adaptive changes) from commits and source code.

  * •

Preventive change refers to architectural changes that take place when
restructuring or reengineering the software system (Williams and Carver,
2010). For instance, Mondalet al. [S41] conducted experiments with commits to
compare three approaches, which are based on Latent Dirichlet Allocation (LDA)
technique, about their performance in categorizing architectural changes into
four subcategories, including preventive changes.

  * •

General architectural change refers to the studies that have a generic
description of architectural changes (e.g., in [S46]). In other words, these
studies do not explicitly specify a concrete category or subcategory of mined
architectural changes that belongs to one of the abovementioned categories or
subcategories.

Design relationship: An architecture of a system is designed with the aid of
architectural elements (e.g., architectural patterns, tactics, and
components). However, there are complex interdependency relationships between
these elements, and many of the tasks faced by developers today are to deal
with an efficient and effective use of these elements together during the
application design to meet both functional and non-functional goals (Bi et
al., 2018). For example, designing with a certain architecture pattern to
address specific quality attribute requirements cannot be considered in
isolation, but there might be other key architectural elements to be
considered, such as design contexts (Bedjeti et al., 2017). 14.9% (13 out of
87) studies researched on mining design relationships from various sources to
aid the development. The mined architectural information in this category
provides developers the relationships between architectural elements (e.g.,
incompatibility between architectural elements (Karthik and Medvidovic, 2019))
to consider when designing with these elements.

  * •

Component-Component relationship: Software applications may comprise a number
of heterogeneous components, such as software components and hardware
components, and these components need to interact and communicate with each
other in order to satisfy various design concerns. Nevertheless, combining
these components together as part of the application stack to meet both
functional and non-functional requirements of systems is not easy for
developers (Karthik and Medvidovic, 2019). 5 out of 87 studies focused their
research on mining component-component relationships to assist the development
process. For instance, Karthik et al. [S20] presented an automatic ML-based
approach for mining the relationships between architectural components from
the unstructured text of Q&A site posts. Moreover, the mined component-
component relationships were categorized into three types, namely
incompatibility, required, and recommended. This mined architectural
information can help developers who work with component-based systems to
effectively understand Component-Component relationships of certain
applications similar to developers’ systems discussed on Stack Overflow.

  * •

Architectural tactic-Context relationship: Whilst applying Architectural
Tactics (ATs) to address Quality Attributes (QAs) is well explored in existing
works, e.g., (Mirakhorli et al., 2013; Bi et al., 2021b), there is no
guidelines for architects, who look for information on what considerations
(e.g., design contexts) they need to consider when applying ATs to address QA
concerns. In order to provide architects with such information, Gopalakrishnan
et al. [S27] proposed an ML and text mining based approach for extracting and
mining typical design contexts in the source code in which architectural
tactics are implemented. The authors went on to perform an in-depth analysis
of the relationship between the design contexts in source code and
architectural tactics implemented in that code through building inference
models, which can be used to predict and recommend the placement of
architectural tactics within source code packages/modules.

  * •

Architectural pattern-Architectural pattern relationship: Architectural
patterns have increasingly become an integral part of architecture design
practices (Buschmann et al., 2007). Architectural patterns are seldom applied
in isolation within an architecture. Individual architectural patterns can
only solve specific parts of the design problem (Kamal and Avgeriou, 2010).
Architectural patterns are often combined with other relevant architectural
patterns during architecture design, which gives these patterns the potential
to address multiple architecturally significant requirements (Avgeriou and
Zdun, 2005). For example, the Client-Server and Broker patterns are often used
in combination to design distributed system architectures (Buschmann et al.,
1996b). However, combining architectural patterns effectively during
architecture design remains a challenging task for software engineers because,
for example, the integration of any two architectural patterns can take
several forms, and existing architectural pattern languages only mention
generic architectural pattern to architectural pattern relationships and do
not go into the details of their combination (Kamal and Avgeriou, 2010). In
[S78], Kamal et al. conducted a qualitative data analysis and mined the design
relationships between various architectural patterns from several sources of
software repositories, such as architecture design documents and cases
studies. The mined architectural information in this subcategory can assist
software engineers who work with software systems that require combinations of
more than one architectural pattern during the development.

  * •

Architectural pattern-Context relationship: Design contexts are factors that
shape an architecture design. Whilst design contexts are recognized by
developers, there is little or no systematic research on how design contexts
impact decisions about architectural patterns in practice. In order to provide
developers with such information, Wijerathna et al. [S87] proposed an
automatic ML-based approach for mining design contexts from Stack Overflow
posts. In addition, the authors went on to manually structure the design
relationships between design contexts and architectural patterns used in
practice, and built a knowledge base of how developers use architectural
patterns with respect to design contexts to address QA concerns.

  * •

Architectural pattern-Quality attribute relationship: Many architecture design
methods consider the use of architectural patterns as a fundamental design
concept (Rozanski and Woods, 2012). When making an effective architectural
pattern selection, developers should consider, among other aspects, the impact
of that pattern on promoting or inhibiting specific quality attributes.
However, for inexperienced developers, this task often requires significant
time and effort. The reasons include a plenty number of architectural
patterns, the emergence of new architectural patterns, and the lack of
techniques and tools for automatically identifying the most suitable
architectural patterns for specific software systems (Velasco-Elizondo et al.,
2016). Velasco-Elizondo et al. [S31] proposed an approach based on an
information extraction technique (i.e., entity extraction) and knowledge
representation (i.e., ontology) to automatically analyze and mine architecture
patterns considering specific quality attributes (e.g., performance) from
architectural pattern descriptions. To be specific, the ontology contains two
subtypes of ontologies: one is English grammar-based ontology, which is
further categorized into promotes verb, modal verb, etc, and the other is
performance ontology that defines performance-specific concepts (e.g.,
throughput). The mined architectural in this subcategory can facilitate
developers to select architectural patterns through knowing whether specific
quality attributes are promoted or inhibited.

  * •

Architectural tactic-Quality attribute relationship: Software systems
typically have multiple QAs, and ATs provide established design solutions to
address these QAs (Bass et al., 2012). However, the information about the
relationships between ATs and QAs have not been explored systematically (Bi et
al., 2021b). To gather such QA-AT relationship information and help architects
make informed design decisions when they apply ATs to address QAs in practice,
Bi et al. [S6] developed a semi-automatic approach for mining QA-AT
discussions from practitioners (i.e., Stack Overflow) efficiently. Moreover,
the authors analyzed the mined QA-AT discussions to structure the design
relationships between ATs and QAs used in practice.

  * •

Design pattern-Architectural tactic relationship: Design patterns are often
used to design and implement architectural tactics for addressing QAs
(Mirakhorli et al., 2012a). Mirakhorli et al. reported that an architectural
tactic can be designed and implemented differently from one system to another
(Mirakhorli et al., 2012b). In other words, there are multiple ways for
designing and implementing an architectural tactic in a system for addressing
a specific QA. For example, the heartbeat tactic is a relatively simple tactic
used to monitor the availability of a critical component (Bass et al., 2012).
However, Mirakhorli et al. (2012b) observed numerous variations in how
heartbeat tactic could be implemented. Therefore, Mirakhorli et al. [S21]
proposed an approach for mining the way developers use design patterns within
the implementation of architectural tactics from OSS projects. Moreover, the
authors went on to analyze and mine architectural information, and they
reported various relationships between design patterns and architectural
tactics that other developers should consider.

Architectural technical debt: Architectural Technical Debt (ATD) is incurred
by architecture decisions that consciously or unconsciously compromise system-
wide QAs, particularly maintainability and evolvability (Li et al., 2014).
Typical ATD includes violations of architecture design principles or rules (Li
et al., 2015). ATD needs to be identified and removed since it is harmful to
the system’s long-term health (Li et al., 2015). 10 out of 87 studies
developed approaches for identifying and mining ATD for different purposes,
such as mining ATD for further management in order to keep the accumulated ATD
under control [S37]. We categorized the mined architectural information
related to ATD into three subcategories (see Table
LABEL:minedArchitecturalInfo).

  * •

Architectural compliance issue occurs when the implemented architecture
deviates away from the intended architecture. The phenomenon of divergence
between the intended and implemented architecture is regarded as architecture
erosion (Perry and Wolf, 1992). An eroded architecture can aggravate the
brittleness of the system and decrease architecture sustainability (Li et al.,
2022). For instance, a software system with an eroded architecture may lead to
the deterioration of the engineering quality of the system (De Silva and
Balasubramaniam, 2012), and make it difficult for developers to understand the
internal structure of the system (Perry and Wolf, 1992). Even though the
consequences of architectural conformance issues have been clearly defined in
the literature, there have been less attention to mining architectural
information related to architectural conformance issues in order to assist the
development. Only 4 out of 87 studies researched on mining architectural
compliance issues. For example, Maffort et al. [S34] presented an approach
that relies on four heuristic models for detecting and mining the absences
(i.e., something expected is not found) and divergences (i.e, something
prohibited is found) in source code based architectures. The mined
architectural compliance issues can be used to rapidly raise architectural
deviation warnings, without deeply involving architects.

  * •

Architectural anti-pattern may occur due to the violation of design principles
(Baldwin and Clark, 2000). Mo et al. [S64] presented an approach for
automatically detecting and mining six types of architecture anti-patterns
(e.g., unstable interface, modularity violation groups, and unhealthy
inheritance hierarchy), defined as connections among files that violate design
principles. Through analyzing the mined architectural anti-patterns, the
authors demonstrated that the identified architectural anti-patterns have a
significant impact on file bug-proneness and change-proneness. This mined
architectural information can be used by architects to pinpoint architectural
anti-patterns in the systems, quantify their severity, and determine
priorities for refactoring.

  * •

Architectural smell may be caused by applying architecture design solutions in
an inappropriate context, mixing design fragments that have undesirable system
behavior, or applying design abstractions at the wrong level of granularity
(Garcia et al., 2009). Díaz-Pace et al. [S71] developed an approach based on
Link Prediction (LP) techniques and an ML classification model to mine two
types architectural smells, namely cyclic dependency and hub-like dependency
from OSS projects.

General architectural information refers to the studies that have a generic
description about architectural information (e.g., design information [S17],
design discussion [S48], architectural information [S60]). In other words,
these studies do not explicitly specify a concrete category or subcategory of
mined architectural information that belongs to one of the abovementioned
categories or subcategories.

Key Findings of RQ1 Finding 1: 8 main categories and 30 subcategories of
architectural information are mined from various sources to assist
architecting activities. Finding 2: Architectural description (59.8%, 52 out
of 87 studies), architectural decision (24.1%, 21 out of 87 studies),
architectural solution (18.4%, 16 out of 87 studies), and system requirement
(18.4%, 16 out of 87 studies) are the top four most mined categories of
architectural information.

###  4.3 Results of RQ2: Sources used for Mining Architectural Information

We observed that the selected studies utilized various sources for mining
architectural information in order to support the development. We applied open
coding & constant comparison and descriptive statistics to analyze the
extracted data item D5 (see Table 4) for answering RQ2 and categorize the
reported sources into thirteen core categories (see Figure 5). Note that,
GitHub describes itself as “an Internet hosting service for software
development and version control using Git”, and in this SMS we opted to encode
it as a Version Control System (VCS). As shown in Figure 5, the most
frequently used source when mining architectural information is VCS, e.g.,
GitHub, (used by 51 studies), followed by Online Q&A sites, e.g., Stack
Overflow, (used by 33 studies), and Wiki (used by 21 studies). In the
following paragraph, we provide examples for the five most employed sources in
mining architecture information.

Ghorbani et al. [S30] collected a set of open source Java applications from
VCS (specifically, GitHub) and mined architectural inconsistencies from those
applications. Soliman et al. [S4] collected architecture related posts in
Online Q&A sites (specifically, Stack Overflow) and mined architectural
knowledge for technology decisions. Gilson et al. [S14] gathered data from two
sources (i.e., Wiki and Online Q&A sites (specifically, Stack Exchange)) for
mining alternative architectural solutions to a given architectural problem,
in order to provide assistance to the decision-making architect. Shahbazian et
al. [S35] crawled several open source projects from Issue tracking systems
(specifically, Jira) to detect and mine implementation issues that lead to
possibly unintentional design decisions and subsequently change the system’s
architecture. In [S78], Kamal and Avgeriou proposed a manual approach for
mining architectural pattern-architectural pattern relationships from books,
research articles, and white papers. Note that, since some studies used more
than one source, the sum of the sources reported in Figure 5 exceeds the total
number of the selected studies (i.e., 87 studies).

Figure 5: The sources used for mining architectural information

Key Findings of RQ2 Finding 3: VCS, e.g., GitHub, (used by 51 studies), Online
Q&A sites, e.g., Stack Overflow, (used by 33 studies), and Wiki (used by 21
studies) are the top three most frequently used sources for mining
architectural information.

###  4.4 Results of RQ3: Supported Architecting Activities

We collected the architecting activities that can be supported by the mined
architectural information (the results of RQ1). As mentioned in Section 3.3.2,
a hybrid approach (which is a combination of open coding & constant
comparison, predefined classifications of architecting activities by
Hofmeister et al. (2007), Tang et al. (2010), and Li et al. (2013), and
descriptive statistics) was used for analyzing the data item D6 to answer RQ3.
Hofmeister et al. (2007) presented a general model for architecture design,
which consists of three main activities: architecture analysis, architecture
synthesis, and architecture evaluation. Tang et al. (2010) extended this
general model by adding two architecting activities (i.e., architecture
implementation and architecture maintenance) in the architecture life cycle.
These two architecting activities emphasize that architecture is not only an
important asset for design but also for the later stage in the software
development life cycle. Li et al. (2013) extended the aforementioned
architecting activities by adding five more activities (i.e., architecture
recovery, architectural description, architecture understanding, architecture
impact analysis, and architecture reuse) from the perspective of Architecture
Knowledge (AK) management (e.g., AK reuse) to support the architecture life
cycle.

As mentioned above, we followed the ten predefined architecting activities
when answering RQ3. However, through our qualitative data analysis (see
Section 3.3.2), we obtained two additional architecting activities, namely
“architecture conformance checking” and “all activities” (means that a study
can support all the eleven architecting activities reported in this SMS).
Thus, we used twelve architecting activities to answer RQ3 in this SMS. Table
LABEL:MappingBetweenArchActivitiesMinedArchInfoAndStudies presents the
frequency of occurrence of the 12 architecting activities supported by the
mined architectural information. Moreover, Table
LABEL:MappingBetweenArchActivitiesMinedArchInfoAndStudies provides the
relationships between mined architectural information and supported
architecting activities as well as relevant studies. These relationships act
as a panorama to comprehensively understand the state of the research in
mining architectural information. As shown in Table
LABEL:MappingBetweenArchActivitiesMinedArchInfoAndStudies, the top three most
supported architecting activities are architecture understanding (43 out of 87
studies), architecture recovery (28 out of 87 studies), and architectural
description and documentation (23 out of 87 studies). We describe the twelve
supported architecting activities below.

Table 6: Relationships between mined architectural information and supported architecting activities Supported architecting activity | Mined architectural information | Studies |  Count (%)
---|---|---|---
Architecture Understanding |  Architectural description, Architectural decision, System requirement, Architectural solution, Architectural change, Architectural technical debt, General architectural information, Design relationship |  [S14] [S18] [S19] [S21] [S25] [S26] [S28] [S29] [S30] [S32] [S33] [S34] [S35] [S36] [S37] [S38] [S39] [S40] [S41] [S42] [S46] [S47] [S50] [S51] [S52] [S57] [S58] [S59] [S60] [S61] [S64] [S65] [S67] [S68] [S69] [S71] [S72] [S73] [S74] [S77] [S80] [S82] [S86] | 43 (49.4%)
Architecture Recovery |  Architectural solution, Architectural description, System requirement, Architectural change, Architectural decision, Architectural technical debt, General architectural information |  [S1] [S12] [S17] [S19] [S24] [S26] [S28] [S33] [S35] [S36] [S37] [S39] [S40] [S41] [S46] [S47] [S51] [S53] [S57] [S59] [S69] [S73] [S75] [S77] [S80] [S86] | 28 (32.2%)
Architectural Description and Documentation |  Architectural solution, System requirement, Architectural description, Architectural change, Architectural decision, General architectural information |  [S1] [S7] [S11] [S14] [S19] [S21] [S25] [S29] [S30] [S32] [S38] [S39] [S40] [S47] [S50] [S51] [S52] [S58] [S60] [S61] [S63] [S73] [S77] [S80] | 24 (27.6%)
Architecture Maintenance and Evolution |  Architectural decision, Architectural solution, Architectural description, Architectural technical debt, Architectural change, System requirement, Design relationship |  [S5] [S19] [S25] [S29] [S30] [S32] [S36] [S37] [S41] [S46] [S57] [S59] [S64] [S65] [S67] [S68] [S69] [S71] [S74] [S76] [S79] | 21 (24.1%)
Architecture Implementation |  Architectural decision, General architectural information, Architectural description, Architectural solution |  [S7] [S13] [S19] [S20] [S21] [S27] [S32] [S33] [S34] [S35] [S38] [S49] [S56] [S63] [S65] [S73] [S79] [S84] | 18 (20.7%)
Architecture Analysis |  System requirement, Architectural description, Architectural technical debt, General architectural information, Architectural decision |  [S2] [S8] [S14] [S29] [S31] [S32] [S37] [S42] [S43] [S45] [S55] [S66] [S76] [S79] [S83] | 14 (17.2%)
Architecture Reuse |  Architectural decision, System requirement, Architectural solution, Architectural description, Architectural change, General architectural information |  [S1] [S13] [S19] [S24] [S25] [S26] [S28] [S33] [S36] [S55] [S56] [S60] [S78] [S80] [S82] | 15 (16.1%)
Architecture Evaluation |  General architectural information, Architectural solution, Architectural decision, System requirement, Architectural description |  [S1] [S2] [S3] [S4] [S7] [S14] [S15] [S34] [S36] [S55] [S79] [S87] | 12 (13.8%)
Architecture Synthesis |  Architectural solution, System requirement, Architectural decision, Architectural description, Architectural change, Design relationship |  [S1] [S2] [S3] [S4] [S7] [S10] [S14] [S15] [S31] [S38] [S79] | 11 (13.9%)
Architecture Impact Analysis |  General architectural information, Architectural change, Architectural solution, System requirement, Architectural description |  [S32] [S34] [S35] [S39] [S40] [S41] [S47] [S49] [S55] [S57] | 10 (11.5%)
Architecture Conformance Checking |  Architectural description, Architectural technical debt |  [S12] [S30] [S34] [S51] [S73] [S81] [S85] [S86] | 8 (9.2%)
All Activities |  Architectural description, General architectural information |  [S16] [S44] [S54] | 3 (3.4%)

Architecture Understanding (AU) is normally conducted to comprehend elements
of an architecture design, such as architectural solutions and relationships
between architectural elements (e.g., the relationships between architectural
components (Stevanetic and Zdun, 2014)), as well as the corresponding design
decisions (why the architecture is designed the way it is). AU assists
architects and concerned stakeholders to get a comprehensive knowledge about
the architecture of a system (Li et al., 2013). The information gained during
AU can be used as input for other architecting activities (such as
architecture analysis and implementation) (Bengtsson et al., 2004). The AU
activity is supported by 49.4% (43 out of 87) of the studies, and eight types
of architectural information (e.g., Architectural description, Architectural
decision, System requirement) were mined to assist AU (see Table
LABEL:MappingBetweenArchActivitiesMinedArchInfoAndStudies). For example, In
[S19], Bhat et al. developed an approach and a prototype tool (Amelie -
Decision Explorer (ADeX)) for automatic curation design decision knowledge for
architectural decision recommendations. One component of Amelie tool can mine
architectural decisions from project artifacts (e.g., project description in
MS Excel files, issues), and this information can support stakeholders to get
a comprehensive understanding of the entire project’s architectural elements.
In addition, Kazman et al. [S37] developed an automatic approach for
extracting and mining the files implicated in architecture flaws (i.e.,
architectural debt) from the development artifacts. This mined architectural
information (i.e., architectural debt) can guide developers to explore and
understand the underlying architecture of a system for further analysis (e.g.,
quantifying the return on investment of removing those architecture flaws).

Architecture Recovery (ARec) is mainly performed to recover architectural
elements from system implementation (Riva, 2000) or documentation (Shahin et
al., 2013). The ARec activity is supported by 32.2% (28 out of 87) of the
studies, and seven types of architectural information (such as Architectural
solution, Architectural description, System requirement) were mined to aid
ARec (see Table LABEL:MappingBetweenArchActivitiesMinedArchInfoAndStudies).
For instance, Shahbazian et al. [S47] developed an approach named RecovAr that
relies on two existing techniques (i.e., Algorithm for Comprehension-Driven
Clustering (ACDC) (Tzerpos and Holt, 2000) and Architecture Recovery using
Concerns (ARC) (Garcia et al., 2011)) for automatically recovering design
decisions from the readily available history artifacts of projects, such as
issue tracker and version control repository. In [S53], Musil et al. developed
an approach named Continuous Architectural Knowledge Integration (CAKI) that
combines the continuous integration of internal and external Architecture
Knowledge (AK) sources (such as Wiki, Technical blogs and tutorials,, Q&A
sites) together with enhanced semantic reasoning and personalization
capabilities dedicated to large organizations. The evaluation results showed
that CAKI potentially reduces AK search effort by concurrently yielding more
diverse and relevant results. Furthermore, CAKI facilitates architects to
efficiently recover and gradually explore relevant AK from various AK sources.

Architectural Description and Documentation (AD): Architecture is described
and documented using a collection of architectural elements, such as
architectural views and models (Clements et al., 2003). The main goals of AD
are facilitating the expression and evolution of software systems, providing a
blueprint for system development, and supporting the communication between
stakeholders (ISO/IEC/IEEE, 2011). The AD activity is mentioned in 27.6% (24
out of 87) of the studies, and six types of architectural information (such as
Architectural solution, System requirement, Architectural description) were
mined to support ARec (see Table
LABEL:MappingBetweenArchActivitiesMinedArchInfoAndStudies). In [S60], Happel
et al. presented an approach and a prototype tool named Ontobrowse that can
allow software engineers (e.g., architects and developers) to search and
document architectural elements (descriptions of services, serviceLayers,
interfaces, domain concepts, and architectural design rules) of a Service-
Oriented Architecture (SOA) based system from different sources, for instance,
issue tracking system and Subversion.

Architecture Maintenance and Evolution (AME): Architecture may evolve during
its lifetime to respond to changing requirements (Tang et al., 2010).
Architectural maintenance is to adjust the architecture according to changes
and faults (Tang et al., 2010). The AME activity is supported by 24.1% (21 out
of 87) of the studies, and seven types of architectural information (such as
Architectural decision, Architectural solution, Architectural description,
Architectural technical debt) were mined to support AME (see Table
LABEL:MappingBetweenArchActivitiesMinedArchInfoAndStudies). In [S37], Kazman
et al. presented a tool called Titan for mining architectural technical debts
(specifically, architectural anti-patterns), which incur high maintenance
penalties. The mined architectural information can help architects and
developers locate architectural anti-patterns in the systems during
architectural maintenance, determine the penalty incurred by these debts, and
the expected benefits if the debts are removed.

Architecture Implementation (AI): During this activity, developers transform
and implement architecture design into code (Tang et al., 2010). The AI
activity is supported by 20.7% (18 out of 87) of the studies, and four types
of architectural information (such as Architectural decision, General
architectural information) were mined to facilitate AI (see Table
LABEL:MappingBetweenArchActivitiesMinedArchInfoAndStudies). For example, Ali
et al. [S79] developed an ML-based approach for mining AK (such as system
requirements, design decisions, architectural solutions) from Stack Overflow
in order to assist five architecting activities. Among other usages, the mined
AK, including detailed design of module (such as module decomposition,
coupling, cohesion), can be utilized by developers during AI.

Architecture Analysis (AA) aims to define the problems that an architecture
needs to address. During AA, Architecturally Significant Requirements (ASRs)
are identified from given architectural concerns and context (Hofmeister et
al., 2007). The AA activity is supported by 17.2% (15 out of 87) of the
studies, and five types of architectural information (such as System
requirement, Architectural description) were mined to aid AA (see Table
LABEL:MappingBetweenArchActivitiesMinedArchInfoAndStudies). In [S42], Gilson
et al. proposed an approach for extracting and mining quality attributes from
use stories in agile development projects. The mined architectural information
guides developers to get relevant quality attributes and potential
architectural key drivers during AA, and provides a “bigger picture” of
potential architectural drivers for early architecture decision making.

Architecture Reuse (AReu) is to reuse existing architectural elements, such as
architectural solutions, architectural decisions and their rationale for
addressing various architectural problems, and this activity can help achieve
an architecture of better quality at a lower cost (Li et al., 2013). The AReu
activity is facilitated by 16.1% (14 out of 87) of the studies, and six types
of architectural information (such as Architectural decision, System
requirement, Architectural solution) were mined to support AReu (see Table
LABEL:MappingBetweenArchActivitiesMinedArchInfoAndStudies). In [S1], Soliman
et al. used a semi-automatic approach to mine AK, including architectural
solutions (e.g., architectural patterns and tactics) and architectural
decisions, from issue tracking systems. Among other support, the mined AK can
support practitioners (e.g., architects) to determine architectural scenarios,
in which the reuse of the AK (e.g., architectural solutions) from issue
tracking system could be the most suitable.

Architecture Synthesis (AS): In this architecting activity, a set of candidate
architectural solutions are proposed to address the ASRs that have been
gathered during AA, and the AS activity essentially links the problem space to
the solution space during architectural design (Hofmeister et al., 2007). The
AS activity is supported by 12.6% (11 out of 87) of the studies, and six types
of architectural information (such as Architectural solution, System
requirement, Architectural decision) were mined to aid AS (see Table
LABEL:MappingBetweenArchActivitiesMinedArchInfoAndStudies). For instance, in
[S4], Soliman et al. used a qualitative analysis approach to mine
architectural knowledge for technology decisions from Stack Overflow and
classified the AK into several categories, including solution synthesis. The
mined AK in the solution synthesis category provides developers a set of
technology solutions, which have certain characteristics (such as technology
features, quality attribute evaluation) for addressing architectural concerns.

Architecture Evaluation (AE) aims to assess the architectural solutions that
are proposed during AS against the ASRs (Hofmeister et al., 2007). During the
AE activity, various factors are considered, such as pros and cons, trade-
offs, and constraints of architectural solutions that result in the selection
of appropriate architectural solutions for satisfying ASRs. The AE activity is
supported by 13.8% (12 out of 87) of the studies, and five type of
architectural information (such as General architectural information,
Architectural solution, Architectural decision) were mined to guide AE (see
Table LABEL:MappingBetweenArchActivitiesMinedArchInfoAndStudies). For
instance, Andrzej et al. [S14] proposed an Architecture Decision Support
System (ADSS) prototype tool that can mine various types of architectural
information (e.g., architectural solutions, architectural decisions and their
rationale) from diverse sources, such as Q&A sites and Wiki. The mined
architectural information can help architects evaluate (among others)
candidate architectural solutions to a given architectural problem by making
architectural decisions when conducting AE.

Architecture Impact analysis (AIA) aims to identify directly affected and
indirectly-influenced elements of an architecture due to an architectural
change scenario (Bengtsson et al., 2004). The outcome of this activity aids
architects to comprehend the dependencies between the changed elements and the
affected elements in the architecture (Li et al., 2013). The AIA activity is
supported by 11.5% (10 out of 87) of the studies, and five types of
architectural information (such as General architectural information,
Architectural change, Architectural solution) were mined to aid AIA (see Table
LABEL:MappingBetweenArchActivitiesMinedArchInfoAndStudies). For example, Anish
et al. [S55] developed an ML-based approach for automatically identifying and
mining ASRs from requirements documents, and classifying the mined ASRs into
several subcategories based on the types of architectural impacts the mined
ASRs can have on the system components, which helps architects perform AIA.

Architecture Conformance Checking (ACC) is conducted to check and evaluate if
the implemented architecture conforms to the intended architecture, and this
activity can help architects identify and correct architectural violations and
further avoid constant architecture erosion during the software development
life cycle (Pruijt et al., 2013). The ACC activity is supported by 9.2% (8 out
of 87) of the studies, and two types of architectural information (i.e.,
Architectural description, Architectural technical debt) were mined to
facilitate ACC (see Table
LABEL:MappingBetweenArchActivitiesMinedArchInfoAndStudies). For example,
Ghorbani et al. [S30] presented DARCY, an approach that automatically detects
and mines inconsistent module dependencies within Java applications. The mined
architectural information can be used to check the inconsistency and evaluate
the consistency in the implementation of architecture when conducting ACC.

All Activities means that the mined architectural information can support all
the abovementioned eleven architecting activities. Only 3.4% (3 out of 87) of
the studies claimed that they can support all the aforementioned architecting
activities, and two types of architectural information (such as Architectural
description) were mined (see Table
LABEL:MappingBetweenArchActivitiesMinedArchInfoAndStudies). For instance, in
[S54], Weinreich and Buchgeher presented an approach named Language for
Integrated Software Architecture (LISA), which consists of the LISA model and
the LISA toolkit to facilitate the eleven abovementioned architecting
activities. The LISA model supports the description of both static and dynamic
structures at different levels of abstraction, from classes, packages and
modules to components. Among other things, LISA model supports the description
of architectural solutions, architectural decisions, and architecting
activities. The LISA toolkit, which is an extension of the Eclipse IDE, can
facilitate the incremental and concurrent development of requirements,
architecture, and implementation. For example, to facilitate AA, the approach
can be used to capture ASRs mined from various sources (such as issue
management systems).

Key Findings of RQ3 Finding 4: Twelve architecting activities can be supported
by the mined architectural information, in which architecture understanding
(49.4%, 43 out of 87 studies), architecture recovery (32.2%, 28 out of 87
studies), and architectural description and documentation (27.6%, 24 out of 87
studies) are the top three most supported architecting activities. A few
numbers of the selected studies (i.e., 3, 3.4%) have proposed approaches for
mining architectural information to support all the architecting activities
reported in this SMS.

###  4.5 Results of RQ4: Approaches and Tools

We describe the approaches and tools used for mining architectural information
in Section 4.5.1 and Section 4.5.2, respectively.

####  4.5.1 Approaches

The selected studies have proposed and used different approaches to mine
architectural information from different sources. We analyzed and synthesized
the approaches used for mining architectural information by following the
coding steps for data synthesis as described in Section 3.3.2. We observed
that some selected studies did name their approaches used to mine
architectural information and others did not. If a study does not provide a
name for the proposed approach, we labelled it as “Unnamed” (see Table
LABEL:SammazationOf_Autom_Approaches and Table LABEL:SammazationOfSemi-
Autom_Approaches). Moreover, we found that the proposed approaches have
different level of automation (e.g., automatic, semi-automatic). In addition,
these approaches have been applied in different types of tasks (e.g.,
classification, clustering) for mining architectural information from various
sources. Thus, in this SMS, we utilized a two-level categorization to
categorize the approaches for mining architectural information. At the first
level, we divided the approaches in three high-level categories based on the
level of automation: (1) automatic, (2) semi-automatic, and (3) manual. At the
second level, we further categorized the approaches in three categories based
on different types of tasks in mining architectural information: (1)
architectural information classification, (2) architectural information
clustering, and (3) architectural information retrieval. In total, we gathered
89 approaches from the selected studies of which 58.4% (i.e., 52 out of 89)
(see Table LABEL:SammazationOf_Autom_Approaches) are automatic approaches,
35.9% (i.e., 32 out of 89) are semi-automatic approaches (see Table
LABEL:SammazationOfSemi-Autom_Approaches), and 6.2% (5 out of 89) are manual
approaches (see Table LABEL:SammazationOfManual_Approaches). Architectural
information classification based approaches count 56.2% (50 out of 89),
followed by clustering based approaches (25.8%, 23 out of 89), and information
retrieval based approaches (17.9%, 16 out of 89). In this SMS, one selected
study may make use of two or more approaches, and consequently the number of
collected approaches (89) is greater than the number of the selected studies
(87). Furthermore, to help and ease the applicability of these approaches in
practice, we also present the sources that the proposed approaches can be
applied on.

Architectural information classification refers to the approaches that mine
architectural information based on classification techniques. Architectural
information classification based approaches are the most frequently 56.2% (50
out of 89) used approaches in mining architectural information. Classification
techniques learn from the data input provided to them and then use this
learning to classify new observations (Kotsiantis et al., 2006). There are two
types of architectural information classification based approaches that were
employed in the selected studies: binary classification based approaches
(e.g., [S6][S48][S59]) and multi-class classification based approaches (e.g.,
[S40][S41][S76][S79]). Architectural information classification based
approaches are applied either automatically (see Table
LABEL:SammazationOf_Autom_Approaches), semi-automatically (see Table
LABEL:SammazationOfSemi-Autom_Approaches), or manually (see Table
LABEL:SammazationOfManual_Approaches). For example, in [S40], Mondal et al.
proposed an approach named Archinet for mining and classifying architectural
changes from code and commit messages in GitHub. Specifically, they formulated
this problem as a multi-class classification task that automatically
classifies architectural changes into four categories. In [S6], Bi et al.
developed a semi-automatic dictionary-based mining approach to extract Quality
Attribute (QA) and Architectural Tactic (AT) related discussions in Stack
Overflow (SO) posts. They formulated this problem as a binary classification
task, which automatically identifies QA-AT related discussions from SO posts.
Then the authors went on to manually structure the design relationships
between QAs and ATs used in practice, and build a knowledge base of how
developers use ATs with respect to QA concerns. In [S6], Malavolta et al.
employed a manual approach (i.e., qualitative analysis) to mine and elicit
evidence-based architectural guidelines for open-source ROS-based software
from GitHub, BitBucket, and GitLab. Their qualitative data analysis yielded 39
guidelines for architecting robotics software, which can be used by
roboticists to architect their systems to achieve particular quality
requirements.

Architectural information clustering denotes the approaches that employ
clustering based techniques to mine architectural information from various
sources. Architectural information clustering based approaches count 25.8% (23
out of 89 approaches). The clustering techniques aim at reducing the amount of
data by categorizing or grouping similar data items together into subsets or
clusters (Saxena et al., 2017). The goal is to create clusters with internal
coherence, placing similar objects in the same group, and assigning dissimilar
objects to different groups. In this sense, documents that belong to a certain
cluster should be as similar as possible and dissimilar from documents in
other clusters. We identified two types of architectural information
clustering based approaches employed in the selected studies: partitional
clustering (e.g, [S43]) and hierarchical clustering (e.g., [S68]). In
addition, these approaches operate in two ways: automatic and semi-automatic
(see Table LABEL:SammazationOf_Autom_Approaches and Table
LABEL:SammazationOfSemi-Autom_Approaches).

Partitional clustering approaches output an initial partition with a certain
number of clusters. Specifically, partitional clustering approaches divide a
dataset into a number of groups based upon a certain criterion known as
fitness measure (Nanda and Panda, 2014). The fitness measure directly affects
the nature of the formation of clusters. Once an appropriate fitness measure
is selected, the partitioning task is converted into an optimization problem
(e.g., grouping based on minimization of distance or maximization of
correlation between patterns) (Nanda and Panda, 2014). For example, Casamayor
et al. [S43] used a partitional clustering based approach to semi-
automatically mine potential responsibilities of components in a software
system to be developed. Specifically, NLP techniques and the K-means algorithm
have been applied to automatically cluster candidate responsibilities into
groups. Firstly, this approach processes requirements documents by the Part-
Of-Speech (POS) tagging technique to detect actions, activities, or tasks that
will become responsibilities of certain components in the architecture of a
system. Afterward, K-means is utilized to group (i.e., partitional clustering)
similar responsibilities into architectural components.

Hierarchical clustering approaches work by iteratively merging smaller
clusters into larger ones, or by splitting larger clusters onto small ones
(Saxena et al., 2017). The key point is the rule used by the approach to
decide which small clusters are to merge or which larger clusters are to
split. The result is a tree of clusters showing how clusters are related. The
output of hierarchical clustering is a hierarchy, a structure that is more
informative than an unstructured set of clusters (Saxena et al., 2017). For
instance, in [S68], Mitchell and Mancoridis presented an approach that
automatically searches and hierarchically groups highly interdependent modules
into same subsystems/clusters and, conversely, groups independent modules into
separate subsystems/clusters. The approach was developed to help software
engineers perform a variety of large and complex system understanding and
maintenance activities.

Architectural information retrieval covers the approaches that employ
Information Retrieval (IR) based techniques (Singhal, 2001) to search and
retrieve architectural information in various sources. Around (17.9%, 16 out
of 89) of the approaches are based on IR techniques, which have been utilized
to (semi-)automatically search and retrieve architectural information by using
two types of searching mechanisms, namely keyword-based search and semantic
search.

On the one hand, keyword-based search approaches do not understand polysemy
and synonymy (Guha et al., 2003). Specifically, when looking at a repository,
a keyword-based approach looks for the distribution of documents within the
repository to find how relevant a document is to the search query of the user
(Guha et al., 2003). Basically, this means that a document with similar
words/terms to those the user types into a search engine will be thought to be
more relevant and will appear at a higher position in the search results (Guha
et al., 2003). For example, Aman-ul-haq and Ali Babar [S23] proposed a tag-
based/annotation-based knowledge identification and extraction approach. One
of the functionalities of this approach is the keyword-based search
functionality, which can search and retrieve desired architectural artifacts
in the repository using the keywords attached to each architectural artifact.
This approach has been developed to reduce the time and effort required for
searching and capturing architecture knowledge in a software repository.

On the other hand, semantic search-based approaches understand polysemy and
synonymy and know the meaning of the words/terms (Guha et al., 2003).
Specifically, semantic search-based approaches are designed to understand the
context the words/terms are used within the documents in order to match these
words/terms more accurately to the user search queries (Guha et al., 2003).
For instance, in [S13], Mujhid et al. presented an approach based on IR and
program analysis techniques for searching and reusing architectural tactics
implemented in OSS projects. Among other algorithms used by this approach, the
approach uses a novel information retrieval algorithm that automatically
searches and ranks the source files implementing architectural tactics in the
software repositories based on (i) the semantic similarity of a source file to
a searched architectural tactic and (ii) the semantic similarity of a source
file and its direct dependent files to a technical problem represented in the
search query. The approach was developed to help developers search and reuse
implementation examples of architectural tactics for a given technical
context.

Table 7: Summarization of tasks, automatic approaches, used sources, and mined architectural information from the selected studies Task |  Automatic approaches |  Used source |  Mined architectural information |  Studies
---|---|---|---|---
Architectural information classification (33 approaches) |  Unnamed |  Q&A sites |  General architectural information |  [S5]
Unnamed |  Q&A sites |  Design relationship (i.e., Component-Quality attribute relationships) |  [S10]
Unnamed |  Version control systems |  Component-Component relationship |  [S12]
Unnamed |  Mixed artifacts (Q&A sites, Technical blogs and tutorials, etc.) |  Architectural solution (i.e., architectural tactic) |  [S15]
Unnamed |  Chat messages |  General architectural information |  [S16]
|  Unnamed |  Version control systems |  General architectural information |  [S17]
|  Unnamed |  Version control systems |  General architectural decision, Architectural description (i.e., Architectural view), General architectural change |  [S18]
|  Unnamed |  Mixed artifacts (Wiki, Issue tracking systems, etc.) |  Architectural decision (i.e., Structural, Behavioral, Ban or non-existence decisions) |  [S19]
|  Unnamed |  Q&A sites |  Design relationship (i.e., Component-Component relationship) |  [S20]
|  Unnamed |  Issue tracking systems |  Architectural decision (i.e., Structural, Behavioral, Ban or non-existence decisions) |  [S22]
|  Unnamed |  Wiki |  General architectural information |  [S23]
|  Unnamed |  Wiki |  General system requirement, General architectural decision, Architectural description (i.e., Architectural rationale), Architectural solution (i.e., Architectural tactics, architectural patterns) |  [S25]
|  Unnamed |  Version control systems |  Design relationship (i.e., Architectural tactic-Context relationship) |  [S27]
|  DARCY |  Version control systems |  Architectural technical debt (i.e., Architectural compliance issue), Architectural description (i.e., Architectural model) |  [S30]
|  Formal elicitation |  Wiki |  General architectural decision |  [S33]
|  Lightweight top-down capture |  Wiki |  General architectural decision |  [S33]
|  Lightweight bottom-up capture |  Version control systems |  General architectural decision |  [S33]
|  Unnamed |  Issue tracking systems |  Architectural description (i.e., Architectural view), General architectural decision, General architectural change |  [S35]
|  ArchiNet |  Version control systems |  Architectural change (i.e., Perfective change, Corrective change, Adaptive change, Preventative change) |  [S40]
|  Unnamed |  Mixed artifacts (Version controls systems, Developer mailing lists) |  Architectural change (i.e., Perfective change, Corrective change, Adaptive change, Preventative change) |  [S41]
|  Unnamed |  Wiki |  System requirement (i.e., Quality attributes) |  [S42]
|  Unnamed |  Mixed artifacts (Q&A sites, Issue tracking systems, Chat messages) |  General architectural information |  [S48]
|  Unnamed |  Version control systems |  General architectural information |  [S49]
|  Unnamed |  Wiki |  System requirement (i.e., Functional requirements) |  [S55]
|  Unnamed |  Code search engine |  Architectural solution (i.e., Architectural tactics) |  [S56]
|  Unnamed |  Version control systems |  Architectural solution (i.e., Architectural tactics) |  [S57]
|  IMEAV |  Wiki |  General architectural information |  [S62]
|  NFR2AC |  Wiki |  Architectural description (i.e., Architectural concerns), System requirement (i.e., Quality attributes) |  [S66]
|  Unnamed |  Version control Systems |  Architectural technical debt (i.e., Architectural smells) |  [S71]
|  SOMAD |  Version control systems |  Architectural technical debt (i.e., Architectural anti-patterns) |  [S72]
|  Unnamed |  App Stores |  System requirement (i.e., Quality attributes, Functional requirements) |  [S76]
|  Unnamed |  Q&A sites |  General architectural information |  [S79]
|  DPWord2Vec |  Mixed artifacts (Q&A sites, Wiki, Online books, research articles, and white papers) |  Architectural pattern |  [S82]
Architectural information clustering (10 approaches) |  Unnamed |  Version control systems |  Architectural solution (i.e., Architectural patterns), Architectural technical debt (i.e., Architectural compliance issues) |  [S34]
POPCON |  Version control systems |  Architectural description (i.e., Architectural models), Architectural technical debt (i.e., Architectural compliance issues) |  [S36]
Unnamed |  Mixed artifacts (Version control system, Issue tracking systems) |  Architectural technical debt (i.e., Architectural anti-patterns), Architectural description (i.e., Architectural models, Architectural views) |  [S37]
ARCADE |  Version control systems |  Architectural description (i.e., Architectural views, Architectural models), General architectural change |  [S46]
RecoverAr |  Mixed artifacts (Version control system, Issue tracking systems) |  Architectural description (i.e., Architectural views, Architectural models, Architectural rationale), General architectural decision, General architectural change |  [S47]
|  Architecture Mining |  Version control systems |  Architectural description (i.e., Architectural models, System of interest) |  [S51]
|  Unnamed |  Mixed artifacts (Version control systems, Issue tracking systems) |  Architectural description (i.e., Architectural models, Architectural views), Architectural technical debt (i.e., Architectural anti-patterns) |  [S64]
|  R3 |  version control systems |  Architectural description (i.e., Architectural models, Architectural views) |  [S65]
|  ROMANTIC |  Version control systems |  Architectural description (i.e., Architectural models) |  [S75]
|  mACS |  Version control systems (i.e., GitHub) |  Architectural anti-pattern |  [S84]
Architectural information retrieval (9 approaches) |  ArchEngine |  Version control systems |  Architectural solution (i.e., Architectural tactics) |  [S13]
Unnamed |  Wiki, Q&A sites |  General architectural information |  [S14]
SAK-SIR |  Mixed artifacts (Technical blogs and tutorials, Online books, research articles, and white papers) |  General architectural information |  [S32]
Unnamed |  Q&A sites |  Architectural description (Architectural rationale), General architectural decision |  [S38]
Unnamed |  Wiki |  General architectural information |  [S44]
|  CAKI |  Mixed artifacts (Wiki, Q&A sites, Technical blogs and tutorials, etc.) |  General architectural information |  [S53]
|  Ontobrowse |  Version control systems |  General architectural information |  [S60]
|  Unnamed |  Version control systems |  Architectural description (i.e., Architectural models), Design relationship (i.e., Component-Component relationships) |  [S67]
|  Unnamed |  Version control systems |  Design relationship (i.e., Component-Component relationships) |  [S74]
Table 8: Summarization of tasks, semi-automatic approaches, used sources, and mined architectural information from the selected studies Task |  Semi-automatic approaches |  Used source |  Mined architectural information |  Studies
---|---|---|---|---
Architectural information classification (12 approaches) |  Unnamed |  Issue tracking systems |  System requirement (i.e., Quality attribute, Functional requirement), General architectural decision, Architectural description (i.e., Architectural rationale), General architectural change, Architectural solution (Architectural pattern, architectural tactic) |  [S1]
Unnamed |  Mixed artifacts (Version control systems, Q&A sites, Technical blogs and tutorials, Online books, research articles, and white papers, etc.) |  General architectural information |  [S2]
Unnamed |  Q&A sites |  General architectural information |  [S4]
Unnamed |  Q&A sites |  Design relationship (i.e., Architectural tactic-Quality attribute relationship) |  [S6]
Unnamed |  Version control systems |  Design relationship (i.e., Design pattern-Architectural tactic relationship) |  [S21]
|  TREx |  Mixed artifacts (Chat messages, Wiki, etc.) |  Architectural description (i.e., Architectural rationale) |  [S24]
|  Unnamed |  Wiki |  Design relationship (i.e., Architectural pattern-Quality attribute relationship) |  [S31]
|  Unnamed |  Mixed artifacts (Wiki, Technical blogs and tutorials, etc.) |  Architectural description (i.e., Architectural view) |  [S52]
|  LISA |  Mixed artifacts (Issue tracking system, Wiki) |  Architectural description (i.e., Architectural model, Architectural view) |  [S54]
|  Keecle |  Version control systems |  Architectural description (i.e., Architectural view), Design relationship (i.e., Component-Component relationship) |  [S59]
|  Unnamed |  Version control systems (i.e., GitHub) |  Architectural decision |  [S85]
|  DCP Miner |  Online Q&A sites (e.g., Stack Overflow) |  Architectural pattern-Context relationship |  [S86]
|  Unnamed |  Version control systems (i.e., GitHub) |  Architectural description (i.e., architectural model) |  [S87]
Architectural information clustering (12 approaches) |  Unnamed |  Mixed artifacts (Version control system, Wiki, Q&A sites) |  Architectural solution (i.e., Framework for addressing scalability, Architectural tactics), General architectural decision, Architectural description (i.e., System of interest, Architectural rationale) |  [S26]
Unnamed |  Version control systems |  Architectural description (i.e., Architectural model, System of interest), Architectural technical debt (Architectural smell) |  [S29]
MicroART |  Version control systems |  Architectural description (i.e., Architectural model, Architectural view, System of interest) |  [S39]
Unnamed |  Wiki |  System requirement (i.e., Functional requirement) |  [S43]
Unnamed |  Q&A sites |  Design relationship (i.e., Architectural pattern-Architectural pattern relationship), Architectural solution (i.e., Architectural pattern) |  [S63]
|  Unnamed |  Version control systems |  Architectural description (i.e., Architectural model, Architectural view) |  [S68]
|  Unnamed |  Version control systems |  General architectural information |  [S69]
|  Unnamed |  Version control systems |  Architectural description (Architectural model), Architectural technical debt (i.e., Architectural compliance issue) |  [S73]
|  Unnamed |  Version control systems |  General architectural change |  [S77]
|  ROSDiscover |  Version control systems (i.e., GitHub) |  Architectural model |  [S80]
|  Arch-KDM |  Version control systems (i.e., GitHub) |  Architectural conformance |  [S81]
|  Unnamed |  System requirement (i.e., Functional requirements, Quality attributes) |  Wiki |  [S83]
Architectural information retrieval (7 approaches) |  Unnamed |  Q&A sites |  General architectural information |  [S3]
Unnamed |  Mixed artifacts (Chat messages, Wiki, etc.) |  General architectural information |  [S28]
MaRK-II |  Wiki |  System requirement (i.e., Functional requirement) |  [S45]
KaitoroCap |  Wiki |  General architectural information |  [S50]
Semantic-based |  Mixed artifacts (Online books, research articles, and white papers) |  Architectural solution (i.e., architectural pattern, tactic), system requirement (i.e., quality attributes) |  [S58]
|  unnamed |  Wiki |  General architectural information |  [S61]
|  unnamed |  Wiki |  General architectural information |  [S70]
Table 9: Summarization of tasks, manual approaches, used sources, and mined architectural information from the selected studies Task |  Manual approaches |  Used source |  Mined architectural information |  Studies
---|---|---|---|---
Architectural information classification (5 approaches) |  Qualitative data analysis |  Mixed artifacts (Version control systems, Q&A sites, ROS answers sites) |  Architectural solution (i.e., Architectural tactic) |  [S7]
Qualitative data analysis |  Version control systems |  System requirement (i.e., Quality attribute), General architecture solution, General architectural decision, Architectural description (i.e., Architectural view, Architectural concern) |  [S8]
Qualitative data analysis |  Mixed artifacts (Version control systems, Q&A sites, ROS answers sites) |  Architectural solution (i.e., Architectural tactic) |  [S9]
Qualitative data analysis |  Developer mailing lists |  System requirement (i.e., Quality attribute), Architectural description (i.e., Architectural model, Architectural rationale, Architectural concern, System of interest) |  [S11]
Qualitative data analysis |  Online books, research articles, and white papers |  Design relationship (i.e., Architectural pattern-Architectural pattern relationship) |  [S78]

####  4.5.2 Tools