*RQ-1: Which topics are addressed by successful plug-ins?* Developers often use several and synonymous terms to describe their plug-ins, hampering an automated categorization. Thus, we manually assessed the purpose of the 100 most downloaded plug-ins and derived 10 initial categories: Revision control, IDE extension (integrating programming languages or frameworks), code analysis, build tool, user interface, database, editor, optimization (of Eclipse), documentation, and server. As we show in Fig. 1, revision control and

**Fig. 1.** Categorized projects of the 100 most downloaded plug-ins in the Eclipse marketplace.

IDE extensions accumulate most downloads. However, we also see that there is a difference in the number of available plug-ins for these two categories.

Only 8 plug-ins address revision control but are responsible for over a quarter of all downloads. This might be explained with a small set of established approaches, similar to the situation for user interfaces. Still, this result can be distorted, for instance because communities may require identical tooling, forcing participants to use a specific system even if they prefer another one.

In contrast, IDE extensions, similar to several remaining categories (e.g., code analysis, or build tools), provide far more plug-ins (28) while accumulating fewer downloads. A potential explanation is that these plug-ins often require adaptations to specific programming languages and development processes. Hence, their overall applicability is limited to specific communities and they compete with other IDEs.

*RQ-2: Who contributes to these plug-ins?* To gain an impression of Eclipse's community, we categorized the owners of plug-ins into four groups: open-source, private, industrial, and university. As we see in Fig. 1, open-source communities are leading in revision control and build tools, accumulating most plug-ins and downloads. Especially in revision control, which is used for distributed work and collaboration, they seem to benefit from their diversity [5]. However, opensource communities do not, or rarely, contribute to some other topics in our sample.

In contrast, private and industrial owners provide plug-ins for almost all topics. Private developers seem to dominate the development of user interfaces, optimization, and server integration, which facilitate using Eclipse. However, such plug-ins provide less utility besides comfort than, for example, customized build tools or revision control. For this reason, industrial and open-source communities might be less interested in developing these. We support this argument due to the fact that industrial owners provide most plug-ins in IDE extensions, databases, and documentation. These are essential aspects of software development in companies. Finally, we remark that universities own only a single plug-in in our sample, potential reasons being that they provide innovative but immature projects.

#### **4 Research Agenda**

The results presented in this work are preliminary and further analysis on how such marketplaces are composed are necessary. More detailed investigations may help to understand, why communities do or do not provide plug-ins and what motivates them. We emphasize that this requires extensive analyses and additional empirical studies. In future work, we aim to assess the following aspects:

*How to assess the success of a plug-in?* In this paper, we solely focused on the number of overall downloads. This is a significant limitation and other metrics are necessary to provide a more detailed view. For instance, to consider the marketplace's evolution, a plug-in's downloads in recent periods, integration into Eclipse packages, or the number of its developers are interesting.

*Are there common characteristics of successful plug-ins?* Different communities successfully participate in the Eclipse marketplace. Hence, success may depend on the topic and also certain characteristics, for example, used licenses [13,17,18], necessity to pay, or maturity. For community managers and developers such information are important to design and provide their software.

*How do users select plug-ins?* The previous two questions may indicate, and can be validated by investigating how, users select plug-ins. Besides these points, the users' selection also depends on their experiences and background. The results can help to understand how new techniques emerge and establish.

*Why and how are plug-ins developed?* An important question in the context of open-source software is the motivation of developers. In particular, it is interesting who initiates plug-ins for which reason, who contributes to these, or which connections exist. The corresponding results may provide insights into motivations to collaborate and participate in communities and open marketplaces.

#### **5 Related Work**

Several authors investigate the evolution of Eclipse or its plug-ins, focusing, for instance, on their architecture or API usage [3,4,14,22]. Further works investigate the laws of software evolution [12] in open-source systems [8,11]. In contrast to the systems themselves, our scope is how a potential marketplace for these evolves and is composed. Still, both approaches are complementary and can be combined, for instance to assess how a plug-ins evolution affects its popularity and status. Finally, Murphy et al. [15] empirically evaluate how developers use the Eclipse IDE. It seems interesting to utilize this approach to also assess how plug-ins are used.

### **6 Conclusions**

Eclipse is one of the most prominent and widely used Java IDEs. Due to its success, a large and diverse community of plug-in developers established. They provide their plug-ins at the Eclipse marketplace, allowing other users to use them.

In this paper, we proposed to analyze the Eclipse marketplace. The preliminary results show that some topics, such as revision control or IDE extensions, are often demanded by users and that different owners participate. Finally, we described further research directions to deepen the understanding of open marketplaces and their communities.

**Acknowledgments.** This research is supported by DFG grant LE 3382/2-1 and Volkswagen Financial Services AG. We thank Heike Fischbach for commenting on drafts of this paper.

### **References**

**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

# Licensing, Strategies, and Practices

### **How are Developers Treating License Inconsistency Issues? A Case Study on License Inconsistency Evolution in FOSS Projects**

Yuhao Wu1(B) , Yuki Manabe2, Daniel M. German3, and Katsuro Inoue1

1 Graduate School of Information Science and Technology, Osaka University, Osaka, Japan<EMAIL_ADDRESS>2 Faculty of Advanced Science and Technology, Kumamoto University, Kumamoto, Japan<EMAIL_ADDRESS>3 Department of Computer Science, University of Victoria, Victoria, Canada<EMAIL_ADDRESS>
**Abstract.** A license inconsistency is the presence of two or more source files that evolved from the same original file containing different licenses. In our previous study, we have shown that license inconsistencies do exist in open source projects and may lead to potential license violation problems. In this study, we try to find out whether the issues of license inconsistencies are properly solved by analyzing two versions of a FOSS distribution—Debian—and investigate the evolution patterns of license inconsistencies. Findings are: license inconsistencies occur mostly because the original copyright owner updated the license while the reusers were still using the old version of the source files with the old license; most license inconsistencies would disappear when the reusers synchronize their project from the upstream, while some would exist permanently if reusers decide not to synchronize anymore. Legally suspicious cases have not been found yet in those Debian distributions.

**Keywords:** Software license *·* Code clone *·* License inconsistency

#### **1 Introduction**

Free and open source software (FOSS) is software that can be freely used, changed, and shared (in modified or unmodified form) by anyone. FOSS should always be distributed under certain open source software licenses, otherwise they cannot be reused by others. The license of a source file usually resides in its header comment.

Many studies in software engineering have been done on software license. Some approaches for software license identification have been proposed [3,6,8]. Vendome et al. investigated the reasons on when and why developers adopt and change licenses by conducting a survey with the relevant developers [10].

We have developed a method to detect *license inconsistencies* in large-scale FOSS projects [11]. A license inconsistency is defined as two or more source files that evolved from the same original file containing different licenses. For example, file A and file B have the same program code, but the header of A includes GPL-2.0 and B includes Apache-2.01. We have applied our detection method to Debian 7.52, one of the Linux distributions, and discovered different reasons that caused license inconsistencies and determined whether they are potentially illegal or not.

Although this work has provided us clues that some license inconsistencies indicate potential legal issues, this one-time analysis is not enough to understand how developers are dealing with the issues: whether developers are putting their efforts on eliminating license inconsistencies or they simply ignore the issues. The investigation of the evolution of license inconsistencies can help us understand whether the legal issues of license inconsistency discovered in the previous work are handled by developers. Therefore, we conducted this research and try to answer the following questions:

In order to address these questions, we apply our license inconsistency detection method to Debian 8.2 (which was the newest version when we started this study) in addition to Debian 7.5, and investigate the evolution of license inconsistencies between theses two versions.

The rest of this paper is organized as follows: Sect. 2 introduces the methodology we employ to address our research questions. Section 3 describes the result we got from our analysis. A discussion of this result is given in Sect. 4. Section 5 introduces the related work. Finally, we conclude our paper in Sect. 6.

1 Note that pairs of no license and any license, and different license versions are reported as license inconsistency.

2 https://www.debian.org/.

#### **2 Methodology**

In this section we describe how we design our study.

#### **2.1 Obtain License Inconsistency Groups for Debian 7.5 and Debian 8.2**

A *license inconsistency group* is a group of multiple files that have same code contents but with different licenses. As introduced in our previous paper [11], we use the following steps to reveal license inconsistency groups:

We apply these steps to Debian 7.5 and 8.2 respectively and obtain the file groups that contain license inconsistencies.

#### **2.2 Compare the Difference of Groups**

All the files in one group have the same token sequences, from which we calculate the hash value as the id for that group. We then compare the id of the groups in each version and get the set of groups that exist: (*a*) only in Debian 7.5; (*b*) only in Debian 8.2; (*c*) in both versions.

The result of this step would be three sets of file groups.

**Table 1.** Number of packages and files in Debian 7.5 and 8.2.

aGroups that exist in both versions

bGroups that exist in either version only

#### **2.3 Investigate the Groups Manually**

The groups that only exist in Debian 7.5 imply that they are eliminated from the new version; those that only exist in Debian 8.2 imply that new license inconsistency cases appears in this new version; those that exist in both versions indicate that they remain during these two versions. We manually examine the different groups and present the results in the next section.

#### **3 Results**

The number of packages and files in Debian 7.5 and 8.2 are shown in Table 1. We only consider .c, .cpp and .java files which are the supported types of our detection method. The detection result of license inconsistency groups in Debian 7.5 and 8.2 is shown in Table 2. In this table, intersection means that both two versions of Debian contain that group of license inconsistency; relative complement means only that version of Debian contain that group of license inconsistency. Thus there are 4062 groups of license inconsistencies detected in both versions; 2701 groups only in Debian 7.5; and 2947 groups only in Debian 8.2. By examining the groups that are only in Debian 7.5 we can find out how and why license inconsistencies disappeared in Debian 8.2; while examining the groups that are only in Debian 8.2 we can understand how and why license inconsistencies appeared. The intersection part indicates that these groups of license inconsistencies persisted in Debian 8.2.

**Fig. 1.** A case of license inconsistency evolution which involves project make, remake and kbuild.

#### **3.1 Why Do License Inconsistencies Appear?**

There are several reasons that license inconsistencies appear. The key point is that there are multiple copies of the same source file with different licenses in the same distribution. We describe the general reasons that license inconsistency appears, thus the examples given in this section are not limited to those that appear in Debian 8.2. We categorize them into three types according to the reason that caused the copy of files.

– **Internal copy-and-paste of source files but their licenses are different.** If copies of the same file exist in one project (we call these internal copies), they should also exist in the final distribution (e.g. Debian 7.5). Thus a case of license inconsistency will be reported by our method if they contain different licenses.

For example, in a project named FreeMedForms, some source files in a plugins directory are under BSD3 license. These files are copied to other directories with their licenses changed to GPL-3.0+.

This type of license inconsistency could exist for just a short time, if the difference of license was due to mistakes and was later on fixed by developers. On the other hand, if developers decided to distribute these source files under different licenses, it would exist permanently.

– **Different versions of the same project are included in the same distribution.** Similar to the previous reason, if different versions of the same project which causes license inconsistencies are included in the final distribution, those license inconsistencies will be reported by our method.

For example, there is a project named groovy which contains files that had no license in version 1.7.2, and were then added a Apache-2.0 license in version 1.8.6. Both of these two versions of this project are included in Debian 7.5, thus this license inconsistency is reported.

– **Upstream and downstream projects both exist in the same distribution.** In this research, if project B reuses source files from project A by copy-and-paste, we call project B the downstream project and project A the upstream project. While the previous two reasons are about the same project, this reason involves multiple projects. Files from the upstream project are reused in the downstream project, and the license of these files were changed either by the original author or the reuser. If both of these projects are included in the same distribution, the license inconsistency will be reported by our method.

An example is shown in Fig. 1. As we can see from this graph, several source files in remake and kbuild project were originally from the make project, where license upgrade occurred in year 2010. Debian 7.5 includes older versions of make and remake, where the license was still GPL-2.0, while the newer version of kbuild contains the GPL-3.0 license. Thus license inconsistency is reported for this case.

**Fig. 2.** A case of license inconsistency evolution which involves project EasyMock and Mockito.

#### **3.2 Why Do License Inconsistencies Persist?**

License inconsistencies usually exist in several continuous versions of a distribution. Some of them disappear soon, some exist for a long time, while some even persist forever. From the result in this research, we can summarize them into two reasons.

– **Source files are not yet synchronized, but license inconsistencies will eventually disappear when they are.** This type of license inconsistencies occurred because the downstream project has not yet been synchronized with the upstream project where the license of source files were changed. However, since the developers of the downstream projects are still synchronizing the project from the upstream regularly, this type of license inconsistencies will be eliminated eventually.

For example, in the case of the project JSON-lib and jenkins-json described earlier, although license inconsistencies appeared in Debian 7.5 (where jenkins-json still uses source files under Apache-2.0/MIT dual license), they disappeared in Debian 8.2, where developers of jenkins-json project synchronized from the upstream project and the license all become Apache-2.0 only. Though this case of license inconsistency disappeared in Debian 8.2, we consider it as a typical example to explain why license inconsistencies would exist for only a period of time and disappear when the source files are synchronized.

– **Downstream project no longer synchronizes from upstream, and license inconsistencies will likely exist permanently.** In this case, developers of downstream project chose to no longer synchronize from the upstream project, thus the license inconsistencies are likely to exist forever, unless the synchronization resumes.

As shown in Fig. 2, among the results we found a project named Mockito which copy-and-owned several source files from a project named EasyMock. The license of these files in the upstream project were changed from MIT to Apache-2.0 in year 2009, however the Mockito project still uses the original MIT license. Besides, Mockito project made some changes to the source code of these files by their own, and never again synchronized from EasyMock. After checking the history of these files in Mockito project, we found that one of the commit in year 2007 contains the following commit message: "*umbilical cord between mockito package and easymock package is cut!* ", which implies that they will never synchronize from the upstream project. Thus this case of license inconsistency is likely to exist permanently, unless Mockito project decides to synchronize from EasyMock again.

#### **3.3 Why Do License Inconsistencies Disappear?**

We observed several cases where license inconsistencies disappeared from Debian 8.2. The reasons are summarize as follows.

– **Downstream project synchronized from upstream project.** When the downstream project synchronized from the upstream project, the license of the source files becomes the same, thus the license inconsistencies disappear.

Again, from Fig. 1 we can see that Debian 8.2 updated all these three projects to a newer version where all of their licenses are upgraded to GPL-3.0, thus this license inconsistency disappears.

The case of project JSON-lib and jenkins-json discussed earlier also applies here.

– **The source code that contained the license inconsistency was removed or changed—thus no longer identical.** In this research since we only inspect identical files, only files that contain the same token sequences are considered that they are from the same origin. If the source code of a file changed dramatically which made their token sequences different from the corresponding files, or if the relevant source file was removed in the new version, then the license inconsistencies will no longer be reported by our method.

For example, there is a file in project icu which is under IBM copyrights in both versions of Debian. This file was reused in project openjdk-7 but with a GPL-2.0 license in Debian 7.5. Thus this case of license inconsistency was reported in Debian 7.5. However, the source code of the file in openjdk-7 was changed in Debian 8.2 while the license remained the same. Our method no longer not consider them as file clones since they have different token sequences, thus the license inconsistency disappears in the newer version of Debian.

### **4 Discussion**

#### **4.1 Revisiting the Research Questions**

The answer to our research questions are as follows:

**RQ1:** *What are the evolution patterns of license inconsistencies and the underlying reasons?*

**RQ2:** *Is the issue of license inconsistency properly handled by developers?* The evolution of license inconsistencies in Debian shows that, they are mainly caused by distribution latency and will be eliminated when the downstream projects get synchronized. We found license inconsistencies appear mostly because the original author modified the license while the reusers were still using the old version of the file; they disappear because the files are synchronized with the upstream projects; and they persist if the downstream projects were not synchronized.

#### **4.2 Effectiveness of This Approach**

This approach is effective to analyze how developers are addressing the issues of license inconsistency in a certain software ecosystem. As shown in the results, we could use this method to reveal how license inconsistency groups evolved over time. By analyzing different groups we could have a basic idea on how developers are treating license inconsistency issues. For example, by analyzing the license inconsistency groups that only exists in the old version, we could learn why license inconsistencies disappeared in the newer version and what efforts did the developers put to eliminate them.

We could apply this same approach to other software systems to study how different communities are dealing with license inconsistency issues.

#### **4.3 Threats to Validity**

**Internal Validity.** We use the same methodology as our previous study to detect license inconsistency cases, which relies on the token sequence generation from CCFinder and the license identification from Ninka. For file clone detection, we use normalized token sequences as the metric to decide clones. If source files are modified a lot (e.g. add/remove several statements), they might not be recognized as clones. We could use approaches that detect similar source files to mitigate this problem. Regarding license identification, Ninka is state-of-the-art license identification tool which has an accuracy of 93% [3]. As shown in Sect. 3, our manual analysis also proves its high accuracy and precision.

**External Validity.** This study focuses on the license inconsistency issues in Debian. The results and analysis may not be generated to other software systems. And we plan to study other software systems in the future.

#### **5 Related Work**

Many studies in software engineering have been done on software license. Some approaches for software license identification have been proposed [3,6,8]. Using these approaches, some researches analyzed software licenses in open source projects and revealed some license issues. Di Penta et al. provided an automatic method to track changes occurring in the licensing terms of a system and did an exploratory study on license evolution in six open source systems and explained the impact of such evolution on the projects [2]. German et al. proposed a method to understand licensing compatibility issues in software packages [4]. They mainly focused on the compatibility between license declared in packages and those in source files. In another research, they analyzed license inconsistencies of code siblings (a code clone that evolves in a different system than the code from which it originates) between Linux, FreeBSD and OpenBSD, but they did not explain the reasons underlying these inconsistencies [5]. Alspaugh et al. proposed an approach for calculating conflicts between licenses in terms of their conditions [1]. Vendome et al. performed a large empirical study of Java applications and found that changing license is a common event and a lack of traceability between when and why the license of a system changes [9]. In their following research [10], they investigated the reasons on when and why developers adopt and change licenses during evolution of FOSS Java projects on GitHub by conducting a survey with the relevant developers. They concluded that developers consider licensing as an important task in software development.

In our previous research [11], we proposed a method to detect license inconsistencies in large-scale FOSS projects. We then applied this method to Debian 7.5 and examined the results manually and discovered various reasons that caused these license inconsistencies, among which some were legally suspicious and deserved further investigation. As far as we know, no research has been done to investigate the evolution of license inconsistencies.

#### **6 Conclusions**

In this research we have applied our license inconsistency detection method to both Debian 7.5 and Debian 8.2. By comparing the results of these two versions of Debian, we identified three evolution patterns of license inconsistencies. With a manual analysis of the license inconsistency cases we discovered various reasons that caused the evolution. Although license inconsistencies are detected in both versions of Debian, from our manual analysis we concluded that these reported license inconsistencies are caused because the upstream project updated the license while the downstream project has not been synchronized yet. These findings suggest that in our target ecosystem, Debian, license inconsistencies are caused by distribution latency and they will disappear when the downstream projects get synchronized.

For future work, we would apply this approach to other software ecosystems and see whether there are different patterns of license inconsistency evolution and whether the license inconsistency issues are properly handled.

**Acknowledgments.** This work is supported by Japan Society for the Promotion of Science, Grant-in-Aid for Scientific Research (S) "Collecting, Analyzing, and Evaluating Software Assets for Effective Reuse"(No.25220003) and Osaka University Program for Promoting International Joint Research, "Software License Evolution Analysis".

#### **References**

1. Alspaugh, T., Asuncion, H., Scacchi, W.: Intellectual property rights requirements for heterogeneously-licensed systems. In: Proceedings of the 17th International Requirements Engineering Conference (RE2009), pp. 24–33 (2009)

**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

### **Addressing Lock-in, Interoperability, and Long-Term Maintenance Challenges Through Open Source: How Can Companies Strategically Use Open Source?**

Björn Lundell1(✉) , Jonas Gamalielsson1 , Stefan Tengblad1 , Bahram Hooshyar Yousefi1 , Thomas Fischer1 , Gert Johansson2 , Bengt Rodung3 , Anders Mattsson4 , Johan Oppmark5 , Tomas Gustavsson6 , Jonas Feist7 , Stefan Landemoo8 , and Erik Lönroth9

> 1 University of Skövde, Skövde, Sweden {bjorn.lundell,jonas.gamalielsson,stefan.tengblad<EMAIL_ADDRESS>2 Combitech AB, Linköping, Sweden<EMAIL_ADDRESS>3 Findwise AB, Gothenburg, Sweden<EMAIL_ADDRESS>4 Husqvarna AB, Huskvarna, Sweden<EMAIL_ADDRESS>5 JAK, Skövde, Sweden<EMAIL_ADDRESS>6 PrimeKey Solutions AB, Solna, Sweden<EMAIL_ADDRESS>7 RedBridge AB, Stockholm, Sweden<EMAIL_ADDRESS>8 Saab AB, Linköping, Sweden<EMAIL_ADDRESS>9 Scania IT AB, Södertälje, Sweden<EMAIL_ADDRESS>
**Abstract.** This industry paper reports on how strategic use of open source in company contexts can provide effective support for addressing the fundamental challenges of lock-in, interoperability, and longevity of software and associated digital assets. The fundamental challenges and an overview of an ongoing collab‐ orative research project are presented. Through a conceptual model for open source usage in company contexts we characterise how companies engage with open source and elaborate on how the fundamental challenges can be effectively addressed through open source usage in company contexts.

#### **1 Introduction**

Over the years, researchers have identified a number of different motivations for compa‐ nies and individuals to engage with open source [4, 21, 29]. Open Source Software (OSS) is software made available under a software license which has been approved by the Open Source Initiative [25] and with OSS the nature of development and competition in the software sector has significantly impacted on this sector [7]. This, in turn, has led to increased opportunities for organisations basing their business on open source and in 2007 it was found that in the context of the Norwegian software industry "more than 30% of the companies get over 40% of their income from OSS related services or soft‐ ware" [11]. Further, open source goes beyond software and important aspects of open source have influenced licensing in the hardware domain [12] and as a development model for standardisation [19].

Open source constitutes an early exemplar of open innovation [20] and its strategic use by companies is increasing [23]. Software maintained in open source projects is external to specific company contexts. Therefore, to maintain control of R&D resources it is critical for any company to participate and utilise effective work practices for involvement in open source communities, something which calls for technical and stra‐ tegic considerations. Hence, companies must be prepared to counter and adapt their way of working in order to stay competitive in an increasingly competitive market.

Building and organising communities around OSS projects involves a number of challenges (e.g. [1, 3, 8]) and previous research shows "that effective governance and work practices that are appreciated by community members is fundamental for longterm sustainability" for open source projects [8]. A number of OSS projects are governed by a foundation (e.g. the Eclipse Foundation, the Linux Foundation, and the Document Foundation), which "are associations that serve to protect, not define, their respective communities' core technologies as well as the advancement of the commons through technical, social, and educational services" [9]. Previous studies have analysed Eclipseprojects and investigated how "software firms can apply different types of governance approaches to open source software development projects" [27]. Further, several widely adopted OSS projects are also governed in other ways and many companies seek guid‐ ance concerning how to establish effective strategies for governance and company involvement in open source projects.

This industry paper makes three principle contributions. First, we elaborate on the three fundamental challenges: lock-in, interoperability, and long-term maintenance of software and associated digital assets. Second, we present an overview of an industrial collaborative research project which aims to establish effective strategies for how companies can (and should) strategically engage with open source projects in order to successfully address the three fundamental challenges. Third, through an evolved conceptual model and associated strategies for company involvement with open source we illustrate how companies seek to strategically use and engage with open source projects in different ways.

#### **2 On Fundamental Challenges: Lock-in, Interoperability, and Long-Term Maintenance**

Over the years we have witnessed more widespread deployments of complex IT-systems and associated increased demands for longevity of software [8, 19]. This causes organ‐ isations to vary concerning different types of lock-in and inability to provide long-term maintenance of critical systems and digital assets [16, 19]. Further, OSS is often deployed in a legacy situation where there is a lot of proprietary software and closed file formats. This leads to the need for migrations between different file formats and establishment of interoperable systems. Interoperability supports systems heterogeneity, thereby increasing options for organisations [2, 10, 13].

Use of an open standard, especially when implemented in OSS, contributes to inter‐ operability in that it "ensures that data and systems can be interpreted independently of the tool which generated it" [17]. This is relevant for scenarios in both commercial [18] and public sector [19] organisations. The Swedish National Procurement Services has published a list of open standards, which all can be referenced as mandatory require‐ ments in public procurement [24]. An open standard is a standard which conforms to the definition presented in the European Interoperability Framework (EIF) version 1.0 [24] and such standards can be implemented and distributed under different licenses for proprietary software and under all OSI-approved licenses for OSS.

To allow for competition and stimulate innovation the Framework agreements estab‐ lished by the Swedish National Procurement Services [24] require that only open stand‐ ards can be referenced as mandatory requirements in public procurement. A recent study commissioned by the Swedish Competition Authority found that many IT-projects in the Swedish public sector refer to closed standards which cannot be implemented in open source software, and the Director General for the Swedish Competition Authority states in the foreword to the report from the study that "From a competition perspective it is often problematic when public sector organisations conduct IT procurement and express requirements for closed standards" [19]. Further, the study shows that there is a widespread practice to refer to standards, trademarks, and proprietary software and IT-solutions which only specific manufacturers and suppliers can provide [19].

Many companies and organisations need to preserve their systems and associated digital assets for more than 30 years [22], and in some industrial sectors (e.g. avionics) even more than 70 years [3, 26]. In such usage scenarios "there will be problems if the commercial vendor of adopted proprietary software leaves the market" with increased risks for long-term availability of both software and digital assets [22].

Open source projects with healthy ecosystems can be an appropriate way to address risks related to lock-in and long-term maintenance of commodity software [15]. Further, use of open standards reduces the risk to an organisation of being technologically lockedin since they increase control by supporting migration, thereby reducing an organisa‐ tion's reliance on a single product or supplier [6, 10, 13, 16, 28]. Further, previous research claims that support for interoperability should not require compatibility with existing software systems (since it tends to favour specific providers of those systems), but rather promote "interoperability with software from multiple vendors", something which can be achieved through use of OSS and open standards [10]. Despite such recommendations, recent results from a study commissioned by the Swedish Competi‐ tion Authority show that many Swedish governmental organisations undertake projects for development and procurement of IT-systems which inhibit competition and use of open source solutions since there are mandatory requirements for compatibility with specific proprietary technology and closed standards that (perhaps) unintentionally inhibit interoperability and create a number of very problematic lock-in effects.

There is limited prior research concerning effective strategies for how companies can successfully address the three challenges through use and involvement in open source software projects. The ongoing collaborative research project LIM-IT is one notable exception which aims to advance the existing body of knowledge concerning effective strategies for company involvement with open source in order to successfully address these challenges.

#### **3 Addressing Fundamental Challenges Through Open Source: An Overview of the LIM-IT Project**

The overarching goal of the LIM-IT project is to develop, use, and scrutinise effective work practices and strategies for development, procurement, and organisational imple‐ mentation of software systems in a number of complex application domains, where such software systems with associated digital assets (provided in a number of different open and closed standards and file formats) typically involve several open source projects as well as proprietary software, often depending on many different legacy systems. To address this goal, practitioners and researchers are committed to investigate the following core question: *"How can companies develop and utilise effective work prac‐ tices for achieving long*-*term strategic benefits from participation in open collaborative projects?"*

To successfully address the core question there is a need to thoroughly understand motivations and expectations from all relevant (private and public sector) stakeholder groups, which inherently encompass a complex web of interdependencies (with a variety of different, sometimes conflicting, motivations and goals) between different stake‐ holders involved in, and affected by, a coopetative marketplace, i.e. a marketplace with competition on top of (open) collaboration.

Researchers from different research specialisations (including open source, software engineering, organisational science, and intellectual property rights) address the funda‐ mental challenges from multi-disciplinary perspectives in collaboration with practi‐ tioners in a number of innovative organisations. Partner organisations include (small and large) companies from the primary software sector that all have extensive experience from development and use of (proprietary and open source) software, as well as software intensive organisations from the secondary software sector, representing businesses in different domains such automotive, avionics, finance, and outdoor power products. The primary software sector companies are all service providers with many years of expe‐ rience from providing services and delivering open source solutions to the private- and public sector. Several of these are international and have also been contracted in frame‐ work agreements by the Swedish National Procurement Services at Kammarkollegiet (a governmental authority) for delivering open source solutions to the Swedish public sector organisations in public procurement. Amongst the companies in the software intensive secondary sector, several are recognised as globally leading enterprises in their respective fields.

The research project seeks to investigate how the fundamental challenges lock-in, interoperability, and long-term maintenance of software and associated digital assets can be successfully addressed in different usage context. Investigation involves system‐ atic scrutiny, through use of an action-case study approach [5], of a number of usage contexts which are of relevance for partner organisations.

### **4 Strategic Use of Open Source in Different Company Contexts**

When companies succeed in establishing long-term symbiotic relationships with external open source communities and projects, such relationships can significantly strengthen the company's own missions in addition to strengthen involved open source communities which govern development in open source projects that may be of strategic importance for the own company.

In Fig. 1 we present a conceptual model for how companies can exploit, and engage with, open source based on previous and ongoing research. The model, which initially was presented in the context of product lines [14], has evolved through collaborative research with representatives for partner organisations involved in the LIM-IT project. Arrows (labelled 1, 2, 3, 4a, 4b, and 5 in Fig. 1) represent principle strategies (strategies 1, 2, 3, 4a, 4b, and 5 in Table 1) for how companies can exploit, and engage with, open source in order to seek long-term benefit for their own organisation in different usage contexts. It should be noted that our research has in particular evolved strategy 4a compared to the initial model [14].

**Fig. 1.** Leveraging Open Source opportunities

Through conduct of research in the LIM-IT project, we have for all these strategies identified a number of specific instances for how companies exploit and are engaged with open source in different usage contexts in the primary and secondary software sector. By drawing from the identified instances in which partner companies exploit and engage with open source, we present some illustrative examples of the identified instances in order to characterise all principle strategies which have been evolved in collaboration with partner companies. The presented strategies and illustrative examples show how companies utilise (and can utilise) open source software for long-term stra‐ tegic benefits.

**Table 1.** Principle strategies for how a company can engage with open source projects

a Stable releases from open source projects that are deployed for use in different contexts are often referred to as software products

All five principle ways for use and engagement with open source projects can be identified amongst the partner organisations in the LIM-IT research project. All compa‐ nies have, to some extent, adopted open source work-practices into their own develop‐ ment process (strategy 1), and all use a variety of different open source software in their own company context (strategy 2). For example, several companies use the Eclipsebased modelling tool Papyrus in development projects undertaken in their own company context, and companies in the primary software sector also provide services and training related to specific open source tools. Further, companies also internally use the open source licensed ROS (Robotic Operating System) for simulations in their own devel‐ opment processes and there are also experiences from use of many other open source licensed solutions, such as Apache Solr, Elastic, PostgreSQL, and MongoDB.

Companies use various open source implementations in their own software and hardware offerings (strategy 3). For example, open source implementations of the LWM2 M protocol is adopted for wireless communication with machines for use in different product offerings and several companies include several open source software components in solutions provided to customers. Several companies use a combination of proprietary and open source software, and organisations have experience from distri‐ bution of solutions under different terms (including open source software, SaaS, and a combination of proprietary and open source software).

Amongst partner organisations there are experience from releasing proprietary soft‐ ware as open source software (strategy 4b) and also from providing contributions to established open source projects (strategy 4a), such as Apache Solr, SignServer, and EJBCA.

Organisations seek to establish a long-term symbiotic relationship between different interests in open and closed contexts which is beneficial for both companies and open source projects (strategy 5). Such relationships can be effective means for influencing the formation of long-term goals and thereby support strategic collaborations. For example, organisations in the LIM-IT project has experience from involvement in open source industrial working groups, such as Eclipse and the Papyrus Industry Consortium. Further, amongst partners there are also experiences from initiation of and engagement with independent open source projects (e.g. SignServer and EJBCA) which include all relevant roles.

It should be noted that a combination of the above strategies may be utilised for utilising opportunities with open source in a specific company context. Besides strategic and technical benefits from utilising the five strategies for engaging with open source there are also other potential benefits. For example, from a perspective of personnel policy, a company engaged with open source may be perceived as a much more attractive employer, which may significantly ease recruitment of new staff.

### **5 Conclusion**

This paper has elaborated on how companies can strategically use open source for addressing lock-in, interoperability, and long-term maintenance of software and asso‐ ciated digital assets. Through a conceptual model for how companies can use and leverage from open source we have presented an overview of a collaborative research project which aims to establish effective strategies for how companies can use and stra‐ tegically engage with open source projects in different ways. In so doing, we have illus‐ trated how organisations, through adoption of different strategies, currently use and are involved with open source projects in order to achieve long-term strategic benefits for their own organisation.

Through systematic research investigations of how companies can successfully address the fundamental challenges lock-in, interoperability, and long-term maintenance of software and associated digital assets in different usage contexts we seek to advance practically useful recommendations for use and engagement with open source projects, whilst also contributing to advancing the existing body of knowledge in the field.

**Acknowledgements.** This research has been financially supported by the Swedish Knowledge Foundation (KK-stiftelsen) and participating partner organisations in the LIM-IT project. The authors are grateful for the stimulating collaboration and support from colleagues and partner organisations.

#### **References**

**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

### **Understanding the Effects of Practices on KDE Ecosystem Health**

Simone da Silva Amorim1(B) , John D. McGregor2, Eduardo Santana de Almeida3, and Christina von Flach Garcia Chavez3

1 Federal Institute of Education, Science and Technology of Bahia, Salvador, Bahia, Brazil<EMAIL_ADDRESS>2 Clemson University, Clemson, SC, USA<EMAIL_ADDRESS>3 Federal University of Bahia, Salvador, Bahia, Brazil<EMAIL_ADDRESS><EMAIL_ADDRESS>
**Abstract.** Open source software ecosystems have adjusted and evolved a set of practices over the years to support the delivery of sustainable software. However, few studies have investigated the impacts of such practices on the health of these ecosystems. In this paper, we present the results of an ethnographic-based study conducted during the Latin-American KDE users and contributors meeting (LaKademy 2015) with the goal of collecting practices used within the KDE ecosystem and understanding how they affect ecosystem health. The analysis was based on softgoal interdependency graphs adapted to represent practices and relate them to non-functional requirements and goals. Our results provide a preliminary insight to understand how KDE ecosystem community interacts, which working practices have been adopted and how they affect ecosystem health.

**Keywords:** Open source software ecosystems *·* Ethnographic studies *·* Software practices *·* Software ecosystem health

#### **1 Introduction**

The software ecosystem (SECO) strategy has been used with great success for several years [1,2]. This strategy has facilitated the emergence of communities surrounding hardware and software computing platforms, such as the Apple iOS community, and certain configurable products, such as the Eclipse Rich Client Platform. Each organization joining a SECO is looking to fill a role in the community that will foster its business objectives. The community members leverage the work of other members to grow bigger products faster than they could have on their own.

Within a SECO, one or more platform providers work to attract product developers to use the provider's platform as the foundation for products. The platform organization shares some degree of control over their platform, provides tools that make product development easier on their platform, and provides access to information about the current structure and future changes to the platform. The dominant organization in the ecosystem, usually the platform owner, heavily influences the culture of the ecosystem.

The *health* of a SECO stands for *the growing and continuity of the software ecosystem remaining variable and productive over time* [11]. The assessment of SECO health can be performed with respect to different aspects, including robustness, productivity, and niche creation [11]. Some studies present quantitative assessment on SECO health [13].

Practices can be useful to understand a SECO and assess its health. For instance, the practice *"review all code before accepting into the release"* may impact several quality indicators and contribute to increasing productivity while avoiding rework. However, to our knowledge, there is no qualitative study that focuses on software practices and their relation to SECO health.

In this paper, we present the results of an ethnographic-based study conducted with the purpose of investigating the practices used by a SECO from three perspectives [5] – business, social, and technical – and their impact on SECO health. The selected research method, ethnography, has been used to study software practices in different contexts [16,17]. The object of our study was KDE1, an open source SECO that gathers a set of platforms and products for Linux distributions. During the Latin-American KDE users and contributors meeting (LaKademy 2015), we collected day-to-day practices and analyzed them to understand their impact on SECO health. Softgoal interdependency graphs (SIG) [7] extended with practices, support the analysis of KDE practices and their influence on health aspects of SECO such as productivity and robustness.

The remainder of the paper is organized as follows: Sect. 2 presents background on SECO. Section 3 introduces SIG-P, our customized SIG [7]. Section 4 presents the context and design of our study. Section 5 presents the collected KDE practices and a preliminary analysis of their relation to KDE health. Section 6 presents findings, contributions and limitations of our work. Section 7 presents related work. Finally, Sect. 8 provides a brief description of future work and some concluding remarks.

### **2 Background**

The concept of *software ecosystem health* was introduced by Iansiti and Levien as *the growing and continuity of the SECO remaining variable and productive over time* [11]. They argued that the performance of an organization also depends on its interactions with the whole ecosystem – not only on the relationship with competitors and stakeholders and on the organizational potential. Iansiti and Levien drew an analogy with biological ecosystems, and argued that SECO health should be assessed based on three aspects: robustness, productivity, and niche creation [11].

1 https://www.kde.org/.

Robustness refers to the ability of the ecosystem to survive radical changes and problems resulting from these changes. The ecosystems should face and overcome all inevitable difficulties inherent in the evolutionary process. Productivity is related to the way in which the work is accomplished with the least waste of time and effort. The ecosystem should add value through innovation, management of resources and cost control. Lastly, niche creation represents SECO features that encourage and support diversity among the different species in the ecosystem. The ecosystem should provide the structure for creating new features over time, increasing the diversity among SECO members' products [11].

Campbell [5] introduces three views to classify the features of a SECO: technical, business and social. The Technical View presents the architecture and approaches to implementing software. The Business View captures business strategies at play in the ecosystem. In some cases, organizations will be in direct competition and making decisions on release dates and licensing in response to the actions of others within the ecosystem. The Social View captures relationships among humans: developers, users, managers and other participants.

SECO health should not be evaluated only by considering metrics. Practices also aid in understanding a project and its evolution with greater fidelity. They provide clear goals and forms to assess a good working performance to achieve a successful end [12]. For example, the practice *"Provide continuous integration tools to check code every day and report errors"* will impact several quality indicators as well as contribute to increasing productivity while avoiding rework.

#### **3 The NFR Framework with Practices**

The NFR framework [6,7] provides a systematic approach for defining and representing non-functional requirements (NFR) for software products. A set of goals represent NFR, design decisions and claims to support or not other goals. Goals are decomposed into subgoals and related with AND/OR relationships [7].

The term *softgoal* introduces the concept of *satisficing* [7]. Softgoals may contribute to some degree (positively (+, ++) or negatively (*−*, *−−*), fully or partially), in satisfying other softgoals. A Softgoal Interdependency Graph (SIG) models NFR and their interdependencies to achieve a softgoal (or desired quality). Design decisions are represented by operationalization softgoals. SIG diagrams provide a visual representation for SIG [7].

#### **3.1 SIG with Practices (SIG-P)**

SIG with Practices (SIG-P), our customized SIG, models SECO practices and their influence on SECO non-functional requirements. In SIG-P, each practice is modeled as a *satisficing technique* [7] and the degree of its influence on SECO health is documented.

Aspects of SECO health such as robustness, productivity, and niche creation [11] are modeled as *softgoals*. Each softgoal can be decomposed and refined

**Fig. 1.** SIG-P for business view

into mandatory and optional subgoals/NFR that influence directly the achievement of such aspects of SECO health. Practices are connected by interdependency links to NFR to indicate that they contribute to reach softgoals. Social, business and technical views [5] are presented in SIG-P diagrams.

SIG-P diagrams can be used to enhance general understanding about a SECO and its health. They provide a graphical representation for practices, their interdependencies and impacts on NFR and SECO aspects. Moreover, evaluators can build SIG-P models to reason and provide qualitative assessment on SECO health and use SIG-P diagrams to support discussions about correlations and trade offs among softgoals. Figure 1 presents an example of a SIG-P diagram. Details about this example are presented and discussed in Sect. 6.

#### **3.2 SIG-P Construction**

The construction of a SIG-P graph for an specific SECO view [5] starts with one or more ecosystem health aspects [11] and a set of practices used by some target SECO. A catalogue of documented NFR, such as the one provided in [7] is strongly recommended.

We defined three steps to guide the bottom-up construction of a SIG-P for each SECO view. Given a set of practices of a target SECO, a catalogue of NFR and SECO softgoals: (1) For each practice, identify a subset of NFR and consider its influence on each NFR; (2) For each identified NFR, analyze its influence on softgoals, considering ambiguities, trade offs and priorities; and (3) For each identified NFR, evaluate the impact (positive or negative) of NFR on softgoals.

These steps can be refined with many iterations considering the reasoning about NFR and their different alternatives. The SIG-P diagram from Fig. 1 illustrates a SIG-P for a business view.

#### **4 Methodology**

The KDE ecosystem2 is an international community that provides a Free Open Source desktop environment for many Linux distributions. The KDE ecosystem has several initiatives to promote their projects around world. In Brazil, a branch of the global organization gathers a team of KDE supporters. The Brazilian group promotes LaKademy (Latin American Akademy), an annual event at which KDE users and contributors meet in person to exchange ideas about projects and discuss the future of KDE. At this event, attendees participate in a set of activities such as lectures, hackathon sessions, and meetings on specific topics.

Hackathons are events that produce working software in a limited period of time. They consist of strenuous and uninterrupted intervals of coding, and are very useful as social and educational events [14]. Komssi *et al.* stated that hackathons' participants can also learn new practices, technologies, and attitudes, which can be used in everyday work [14]. During KDE events, hackathons provide informal ways to meet new people and learn new technologies.

#### **4.1 Research Method**

In this study, research techniques commonly used in ethnographic studies were adopted to support the researcher in understanding the practices and communication strategies KDE uses to work collaboratively. Most of the work in the context of open source SECOs, including KDE, is performed by distributed teams and contributors. Therefore, LaKademy provided an appropriate setting for conducting an ethnographically-inspired [9,18], short term study about KDE. The researcher could observe and participate in LaKademy through in-person meetings and activities.

To address practical challenges faced when conducting ethnographic studies [16], we joined the KDE community to gain better access to the setting of the practices and the use of a common vocabulary. We also conducted our investigation without unnecessary formality, explaining our research, who we were and what we were doing, and trying not to interfere with the activities of LaKademy. Verbal consent was captured on audio recordings at the beginning of sessions. Furthermore, we were rigorous in data collection and analysis to avoid bias.

The first days of Lakademy hackathons are used to disseminate KDE practices and knowledge about the ecosystem. Then developers follow a plan to work hard and deliver planned features. Therefore even knowing that developers are not working in their natural setting, we assume that many practices can be transferred to everyday work of participants.

2 http://www.kde.org/.

#### **4.2 Data Collection and Analysis Procedures**

On-site immersion and data collection took place at LaKademy 20153 event held in Salvador, from 3*rd* to 6*th* June, 2015. The event had 15 participants including students, professionals and KDE collaborators. During 4 days, we conducted semi-structured interviews, listened to lectures, analyzed KDE documents, and observed interactions of community members, mainly during hackathon sessions. Observations were documented in the form of field notes and audio records.

*Interviews.* Interviews were semi-structured, with 24 open-ended questions intended to understand the daily practices used in KDE ecosystem. Interviews were audio-recorded and later transcribed. We had interview sessions with 4 KDE members that have contributed actively to KDE for 6 years or more – three of them were members of KDE *e.V.* [3], a group that represents the KDE Community with respect to legal and financial issues. One of the interviewed KDE members is part of the KDE Board responsible for business management. We assumed that these people were heavily engaged and up to date regarding most of KDE decisions and practices. They also provided relevant information about the KDE community, contribution areas, financial issues and future plans. The questionnaire can be found at http://homes.dcc.ufba.br/∼ssamorim/lakademy/.

*Hackathons.* During LaKademy 2015, the researcher joined two hackathon sessions for removing bugs from two KDE applications: Cantor, a mathematical application, and the Plasma Network Manager. These sessions provided a good opportunity to observe developers with different levels of experience performing tasks with their own style and ask questions about reasons behind their actions. Some practices, not identified during interviews, were collected during hackathon sessions.

*Document Analysis.* The researcher also gathered references to online KDE documents. The community maintains a rich website that describes various aspects of KDE ecosystem. We had access to international and Brazilian websites and their documents. Thus, we could explore in detail the dynamics of interactions of the entire KDE ecosystem.

### **5 Results and Analysis**

#### **5.1 Practices in KDE**

We collected 68 practices used in KDE SECO. Although a practice could be associated to more than one SECO view (social, business, or technical), we linked it to a single view for which the practice seemed to be most influential. The collection of 68 practices was validated by two interviewees. Five practices (B14, T3, T21, T27, and T35) were not validated. We also asked the two interviewees to choose five important practices for each view.

3 https://br.kde.org/lakademy-2015.

Table 1 presents 10 Social Practices, related to working together in the community, selected by two interviewees. Accordingly, Table 2 presents 10 Business Practices, related to aspects of management, strategic planning, and innovation, and organized activities such as marketing, making decisions, and so on. Finally, Table 3 presents 10 Technical Practices (out of 40 practices), related to product development (core and applications), technologies used, code rules, and others. Other practices are available at http://homes.dcc.ufba.br/∼ssamorim/ lakademy/.

**Table 1.** Social Practices

**Table 3.** Technical Practices

#### **5.2 Analysis**

We resorted to SIG-P diagrams to understand and analyze the effects of identified practices on KDE health. SIG-P diagrams were built, one for each SECO view, by following steps described previously (Sect. 3).

SIG-P diagram for KDE business view (Fig. 1, introduced in Sect. 3) supports the analysis of KDE business practices. For instance, practice B5 (*Provide a nonprofit corporation to manage legal and financial issues*) may influence Sustainability that, in turn, provides a positive influence on Niche Creation.

**Fig. 2.** SIG-P for Social View

**Fig. 3.** SIG-P for Technical View

Fig. 2 presents a SIG-P diagram for KDE social view that is used to reason about KDE social practices. For instance, Practices S7 (*Conduct a general annual face to face meeting to discuss community issues*) and S8 (*Perform elections to assign responsibility levels for new members*) affect Controllability4 that, in turn, provides a positive influence on Robustness and Productivity

Fig. 3 presents a SIG-P diagram for KDE technical view. For instance, Practice T16 (*Require that all infrastructure must be under the control of the ecosystem and be based on the technologies created by the ecosystem*) may influence Reliability that, in turn, provides a positive influence on Robustness and Productivity. At http://homes.dcc.ufba.br/∼ssamorim/lakademy/ other SIG-P diagrams are available.

#### **6 Discussion**

#### **6.1 Findings**

This study allowed us to uncover implicit features of SECO practices and reveal the context in which they were immersed: (a) Many practices are not documented or standardized; (b) There is a great amount of tacit knowledge sharing during hackathon sessions – this highlights the importance of meeting people face-to-face to strengthen the community; and (c) Practices do have influence on KDE ecosystem health, which can be exploited later to support decisiontaking processes.

#### **6.2 Contributions**

SECO participants can use the knowledge base of practices and related effects in situations such as: (a) SECO adoption, whenever an organization wants to adopt

4 Controllability can be defined as a quality of ecosystems interested in keeping stable and pursuing sustainable development [15].

some ecosystem platform, the NFR with practices can be used to evaluate risks; (b) Getting involved, whenever an individual or team wants to contribute with an open source ecosystem, documented practices and their effects on SECO health can improve the learning curve; and (c) Partnership establishment, in scenarios of feasibility analysis, the NFR with practices can provide useful information to judge the feasibility of partnerships.

Researchers can use our results as input to new evaluation models and quality models for SECOs, and inspiration for performing additional qualitative studies on SECO health. So far additional studies and data are necessary to analyze which practices are more influential or the degree of their influence on SECO health. A deeper understanding on SECO practices will help us to further investigate factors that may contribute to balancing the trade-offs necessary for SECO health.

#### **6.3 Limitations**

The contributions of this study should be balanced against some limitations relating to the timing of the study, observer bias, and observed subject bias. Classical ethnographic studies are performed over a long period of time [10]. However, LaKademy provides a rare opportunity to meet people face-to-face in an ecosystem that carries out many online activities.

Additionally, there are limitations regarding practices not captured, variability of SIG-P configurations, and validation of the results. SIG-P models are supposed to support understanding and analysis of SECO practices with respect to their influence on NFR and different health aspects (robustness, productivity, and niche creation) of KDE. In this paper, we only show one combination for SIG-P, but many other combinations are possible including negative impacts. Moreover, we worked to avoid bias taking a rigorous approach to data collection and analysis. Nevertheless, we should perform more combination of practices and NFR and validate the results to produce a more comprehensive analysis.

#### **7 Related Work**

To our knowledge, there is no qualitative study in the context of SECO health that focuses on software practices. In this section, we outline related work that addresses ethnography and practices.

Sharp *et al.* [17] presented an ethnographic study to identify XP practices, conducted in a small company that developed web-based intelligent advertisements. Their findings included 5 characterizing themes within XP practice. The identified practices indicated that, in the XP culture, individuals and teams are respected, take responsibilities, and act to maintain the quality of working life. This work helped us in the identification of XP practices used by KDE.

Evangelista [8] in his PhD thesis investigated the Free Software Movement (FSM), with the overall goal of studying the relation between free software and topics such as culture, power, labor and ideology. To understand the FSM in the Brazilian context, he conducted an ethnographic study during the 9*th* International Free Software Forum (FISL)5. His study reported several practices used in open source ecosystems. Furthermore, it inspired us to apply ethnography in our work, and provided guidance on observing and understanding practices used by KDE.

#### **8 Conclusions and Future Work**

SECOs have achieved success and attracted the attention of researchers. However, little is known about SECO practices and reasons for their success. In this paper, we presented an ethnographic-based study conducted during LaKademy 2015 to collect practices of the KDE ecosystem and understand how they affect ecosystem health. These practices were linked to non-functional requirements to provide a preliminary support to SECO health assessment.

This work provides a set of practices and a SIG-P catalogue of the analyses of the impact of practices on SECO health. These results are important steps for our ongoing research proposal of an assessment method for SECO health based on practices [4]. Our results can also support researchers in understanding operational aspects of open source SECOs.

**Acknowledgments.** We are thankful to Lakademy 2015 organizers and participants. John McGregor is partially funded by the NSF grant #ACI-1343033.

#### **References**

5 http://softwarelivre.org/fisl17.

**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

### **Challenges in Validating FLOSS Configuration**

Markus Raab1(B) and Gerg¨o Barany2

1 Institute of Computer Languages, Vienna University of Technology, Vienna, Austria<EMAIL_ADDRESS>2 Inria, Paris, France<EMAIL_ADDRESS>
**Abstract.** Developers invest much effort into validating configuration during startup of free/libre and open source software (FLOSS) applications. Nevertheless, hardly any tools exist to validate configuration files to detect misconfigurations earlier. This paper aims at understanding the challenges to provide better tools for configuration validation. We use mixed methodology: (1) We analyzed 2,683 run-time configuration accesses in the source-code of 16 applications comprising 50 million lines of code. (2) We conducted a questionnaire survey with 162 FLOSS contributors completing the survey. We report our experiences about building up a FLOSS community that tackles the issues by unifying configuration validation with an external configuration access specification.

We discovered that information necessary for validation is often missing in the applications and FLOSS developers dislike dependencies on external packages for such validations.

#### **1 Introduction**

*Configuration settings* influence the behavior of software and are used ubiquitously today. *Configuration access* is done by the part of applications concerned with fetching configuration settings from configuration files, environment variables, etc. at run-time. *Configuration validation* detects configuration settings which do not fulfill the user's expectations, for example, setting a web browser's proxy to a server that is not reachable in the currently connected network.

While configuration access seems to be straightforward, system administrators experience many surprises on a daily basis. In the systems community the issue is well-known as *misconfiguration* [1,30,36,37]. Misconfigurations cause large-scale outages of Internet services [19]. Yin et al. [37] claim that *"a majority of misconfigurations* (70.0%–85.5%) *are due to mistakes in setting configuration"*.

Xu et al. argue that often configuration access code and not system administrators are to blame [35]. Often (38.1%–53.7%) misconfiguration is caused by illegal settings which clearly violate syntactic or semantic rules [37]. Thus most

c The Author(s) 2017

DOI: 10.1007/978-3-319-57735-7 11

Gerg¨o Barany—This work was performed while the author was at CEA LIST Software Reliability Laboratory, France, and supported by the French National Research Agency (ANR), project AnaStaSec, ANR-14-CE28-0014.

F. Balaguer et al. (Eds.): OSS 2017, IFIP AICT 496, pp. 101–114, 2017.

errors could be caught with a consistency checker executed before configuration changes. Nevertheless, only in 7.2% to 15.5% cases do error messages pinpoint the error [37]. Free/libre and open source software (FLOSS) applications often do not validate their settings before startup or even later [34]. System administrators have to find their own ad-hoc ways [3,4,13,31,39].

Other factors also influence configuration settings. We will call validation that considers more than the settings of a single application *global validation*. Faulty global validation causes issues in 46.3%–61.9% of cases [37]. For example, when a web browser is started in a different network, previously working proxy settings will fail to work. Our holistic approach rejects misconfigurations early on.

These issues lead to our **research question**: Why do we lack tools for global validation, and how can we help developers provide them?

Our contributions are as follows:

#### **2 Methodology**

Our methodological foundation builds on *theory building from cases* [10,11]. In the present paper we will use two different methodologies embedded in a framework: source-code analysis and a questionnaire.

#### **2.1 Source-Code Analysis**

We study getenv, which is an application programming interface (API) to access environment variables. We chose it because it is the only widely standardized configuration access API (included in C, C++, and POSIX standards) and available in many programming languages. In earlier work [26], we showed that getenv is used at run-time ubiquitously. getenv is often combined with other techniques, for example, overriding configuration file settings. Furthermore, environment variables are not part of configuration settings dialogues, i.e., they are usually not validated before reaching the application.

We carefully selected 16 applications across different domains. We included large applications with a thriving community but also others for diversity. We used the versions of the applications as included in Debian 8 (Jessie) as shown later in Table 1.We downloaded package sources from http://snapshot.debian.org. To determine the code size we used Cloc 1.60 [7].

We manually counted all getenv occurrences for the version specified in Table 1. Then we categorized the resulting 2,683 code snippets around getenv. We looked if getenv occurrences *depend* on some other configuration. Such situations occur when configuration settings interact; for example, fallback chains of configuration access points depend on each other. Such fallback chains are hints to global configuration access, which we wanted to find. As our last experiment, we searched for places where global validation would be useful, and investigated how helpful the documentation of the getenv parameters is.

**Threats to Validity:** For evaluating usefulness (as only done in the last experiment), by nature, subjectivity is involved. In particular, it is possible that we overlooked dependences. We will report the numbers we found but we consider the experiment as exploratory and not as something that could be repeated with the same numbers as outcome. The individual examples, however, are insightful.

#### **2.2 Questionnaire**

We carefully prepared a questionnaire with FLOSS developers in mind. Then we conducted pilot surveys with developers, colleagues and experts for surveys. In the iterations we improved the questions and made the layout more appealing.

In order to reach the target group, we posted requests to fill out the survey in the respective FLOSS communication channels. To obtain a higher quality, we awarded non-anonymous answers with small donations to FLOSS-related projects. We used the non-anonymous answers to cross-check statistics.

We asked some personal questions about age, education, occupation, and FLOSS participation to have the important characteristics of our participants.

We used Limesurvey version 2.50+ for conducting the survey. We will report the percentages relative to the number of persons (*n*) who answered a particular question. We report means and standard deviations (*s*) of samples for *n* ≥ 95. We used the Kolmogorov-Smirnov test [15] for smaller samples.

**Threats to Validity:** For the validity of our survey it is important that only FLOSS contributors participate. The donation might have persuaded some participants to fill out parts of the survey even though they had no particular experience. Thus we explicitly asked about contributions to specific projects.

The survey reflects the beliefs of participants. Thus we used other methods to distill facts about the applications. Because opinions help to understand goals and reasons, the survey is an important part of the overall study. It should be considered as supplement to the source-code analysis.

**Demographics:** The front page of the survey was shown to 672 persons, 286 gave at least one answer, 162 completed the questionnaire, and 116 persons entered their email addresses. The age of the population (*n* = 220) has a mean of 32 years (*s* = 9). The degrees in the population (*n* = 244) are: master (38%), bachelor (25%), student (18%), no degree (13%), or PhD (6%). As their occupation, 56% of the persons selected software developer, 21% system administrator, and 16% researcher (multiple choice question, *n* = 287). Participants reported work on up to five different FLOSS projects. For the first project, they estimated their participation with a mean of 5.3 years (*s* = 5, *n* = 180). 60% of them reported a second FLOSS project, 36% a third, 17% a fourth, and 9% a fifth.

Raw data and questions are available at https://rawdata.libelektra.org.

#### **3 Configuration Access**

Before we start exploring our research question, we need to validate that our evaluated configuration accesses are indeed relevant and popular. In this section we investigate which configuration access methods FLOSS developers use.

#### **3.1 Which Methods for Configuration Access are Popular?**

**Finding 1***a***:** *We observed that* getenv *is omnipresent with 2,683 occurrences.* The source code of the applications we analyzed has 4,650 textual getenv occurrences. 2,683 of them were actual getenv invocations, 1,967 were occurrences in comments, ChangeLog, build system, or similar. (See Table 1 for details.)

**Finding 1***b***:** *Three kinds of configuration access are equally popular: Commandline arguments, environment variables, and configuration files. Developers are highly satisfied with them. Others are used less, and are more frustrating to use.*

Command-line arguments (92%, *n* = 222), environment variables (e.g., via getenv) (79%, *n* = 218), and configuration files (74%, *n* = 218) are the most popular ways to access configuration. Other systems, such as X/Q/GSettings, KConfig, dconf, plist, or Windows Registry, were used less (≤ 13%, *n* ≥ 185).

Participants rarely found it (very) frustrating to work with the popular systems: getenv (10%, *n* = 198), configuration files (6%, *n* = 190), and commandline options (4%, *n* = 210). Less-used systems frustrated more (≥ 14%, *n* ≥ 27).

### **3.2 What Is the Purpose of** getenv**?**

**Finding 1***c***:** *Like other configuration accesses,* getenv *is used to access configuration settings (*57*%). Sometimes it bypasses main configuration access.*

Of the 2,683 getenv invocations, 1,531, i.e., 57%, relate to run-time configuration settings and not debugging, build-system, or similar. Other investigations in this paper discuss these 1,531 getenv occurrences. (See Table 1 for details.)

We found occurrences where getenv obviously bypasses main configuration access, for example, to configure the location of configuration files.

Also in the survey we asked about the purpose of getenv (*n* = 177). The reasons to use it vary: in a multiple choice question 55% say they would use it for debugging/testing, 45% would use getenv to bypass the main configuration access, and 20% would use getenv if configuration were unlikely to be changed.

**Finding 1***d***:** *In many cases* getenv *parameters are shared between applications.* In the source code we investigated which parameters were passed to getenv. We found that 716 parameters were shareable parameters such as PATH. In the survey 53% say they use getenv for configuration integration (*n* = 177).

### **Finding 1***e***:** *Parameters of* getenv *are often undocumented.*

The function parameter passed to getenv invocations tells us which configuration setting is accessed. In an Internet search using the application's and getenv parameter's name with https://startpage.com, we found documentation for only 283 of the non-shared getenv parameters but not for the 387 others.

The FLOSS projects deal with the missing documentation of getenv parameters in different ways. Most projects simply claim their getenv usage as internal, saying the environment should not be used for configuration by end users, even if there is no other way of achieving some goal. Often we miss a specification describing which parameters are available.

In other projects, the developers invest effort to create lists of available parameters. For example, in LibreOffice developers try to find getenv occurrences automatically with grep, which fails with getenv aliases1.

**Discussion:** The getenv API has some severe limitations and is sometimes a second-class citizen. One limitation is that return values of getenv invocations cannot be updated by other processes. For example, getenv("http proxy") within a running process will still return the old proxy, even if the user changed it elsewhere. Another limitation is that they do not support persistent changes by applications. Configuration files, however, are not easily shareable.

**Implication:** There is currently no satisfactory solution in FLOSS for global, shareable configuration settings. getenv supports all characteristics of configuration access and can be used to investigate challenges in configuration validation.

#### **4 Configuration Validation**

Having established which configuration accesses are popular (including, but not limited to, getenv), we will investigate challenges of configuration validation.

#### **4.1 Which Are the Concerns Regarding Global Validation?**

**Finding 2***a***:** *Developers have concerns about adding dependencies for global validation (*84*%) and reducing configuration (*30*%) but desire good defaults (*80*%).* Many persons (30%, *n* = 150) think that the number of configuration settings

should *not* be reduced. But 43% said it should be reduced to prevent errors.

We got mixed answers (*n* = 177) to the question "Which effort do you think is worthwhile for providing better configuration experience?" Most persons (80%) agree that proper defaults are important. Most methods exceed the effort considered tolerable by the majority of participants: Only getenv would be used by the majority (53%). System APIs would be used by 44%. Fewer (30%) would use other OS-specific methods, such as reading /proc. Only 21% of the participants would use dedicated libraries, 19% would parse other's applications configuration files, and 16% would use external APIs that add new dependencies.

**Discussion:** To avoid dependencies, FLOSS developers currently expect users to configure their applications to be consistent with the global configuration.

1 https://bugs.documentfoundation.org/show bug.cgi?id=37338.

**Implication:** The results indicate demand for dependency injection to have global validation without direct dependencies.

#### **4.2 Which Challenges Prevent Us from Supporting Validation?**

**Finding 2***b***:** *Present configuration validation is encoded in a way unusable for external validation or introspection tools.*

In none of the 16 applications was the validation code kept separately, e.g., in a library. Instead it was scattered around like other cross-cutting concerns.

**Finding 2***c***:** *Developers are unable to support global validation, even if the problem is well-known and they put effort into it. We found out that information essential to check or fix constraints is not available within the applications.*

In Table 1 we present the list of applications we analyzed. The column *counted getenv* lists our manual count of all getenv invocations. The column *config getenv* shows getenv occurrences used for configuration as described in Finding 1*c*. The column *depend getenv* presents manually counted getenv occurrences that depend on, or are used by, other configuration code. The last column *lines per getenv* shows how often manually counted getenv occurs in code.

**Table 1.** Manual count and classification of getenv occurrences.

Most of these places (1,115, i.e., 73%) were dependent on some other configuration. We found 204 places where some kind of configuration dependencies were forgotten. In 58 cases we found several hints, e.g., fallback chains with missing cases or complaints on the Internet about the not considered dependency.

We give a real-life example from the Eclipse source of how easily dependencies are forgotten. The missing dependencies lead to missing validation, which leads to frustrating user experience. If Eclipse wants to instantiate an internal web browser, some users get an error saying that MOZILLA FIVE HOME is not set. On GitHub alone, issues mentioning the variable were reported 71 times. The underlying issue usually is that the software package webkitgtk is missing2. The developers even considered the dependency (installation paths) for RPM-based systems by parsing gre.conf. But for other users on non-RPM-based systems the fallback is to query MOZILLA FIVE HOME which leads to the misleading error. In Eclipse the workarounds (including parsing gre.conf) needed 213 lines of code. Furthermore, most of the 9006 code snippets we found on GitHub referring to the variable are small wrappers trying to set MOZILLA FIVE HOME correctly.

**Discussion:** While the package managers easily answer where the missing files are located, within the application there is no reasonable way to find out. We found similar issues concerning network, firewall, hardware settings, etc.

**Implication:** Applications have a need to access global configuration settings.

### **5 Experience Report on Supporting Global Validation**

*Elektra* is a library that aims at providing unified access to configuration settings as key/value pairs. It integrates a specification language for global validation. Here we will discuss how Elektra fulfills the requirements unearthed by the study before describing the challenges to adoption that Elektra faced in the past. We summarize requirements derived from the findings of Sects. 3 and 4.

#### **5.1 Unify Configuration**

**Requirement 1***a*−*c***:** *Developers use different mechanisms for configuration accesses interchangeably or to bypass limitations of others.* To avoid the need for bypasses, Elektra bootstraps itself at startup, making it possible for configuration settings to describe the configuration access, for example, which configuration files should be used. To allow administrators to use all popular techniques, Elektra reads from different sources such as configuration files, system settings, and environment variables. Elektra integrates many different configuration file formats such as INI, XML, etc., and it supports notifications to always keep application's configuration settings in sync with the persistent configuration settings.

**Requirement 1***d***:** *FLOSS developers demand a way to share configuration settings.* We implemented a layer similar to a virtual file system, which enables

2 https://groups.google.com/forum/#!topic/xmind/5SjPTy0MmEo.

applications and system administrators to mount configuration files [29]. This technique facilitates applications to access configuration settings of any other application. Using links and transformations [22] one can even configure applications to use other settings without any support from the application itself.

**Requirement 1***e***:** *There should be a way to document configuration settings.* Elektra introduces specifications for configuration settings [24]. These specifications should also include documentation. But even if they do not, users at least know which configuration settings exist, and which values are valid for them.

#### **5.2 Validate Configuration**

**Requirement 2***a***:** *Dependencies exclusively needed for configuration settings should be avoided.* Elektra introduces plugins that enable a system-level dependency injection. Developers specify validations in specifications, without the need for their application to depend on additional external libraries. In plugins executed on configuration access, the settings get validated or default settings get calculated. Elektra only uses the C standard library and no other external dependencies [20]. Nevertheless, even the dependency on Elektra itself can be avoided. Elektra supports intercepting of library calls such as getenv [26,27]. Using this technique, applications think they use environment variables, while in reality they query Elektra.

**Requirement 2***b***:** *Configuration settings and validations should be open to introspection.* Similarly to getenv, Elektra provides an API, but it aims to overcome the limitations of previous abstractions and interfaces. Elektra allows many configuration files to be integrated with a uniform key/value API. Even the specifications of accesses, dependencies, and validations are accessible via the same API. Thus system administrators and applications can use the API to introspect configuration settings. Elektra relies on file system permissions to restrict access to configuration files.

**Requirement 2***c***:** *Global validation should be supported.* Elektra supports global validation through a range of different checker plugins. These plugins do not only check data for consistency but also check if configuration settings conflict with reality. For example, one plugin checks for presence of files or directories, while another plugin checks if a host name can be resolved. Checks are executed whenever Elektra's API is used for writing. This way also all administrator tools sitting on top of Elektra reject invalid configuration settings. Elektra also allows to integrate system information such as hardware settings via plugins.

#### **5.3 Community Building**

The *Elektra Initiative* is a community that started with the straightforward idea to have a single API for configuration access. Other projects watched how it progressed, but adoptions occurred rarely. Due to various grave issues in the first versions, the API needed several redesigns. Obviously, API changes are not very popular and Elektra lost most of its users at this time. Despite many marketing efforts to change the situation, it was predominantly companies and not FLOSS software that used Elektra. This slow adoption was unfortunate but an opportunity to continue making changes. Unfortunately, the changes were not done wisely, instead we introduced mostly unnecessary features. Here the Elektra Initiative had its low and turning-point.

Then the goals shifted towards a more technical solution: We avoid marketing campaigns to persuade everyone to use the API with arguments like "it will give benefits once everyone migrates". Instead it should offer immediate advantages previous APIs did not have. This meant Elektra went into a highly competitive market facing the difficulty of being better than any other configuration library. As a distinctive feature, we started to aim for global validation but without giving up compatibility to current configuration files. We avoid an API that forces our ideology, like our favourite configuration file format or validation, onto users.

These changes made the core more complicated, which led to a recruiting problem. The documentation was for a long time only a master thesis [20], which was a very unsatisfactory situation. The next efforts were to make the project community-friendly again. We started to improve quality by regression tests, fixing reports of code analysis tools, and adding code comments and assertions. Then we started overhauling documentation, writing tutorials, and created a website. Last but not least, we started releasing every two months with fixes and new features in every release. These changes led to more than a dozen contributors, Elektra being packaged for many distributions, and acceptance of a paper on Elektra in "The Journal of Open Source Software" [23].

#### **6 Community Feedback and Future Work**

The survey validated Elektra's goals: Many agreed (80%, *n* = 153) that a solution must be lightweight and efficient; and that a configuration library must be available anywhere and anytime (84%, *n* = 153). Many persons (70%, *n* = 150) consider it important that the community is supportive. Even more persons want bugs to be fixed promptly (88%, *n* = 150). Because 76% persons find it important that applications directly ship documentation (*n* = 157), external specifications should have documentation. Nearly everyone (96%, *n* = 173) agrees that configuration integration, such as global validation, would at least moderately improve user experience. Thus we will continue research in this area.

A participant said: *"Must be extensible/adaptable. If it is, users can take care of many of the above aspects themselves"*. We agree and continue to pioneer modularity. For example, many persons found readability of configurations important (65%, *n* = 152) but could not agree which formats are readable.

Another person wrote: *"It must offer a compelling reason to switch from e.g. gsettings. For example a killer feature that others don't have, etc. Otherwise, the status quo wins."* Elektra's "killer feature" can be global validation.

From our experience with Elektra, it was also clear that we need to put much more effort into API stability. Thus we avoid breaking changes to the API. We are about to provide easy-to-use high-level APIs for different use cases.

The 1.0 release of Elektra is still pending: (1) The specification language for validation/transformation/dependency injection is not completely defined. (2) The configuration parsers have limitations, e.g., they do not always preserve comments or order. (3) Elektra puts some unnecessary limitations on the plugins.

#### **7 Related Work**

Many other configuration libraries have validation capabilities, for example, Apache Commons Configuration. Unlike Elektra they do not have external specifications. Instead they require developers to hardcode them into the applications.

Other papers describe the technical details of Elektra [20,23,29]. In particular frontend code generation avoids errors in configuration access [21,28]. Other work describes Elektra's specification language [22,24] and how applications participate without code modifications [25,26].

Crowston et al. [6] created a survey of empirical research for FLOSS. Michlmayr et al. [16] investigated quality issues of FLOSS using interviews. We were able to confirm that documentation often is lacking. Barcomb et al. [2] used a questionnaire to investigate how developers acquire FLOSS skills.

PCheck [34] validates configuration files early. Unlike Elektra, it is not free software and does not support application-specific checks or plugins. Some work was done to automatically resolve misconfiguration [1,30,33,38]. These approaches aim at solving already manifested issues, Elektra aims at resolving them earlier. Xu et al. [36] surveyed further possibilities.

Nos´al et al. [17,18] investigated similar abstractions but with a focus on language mapping. Denisov [8] collected requirements for configuration libraries.

Berger et al. [5] and Villela et al. [32] created a questionnaire that asks about variability modeling. Our survey focused on a different target group.

#### **8 Conclusions**

In this paper we examined challenges in configuration access and presented a solution. We addressed the research question: Why do we lack tools for global validation and how can we help developers provide them? The answer is that validations are encoded in the software in a way (1) unusable by external tools, and (2) incapable of using global knowledge of the system. The answer is backed up by both a questionnaire and a source analysis.

To overcome developers' configuration issues, we need to externalize configuration access specifications and use a unified configuration library. The empirical data backs up that this is possible and wanted. It is *possible*, because currently different configuration accesses are used interchangeably. It is *wanted*, because users stated that different forms of configuration access sources should be able to override each other.

Based on our survey we might have to rethink how to reduce the number of configuration settings because many developers do not agree with complete removal of less-used settings. The survey also showed that external dependencies in configuration access code are a contradictory topic: Developers want good defaults, but do not want to pay for them with dependencies. Elektra's way of implementing dependency injection and globally calculating default settings fulfills both goals. Because of the externalization of configuration access specifications, users can even introspect the (default) settings that applications receive.

Finally, we described FLOSS community efforts to improve on the issues. The results show that a dependency injection at the system level is feasible and practical. It has the potential to be accepted by developers if they perceive global integration and validation as "killer feature". The current status of the FLOSS project can be tracked at https://www.libelektra.org.

**Acknowledgments.** We thank the anonymous reviewers, Tianyin Xu, Franz Puntigam, Stefan Winter, Milan Nos´al, and Harald Geyer for detailed reviews of this paper. Additionally, many thanks to all the people contributing to Elektra.

#### **References**

**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

# Case Studies

### **Progression and Forecast of a Curated Web-of-Trust: A Study on the Debian Project's Cryptographic Keyring**

Gunnar Wolf1(B) and V´ıctor Gonz´alez Quiroga2

1 Instituto de Investigaciones Econ´omicas, Universidad Nacional Aut´onoma de M´exico, Mexico City, Mexico<EMAIL_ADDRESS>2 Facultad de Ciencias, Universidad Nacional Aut´onoma de M´exico, Mexico City, Mexico

**Abstract.** The Debian project is one of the largest free software undertakings worldwide. It is geographically distributed, and participation in the project is done on a voluntary basis, without a single formal employee or directly funded person. As we will explain, due to the nature of the project, its authentication needs are very strict—User/password schemes are way surpassed, and centralized trust management schemes such as PKI are not compatible with its distributed and flat organization; fully decentralized schemes such as the PGP Web of Trust are insuficient by themselves. The Debian project has solved this need by using what we termed a "curated Web of Trust".

We will explain some lessons learned from a massive key migration process that was triggered in 2014. We will present the social insight we have found from examining the relationships expressed as signatures in this curated Web of Trust, some recommendations on personal keysigning policies, and a statistical study and forecast on aging, refreshment and survival of project participants stemming from an analysis on their key-handling.

**Keywords:** Trust management *·* Cryptography *·* Keyring *·* Survival *·* Aging *·* curated Web of Trust

#### **1 Introduction**

The Debian project is among the most veteran surviving free software projects; having been founded in August 1993 by Ian Murdock [5], it has grown to be one of the most popular Linux distributions by itself, as well as the technical base for literally hundreds of others. It is the only distribution that produces an integrated operating system capable of running on different operating system kernels – Although an overwhelming majority of Debian users use Linux, it hs been *ported* to the FreeBSD and GNU HURD kernels as well [15,16].

But besides all of its the technical characteristics, what makes Debian really stand out as a project is its social composition: It is, since its inception, a completely volunteer-driven, community-run project, with very big geographic dispersion [13].

Since Debian's early days, cryptographically strong identification was deemed necessary to guarantee the security guarantees Debian's users have; as the project grew, a viable trust management strategy had to be envised as well; we call it the *curated Web-of-Trust* model [20].

But cryptographic parameters that were deemed safe for long-term use in the mid nineties are now considered to be unsafe. By 2014, the Debian project underwent a large key migration to keep up with the security recommendations for the following years [14]. We described the full reasoning for this migration and an oveview of the process and its numeric impact in the project in [20].

The aforementioned migration prompted a study of the direct metrics of the keyring's health, such as those detailed by [19], as well as a more transdisciplinary analysis of the keyring as a social network. Throughout this work, we will present an overview of the *trust aging* that had started manifesting since around 2010, as well as its forceful re-convergence, and a statistical analysis on key survival expectations.

#### **2 Trust Models in Public Key Cryptography**

Besides encryption and signing, public key cryptography provides several models for identity assessment, called *trust models*. The most widespread model is the *Public Key Infrastructure* (PKI) model, a hierarchical model based on predetermined *roots of trust* and strictly vertical relationships (certificates) from *Certification Authorities* (CAs) to individuals. This model is mostly known for being the basis for secure communication between Web browsers and servers using the https protocol.

As we have presented [20], the Debian project, being geographically distributed and with no organizational hierarchy, bases its trust management upon the *Web of Trust* (WoT) model, with an extra step we have termed *curatorship*. The WoT model has been an integral part of OpenPGP since its inception [21]. For this model, there is no formal distinction between nodes in the trust network: All nodes can both receive and generate certificates (or, as they are rather called in the WoT model, *signatures*) to and from any other node, and trust is established between any two nodes that need to assert it by following a *trust path* that hopefully links them in the desired direction and within the defined tolerable distance [19].

Beside the aforementioned work, several other works have studied the information that can be gathered from the total keyring in the SKS keyserver network1 [3]. The work we will present in this paper is restricted to a small subset

1 For a WoT model to be able to scale beyond a small number of participants, *key servers* (systems that store and allow for retrieval of public key material) are needed. The *Synchronizing Key Server* (SKS) network is the largest network of OpenPGP key servers.

thereof—As of December 2016, the SKS network holds over 4 million keys, while the active Debian keyrings hold only around 1500.

### **2.1 Cryptographic Strength**

Public key cryptography works by finding related values (typically, very large prime numbers). The relation between said numbers, thanks to mathematical problems that are *hard enough* to solve to be unfeasible to be attacked by brute force, translates to the strength of the value pair.

Said schemes' strength is directly related to the size of the numbers they build on. Back in the 1990s, when Internet connectivity boomed and they first became widely used [21], key sizes of 384 through 1024 bits were deemed enough; using longer keys demanded computing resources beyond what was practical at the time.

Of course, computers become more powerful constantly; cryptographic problems that were practically unsolvable 10 or 20 years ago are now within the reach of even small organizations [14, p. 11]. Cryptographic keys used for RSA and DSA algorithms should now be at least 2048 bits, with 4096 becoming the norm.

By 2009 (when the need to migrate to stronger keys was first widely discussed within the Debian project) the amount of 1024-bit keys was close to 90% of the total keyring; the upcoming need of migration was repeatedly discussed, and due to the threat of an attack becoming feasible for a medium-sized organization [14, pp. 30, 32], by July 2014 a hard cutoff line for expiring keys shorter than 2048 bits was set for January 2015, setting a six month period for key migration. We published a analysis on that migration process [20], which prompted the present work.

#### **3 Trust Aging and Reestablishment**

The work done for the described keyring migration, as well as the migration process itself, presented a great opportunity to understand the key migration as a social phenomenon as well, using the keyring as a way to measure social cohesion and vitality.

We prepared graphic representations of the keyring at its various points in time, in the hope to learn from it patterns about its growth and evolution that can warn about future issues. For the trust-mapping graphs, we use directed graphs, where each key is represented by a node and each signature by an edge from the signer to the signee. For starters, we were interested in asserting whether the characterstics observed on the whole OpenPGP WoT [19] repeated in the subset of it represented by the Debian keyrings. Of course, said work was done as a static analysis on the keyring back in 2011; back then, the whole OpenPGP keyring stood at 2.7 million keys; at the time of this writing there are 4.5 million keys, growing by 100 to 400 keys every day [17].

**Fig. 1.** Snapshots of the Debian keyring evolution at different points in time

Figure 1 presents seven snapshots of the main developers keyring, processed by *Graphviz* using the *neato* layout program, which implements the *spring* minimal energy model [6]. Of course, at the scale they are presented, each individual edge or node becomes irrelevant; there is too much density at the center, and the outlying nodes and edges appear as just noise. However, the *shape* of the strong set2 does lend itself to analysis.

Figures 1(a), (b) and (g) present a regular shape, approximately following Ulrich's observations, that the strong set of the WoT exhibits scale-freen. Quoting [19, Sect. 4.3],

Connectivity-wise, scale-free graphs are said to be robust against random removal of nodes, and vulnerable against the targeted removal of hubs (which leads to partitioning). This is usually explained by the hubs being the nodes that are primarily responsible for maintaining overall connectivity.

Ulrich notes that the WoT graph is *similar to a scale-free one and exhibits a hub structure, but is not scale-free in the strict sense*.

Something happened, however, in the course of 2010 that led to the WoT acquiring the shape shown in Fig. 1(c) by the end of the year – Instead of a seemingly uniform *blob*, there is a distinct protuberance. This horn grew throughout the following years, and by 2014, the keyring consisted of two roughly equivalent *blobs* somewhat weakly linked together, as Figs. 1(d) and (e) show.

We find this protuberance to be the portrait of a social migration: The project is often portrayed as unique among free software projects due to the close personal

2 The strong set is defined as the largest set of keys such that for any two keys in the set, there is a path from one to the other [10].

ties among developers; its yearly developers' conference, *DebConf*, has a very high repeating attendance rate. However, given the project has lived for over 20 years, it is understandable many of the original members have grown inactive and moved on to other life endeavors; formal retirement is requested from developers, but many people reduce their engagement gradually, and just never formally retire.

While the geographical dispersion makes it quite hard for some developers to meet others and obtain new certificates, there is a tradition in Debian to announce travels in a (private, developers-only) mailing list, and active developers often will gladly meet people travelling to their region just for a key signature exchange.

Although the number of developers that by late 2010 had migrated to a stronger key was still quite small, the call for key migration was initially answered by those with most active key activity –hence, probably more conscious about the importance of this migration. Of course, although it was not a targetted removal, it was a socially self-selected one: Trust hubs were among the first to migrate to stronger keys. And even though they attempted to re-build their WoT relationships and cross-sign with other developers at gatherings such as DebConf, the group of developers that –as explained in Sect. 3– had drifted away from project activity didn't reconnect with them.

While the migration to keys longer than 1024 bits took much longer than originally expected, the initial push was not bad: During 2010, it reached from practicaly zero to close to 10% of the keys– But many of those keys were *hubs*, people long involved in the project, with many social bonds, and thus very central keys. When those people migrated to newer keys, the signatures linking their long-known fellow developers to the project were usually not updated, and several old keys could have even *become islands*, gradually losing connectivity to the strong set.

Given Debian's longstanding practices, rather than isolated, many such keys started *drifting apart* as a block, growing separated from the center of mass. This explains why the migration started as a *lump* to later become two large, still somewhat strongly connected bodies, mostly stable over the years. Of course, as more developers migrated to strong keys, by late 2014 the remaining group started losing cohesion, and by December 2014 (before it was completely removed), it is barely noticeable – All of its real *hubs* had migrated to the new center of mass, with many previously connected keys becoming isolated, as Fig. 1(f) shows.

In order to prove this hypothesis, we generated again the same graphs, but factoring in the *trust aging*: If individual signatures are colored by their age, it is possible to visually identify if a significant portion of the group's trust is aging – That is, if social bonds as reflected by intra-key signatures are over a given edge. The seven subfigures of Fig. 2 correspond with those of Fig. 1, but with color-coded edges (according to the image caption).3

3 Some care should be taken interpreting the presented graphs. Particularly, chosen colors are not equally strong and visible against white background; mid-range (orange, yellow) signatures appear weaker than red or blue ones. Also, the drawing algorithm overlays lines, and in high density areas, only the top ones prevail. Still, we believe our observations to hold despite these caveats.

**Fig. 2.** Snapshots of the Debian keyring evolution at different points in time, showing signature age: Blue, ≤1 year; green, between 1 and 2 years; yellow, between 2 and 3 years; orange, between 3 and 4 years; red, ≥4 years. Signature coloring is relative to each of the snapshots: Blue edges in graph (a) represent signatures made throughout 2008. (Color figure online)

Surprisingly, even Fig. 2(b) shows a clear grouping of keys by signature age? But this grouping does not appear a year earlier, in Fig. 2(a). This can, again, be indicative that the first people to migrate to stronger keys, even before it altered the overall shape of the WoT, migrated during 2009; by early 2010, they might constitute te tight, new (blue) group still in the peripheria, that eventually became the core of the newer *blob*.

#### **4 Expectations on Key Survival**

Following from the same data set, we started a further statistical analysis; this section presents the preliminary results we gathered from applying survival analysis techniques.

The general focus of survival analysis is on the modeling the time it takes until a specific event occurs, in social sciences one often speaks of *event history* [18]. We have found interesting findings from studying how many people keeps participating in the Debian project throughout the time, that is, to model the time until departure from the Debian project. The main motivation comes from the need to understand keyring population along time. Our sampled data is defined by the PGP keys that make up the curated WoT from the Debian Developers keyring [20].

The analyzed data is treated as a longitudinal study. We point out that intervals are not of the same length in time: each data point is a *tag* in the keyring's Git history,4 and the period of analysis spans between July 2008 and July 2016. During said period, 124 tags were recorded, averaging to 23.96 days each, with a standard deviation of 27.95, with a maximum of 69 days and a minimum of one day.

Given the way the keyring is structured, we used key the long key ID (the lowest 64 bits of its fingerprint, in hexadecimal representation) as a unique identifier for each key. For each tag and key we counted the number of signatures made to that key by counting the number of non-zero entries in the corresponding key column of an adjacency matrix at a specified tag.

We identified people's participation in Debian using their PGP key activity record (has the key stopped getting signatures?) and keyring membership (has the key stopped being part of the keyring of interest?) which makes our data is right-censored because no further information about keyring membership is known afterwards; right censoring scheme constitutes data where all that is known is that the individuals are still *alive* (the keys are still active) at a given time, [8].

For this analysis, we must highlight a drawback from our failure definition: in real life there's no 1-to-1 correspondence of a key-person pair; key migrations – regarding our current study logic– mean one key dies (leaves the keyring) and another key enters as a completely independent individual. This will be refined in further analysis.

We used the R programming language with its survival, flexsurv packges; unless otherwise stated, significance level is assumed to be 5%.

Our first line of approach was first to show through the survival function the proportion of remaining keys in the keyring along time, that is keyring permanency. Then using the cumulated hazard function we get the expected exits per key that remains in the keyring until the endtime (in perpetuity). Finally, for the hazard rate function, we get the departure rate from the keyring.

For the non-parametric or observed curves we used the Kaplan-Meier product limit estimator for the survival function, [7], the Nelson-Aalen moments estimator for the cumulated hazard function [1], and the kernel density estimator for the hazard rate function, [9].

A parametric estimation to see the *mortality law* fitting our data was made using a Generalised Gamma distribution through maximum-likelihood estimation [11]. The motivation for using the Gen. Gamma model is due to the closeness it has to the observed hazard rate function obtained non-parametrically. Proper justification for said model comes from the fact that it minimizes Akaike's Information Criterion when compared to the other models, making it a better model in terms of information loss, [2], while also rejecting other models using a log-likelihood test of –5790.042 at 3◦ of freedom, [12]. The estimated parameters found for our model

4 Version control systems handle the concept of *tags* in a repository: Specific points of a project's development that are in some way relevant or significant; many projects use *tags* to mark their releases. This is the case of the Debian keyring maintainers' repository: Tags mark each keyring version that was put in production. The team attempts to put a new version in production roughly once a month.

were μ = 4.6808, σ = 0.1990, Q = 3.4078, with standard errors of 0.0134, 0.123, 0.2205 respectively.

In the non-parametric plot of Fig. 3, we observe downward steps when at least one key stops getting signatures. The crosses represent the followup time for censored observations (for which no further information is known and thus the proportion of keys remains). This plot does reflect the fact that many keys were dropped during the 1024-bits key removal, at tag 107 (January 2015). Observed proportion of keys being above the theoretical model from tag 40 to tag 100 (around 4 years) suggests that after four years the keys wouldn't be much likely to leave; at least not until tag 95 where the probability of remaining afterwards is less than the 50% chance of heads in a coin flip.

**Fig. 3.** Probability of key permanency. The black line follows observed (nonparametric) data from the keyring, with crosses representing the followup time for right-censored observations; the red line is the parametric estimation; dotted lines represent confidence bands. (Color figure online)

As we mentioned, due to the 1024-bit key migration, there is a clear skew that introduces a sharp drop around 100 tags; work is underway, as will be outlined in Sect. 5, to do a similar analysis based on personal identities instead of only key IDs.

Figure 4 shows the key exits given one key in perpetual risk, that is, if it is to remain in the keyring for all its time span. The increasing steps from the non-parameric exits is natural being the cumulated sum per tag of the exits over remaining keys ratios. The similarity from previous plot is expected since cumulated hazard is a logarithmic transformation from survival function. We see again that the observed plot lies below the theoretical model starting from tag 40 to 100 (about 4 years), quickly increasing afterwards more than expected. It is not until near tag 100 that a key is expected to leave, but if for any reason it didn't or inmediatly enters again afterwards, it would be expected to leave again shortly after 3 tags (about 2 months).

**Fig. 4.** Cumulated hazard of key exits. The black line follows observed (nonparametric) data from the keyring, with crosses representing the followup time for right-censored observations; the red line is the parametric estimation; dotted lines represent confidence bands. (Color figure online)

**Fig. 5.** Hazard rate of key exits. The black line follows observed (non-parametric) data from the keyring, the red line is the parametric estimation; dotted lines represent confidence bands. (Color figure online)

Figure 5 shows the departure rate is analogous to a *mortality* rate. The observed behaviour suggest that coming of age there's a sudden increase on the risk i.e. keys "wear out" to their age around tag 90 (close to 6 years). Yet the parametric departure rate being under the non-parametric rate at the final tags shows the dramatic effect from the 1024 removal. Another remarkably finding was that departure rate in general doesn't grow far from 0 giving empirical evidence to say that 5 out of 1000 keys will leave "any time now" (from the fact that hazard rate is the instantaneous probability of failure at a specified time) in a 8 year lapsus.

In general keys in the project will constantly remain for about 6 years, as long as they went through 4 years, which in turn suggests further confirmatory analysis. It is just after the six years where keys effectively don't survive until the end.

#### **5 Conclusions and Future Work**

The Debian keyring is a very peculiar subset of the whole OpenPGP Web of Trust analyzed in [19]. The work we present here provides data empirically supporting the theoretical observations, particularly regarding the robustness of what he defines as the LSCC (Largest Strongly Connected Component). The migration away from 1024-bit keys provided an opportunity to follow the progression of the connectivity in our WoT after several of its hubs were removed.

The preliminary results for this work have been shared with a group of Debian developers. Historically, the usual practice for key signing has been to generate non-expiring signatures; people that have already cross-signed their keys don't have an incentive to refresh their trust. There is an ongoing discussion as to whether this practice should change towards time-limited signatures, better modeling ongoing social relationships, or to stick to current practice.

As mentioned in Sect. 4, the survival study has so far been done around individual keys; work is underway so that a person's activity can be properly represented instead of following keys as separate individuals.

The resulting survival analysis can be used to generate cohort tables that further explain the keyring population for demographics, [4]. The outlined methodology can also be extended by introducing covariates such as the number of signatures received or network measures; this study was done only on the *Debian Developers* keyring, it would be interesting to compare with the more loosely connected *Debian Maintainers* keyring. We also want to further explain the keyrings by stratification. The survival analysis showcases good health of the *Debian Developers* keyring (which makes up the mass of Debian's WoT).

Finally, the methodology followed for this study could be applied to other free software projects, aiming to correlate with events and trends spanning a wider population than Debian's; the applicability of our work to other projects, however, depends on having a proper data set to base the work off. We are not aware of other projects having curated keyrings in a fashion that allows for analysis of their evolution over time.

**Acknowledgments.** We wish to thank GPLHost for donating the computing resources needed for this work.

#### **References**

**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

### **Understanding When to Adopt a Library: A Case Study on ASF Projects**

Akinori Ihara1(B) , Daiki Fujibayashi1, Hirohiko Suwa1, Raula Gaikovina Kula2, and Kenichi Matsumoto1

1 Nara Institute of Science and Technology, Ikoma, Japan<EMAIL_ADDRESS>2 Osaka University, Suita, Japan<EMAIL_ADDRESS>