Lost in Zero Space
CC Attribution 2.0 Belgium:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Mens, Tom
When developing open source software end-user applications or reusable software packages, developers depend on software packages distributed through package managers such as npm, Packagist, Cargo, RubyGems. In addition to this, empirical evidence has shown that these package managers adhere to a large extent to semantic versioning principles. Packages that are still in major version zero are considered unstable according to semantic versioning, as some developers consider such packages as immature, still being under initial development. This presentation reports on large-scale empirical evidence on the use of dependencies towards 0.y.z versions in four different software package distributions: Cargo, npm, Packagist and RubyGems. We study to which extent packages get stuck in the zero version space, never crossing the psychological barrier of major version zero. We compare the effect of the policies and practices of package managers on this phenomenon. We do not reveal the results of our findings in this abstract yet, as it would spoil the fun of the presentation. This empirical study builds further on our earlier work, in which we have studied different kinds of dependency management issues in software package distributions. The current empirical evolutionary study is based on recent package management metadata of 1.5 million packages, totaling 12 million package releases and 56 million package dependencies. We analyse dependency version constraints to determine: * to which extent packages depend on 0.y.z releases of other packages; * whether packages with major version zero ever cross the psychological barrier of 1.0.0; * whether there is any reluctance to depend on 0.y.z packages; * whether dependency constraints are more permissive than what semantic versioning dictates for packages in major version zero.
 Hello everybody, my name is Tom Lenz. I'm a professor at the University of Moles in Belgium in the computer science department in the software engineering laboratory. The focus of this presentation will be on the relation between package dependency management and reliance on packages that still have releases with a zero major version component. This research is conducted as part of a project called Seiko Assist, which is a Belgian inter-university project. I will refer to the problem of depending on packages in version, major version zero as the zero space problem. So this research has been conducted together with another resource associated with called Alexandre de Comte. This presentation has also been published or will be published soon in a journal article. The link to the full details of this publication can be found below in this download that's available on archive. Next I will report on the results of some empirical analysis that we conducted on four different package management systems, cargo for rust packages, ruby jumps for ruby packages, packages for PHP packages and NPM for the Node JavaScript ecosystem. In total there is lots of packages available in these ecosystems. You can see the number of packages that's range between 35,000 in the smallest ecosystem and up to over one million different packages in the largest one, which is NPM. So we have package ecosystems of different orders of magnitude. In terms of number, each of these packages tend to have multiple releases, so the number of releases is sometimes even a tenfold of the number of packages ranging from 183,000 to almost 9 million in NPM and even more dependencies going up to 48 million dependencies in NPM. In total this data we have downloaded from the open source repository and dependency metadata in the libraries.io dataset, which if you are interested in, it's a very useful source of data that you can access through this Zenodo digital object identifier. If you study the different packages in these four different package managers, obviously you will find packages with many different releases and each of these different releases they will have a particular version number. All of these packages, they claim that they follow the semantic versioning policy, so basically they will have a three-component versioning system composed of a major version, a minor version and a patch version. The packages will have a version number that is above 1.0.0, so you will have versions like 1.0, 4.0.6, 7.1.3 and so on. This is what we call the one-plus space. We will also find lots of packages that are still in what we call the zero space, which is versions where the major component is zero. For example, 0.0.1, 0.1.0, 0.12.3 and so on. What we wanted to study was to which extent are packages in these four different package managers making use of releases in the one-plus space or of releases in the zero space and how this affects the dependency management in these different package managers. Why did we want to do this? Basically because it is generally considered that if you have packages that have a major version component 1.0, that they are considered to be stable packages, also more mature packages, and hence they are more likely to be popular. On the other hand, if we look at packages whose releases are still in the zero space, then they are often considered as being unstable and still under development. So they are not yet mature. But is this really the case? Is this common wisdom really the case? Well, let us just have a look at two examples. Take for example one package in NPM, which is zero kit web SDK, which is currently in version 4.0.6. So it's already a pretty high first version number. So one could consider this package to be already mature and stable. Is this really the case? I would say not really because in total there has been only five releases of this package over time. In total only 16 commits and there's been one contributor active for this particular package. In fact, it even turns out that this package has no longer been maintained since July 2017. And because of this, it has been archived on GitHub in June 2018, even though it's still available on NPM. If we take another example in the zero space, that we have for example Axios, which is a package that is still in zero release, 0.21.1 to be precise. So one could consider that because it's still in a zero version, it's not production ready yet. But on the other hand, if we look at this package, if we look at this package history, it has had over time 46 different releases, a total of almost 1000 commits and 257 different contributors. Moreover, if you look at the details on GitHub, we found that it has on average 10 million weekly downloads, about 50,000 different packages, other NPM packages that depend on it. And moreover, on GitHub, we also can see that it is used by roughly 3.5 million other GitHub repositories. So one cannot really say that this package is not production ready. So one could argue that the fact that there are packages in the zero space that are already production ready, they tend to follow the zero based versioning scheme, which is actually some term that has been coined by a person that developed this satiric website, zerover.org, where he's making mocking of people that stick to only major version zero for their releases. And use this as a statement to say that if you have a major, that your software's major version should never exceed the first and most important number in computing, namely zero. In fact, it's a satirical website, but nevertheless, there is lots of popular packages that are actually following pre parenthesis, this zero based versioning strategy. In fact, if you again look on that website, you will look, it's fine. Lots of packages like react native that have been developed over many different years. For example, react native, it's very popular project. It has had 345 different releases. It has been existing over 5.5 years, and it's still currently under major version zero today. The same for the project I just mentioned before, Axios, which is still under its version zero. Even though it has been developed over six years, and it has a lot of stars and lots of dependence, and so on, we can go over this list for quite a long time and find lots of extremely popular packages that are still in their major version zero. So this led us to consider the following research question. Is it true that many packages tend to get stuck in zero space? To find these out, we did some analysis to compute the proportion of packages in each of the four studied ecosystems that still have a release in a particular version range. So either only the version range belonging to the zero space, the version range belonging only to the one plus space, or a version range belonging to both. So if a package has releases in both zero dot y dot z, and other releases that have a version superior to one, then we would consider that they are in the both. If we do this for the four different ecosystems, then we actually find that there is only a very small minority of packages in all the ecosystems that have actually crossed the one dot zero dot zero barrier. So in the beginning of their lifetime, they were a package with major version zero, and then they migrated to a version that was superior to one. In many packages in the different ecosystems, we found that they actually got stuck in the zero space and never migrated to version one. The proportion of such packages really depends on the considered ecosystem. In fact, we can find that there is two types of package managers, those like packages and npm, in which there is only a minority of packages that are stuck in zero version space. And then we have on the other hand, Ruby gems and cargo, where large majority of packages are always and will continue to remain in zero space. This is about 75%, so three out of four packages always have had zero release. And for cargo, it's even worse because it's up to 92%. So for this, it seems to be the case that they are actually not following a semantic versioning, but zero case versioning, because they never crossed the one dot zero version barrier. So now let's focus again on those packages that are actually in the both category. So packages that have actually migrated from zero version to one version. Then we did some survival analysis to find out how long did it take for these packages to migrate to the one plus space. So if we compare the time between their first zero release and their first release superior to one, then this survival analysis shows the proportion of packages having done this. And actually what we found that in the major for the majority of packages, it only took a few months to cross the mythical one dot zero barrier, but still for one out of five packages, it took over more than a year. And for the worst ecosystem, namely in RubyGems, in many cases, it took over two years before, if they migrated actually to the one plus space, it took over two years to do this. So now this is about the prevalence of zero releases in these four different package distributions, package managers. Now let's have a look at the effect of this on the dependency management in these four different packages distributions. So of course we know that in these four different package managers, you have lots of dependencies between different packages, you can have packages, you can have dependencies between packages that are all belonging to one plus space. You can also have lots of dependencies between packages between the zero space, but more importantly, there is of course also dependencies that go from packages that are considered stable and that are in the one plus space to packages that are still in the zero space and vice versa. So we wanted to find out to which extent this is the case, and to which extent is this a problem. More in particularly, let us focus on the red arrows, which are dependencies from packages that have a release that is above one dot zero, and that depends on other packages that still have a release with major version zero. So we asked ourselves the question, is it actually a problem for a mature, popular package to depend on a package that still have a zero dot y dot z version. Why did we ask ourselves this question? Because we found several citations of popular books for different ecosystems, that this is considered to be a problem. For example, Jeremy Khan in his blog for NPM said that you cannot trust a project that depends on another project that is still in a zero dot y dot z version, because such projects are not meant to be ready for use. For cargo, Rust, there was a similar reference that you can find on this blog here, where they consider, depending on a package that is still in version zero dot y dot z, that's a bad thing, and that should be avoided. So of course, this is just two different blogs where some people say that it's a problem. Is this generally the case? Difficult to say. So to get some more insight in this, we try to find more qualitative evidence of this by doing a survey on LinkedIn and on Twitter with in total 102 respondents, where we asked them the following question. If you are a developer, both of you need to depend on an open source package distributed through one of these package managers, would you actually trust depending on a package that has major version zero? There were four possible answers. Obviously, I'm sure I don't mind depending on this or no, I would never depend on such a version, and then some intermediate responses which say I would only do this after checking the package or the package history for this particular package I would like to depend on, or I would only do this if there is no real alternative. So what we find in the responses is that the answer about is it can be, can one trust to depend on zero y z versions is really mitigated because more than half of all responses are in these categories where they say I would only do it after checking or if there is not really an alternative. So actually, the one where they say no, there is no problem at all of depending on such a package, it's only one out of three. So two thirds of all the respondents said it's, it might be problematic to depend on such a package show so you shouldn't do it without any checking or without looking for alternatives. Of course, all of this is related to the use of semantic versioning. I don't think I have to explain semantic versioning to most of you, but what is important to recall, if you look at the details of the semantic versioning policy is that in this policy they actually say something explicitly about using a major version zero. A major version zero is actually intended for initial development only. Why? Because if you have a major version zero, then basically anything may change at any time. Even if you do a patch update, you might still introduce breaking changes. Just for a quick reminder for those people in the audience that don't know what semantic versioning is, if we take this traditional three component version numbering, then if you would like to depend on some package that has a particular version number, and you want to be semantic versioning compatible, if you use, you can use dependency constraints to depend on other packages. If you use a tilde constraint, then you signal that the versions of the package you depend on, you allow any increase of their patch number, and you assume that it will not introduce any breaking changes. If you use a carrot constraint to depend on another package version, then you say that you also allow minor updates of the package you depend upon, and semantic versioning implies that also minor updates are still supposed to be backwards incompatibilities, so it will not introduce breaking changes. If, on the other hand, you use a very permissive constraint where you also allow automatic upgrades of the major version component of your dependencies, in that case, it might be that you will have to face backwards incompatibility problems, because a change in the major release number might introduce breaking changes. The problem with this is that not all of the four studies package managers stick to semantic versioning in the same way, and also they also don't use the dependency constraints in the same way. So here on the left we have listed the different common types of dependency constraints that are being used, and the four different package distributions being considered. Since we wanted to do an analysis about the relation between using dependency constraints and how this affects the reliance on zero major versions, we first needed to translate the dependency constraint notation into a uniform interval-based notation in order to have the exact meaning of the notation that might differ between the different ecosystems. For example, if you take cargo and you would specify 1.0, and this would actually mean that you accept any version number between 1.0.0 and the next major release, well, if you would do this for npm, then this would actually mean only all versions between 1.0.0 and 1.1.0 excluded. And even for packages, it actually would mean just one single version that's allowed. So we can see that there is differences across different package distributions about how a particular version number is interpreted. The most important version constraints being used, however, they are used in a more or less systematic way across different package managers. For example, till the 1.3 is interpreted in the same way for all package managers, and the same is true for carret constraints, except that it's not available for RubyGems. So if you now relate this to semantic versioning, then basically everything that's showing in red are constraints that are more restrictive than semantic versioning, because semantic versioning that says that everything except major upgrades are expected to be backwards compatible. So for example, using carret 1.3 is semantic versioning compliant, but using till the 1.3 is more restrictive than somewhere, and then you have the more permissive constraints, which are things like for example anything that's bigger than 1.2.3, or using a stock constraint will be more permissive than somewhere. Now the interesting part here is that if you look at constraints, dependency constraints to a version number with a major version 0 component, then they are always considered as being more permissive than somewhere. Why is the case? Basically, because somewhere dictates that if you have a zero version, any upgrade, even a patch upgrade of your zero version will be considered to be potentially introducing a breaking change. The different ecosystems we studied tend to be more permissive, since they do allow for patch updates, and they assume that patch updates are still considered to be non-breaking. Here we looked at the different documentation in the four different packaging ecosystems to find evidence of this, and they actually do say this, and they do show the deviation against semantic versioning, but not in an extremely explicit way. For example, if you have a version 0, then you can consider actually the minor upgrade as a breaking change indicator, which implicitly assumes that patch updates are non-breaking. Here again the same for the sort of NPM, for cargo it's the same. If you make breaking changes, you can increment the minor versions, so this implicitly assumes that if you upgrade patches, they are not breaking. And here again, this is for packages where if you have a version that's in the zero space, then the current constraint is considered as a version that is allowing patch upgrades as well, and so on. So it appears that the four different packaging ecosystems systems are more permissive than some ver specifies. And when we analyze this, we indeed see that this is the case in all of the four different study ecosystems, the percentage of packages that have a dependency to some other package that's still in version 0, they have no problem with accepting patch updates. You can see here in green the proportion of dependencies that allow for patch updates, which is a big majority. And in the case of Ruby jumps, they even allow for minor updates, even if this would clearly be considered as breaking changes in all of the other ecosystems. So for sure, with respect to dependency management and dependency constraints, the packaging ecosystems are more permissive than what some ver specifies. Now the next question is, if you remember all the pictures I have shown before, is that we found a difference between two of the ecosystems that have lots of packages that are still in zero version space, while in the other two ecosystems, there was many much less packages that are still in zero version space. Why is this the case? We looked at the default initial version that was set for newly created packages in each of the package managers. If you take the cargo package manager, and if you do a cargo init, then it will create an initial version with as default value 0.1.0. So it's clearly in the zero version space. In NPM, it's different. In fact, it's different since April 2014, where if you would use NPM init, it would set the default package version of a newly created package to 1.0.0. Exactly because the ecosystem managers do not agree with the fact that the same ver specification does not adhere to or is different for 0.ex.y versions, which only leads to confusion. For packages, there is no specific initial version that's set. Basically, the version number used for a newly created package is used from the Git tag that can be found in the Git repository where the package is being developed. And for RubyGems, it's like cargo, the initial version will be set as 0.1.0. So cargo and RubyGems, they set as a default 0 version as initial version, while NPM sets a 1 version, and while for packages, it's basically the developer that decides. So because of this, we do find a big difference across the four different package distributions in what is the initial version being set for different packages. So for packages and NPM, we see that a majority starts in version 1.0.0. In NPM, it's maybe still a majority, but I guess it will increase over time since the new policy adopted by NPM started in 2014. While for cargo and RubyGems, the proportion of packages newly created that are starting in 1.0 is really minor simply because the default policy is to set 0.1.0 version. So this explains why basically these two different ecosystems are majorly in zero space because of the default 0.1.0 version that is being set upon creation of a new package. This is actually showing the same thing, but in evolution over time. So there is nothing special to say about this except for the fact that for example in NPM, we see that since 2014, where NPM changed its policy of specifying the creation of a new package to 1.0.0. The proportion of releases being in zero space is actually decreasing. So we see that the effect of a policy can make a big impact. What we also notice if you look at the percentage or the proportion of releases that are still in zero space, that it's really abundant and there is lots of releases that are still in zero space ranging from 2 out of 10 for packages, the green one up to 9 out of 10 for cargo, the blue one. So 0.1.0 releases are really prevalent everywhere. What can we reduce from all of this or recommend from all of this? Basically, if it is the goal of package managers like cargo and RubyGems to stick better to the semantic versioning policy or to inside their package maintainers to move out of the zero version space sooner, then they should probably change their release policies and more in particular to the way of specifying the initial version number of a newly created package. Now let's focus on dependencies. We have lots of packages that are in zero version space and we want to know how much to which extent other packages depend on such packages. So what we see here is the distribution of the number of dependent packages that depend either on a package still in version zero or a package that is in one space. And we see this for the different ecosystems. If you look at these different distributions, it's difficult to see a difference. In fact, we don't really see a difference between depending on a zero version package or one package. There is lots of packages that depend on packages in one space, but there is also lots of packages depending on packages that are still in the zero space. And in fact, from a practical point of view, we try to find out, do we find the difference between those packages that depend on packages with major version zero and those packages that depend on packages that are in major version one. And we couldn't really find the big difference. They had a comparable release frequency, they had a comparable number of dependencies, a comparable number of stores, number of folks and so on. So in practice, there is developers that use dependent packages don't seem to make a big distinction between depending on a one plus package or depending on a zero version package. So if we consider zero packages to be problematic, then in that case, the maintainer of those packages should be invited to cross the one of zero barrier as soon as possible, in order for packages that are already in the one space to safely depend on them without having the problem of having any breaking changes or not being able to respect the semantic versioning policy. From all of this, what can we conclude? Well, there is many different things that we can conclude from this presentation and the findings of our analysis. First of all, if you look at semantic versioning, then the rules are a bit confusing because semantic versioning makes an explicit distinction between its rules for one plus packages and for packages that are still in the zero space. And moreover, the distinct package managers that we studied, they all have a different way of following the semantic versioning. They are not fully semantic versioning compliant, especially when it concerns packages that are still in the zero version space. What we also found that, well, although people believe or think that packages that are still using a zero version number are not yet mature or production ready, in practice, that doesn't seem to be the case. There is lots of packages in the zero version space that are popular, that are mature, and in terms of dependencies to packages in the zero space or run space, we couldn't observe any practical difference between how they are used. Still, there appears to be a big artificial psychological barrier that makes it so that lots of packages never are only very slowly traverse the mythical 1.0 barrier. So what can we do to improve upon this? First of all, we believe that there is a need to increase the awareness of the specific versioning rules and dependency constraints being used by particular package manager. We also believe that there should be more uniformity in these rules across package managers. We also believe that there should be a better alignment between the semantic versioning rules and the rules being used by different package managers. And if there is a difference, this difference should be made more explicit and should be documented more clearly to the package manager community. And finally, the last recommendation is that if package maintainers consider their package to be production ready, they should be incited to move out of the zero space as soon as possible. Okay, so this is the conclusion of my presentation. I hope you have any questions and I would be delighted to respond to them. Thank you for your attention.
 Hello everybody, my name is Tom Lenz.  I'm a professor at the University of Moles in Belgium in the computer science department  in the software engineering laboratory.  The focus of this presentation will be on the relation between package dependency management  and reliance on packages that still have releases with a zero major version component.  This research is conducted as part of a project called Seiko Assist, which is a Belgian inter-university project.  I will refer to the problem of depending on packages in version, major version zero as the zero space problem.  So this research has been conducted together with another resource associated with called Alexandre de Comte.  This presentation has also been published or will be published soon in a journal article.  The link to the full details of this publication can be found below in this download that's available on archive.  Next I will report on the results of some empirical analysis that we conducted on four different package management systems, cargo for rust packages, ruby jumps for ruby packages,  packages for PHP packages and NPM for the Node JavaScript ecosystem.  In total there is lots of packages available in these ecosystems. You can see the number of packages that's range between 35,000 in the smallest ecosystem  and up to over one million different packages in the largest one, which is NPM.  So we have package ecosystems of different orders of magnitude.  In terms of number, each of these packages tend to have multiple releases, so the number of releases is sometimes even a tenfold of the number of packages ranging from 183,000 to almost 9 million in NPM  and even more dependencies going up to 48 million dependencies in NPM.  In total this data we have downloaded from the open source repository and dependency metadata in the libraries.io dataset,  which if you are interested in, it's a very useful source of data that you can access through this Zenodo digital object identifier.  If you study the different packages in these four different package managers, obviously you will find packages with many different releases  and each of these different releases they will have a particular version number.  All of these packages, they claim that they follow the semantic versioning policy, so basically they will have a three-component versioning system  composed of a major version, a minor version and a patch version.  The packages will have a version number that is above 1.0.0, so you will have versions like 1.0, 4.0.6, 7.1.3 and so on.  This is what we call the one-plus space.  We will also find lots of packages that are still in what we call the zero space, which is versions where the major component is zero.  For example, 0.0.1, 0.1.0, 0.12.3 and so on.  What we wanted to study was to which extent are packages in these four different package managers making use of releases in the one-plus space  or of releases in the zero space and how this affects the dependency management in these different package managers.  Why did we want to do this?  Basically because it is generally considered that if you have packages that have a major version component 1.0, that they are considered to be stable packages,  also more mature packages, and hence they are more likely to be popular.  On the other hand, if we look at packages whose releases are still in the zero space, then they are often considered as being unstable and still under development.  So they are not yet mature.  But is this really the case? Is this common wisdom really the case?  Well, let us just have a look at two examples.  Take for example one package in NPM, which is zero kit web SDK, which is currently in version 4.0.6.  So it's already a pretty high first version number.  So one could consider this package to be already mature and stable.  Is this really the case?  I would say not really because in total there has been only five releases of this package over time.  In total only 16 commits and there's been one contributor active for this particular package.  In fact, it even turns out that this package has no longer been maintained since July 2017.  And because of this, it has been archived on GitHub in June 2018, even though it's still available on NPM.  If we take another example in the zero space, that we have for example Axios, which is a package that is still in zero release, 0.21.1 to be precise.  So one could consider that because it's still in a zero version, it's not production ready yet.  But on the other hand, if we look at this package, if we look at this package history, it has had over time 46 different releases,  a total of almost 1000 commits and 257 different contributors.  Moreover, if you look at the details on GitHub, we found that it has on average 10 million weekly downloads,  about 50,000 different packages, other NPM packages that depend on it.  And moreover, on GitHub, we also can see that it is used by roughly 3.5 million other GitHub repositories.  So one cannot really say that this package is not production ready.  So one could argue that the fact that there are packages in the zero space that are already production ready,  they tend to follow the zero based versioning scheme, which is actually some term that has been coined by a person that developed this satiric website,  zerover.org, where he's making mocking of people that stick to only major version zero for their releases.  And use this as a statement to say that if you have a major, that your software's major version should never exceed the first and most important number in computing, namely zero.  In fact, it's a satirical website, but nevertheless, there is lots of popular packages that are actually following pre parenthesis, this zero based versioning strategy.  In fact, if you again look on that website, you will look, it's fine. Lots of packages like react native that have been developed over many different years.  For example, react native, it's very popular project. It has had 345 different releases.  It has been existing over 5.5 years, and it's still currently under major version zero today.  The same for the project I just mentioned before, Axios, which is still under its version zero.  Even though it has been developed over six years, and it has a lot of stars and lots of dependence, and so on, we can go over this list for quite a long time and find lots of extremely popular packages that are still in their major version zero.  So this led us to consider the following research question. Is it true that many packages tend to get stuck in zero space?  To find these out, we did some analysis to compute the proportion of packages in each of the four studied ecosystems that still have a release in a particular version range.  So either only the version range belonging to the zero space, the version range belonging only to the one plus space, or a version range belonging to both.  So if a package has releases in both zero dot y dot z, and other releases that have a version superior to one, then we would consider that they are in the both.  If we do this for the four different ecosystems, then we actually find that there is only a very small minority of packages in all the ecosystems that have actually crossed the one dot zero dot zero barrier.  So in the beginning of their lifetime, they were a package with major version zero, and then they migrated to a version that was superior to one.  In many packages in the different ecosystems, we found that they actually got stuck in the zero space and never migrated to version one.  The proportion of such packages really depends on the considered ecosystem. In fact, we can find that there is two types of package managers, those like packages and npm,  in which there is only a minority of packages that are stuck in zero version space. And then we have on the other hand, Ruby gems and cargo, where large majority of packages are always and will continue to remain in zero space.  This is about 75%, so three out of four packages always have had zero release. And for cargo, it's even worse because it's up to 92%.  So for this, it seems to be the case that they are actually not following a semantic versioning, but zero case versioning, because they never crossed the one dot zero version barrier.  So now let's focus again on those packages that are actually in the both category. So packages that have actually migrated from zero version to one version.  Then we did some survival analysis to find out how long did it take for these packages to migrate to the one plus space.  So if we compare the time between their first zero release and their first release superior to one, then this survival analysis shows the proportion of packages having done this.  And actually what we found that in the major for the majority of packages, it only took a few months to cross the mythical one dot zero barrier, but still for one out of five packages, it took over more than a year.  And for the worst ecosystem, namely in RubyGems, in many cases, it took over two years before, if they migrated actually to the one plus space, it took over two years to do this.  So now this is about the prevalence of zero releases in these four different package distributions, package managers.  Now let's have a look at the effect of this on the dependency management in these four different packages distributions.  So of course we know that in these four different package managers, you have lots of dependencies between different packages, you can have packages, you can have dependencies between packages that are all belonging to one plus space.  You can also have lots of dependencies between packages between the zero space, but more importantly, there is of course also dependencies that go from packages that are considered stable and that are in the one plus space to packages that are still in the zero space and vice versa.  So we wanted to find out to which extent this is the case, and to which extent is this a problem. More in particularly, let us focus on the red arrows, which are dependencies from packages that have a release that is above one dot zero, and that depends on other packages that still have a release with major version zero.  So we asked ourselves the question, is it actually a problem for a mature, popular package to depend on a package that still have a zero dot y dot z version.  Why did we ask ourselves this question? Because we found several citations of popular books for different ecosystems, that this is considered to be a problem.  For example, Jeremy Khan in his blog for NPM said that you cannot trust a project that depends on another project that is still in a zero dot y dot z version, because such projects are not meant to be ready for use.  For cargo, Rust, there was a similar reference that you can find on this blog here, where they consider, depending on a package that is still in version zero dot y dot z, that's a bad thing, and that should be avoided.  So of course, this is just two different blogs where some people say that it's a problem. Is this generally the case? Difficult to say.  So to get some more insight in this, we try to find more qualitative evidence of this by doing a survey on LinkedIn and on Twitter with in total 102 respondents, where we asked them the following question.  If you are a developer, both of you need to depend on an open source package distributed through one of these package managers, would you actually trust depending on a package that has major version zero?  There were four possible answers.  Obviously, I'm sure I don't mind depending on this or no, I would never depend on such a version, and then some intermediate responses which say I would only do this after checking the package or the package history for this particular package I would like to depend on, or I would only do this if there is no real alternative.  So what we find in the responses is that the answer about is it can be, can one trust to depend on zero y z versions is really mitigated because more than half of all responses are in these categories where they say I would only do it after checking or if there is not really an alternative.  So actually, the one where they say no, there is no problem at all of depending on such a package, it's only one out of three. So two thirds of all the respondents said it's, it might be problematic to depend on such a package show so you shouldn't do it without any checking or without looking for alternatives.  Of course, all of this is related to the use of semantic versioning. I don't think I have to explain semantic versioning to most of you, but what is important to recall, if you look at the details of the semantic versioning policy is that in this policy they actually say something explicitly about using a major version zero.  A major version zero is actually intended for initial development only. Why? Because if you have a major version zero, then basically anything may change at any time. Even if you do a patch update, you might still introduce breaking changes.  Just for a quick reminder for those people in the audience that don't know what semantic versioning is, if we take this traditional three component version numbering, then if you would like to depend on some package that has a particular version number, and you want to be semantic versioning  compatible, if you use, you can use dependency constraints to depend on other packages. If you use a tilde constraint, then you signal that the versions of the package you depend on, you allow any increase of their patch number, and you assume that it will not introduce any breaking changes.  If you use a carrot constraint to depend on another package version, then you say that you also allow minor updates of the package you depend upon, and semantic versioning implies that also minor updates are still supposed to be backwards  incompatibilities, so it will not introduce breaking changes. If, on the other hand, you use a very permissive constraint where you also allow automatic upgrades of the major version component of your dependencies, in that case, it might be that you will have to face backwards incompatibility  problems, because a change in the major release number might introduce breaking changes. The problem with this is that not all of the four studies package managers stick to semantic versioning in the same way, and also they also don't use the dependency constraints in the same way.  So here on the left we have listed the different common types of dependency constraints that are being used, and the four different package distributions being considered.  Since we wanted to do an analysis about the relation between using dependency constraints and how this affects the reliance on zero major versions, we first needed to translate the dependency constraint notation into a uniform interval-based notation in order to have the exact meaning of the notation  that might differ between the different ecosystems. For example, if you take cargo and you would specify 1.0, and this would actually mean that you accept any version number between 1.0.0 and the next major release,  well, if you would do this for npm, then this would actually mean only all versions between 1.0.0 and 1.1.0 excluded. And even for packages, it actually would mean just one single version that's allowed.  So we can see that there is differences across different package distributions about how a particular version number is interpreted.  The most important version constraints being used, however, they are used in a more or less systematic way across different package managers.  For example, till the 1.3 is interpreted in the same way for all package managers, and the same is true for carret constraints, except that it's not available for RubyGems.  So if you now relate this to semantic versioning, then basically everything that's showing in red are constraints that are more restrictive than semantic versioning, because semantic versioning that says that everything except major upgrades are expected to be backwards compatible.  So for example, using carret 1.3 is semantic versioning compliant, but using till the 1.3 is more restrictive than somewhere, and then you have the more permissive constraints, which are things like for example anything that's bigger than 1.2.3, or using a stock constraint will be more permissive than somewhere.  Now the interesting part here is that if you look at constraints, dependency constraints to a version number with a major version 0 component, then they are always considered as being more permissive than somewhere.  Why is the case? Basically, because somewhere dictates that if you have a zero version, any upgrade, even a patch upgrade of your zero version will be considered to be potentially introducing a breaking change.  The different ecosystems we studied tend to be more permissive, since they do allow for patch updates, and they assume that patch updates are still considered to be non-breaking.  Here we looked at the different documentation in the four different packaging ecosystems to find evidence of this, and they actually do say this, and they do show the deviation against semantic versioning, but not in an extremely explicit way.  For example, if you have a version 0, then you can consider actually the minor upgrade as a breaking change indicator, which implicitly assumes that patch updates are non-breaking.  Here again the same for the sort of NPM, for cargo it's the same. If you make breaking changes, you can increment the minor versions, so this implicitly assumes that if you upgrade patches, they are not breaking.  And here again, this is for packages where if you have a version that's in the zero space, then the current constraint is considered as a version that is allowing patch upgrades as well, and so on.  So it appears that the four different packaging ecosystems systems are more permissive than some ver specifies.  And when we analyze this, we indeed see that this is the case in all of the four different study ecosystems, the percentage of packages that have a dependency to some other package that's still in version 0, they have no problem with accepting patch updates.  You can see here in green the proportion of dependencies that allow for patch updates, which is a big majority.  And in the case of Ruby jumps, they even allow for minor updates, even if this would clearly be considered as breaking changes in all of the other ecosystems.  So for sure, with respect to dependency management and dependency constraints, the packaging ecosystems are more permissive than what some ver specifies.  Now the next question is, if you remember all the pictures I have shown before, is that we found a difference between two of the ecosystems that have lots of packages that are still in zero version space,  while in the other two ecosystems, there was many much less packages that are still in zero version space.  Why is this the case?  We looked at the default initial version that was set for newly created packages in each of the package managers.  If you take the cargo package manager, and if you do a cargo init, then it will create an initial version with as default value 0.1.0.  So it's clearly in the zero version space.  In NPM, it's different.  In fact, it's different since April 2014, where if you would use NPM init, it would set the default package version of a newly created package to 1.0.0.  Exactly because the ecosystem managers do not agree with the fact that the same ver specification does not adhere to or is different for 0.ex.y versions, which only leads to confusion.  For packages, there is no specific initial version that's set.  Basically, the version number used for a newly created package is used from the Git tag that can be found in the Git repository where the package is being developed.  And for RubyGems, it's like cargo, the initial version will be set as 0.1.0.  So cargo and RubyGems, they set as a default 0 version as initial version, while NPM sets a 1 version, and while for packages, it's basically the developer that decides.  So because of this, we do find a big difference across the four different package distributions in what is the initial version being set for different packages.  So for packages and NPM, we see that a majority starts in version 1.0.0.  In NPM, it's maybe still a majority, but I guess it will increase over time since the new policy adopted by NPM started in 2014.  While for cargo and RubyGems, the proportion of packages newly created that are starting in 1.0 is really minor simply because the default policy is to set 0.1.0 version.  So this explains why basically these two different ecosystems are majorly in zero space because of the default 0.1.0 version that is being set upon creation of a new package.  This is actually showing the same thing, but in evolution over time. So there is nothing special to say about this except for the fact that for example in NPM, we see that since 2014, where NPM changed its policy of specifying the creation of a new package to 1.0.0.  The proportion of releases being in zero space is actually decreasing.  So we see that the effect of a policy can make a big impact.  What we also notice if you look at the percentage or the proportion of releases that are still in zero space, that it's really abundant and there is lots of releases that are still in zero space ranging from 2 out of 10 for packages, the green one up to 9 out of 10 for cargo, the blue one.  So 0.1.0 releases are really prevalent everywhere.  What can we reduce from all of this or recommend from all of this?  Basically, if it is the goal of package managers like cargo and RubyGems to stick better to the semantic versioning policy or to inside their package maintainers to move out of the zero version space sooner, then they should probably change their release policies and more in particular  to the way of specifying the initial version number of a newly created package.  Now let's focus on dependencies.  We have lots of packages that are in zero version space and we want to know how much to which extent other packages depend on such packages.  So what we see here is the distribution of the number of dependent packages that depend either on a package still in version zero or a package that is in one space.  And we see this for the different ecosystems.  If you look at these different distributions, it's difficult to see a difference.  In fact, we don't really see a difference between depending on a zero version package or one package.  There is lots of packages that depend on packages in one space, but there is also lots of packages depending on packages that are still in the zero space.  And in fact, from a practical point of view, we try to find out, do we find the difference between those packages that depend on packages with major version zero and those packages that depend on packages that are in major version one.  And we couldn't really find the big difference.  They had a comparable release frequency, they had a comparable number of dependencies, a comparable number of stores, number of folks and so on.  So in practice, there is developers that use dependent packages don't seem to make a big distinction between depending on a one plus package or depending on a zero version package.  So if we consider zero packages to be problematic, then in that case, the maintainer of those packages should be invited to cross the one of zero barrier as soon as possible, in order for packages that are already in the one space to safely depend on them without having the problem of having any breaking  changes or not being able to respect the semantic versioning policy.  From all of this, what can we conclude?  Well, there is many different things that we can conclude from this presentation and the findings of our analysis.  First of all, if you look at semantic versioning, then the rules are a bit confusing because semantic versioning makes an explicit distinction between its rules for one plus packages and for packages that are still in the zero space.  And moreover, the distinct package managers that we studied, they all have a different way of following the semantic versioning. They are not fully semantic versioning compliant, especially when it concerns packages that are still in the zero version space.  What we also found that, well, although people believe or think that packages that are still using a zero version number are not yet mature or production ready, in practice, that doesn't seem to be the case.  There is lots of packages in the zero version space that are popular, that are mature, and in terms of dependencies to packages in the zero space or run space, we couldn't observe any practical difference between how they are used.  Still, there appears to be a big artificial psychological barrier that makes it so that lots of packages never are only very slowly traverse the mythical 1.0 barrier.  So what can we do to improve upon this? First of all, we believe that there is a need to increase the awareness of the specific versioning rules and dependency constraints being used by particular package manager.  We also believe that there should be more uniformity in these rules across package managers. We also believe that there should be a better alignment between the semantic versioning rules and the rules being used by different package managers.  And if there is a difference, this difference should be made more explicit and should be documented more clearly to the package manager community.  And finally, the last recommendation is that if package maintainers consider their package to be production ready, they should be incited to move out of the zero space as soon as possible.  Okay, so this is the conclusion of my presentation. I hope you have any questions and I would be delighted to respond to them. Thank you for your attention.
dep_depending_packages_major_version_zero_0_167.png dep_depending_packages_major_version_zero_168_803.png dep_depending_packages_major_version_zero_804_923.png dep_depending_packages_major_version_zero_924_1799.png dep_depending_packages_major_version_zero_1800_4055.png dep_depending_packages_major_version_zero_4056_4547.png dep_depending_packages_major_version_zero_4548_5183.png dep_depending_packages_major_version_zero_5184_6887.png dep_depending_packages_major_version_zero_6888_8267.png dep_depending_packages_major_version_zero_8268_9875.png dep_depending_packages_major_version_zero_9876_10991.png dep_depending_packages_major_version_zero_10992_12191.png dep_depending_packages_major_version_zero_12192_13343.png dep_depending_packages_major_version_zero_13344_13931.png dep_depending_packages_major_version_zero_13932_14411.png dep_depending_packages_major_version_zero_14412_14519.png dep_depending_packages_major_version_zero_14520_14567.png dep_depending_packages_major_version_zero_14568_14651.png dep_depending_packages_major_version_zero_14652_15539.png dep_depending_packages_major_version_zero_15540_15791.png dep_depending_packages_major_version_zero_15792_16187.png dep_depending_packages_major_version_zero_16188_17219.png dep_depending_packages_major_version_zero_17220_19043.png dep_depending_packages_major_version_zero_19044_19607.png dep_depending_packages_major_version_zero_19608_20075.png dep_depending_packages_major_version_zero_20076_20855.png dep_depending_packages_major_version_zero_20856_23183.png dep_depending_packages_major_version_zero_23184_24323.png dep_depending_packages_major_version_zero_24324_26867.png dep_depending_packages_major_version_zero_26868_31595.png dep_depending_packages_major_version_zero_31596_31715.png dep_depending_packages_major_version_zero_31716_33563.png dep_depending_packages_major_version_zero_33564_33803.png dep_depending_packages_major_version_zero_33804_34271.png dep_depending_packages_major_version_zero_34272_34931.png dep_depending_packages_major_version_zero_34932_35195.png dep_depending_packages_major_version_zero_35196_38039.png dep_depending_packages_major_version_zero_38040_39719.png dep_depending_packages_major_version_zero_39720_40535.png dep_depending_packages_major_version_zero_40536_40619.png dep_depending_packages_major_version_zero_40620_42083.png dep_depending_packages_major_version_zero_42084_42719.png dep_depending_packages_major_version_zero_42720_42791.png dep_depending_packages_major_version_zero_42792_45035.png dep_depending_packages_major_version_zero_45036_48971.png dep_depending_packages_major_version_zero_48972_49187.png dep_depending_packages_major_version_zero_49188_49599.png