Software Ecosystems as Networks
CC Attribution 2.0 Belgium:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Boldi, Paolo
The goal of the EU project FASTEN is being able to perform a more sophisticated analysis of security-vulnerability propagation, licensing compliance, and dependency risk profiles (among others) by relying on the call-level dependency network of the whole software ecosystem. We outline the purpose and structure of the project, and present some preliminary results.
 Hello, good morning. I am Paolo Baud from the University of Milan and today I'm gonna talk about FASTA. FASTA is a European project started two years ago that I am contributing to and the plan of my talk is to present you with the main goals of FASTA. I will try to explain why we decided to start this project in the first place and what will be the benefits of this project for the community and in the second part I will tell you something about the advances of the project so far. So the FASTA project, FASTA means fine-grained analysis of software ecosystems as networks. As I said, it's an Horizon 2020 project and it's a consortium including many universities, among them the University of Milan that is the place where I work and a number of companies. The idea of the FASTA project to begin with is to study a better way to distribute software libraries. Let me start by telling you something about sharing. Somebody said that the fundamental act of cooperation among programmers is through software sharing and today sharing happens typically distributing software libraries. In a way we can say that internet made the dream of collaborative development a reality because today we can easily use external libraries that are made available in the internet on so-called repositories like SourceForge or GitHub or Bitbucket or on forges like Maven, PyPy, Cipen etc. Through these repositories it is extremely easy for everybody to write new software based on existing libraries. If you want, this is the industrial revolution arriving at the arbor of software development because thanks to the existence of software libraries we have a way to implement the division of labor that according to Kant is in fact the key point of industrial revolution. In a way we can imagine and in fact this is the truth that we are experimenting that the existence of software libraries made it possible for software writing and software development to grow in an incredible way in the last two decades. Now in practice software libraries, the use of software libraries is implemented through dependencies and dependencies form a complex network that we call dependency graph. In reality what you do when you write software that depends on some other library is specifying which libraries you depend upon and which version constrained you want on those libraries. Sometimes you pin a specific version of a specific library but more often than that you just specify a library with a version constraint like any version from 3.0 onwards would be okay. This is a specification of a dependency and if you look at dependencies and their transitive dependencies they form a network not really a graph because versions are specified via constraint and this graph is really huge and I call them graphs because in fact they become graphs when a package manager is taken into consideration. Package managers are there precisely to determine which version will be chosen for each library. This choice is different depending on the package manager you're using and depending also on some external context like the time the point in time when you are building your own artifact. So all in all the set of libraries and the mutual dependencies among libraries is what we call a dependency heaven. As I say this is the place where you can rely on an ecosystem of easy to use well written libraries and reusing their code you can dream to make to build up and build more and more complex libraries. But all of a sudden the dependency heaven can turn into a dependency hell. It's enough that a simple bug or security breach is found in one of the libraries you depend directly or indirectly upon that the whole tower falls down. There are many examples of how dependencies can turn into a nightmare. Some of them are extremely well known and extremely serious probably the two recent examples that everybody knows about are the left pad incident back in 2016 where removing a simple 13 lines of code library from javascript forge affected millions of huge websites or the Equifax breach 2017 that costed about four billion dollars. The problem here is that ecosystems grow ecosystem is a set of libraries in a given context and ecosystems grow at mind-boggling speed. Just to give you an example on the average in 2019 Zimmerman evaluated that javascript projects depend on an average of 80 other libraries and dependencies change very rapidly like more than half of them change in a six month time. So these ecosystems grow at mind-boggling speed but at the same time they deteriorate almost as rapidly because of many reasons one of them is that there exist out there packages that for bottlenecks the removal of those packages can bring down almost 40 percent of the whole system and also because few maintainers really dominate most packages so those ecosystems are very powerful they grow fast but they are also very fragile they tend to deteriorate very rapidly. Just to give you an example of what we mean by deterioration think of the typical situation that we may refer to as epidemics, independency graphs. Suppose that you have a situation like the one depicted here, here you see four libraries with specific versions and the dependencies between those libraries and suppose that at some point a vulnerability alert is issued about version 3.0 of library. Now all the libraries that are in the pictures are potentially infected so in theory as soon as this vulnerability issue is published the developers of libraries A, B and C should take some decision. In particular there are today many systems that offer you alerts. The tab for example recently deployed security alert system that alerts you as soon as there is a known vulnerability issue on some of your transitive dependencies but is this enough? Well really not, really not. In theory those systems are enough to allow developers to take some step but in practice developers simply don't update their dependencies and so vulnerabilities proliferate. If you want to wonder why, why aren't these tools enough our answer is that those tools are not sharp enough for what we want. I want to give you an example of the kind of question developers and maintainers would like to be able to answer and that would make dependency management really more effective. Typically a developer would ask things like does this outdated dependency really break my code or on the side of maintainers how can I update my libraries or my APIs without breaking too many clients or too many of my most important clients? Another completely different question is am I violating anyone's copyright or on the side of maintainers how do I spot instances of my code being distributed without permissions? All these questions are not really talking about dependencies at a package level they are really talking about dependencies at a call level. To understand better what I mean let me go back to my examples. Now let us try to look into the code in this expanded version of the picture instead of just looking at packages we are looking at actual functions or callables or methods as you prefer to call them within each package. Now here you can actually see what method or what function calls which functions. If the vulnerability alert is concerned with function f3 of module d then as you can see not all the functions and in fact not all the packages are really affected. This kind of information is of course much more informative and it avoids the cry-wolf effect because if you had this kind of information then you would know precisely which of your functions are affected. For example maintainers of library a's can simply don't care because in fact even if in theory they are affected in practice they are not whereas developers of library b know exactly where the vulnerability impacts in their code and they know exactly what to do. This idea of bringing dependencies at a call level can answer questions that today cannot be answered simply like how many libraries are affected if I remove a certain method or interface or is my library compliant with the licenses of the libraries that I depend upon or does this vulnerability really affect my code as in the example I showed you before or what are the methods or functions that are more central in a given ecosystems what are the real bottlenecks what are the critical call points. This is the starting point of Fasten if you if you want and from this you can understand the Fasten toolchain. The ideal toolchain is that we take information coming from various sources like repositories, forges, sources of security alerts and so on and all this information is published in a data stream that the Fasten server uses to enrich its own internal knowledge base. To enrich the knowledge base the first and most important step is every time a new package version is published we need to build a call graph. Once we have the call graph we have a storage layer that stores the call graph along with all the metadata information that is needed for further analysis and in fact there is an analysis layer that also enriches the metadata with further metadata that will be used later to answer to users query either through REST APIs that can for example be communicated with a continuous integration server or directly answering to developers queries. Let me show the preliminary results we have obtained in the first two years of the project. First of all some server side highlights. As I said the first and probably the most critical step is the call graph generation. Here you can see the data flow of the call graph generation plugin. In practice whenever a forge publishes a new version of a package this is fed through a custom scraper or through a code feeder to the Fasten server. At this point there is a call graph generator engine that tries to extract the call graph and store it in the Fasten database for later generating what we call call graph stitching. I will get back to what call graph stitching is in a second. Of course call graph generation is language specific but before entering into the details of how call graphs are generated let me highlight that we need in the first place a uniform way to refer to identifiers to functions that is independent on the forge, independent on the language and so forth. For this reason we have a universal function identifier that is a specific kind of URI. We call them Fasten URIs. As you can see in a Fasten URI you have a full specification including the forge, the artifact, the version of the artifact, the namespace that of course changes depending on the language, the function or method depending on the language and its signature and so on. We have a generic URI syntax that then gets specialized on the languages. Then we have a transport layer. For transport layer we use JSON and the JSON scheme that we are using is what we use within the server and also to feed the Fasten server with new data about call graphs. But as I said the the central point here is call graph generation. As you all know generating a call graph is a hard problem. Technically it's impossible to generate a complete and sound call graph but we can try to do our best and on the three languages Fasten is based on that is Java, Python and C. We use different techniques. For Java we use tools based on the OPAL project. For Python we use a new static analysis tool called PyCG and for C we use a mix of static analysis using C Scout and a number of dynamic profiling systems based on Gprof and call graph. Here you can see the number of packages we have analyzed so far. For the moment the language which on which we have the largest number of artifacts analyzed is Java and in fact we have analyzed more than two million almost three millions of artifacts so far. When I say artifact I mean package release so we may have more than one call graph for every package of course. We will have one such call graph for every version of a package. As I said those call graphs are analyzed and stored in a way that decouples package resolution from call graph generation. In a way what we do is every time we generate the call graph the call graph will also include unresolved calls that will be later attached to actual functions or to actual callables in actual call graphs once the resolution will be done. Call graph stitching is the call level resolution of a given dependency tree simulating a given package manager at a given time or within a given context. This is the database schema we use. In the database we store a number of metadata about call graphs. The actual call graphs are stored in another database. I will get back to this in a second but let me show you some examples of a type of queries you can write using only the metadata like this query answers the question what are the largest packages by number of callables or what are the packages that depend on a given vulnerable package. These are queries that are made only at metadata level so they cannot look at the actual calls. Now the actual call graphs as I said are stored in a separate database because the point here is that we need to store them in an efficient way both space efficient and time efficient. And for this we use a state-of-the-art system of graph compression that is called web graph. And currently we are using this system that was originally developed for web graphs and for social networks and in fact we have adapted it to work very well on call graphs. Currently we can store them at 3.6 bits per edge plus some space that is needed to store global IDs. We go up to nine bits per edge which is in any case very good. These formats of graph storage are compression systems that are also very efficient when you use them because you don't need to decompress the graph to traverse it. Currently we estimate that the whole Maven fits in 38 gigabytes which is an extremely good result and the reason why we can do such a good compression is that most artifacts are stored with a deflation that is 50% or less. In this picture here you can see the compression results we get depending on the graph size and the radius of those balls there are the number of packages within of artifacts within Maven that have that specific graph size and that specific deflation. Now let me offer you some other plugins that we use on the server side. First of all we have the vulnerability plugin. The vulnerability plugin gathers information about vulnerabilities from different sources and what the plugin does is producing a normalized vulnerability object. The normalization is needed to smooth out the different sources of information the different places that publish vulnerability information and the plugin continuously pulls updates from new information and keeps storing the results in the metadata database. We also have an analysis plugin that is called Rapid. Rapid also works on the server side and does risk analysis and inspect propagation for security and maintainability risks. The plugin code currently is deployed in its first version. It processed about 200,000 Maven coordinates and on the client side it can be used there is a user application that is able to model and present to the user the risks of its own code. We also have a license and compliance analysis plugin that is used to gather information about all the generated artifacts and stores in the metadata database relevant license and authorship information at a callable level. On the client side we are developing a REST API that exposes canned queries from the metadata database. In development we also plan to have a full DB entity support with custom extension points. As you have seen at the beginning of my presentation we have a number of companies that are part of the consortium three of them and the code CIG and XWiki. They are all offering use cases. All of them at the same time offer part of their code some of their libraries to the project and on the other hand they integrate Fasten in their own solutions. For example Endocode developed a license compliant solutions called Quartermaster. Currently they are integrating Fasten to improve the precision of their compliance offering. CIG on the other hand integrates Fasten in better code hub that is a good GitHub connected code quality monitoring product. Whereas XWiki is doing risk validation in the installed extensions of their XWiki instances and they also discover XWiki components in available extensions and so on. What is the future timeline? We are as I said at two-thirds of the project. In the first quarter of 2021 we plan to end the impact analysis and integration with Maven and PyPy and for the second quarter the industrial cases use cases will be integrated. In the third quarter we plan to fully integrate the licensing and security plugin and by the end of the quarter the project will finish with the external integrations. We may have a second run of Fasten in 2022 but it's too early probably to to mention it now. In a nutshell the take-home message that I wanted to to share with you is that network analysis at a call graph level will be the next step for the future of software development and Fasten can be and hopefully will be a key actor in this new era of software development that will allow developers to have information at a very fine-grained level about the code that they use and also about their own code. With this I end my presentation and I am ready to take your questions. So I certainly agree that in the long run there must be some form of correlation between software quality and popularity but in the short run I think that there are so many reasons why somebody decides to use a specific library in its own code. So many reasons beside quality that it's hard to say that in the short run the number of people that are using directly or indirectly the piece of code can be related to the quality of the writing of the code. It may depend on simply how good was advertised or on the fact that it's the only currently the only library doing that available or depending on the specific ecosystem you decided to put your library in. So as I said I think at the beginning at least quality and popularity are not related. Of course in the long run maintaining code and keeping up to the expectations probably is related to software quality but as I said people that started using a library will sort of delay their change in the library. In many cases people simply decide not to change their dependencies even if they know for sure that there are vulnerabilities of bias because of laziness because after all they are not sure that the vulnerability will impact their code and so on. So one of the purposes of the project that I was describing in the talk is precisely to help people decide better which library to depend upon and there are many ways to do that for example there is going to be a sketch of a quality evaluator by which you can ask how good is this library? Is it a good thing that I decided to use it or not? So I totally agree that in a way popularity and quality are related but not I don't know of any study of any precise study about it but I'm pretty sure that in the short run this is not true. In the long run probabilities FOSDEN will hopefully help people choose the right library. Do you think that there can be specific outcomes that can help move towards a direction that where quality and popularity are more aligned is this something that should be the case and how can this be affected? Well hopefully so in the sense that software quality depends pretty much on a fine-grained evaluation right evaluation at the level of the code at the level of the course. So the fact that the overall idea of FOSDEN is to move all the software development decision to the fine-grained level of course. Now at that level you will have many more tools that will allow you to make decisions in a better way. So I think that if people will use the tools that we are developing certainly this alignment between quality and popularity will be better simply because you will not choose a library it's like having trip advisor right right. So it's like having a tool that will allow you in a better way to know beforehand whether it's a good idea or not to choose something but based on an excellent evaluation of quality of the code. You mentioned quality of the code and I see also some discussion taking on the live stream people talk about other aspects of quality think about support the community and so on how can this be monitored depicted analyzed in a graph structure network. Yes one of the central point in particular one of the things that I personally am contributing in the project is precisely the notion of centrality of things. Now if you have a network like a call graph network or a dependency network one of the most important aspects is to evaluate which nodes in this case nodes are either specific callables or specific packages are more important and the notion of importance can be based on many different aspects like quality is one, popularity is another one, the size of the community that is supporting that specific package is another one and typically in centrality evaluation one tries to find a mix of the various aspects so you will have at the end of the day many different scores about a specific callable or a specific library and you will mix them to find which one better mixes all the aspects that are quality the size of the community and so on. Alex Savras made an interesting comment I didn't mention licensing of course but the licensing has an additional aspect in that some licenses have are either not compatible with each other or affect downstream licensing do you think this could be part of the graph and the monitor and analyzed? Yes we are going to offer a plugin for that so in fact we are in the Fasten server every single callable is sorry it contains metadata about licensing and you can use people will be able to use the software to understand if licenses are compatible if they are coherent and at the level of callable so certainly this is also an important aspect of course related to quality and usability and it is important at this point that you have some button to put that you can click to understand if the license you are using is compatible with your dependencies also transit. Those ecosystems are so complex and the licenses are so difficult even to understand that having an automated way to do that is central. Returning back to the earlier discussion regarding quality and popularity comments was asking if we assume that these are not the main reasons for using a library how should we as a research community assess quantify measure what other reasons are important for adopting a library so that we can also help maintainers improve their libraries. Can you ask again the question so? The question is that there are other reasons apart from quality and popularity for using a library I would for example think functionality or compatibility with other libraries or dependencies on other libraries and the question is how we can as a research community assess quantify and measure those reasons in order to help library maintainers. Yes I think that as I said trying to answer the question in the first place the notion why the notion of popularity can be made precise in a quantitative way the notion of quality is not so clear. There are many notions of quality all of them are justifiable I mean not only software quality but effect as you mentioned that the library is simply effective in solving the problem etc. As a community we first should understand that all these notions of quality that are sometimes of odds are important and they are equally good reasons to adopt a certain library or to use a certain library. At the end of the day we also should say that okay guys this library here exists just because it's the only one for this goal for for this specific purpose but be careful because its quality is so low. So probably highlighting the fact that you should do more about a specific application and we'll have the community understand which are the bottlenecks on which people can work to produce better quality library for that specific purpose. I very much like your comment that quality has many reasons and many facets to it and all are valid because this is the widest definition of quality whatever the customer or in this case the library user is comfortable and thinks is important for them. So which other things would you consider as important to monitor? Certainly for example one thing that is pretty obvious to measure is the notion of popularity and the various notions of quality you can measure if the reason is matched like an outlier. Normally in a best possible world a popular piece of software should be of high quality with a large community it should be useful it should be working very well etc but we know much too well that there will be outliers guys that are extremely popular but of low quality guys that are extremely popular but they have a small community of supporters etc. Helping singling out those outliers will be the starting point I think to understand what I was calling the bottlenecks right probably the better world would be the outliers and as knowing that there are outliers and which ones are there probably is a good starting point to have better ecosystems and more coherent ecosystems. And that's important the idea of ecosystems this goes outside the actual library but as you said it's something very important is this something that we can measure through the network that can be monitored in this way? Yes well the network is necessary to monitor what is going on and fine grained networks are better than coarse grained ones. This is just a starting point anyway right. Another thing that is certainly needed is to gather together all the sources of information for example about vulnerability about licensing about everything those sources should be gathered together and analyzed. Of course the FASTAN project tries to do something but it's not probably it's not enough we will need more together more sources of information to analyze them better in order to build the metadata about libraries and co-libles in a more in a richer way. At the beginning of your presentation there was some discussion regarding bots that inform users of libraries regarding outdated dependencies and so on there was mention of dependent bot and renovate but what elements of automation do you think of automation do you think can be built on top of this network? Well in a nutshell I would say easier to use tools for developers probably the final step in my imagination would be to integrate these two popular IDEs so that as you develop your favorite integrated development environment will tell you guys be careful this is your calling it function that may be vulnerable you'd better use this one which is newer not so popular but of better quality are you sure you want to use that library not instead of this? And of course an important element of FASTAN is that it focuses more deeply not just to libraries but as you mentioned to functions yes this might open many possibilities of doing additional and more interesting suggestions. Yes also because as I said vulnerabilities are about specific callables not general libraries so sometimes this helps in two ways right because you know that a certain function that you're using even coming from a vulnerable package will be saved and on the other hand if you have a security alert you will be certainly sure that you are in fact in danger I think this will help in having a better understanding of how deadly this is.
 Hello, good morning. I am Paolo Baud from the University of Milan and today I'm gonna talk  about FASTA. FASTA is a European project started two years ago that I am contributing to and the  plan of my talk is to present you with the main goals of FASTA. I will try to explain why we decided  to start this project in the first place and what will be the benefits of this project for the community  and in the second part I will tell you something about the advances of the project so far.  So the FASTA project, FASTA means fine-grained analysis of software ecosystems as networks.  As I said, it's an Horizon 2020 project and it's a consortium including many universities,  among them the University of Milan that is the place where I work and a number of companies.  The idea of the FASTA project to begin with is to study a better way to distribute software libraries.  Let me start by telling you something about sharing. Somebody said that the fundamental  act of cooperation among programmers is through software sharing and today sharing happens  typically distributing software libraries. In a way we can say that internet made the dream of  collaborative development a reality because today we can easily use external libraries that are made  available in the internet on so-called repositories like SourceForge or GitHub or Bitbucket  or on forges like Maven, PyPy, Cipen etc.  Through these repositories it is extremely easy for everybody to  write new software based on existing libraries. If you want, this is the industrial revolution  arriving at the arbor of software development because thanks to the existence of software libraries  we have a way to implement the division of labor that according to Kant is in fact the key point  of industrial revolution. In a way we can imagine and in fact this is the truth that we are experimenting  that the existence of software libraries made it possible for software writing and software development  to grow in an incredible way in the last two decades. Now in practice software libraries,  the use of software libraries is implemented through dependencies and dependencies form a  complex network that we call dependency graph. In reality what you do when you write software  that depends on some other library is specifying which libraries you depend upon and which version  constrained you want on those libraries. Sometimes you pin a specific version of a specific library  but more often than that you just specify a library with a version constraint like any version  from 3.0 onwards would be okay. This is a specification of a dependency and if you look at  dependencies and their transitive dependencies they form a network not really a graph because  versions are specified via constraint and this graph is really huge and I call them graphs because  in fact they become graphs when a package manager is taken into consideration. Package managers are  there precisely to determine which version will be chosen for each library. This choice is different  depending on the package manager you're using and depending also on some external context  like the time the point in time when you are building your own artifact. So all in all the set  of libraries and the mutual dependencies among libraries is what we call a dependency heaven.  As I say this is the place where you can rely on an ecosystem of easy to use well written libraries  and reusing their code you can dream to make to build up and build more and more complex libraries.  But all of a sudden the dependency heaven can turn into a dependency hell. It's enough that a  simple bug or security breach is found in one of the libraries you depend directly or indirectly upon  that the whole tower falls down. There are many examples of how dependencies can turn into a  nightmare. Some of them are extremely well known and extremely serious probably the two  recent examples that everybody knows about are the left pad incident back in 2016  where removing a simple 13 lines of code library from javascript forge affected millions of huge  websites or the Equifax breach 2017 that costed about four billion dollars.  The problem here is that ecosystems grow ecosystem is a set of libraries in a given context  and ecosystems grow at mind-boggling speed. Just to give you an example on the average  in 2019 Zimmerman evaluated that javascript projects depend on an average of 80 other libraries  and dependencies change very rapidly like more than half of them change in a six month time.  So these ecosystems grow at mind-boggling speed but at the same time they deteriorate almost as  rapidly because of many reasons one of them is that there exist out there packages that for  bottlenecks the removal of those packages can bring down almost 40 percent of the whole system  and also because few maintainers really dominate most packages so those ecosystems are very powerful  they grow fast but they are also very fragile they tend to deteriorate very rapidly.  Just to give you an example of what we mean by deterioration think of the typical situation  that we may refer to as epidemics, independency graphs. Suppose that you have a situation like the  one depicted here, here you see four libraries with specific versions and the dependencies between  those libraries and suppose that at some point a vulnerability alert is issued about version 3.0  of library. Now all the libraries that are in the pictures are potentially infected so in theory  as soon as this vulnerability issue is published the developers of libraries A, B and C should take some  decision. In particular there are today many systems that offer you alerts. The tab for example  recently deployed security alert system that alerts you as soon as there is a known vulnerability  issue on some of your transitive dependencies but is this enough? Well really not, really not.  In theory those systems are enough to allow developers to take  some step but in practice developers simply don't update their dependencies and so vulnerabilities  proliferate. If you want to wonder why, why aren't these tools enough our answer is that those tools  are not sharp enough for what we want. I want to give you an example of the kind of question  developers and maintainers would like to be able to answer and that would make  dependency management really more effective. Typically a developer would ask things like  does this outdated dependency really break my code or on the side of maintainers how can I update  my libraries or my APIs without breaking too many clients or too many of my most important clients?  Another completely different question is am I violating anyone's copyright or on the side of  maintainers how do I spot instances of my code being distributed without permissions? All these  questions are not really talking about dependencies at a package level they are really talking about  dependencies at a call level. To understand better what I mean let me go back to my examples. Now  let us try to look into the code in this expanded version of the picture instead of just looking  at packages we are looking at actual functions or callables or methods as you prefer to call them  within each package. Now here you can actually see what method or what function calls which functions.  If the vulnerability alert is concerned with function f3 of module d then as you can see  not all the functions and in fact not all the packages are really affected.  This kind of information is of course much more informative and it avoids the cry-wolf effect  because if you had this kind of information then you would know precisely which of your functions  are affected. For example maintainers of library a's can simply don't care because in fact even  if in theory they are affected in practice they are not whereas developers of library b know exactly  where the vulnerability impacts in their code and they know exactly what to do.  This idea of bringing dependencies at a call level can answer questions that today cannot  be answered simply like how many libraries are affected if I remove a certain method or interface  or is my library compliant with the licenses of the libraries that I depend upon or does this  vulnerability really affect my code as in the example I showed you before or what are the methods  or functions that are more central in a given ecosystems what are the real bottlenecks what  are the critical call points. This is the starting point of Fasten if you if you want and  from this you can understand the Fasten toolchain. The ideal toolchain is that we take information  coming from various sources like repositories, forges, sources of security alerts and so on  and all this information is published in a data stream that the Fasten server uses  to enrich its own internal knowledge base. To enrich the knowledge base the first and most  important step is every time a new package version is published we need to build a call graph.  Once we have the call graph we have a storage layer that stores the call graph along with all  the metadata information that is needed for further analysis and in fact there is an analysis  layer that also enriches the metadata with further metadata that will be used later to answer to  users query either through REST APIs that can for example be communicated with a continuous  integration server or directly answering to developers queries. Let me show the preliminary  results we have obtained in the first two years of the project. First of all some server side  highlights. As I said the first and probably the most critical step is the call graph generation.  Here you can see the data flow of the call graph generation plugin. In practice whenever a forge  publishes a new version of a package this is fed through a custom scraper or through a code feeder  to the Fasten server. At this point there is a call graph generator engine that tries to  extract the call graph and store it in the Fasten database for later generating what we call  call graph stitching. I will get back to what call graph stitching is in a second.  Of course call graph generation is language specific but before entering into the details  of how call graphs are generated let me highlight that we need in the first place a uniform way  to refer to identifiers to functions that is independent on the forge, independent on the  language and so forth. For this reason we have a universal function identifier that is a specific  kind of URI. We call them Fasten URIs. As you can see in a Fasten URI you have a full specification  including the forge, the artifact, the version of the artifact, the namespace that of course changes  depending on the language, the function or method depending on the language and its signature and so  on. We have a generic URI syntax that then gets specialized on the languages. Then we have a transport  layer. For transport layer we use JSON and the JSON scheme that we are using is what we use  within the server and also to feed the Fasten server with new data about call graphs.  But as I said the the central point here is call graph generation. As you all know  generating a call graph is a hard problem. Technically it's impossible to generate a  complete and sound call graph but we can try to do our best and on the three languages Fasten is  based on that is Java, Python and C. We use different techniques. For Java we use tools based on the  OPAL project. For Python we use a new static analysis tool called PyCG and for C we use a mix  of static analysis using C Scout and a number of dynamic profiling systems based on Gprof and  call graph. Here you can see the number of packages we have analyzed so far. For the moment the language  which on which we have the largest number of artifacts analyzed is Java and in fact we have  analyzed more than two million almost three millions of artifacts so far. When I say artifact I mean  package release so we may have more than one call graph for every package of course. We will have one  such call graph for every version of a package. As I said those call graphs are analyzed and stored  in a way that decouples package resolution from call graph generation. In a way what we do is every  time we generate the call graph the call graph will also include unresolved calls that will be later  attached to actual functions or to actual callables in actual call graphs once the resolution  will be done. Call graph stitching is the call level resolution of a given dependency tree  simulating a given package manager at a given time or within a given context.  This is the database schema we use. In the database we store a number of metadata about call graphs.  The actual call graphs are stored in another database. I will get back to this in a second  but let me show you some examples of a type of queries you can write using only the metadata like  this query answers the question what are the largest packages by number of callables or  what are the packages that depend on a given vulnerable package. These are queries  that are made only at metadata level so they cannot look at the actual calls.  Now the actual call graphs as I said are stored in a separate database because the point here is  that we need to store them in an efficient way both space efficient and time efficient.  And for this we use a state-of-the-art system of graph compression that is called web graph.  And currently we are using this system that was originally developed for web graphs and for  social networks and in fact we have adapted it to work very well on call graphs. Currently we  can store them at 3.6 bits per edge plus some space that is needed to store global IDs. We go up to  nine bits per edge which is in any case very good. These formats of graph storage are compression  systems that are also very efficient when you use them because you don't need to decompress the graph  to traverse it. Currently we estimate that the whole Maven fits in 38 gigabytes which is an extremely  good result and the reason why we can do such a good compression is that most artifacts  are stored with a deflation that is 50% or less. In this picture here you can see  the compression results we get depending on the graph size and the radius of those  balls there are the number of packages within of artifacts within Maven that have that specific  graph size and that specific deflation. Now let me offer you some other plugins that we use on  the server side. First of all we have the vulnerability plugin. The vulnerability plugin gathers information  about vulnerabilities from different sources and what the plugin does is producing a normalized  vulnerability object. The normalization is needed to smooth out the different sources of information  the different places that publish vulnerability information and the plugin continuously pulls  updates from new information and keeps storing the results in the metadata database. We also have  an analysis plugin that is called Rapid. Rapid also works on the server side and does  risk analysis and inspect propagation for security and maintainability risks.  The plugin code currently is deployed in its first version. It processed about  200,000 Maven coordinates and on the client side it can be used there is a user application  that is able to model and present to the user the risks of its own code. We also have a license  and compliance analysis plugin that is used to gather information about all the generated artifacts  and stores in the metadata database relevant license and authorship information at a callable level.  On the client side we are developing a REST API that exposes canned queries from the metadata  database. In development we also plan to have a full DB entity support with custom extension points.  As you have seen at the beginning of my presentation we have a number of companies that are  part of the consortium three of them and the code CIG and XWiki. They are all offering  use cases. All of them at the same time offer part of their code some of their libraries to the  project and on the other hand they integrate Fasten in their own solutions. For example Endocode  developed a license compliant solutions called Quartermaster. Currently they are integrating  Fasten to improve the precision of their compliance offering. CIG on the other hand integrates Fasten  in better code hub that is a good GitHub connected code quality monitoring product.  Whereas XWiki is doing risk validation in the installed extensions of their XWiki instances  and they also discover XWiki components in available extensions and so on.  What is the future timeline? We are as I said at two-thirds of the project.  In the first quarter of 2021 we plan to end the impact analysis and integration with Maven  and PyPy and for the second quarter the industrial cases use cases will be integrated.  In the third quarter we plan to fully integrate the licensing and security plugin and by the end  of the quarter the project will finish with the external integrations. We may have a second run  of Fasten in 2022 but it's too early probably to to mention it now. In a nutshell the take-home  message that I wanted to to share with you is that network analysis at a call graph level  will be the next step for the future of software development and Fasten can be and hopefully will  be a key actor in this new era of software development that will allow developers to have  information at a very fine-grained level about the code that they use and also about their  own code. With this I end my presentation and I am ready to take your questions.  So I certainly agree that in the long run there must be some form of correlation  between software quality and popularity but in the short run I think that there are so many reasons  why somebody decides to use a specific library in its own code. So many reasons beside quality  that it's hard to say that in the short run the number of people that are using directly or  indirectly the piece of code can be related to the quality of the writing of the code.  It may depend on simply how good was advertised or on the fact that it's the only  currently the only library doing that available or depending on the specific  ecosystem you decided to put your library in. So as I said I think at the beginning at least  quality and popularity are not related. Of course in the long run maintaining  code and keeping up to the expectations probably is related to software quality but as I said  people that started using a library will sort of delay their change in the library. In many cases  people simply decide not to change their dependencies even if they know for sure that there are  vulnerabilities of bias because of laziness because after all they are not sure that the  vulnerability will impact their code and so on. So one of the purposes of the project that I was  describing in the talk is precisely to help people decide better which library to depend upon and  there are many ways to do that for example there is going to be a sketch of a quality evaluator  by which you can ask how good is this library? Is it a good thing that I decided to use it or not?  So I totally agree that in a way popularity and quality are related but not I don't know of any  study of any precise study about it but I'm pretty sure that in the short run this is not true.  In the long run probabilities FOSDEN will hopefully help people choose the right library.  Do you think that there can be specific outcomes that can help move towards a direction that  where quality and popularity are more aligned is this something that should be the case and how  can this be affected? Well hopefully so in the sense that software quality depends pretty much  on a fine-grained evaluation right evaluation at the level of the code at the level of the course.  So the fact that the overall idea of FOSDEN is to move all the software development decision  to the fine-grained level of course. Now at that level you will have many more tools that  will allow you to make decisions in a better way. So I think that if people will use the tools that  we are developing certainly this alignment between quality and popularity will be better  simply because you will not choose a library it's like having trip advisor right right.  So it's like having a tool that will allow you in a better way to know beforehand whether it's  a good idea or not to choose something but based on an excellent evaluation of quality of the code.  You mentioned quality of the code and I see also some discussion taking on the  live stream people talk about other aspects of quality think about support the community  and so on how can this be monitored depicted analyzed in a graph structure network. Yes  one of the central point in particular one of the things that I personally am contributing in the  project is precisely the notion of centrality of things. Now if you have a network like a  call graph network or a dependency network one of the most important aspects is to evaluate which  nodes in this case nodes are either specific callables or specific packages are more important  and the notion of importance can be based on many different aspects like quality is one,  popularity is another one, the size of the community that is supporting that specific  package is another one and typically in centrality evaluation one tries to find a mix of the  various aspects so you will have at the end of the day many different scores about a specific  callable or a specific library and you will mix them to find which one better mixes all the aspects  that are quality the size of the community and so on. Alex Savras made an interesting comment  I didn't mention licensing of course but the licensing has an additional aspect in that  some licenses have are either not compatible with each other or affect downstream licensing  do you think this could be part of the graph and the monitor and analyzed? Yes we are going to offer  a plugin for that so in fact we are in the Fasten server every single callable is  sorry it contains metadata about licensing and you can use people will be able to use the software  to understand if licenses are compatible if they are coherent and at the level of callable so certainly  this is also an important aspect of course related to quality and usability and it is important at  this point that you have some button to put that you can click to understand if the license you are  using is compatible with your dependencies also transit. Those ecosystems are so complex and the  licenses are so difficult even to understand that having an automated way to do that is  central. Returning back to the earlier discussion regarding quality and popularity comments was  asking if we assume that these are not the main reasons for using a library how should we as a  research community assess quantify measure what other reasons are important for adopting a library  so that we can also help maintainers improve their libraries. Can you ask again the question so?  The question is that there are other reasons apart from quality and popularity for using a  library I would for example think functionality or compatibility with other libraries or dependencies  on other libraries and the question is how we can as a research community assess quantify and measure  those reasons in order to help library maintainers. Yes I think that as I said trying to answer  the question in the first place the notion why the notion of popularity can be made  precise in a quantitative way the notion of quality is not so clear. There are many notions of  quality all of them are justifiable I mean not only software quality but effect as you mentioned  that the library is simply effective in solving the problem etc. As a community we first should  understand that all these notions of quality that are sometimes of odds are important and  they are equally good reasons to adopt a certain library or to use a certain library.  At the end of the day we also should say that okay guys this library here exists just because  it's the only one for this goal for for this specific purpose but be careful because its  quality is so low. So probably highlighting the fact that you should do more about a specific  application and we'll have the community understand which are the bottlenecks on which  people can work to produce better quality library for that specific purpose.  I very much like your comment that quality has many reasons and many facets to it and all are  valid because this is the widest definition of quality whatever the customer or in this case  the library user is comfortable and thinks is important for them. So which other things would  you consider as important to monitor? Certainly for example one thing that is pretty obvious to  measure is the notion of popularity and the various notions of quality you can measure  if the reason is matched like an outlier. Normally in a best possible world a popular  piece of software should be of high quality with a large community it should be useful it should  be working very well etc but we know much too well that there will be outliers guys that are  extremely popular but of low quality guys that are extremely popular but they have a small  community of supporters etc. Helping singling out those outliers will be the starting point I think  to understand what I was calling the bottlenecks right probably the better world would be the  outliers and as knowing that there are outliers and which ones are there probably is a good starting  point to have better ecosystems and more coherent ecosystems.  And that's important the idea of ecosystems this goes outside the actual library but as you said  it's something very important is this something that we can measure through the network that  can be monitored in this way? Yes well the network is necessary to monitor what is going  on and fine grained networks are better than coarse grained ones. This is just a starting point  anyway right. Another thing that is certainly needed is to gather together all the sources  of information for example about vulnerability about licensing about everything those sources  should be gathered together and analyzed. Of course the FASTAN project tries to do something  but it's not probably it's not enough we will need more together more sources of information  to analyze them better in order to build the metadata about libraries and co-libles in a  more in a richer way. At the beginning of your presentation there was some  discussion regarding bots that inform users of libraries regarding outdated dependencies and  so on there was mention of dependent bot and renovate but what elements of automation do you  think of automation do you think can be built on top of this network?  Well in a nutshell I would say easier to use tools for developers  probably the final step in my imagination would be to integrate these two popular IDEs  so that as you develop your favorite integrated development environment will tell you  guys be careful this is your calling it function that may be vulnerable you'd better use this one  which is newer not so popular but of better quality are you sure you want to use that library  not instead of this? And of course an important element of FASTAN is that it focuses more deeply  not just to libraries but as you mentioned to functions yes this might open many possibilities  of doing additional and more interesting suggestions. Yes also because as I said  vulnerabilities are about specific callables not general libraries so sometimes this helps in  two ways right because you know that a certain function that you're using even coming from  a vulnerable package will be saved and on the other hand if you have a security alert you will  be certainly sure that you are in fact in danger I think this will help in having a better understanding  of how deadly this is.
software_ecosystems_as_networks_0_179.png software_ecosystems_as_networks_180_1259.png software_ecosystems_as_networks_1260_1319.png software_ecosystems_as_networks_1320_2063.png software_ecosystems_as_networks_2064_2903.png software_ecosystems_as_networks_2904_3779.png software_ecosystems_as_networks_3780_4007.png software_ecosystems_as_networks_4008_4271.png software_ecosystems_as_networks_4272_4463.png software_ecosystems_as_networks_4464_5327.png software_ecosystems_as_networks_5328_5843.png software_ecosystems_as_networks_5844_8099.png software_ecosystems_as_networks_8100_8303.png software_ecosystems_as_networks_8304_8927.png software_ecosystems_as_networks_8928_9011.png software_ecosystems_as_networks_9012_9479.png software_ecosystems_as_networks_9480_11051.png software_ecosystems_as_networks_11052_12719.png software_ecosystems_as_networks_12720_15719.png software_ecosystems_as_networks_15720_17387.png software_ecosystems_as_networks_17388_19775.png software_ecosystems_as_networks_19776_20807.png software_ecosystems_as_networks_20808_20963.png software_ecosystems_as_networks_20964_21083.png software_ecosystems_as_networks_21084_23135.png software_ecosystems_as_networks_23136_23435.png software_ecosystems_as_networks_23436_24911.png software_ecosystems_as_networks_24912_26591.png software_ecosystems_as_networks_26592_26639.png software_ecosystems_as_networks_26640_27167.png software_ecosystems_as_networks_27168_27335.png software_ecosystems_as_networks_27336_29591.png software_ecosystems_as_networks_29592_31079.png software_ecosystems_as_networks_31080_31727.png software_ecosystems_as_networks_31728_35087.png software_ecosystems_as_networks_35088_36011.png software_ecosystems_as_networks_36012_37031.png software_ecosystems_as_networks_37032_37631.png software_ecosystems_as_networks_37632_38063.png software_ecosystems_as_networks_38064_38267.png software_ecosystems_as_networks_38268_40091.png software_ecosystems_as_networks_40092_40163.png software_ecosystems_as_networks_40164_41243.png software_ecosystems_as_networks_41244_41327.png software_ecosystems_as_networks_41328_41459.png software_ecosystems_as_networks_41460_41855.png software_ecosystems_as_networks_41856_42059.png software_ecosystems_as_networks_42060_42347.png software_ecosystems_as_networks_42348_42551.png software_ecosystems_as_networks_42552_42827.png software_ecosystems_as_networks_42828_42887.png software_ecosystems_as_networks_42888_42935.png software_ecosystems_as_networks_42936_65915.png software_ecosystems_as_networks_65916_66031.png