An event based approach for CI/CD pipelines
CC Attribution 2.0 Belgium:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Pettersson, Emelie
Fristedt, Fredrik
How can we listen to when new upstream software has been tested to the extent that we feel comfortable integrating it into our software? How can we communicate about new artifacts available for others to integrate? How can we see what has been integrated where? How can we achieve traceability across pipelines run on different tooling infrastructure? How can we visualize our pipelines to follow changes from source code to customer deployment? We will describe these challenges and show how we tackled them using self-documenting integration pipelines providing traceability and visualization to benefit multiple needs in the organization. We will present based on our experience from large-scale software development.
 The room is very full. We are very glad to be here. My name is Emily. Yes, and my colleague is Fredrik. We only have one mic, so we're going to switch in the middle, sort of. But let's get started. I have an agenda here, and at the beginning we see I planned to walk through the definition of a pipeline, but considering the audience, I think that's unnecessary. So I'm going to skip that. And, May, I'm coming from Ericsson, which is not open source, of course, it's a company. But we do contribute to open source, and that's why we're here today. I'd like to start off with describing what kind of challenges we've seen considering the communication between different CI CD tools. And then I'm going to present the way we solved it. Yes, I'm going to speak a little bit louder. And also how we open source our solution, and how Fredrik and his company Axis started using the same solution. So let's skip this part as promised, and instead go straight for the communication challenges that we've seen at Ericsson. So first of all, why does Ericsson care about CI CD? Well, any company that produces software should care about CI CD, obviously. And we have a very complex environment. We have a lot of different products going out to our customers, and configuration applied on top of them considering if it's going to be to customer A or if customer B wants them. So we have many different pipelines and different pipeline tools used to produce our software. Very often they're integrated into each other. They're called connected pipelines, and that's what we like to call them. And I'll go through what I mean by that later. We are a very large organization. We have hundreds of teams developing our products. Each one of them uses different toolings. And the demands on traceability is very high. We need to keep track of our software. We need to see which stages does it go through in the pipeline, no matter if it stops at delivery or if it goes all the way into deployment to the customer networks. We need to follow and see where does our software go. We also need to be able to monitor it, independent of whichever technology we use in the background. We need to be able to keep track of the important KPIs, and I guess also visualize it in a good way. And one visualization tool would be nice. So I spoke about connected pipelines and what I mean by that is we usually integrate our products into each other, sort of. So they're hierarchies of products. So you've got product A and B to the left here, and they're produced in their own separate pipelines before they're together integrated into the next level, or the next connected pipeline, producing the product AB. This is a very common phenomenon at Ericsson where we have multiple integration spots, and it's tough to keep track of all of these dependencies to integrate upstream and see where does this software come from and how it has been tested before we integrated. And again traceability is important here all the way. So I've tried to summarize into a few questions of what are the exact challenges we see? What is the problems that we need to solve? One thing is when we use different pipeline environments, how do we communicate about when our product has been delivered or released? Let's say our product A is done here. How does somebody downstream become aware of this? How can we announce this to the world? And how can we, if we have another perspective, how can we become aware when others release their software? Automatically the independent of whichever technology stack we use would be nice to become aware of, oh hey, this version X, Y, Z is now available, so we automatically integrate it into our pipeline and go for it. And how can we visualize these connections? How can we see what kind of tests has been done upstream but also downstream on our software later on? How can we see how many has downloaded our products? It would be nice to know. Were their tests successful into their environment or not? And why? So the solution for this was that we focused mostly on the communication. How can we communicate across different tools? So we came up with a solution that's like a common language, if you will, across these different pipelines, across different tools, across different visualization tools that could be based off the same information. And we called it AFL. Now AFL is created in 2012, so we've been using it for a lot of years internally. A couple of years back we open sourced it and it's now available on GitHub. But what it is, it's a message protocol. It's event based. It's defining a set of events that you can use to communicate about different concepts used in your pipeline, be it artifacts, be it tests, source code changes. You can communicate about this independent of whichever technology you use. It doesn't matter if it's a Java application, it's uploaded somewhere to a Maven repository or if it's a Docker image, you can still communicate about it in the same thing, in the same way. And what I think is great about AFL, at least, you can cherry pick whichever events suit your needs. You don't have to take all of the protocol, it's a lot of events. You can cherry pick the ones that you specifically need in your use case. So let's say you're only interested in the test case related events, then you only care about those. And the events are also linked together. And that's very important for us at least to maintain the traceability between them. So you can trace back from the latest event, you can go backwards in time and see the whole pipeline chain. You can represent the pipeline. And with AFL, you can answer some of the questions I posed previously. For example, when we have a new artifact available, you can send out an event saying that I've now created this artifact. You see it to the left of the, in the picture, it's called AFL artifact created event. What comes after that is, could be, a published event, whenever you upload it somewhere, you can send additional events linking back to the previously created artifact. The linking here is important. The protocol itself defines how you can link your events, if they must have or if it's optional links. These are all in JSON format, I guess you're wondering. Next question could be, how can we visualize our pipeline? It's a pipeline consists of many different steps, right? And depending on how it looks, you want to maybe tell the world about what has happened, how do we visualize this? This is a very simple CI pipeline. It's a GitLab syntax. So you build your product, then you test it, you test it some more, and then you upload, you build binary somewhere. Every standard case. And some of the AFL events that could be used for this are, for example, these. You can have an artifact created event to signify the first built. This is where you actually build your binary or whatever it might be. Then you start some tests. And then you might want to send test case events. So you've got the trigger to start and then finish events for this. And these can be sent, of course, these jobs are in parallel in this pipeline. So you can send them in two sets. And then comes the artifact published event as the last one to signify that, hey, now my artifact is available somewhere for downloading. And I'm going to show you the exact same set of events only connected in an event graph to make it clearer. So start to the left, got artifact created event. This is the first one in chronological order. Then comes the test case events, which links back to the artifact because this is what we're testing. And the link name in this case is called item under test. Makes perfect sense to me, at least. Then comes the test case started event, linking back to the triggering. And then we've got the finished event, test case finished event. This one contains the result of the test, if it was successful or not. Now, of course, if you have several test cases, you might want to group them under some sort of label that saying, yes, I've tested it to this extent. And I want to label my artifact now. This is ready for the next level of delivery somewhere. Then you can send a confidence level modified event. This is very free formatted in a way that you can name the confidence level whatever you want to as long as you standardize it, it shouldn't be a problem. And then you point back to the artifact that you've created. So this is very useful for us as listeners or consumers of this product. We know once it has been tested enough, we know when it's reached the confidence level we want it to have before we actually start using it. And once the last event comes in, it's the artifact published event, we know where we can find it. Where is the artifact located? Where can I download it? And I have one last wrap up here. As I said, we have open source this. And it's available on GitHub. And we really hope that the community keeps growing and getting more contributions because we see that a lot of different companies and organizations have the same problems that we faced. And one thing I liked about AFL is that it provides one common language across different technology stacks across different tools you might use. So you can base visualization tools on the events in the state that for example. And what's also important is the links between the events. It enables traceability so you can follow a complete set of pipeline. You know what has happened to your software. And you know where it might go even further downstream. And I'd like to invite Fredrik to this side of the room so we can switch Mike because he's going to tell you about how access started contributing and using AFL. So if we can set this up here. Okay, so I'm from a company called Access Communications and we do surveillance cameras, which is kind of a scary thing. But there's a lot of software running these cameras and somebody said roughly around 20 million lines of code in the camera. So there's a lot of software going into this small thing. It's a kind of a it's basic the Linux box but still. And we have nowhere the scale that Ericsson has. We have about 1,000 developers in total and the team that does the teams that do the cameras are about 400 developers and about 150 testers. And we do we like to think we do continuous integration at least because we commit to master a lot of times every day. And I'm going to try to tell you our AFL story. So Emily gave you a good background on what AFL is and I'm trying to go through some kind of a timeline how we found out about AFL and how we adopted it and how we now very much would like to drive our all of our pipelines using it. That's at least that's the ambition for for the future. And first I would like to give you some context on what exactly it is that we do. So Access is quite an old company and we've done in the history we've done network connected stuff and which it's basically just making some kind of hardware putting in a box and selling it. That's the standard access way that we've been doing it for for a lot of years and it's basically what other companies do also they sell cardboard boxes with tech stuff in it. But more and more we've been seeing a demand from our from our business and from our from our customers that they want complete systems. They want cloud integration. They want they want a lot of stuff which which in turn gives a lot more software that has to be tested has to work together. So that that's where in there that comes the challenge of knowing how which which versions work together with software stacks works best together and so on. So that's that's kind of our the similar challenge that Ericsson has. But back to our story. The first recorded presence of the AFL protocol at Access. As I've heard there were a couple of engineers attending some kind of a conference or a meeting where where somebody from Ericsson was presenting some kind of visualization tool how we could visualize your software. I don't think it was called. I don't even think they was referred to as a pipeline but but some kind of software flow and they found that this was something called AFL which was underneath here and they were they got intrigued. So they they started doing some tinkering on their own and trying to trying to make something with AFL in the access context. And and which was which was fine but really didn't lead anywhere. And then it resurfaced in some kind of a in a research academia industry project that access had where some of the researchers had found out about AFL from Ericsson also. And and it resurfaced and they did some more tinkering and some more engineers got interested but it was still kind of under the radar stuff. Just engineers like technology enthusiasts that did stuff but nothing that really drove the business or was that was a thing. And in like the next next big thing in my opinion was was when Access actually released something that had to do with AFL. So these engineers had been tinkering more and they wanted to to make something of it. They wanted to AFL to produce something that was of value. So so they they we used Jenkins as almost everybody else. So they made a Jenkins plugin which was very basic. It basically produced artifact created event when we we did a build. That was all we did. So it was kind of worthless on its own but still it's kind of a footprint that that now we're actually producing AFL events. We have a message bus where there are events coming out. But anyone there was really nobody listening which was which is kind of a problem because it's it's when doing this type of thing. So the whole point is to somebody listening in and acting on the events and there is like a chain created and a flow. So so our biggest problem here was was nobody was listening in and if nobody's listening in a pub subculture it's kind of hard to know if you're making sense if you if you in the beginning at least because because you have to know that that's in the beginning you have to have some kind of acknowledgement that your event it was useful. There's a value to it. So we had to to create some kind of value in our minimal AFL implementation. And this is basically where I where I came in in the in the AFL thing. I was a part of a team that does test automation and we were struggling with our CI part of the testing part of our CI pipeline. It was kind of old and we didn't it didn't really want to do what we wanted to do with our testing. So we we started looking to rebuilding it basically or changing it and so on. And I had a coffee discussion with a colleague of mine. He he was one of those AFL tinkering engineers. So he said you should look at AFL if you're going to do things from scratch. You might as well look at AFL because it's pretty cool. You can do cool stuff with it. And we looked into AFL and we a colleague me and a colleague and we really liked it. We saw that this was this is something that good that's good. Probably not solve our immediate problems but there was a bigger issue of the system of systems talk and knowing information about the software that goes in from from everywhere. So so we decided to make our new testing part of the CI CD pipeline talk AFL and understand AFL. And in this we started listening to the Jenkins events the lonely guy over here that was playing AFL music and nobody was listening. So we actually started listening to that and we started triggering our tests based on those those events which which gave us a much more understanding of what it is like to have an event driven pipeline and what we would have us like to adhere to a protocol like AFL. And we actually wound up being a great listener today for protocol. And during during the development of our testing pipeline we had to learn AFL basically. So we we we wrote a bunch of Python libraries and two of them which were kind of important at least we thought we we said we have to we have to contribute back to since it was an open source if it was an open source community and open source protocol we said we can contribute here we can learn people how to teach people how to how to use AFL. And these two libraries the AFL Python lib which is a kind of small library that lets you publish AFL messages and listen to AFL messages on the message plus. It's it's very useful if you want to get started with AFL. And then there is something called the AFL graph QL API which is basically an API on top of the the event repository which is basically a storage for events because if you want to look at events back in history you have to store them somewhere you can't just keep them on the bus. So that's basically our graph QL which is kind of useful if you want to start tinkering with with the storing events and looking back in history and so on. And this this open sourcing of this got access more engaged in the community and we actually there's a there's a conference called called AFL Summit. It's very small conference where we actually access actually hosted it this fall. And in moving forward we're going to try to to keep the backlogs working with an open backlog concept as we learned about AFL how we are our journey we're trying to contribute directly to the to the community rather than just doing stuff inside and then just dumping it outside which is kind of a bad thing. And then moving forward as AFL is now a thing at Axis we actually have a small team that work with AFL and we're we're going to really going to try to to drive all the all of our pipelines we see them being event-driven in some way moving forward. And also also we're going to try to to contribute more to the AFL community trying to to like Emily said want to spread it want to make more people try it out and see if it can become a thing. So we'll try to to contribute more of our experiences moving forward in 2020. And that's that's that's my access AFL journey basically. And we'd like to thank you for listening to us and here's some some contact information the AFL community on GitHub and we have a Google group and a Slack channel and a YouTube channel also with some some videos if you want to learn more or get a more basic tutorial of what AFL is. And now I guess it's questions. So the question was is there is some kind of a GitLab integration for for the AFL. Not that I know of currently on GitHub I don't know if not not yet. So and I'm I'm not quite sure how GitLab works. So I it should be possible there's no I don't see any like big technical blocking it it's just that somebody has to to contribute it and do it. Yeah. How would there any like this there any kind of stuff how to do it like manually like triggering events. They're awesome if you look at the Python library there is some there is some basic basic tutorial how to set that up how to send events and and receive events. And then there's also also something called I think it's called a full easy to use or which is basically a set of Docker containers that you could just spin up you get a full like a small pipeline which produces events but I don't it's it's that's something that Eric's been contributing and I don't know if all of it is open source yet might be but it's basically something you can try out to get some messages going and try to get a feel for how it how it works. Yes absolutely so the question was since we've been sending events for several years now with an Ericsson we have of course a lot of data accumulated so we the question was if we can look at lead times and see where the bottlenecks in the pipelines and yes of course we do have that possibility and that's one of the great benefits that we've seen internally at least and that's why I think I don't know how many millions of events we have stored but there's a lot of them. So it's indeed possible if you have like a implementation of a storage somewhere and then you can look it up and you can see because all the events are timestamped so you can also see lead times between the events between different activities in your pipeline which is very useful and yes Fredric mentioned and just like to add because aside from the protocol we have some example implementations of services surrounding the protocol helping to send the events helping to visualize them I think we also have open source and there are more coming the more planned for that is just a matter of prioritization and so on. Any more questions? Yes in the back. Have you also thought about using Eiffel or the outside of development like I know triggering some team to rewrite the manual on new release or something? So the question was have we tried to use Eiffel outside of development to trigger some manual activities if I understood it correctly and I'm not aware that we listen and act upon these events and then doing some sort of manual step of course it's at all possible but the listeners usually are performing some sort of automatic task afterwards and as far as I know we mostly focus on up until continuous delivery sort of deployment stages don't really trigger on Eiffel events later on yet at least. So there are like more generic Eiffel events like announcement events that say something that something has happened and you could definitely use that for signaling other people in the organization and we actually use that to sort of a debugging thing so we know that we've gone a certain path through our testing CI system. We do announcements here and there to see okay we've gone this far because it's a microservice architecture and it's kind of hard to keep track of everything so it's a handy thing but there are announcements events that are very generic so you could use them for almost anything. Any more questions? Okay, thank you. Thank you. Thank you. Thank you.
 The room is very full. We are very glad to be here. My name is Emily. Yes, and my colleague  is Fredrik. We only have one mic, so we're going to switch in the middle, sort of. But  let's get started. I have an agenda here, and at the beginning we see I planned to walk  through the definition of a pipeline, but considering the audience, I think that's unnecessary.  So I'm going to skip that. And, May, I'm coming from Ericsson, which is not open source,  of course, it's a company. But we do contribute to open source, and that's why we're here  today. I'd like to start off with describing what kind of challenges we've seen considering  the communication between different CI CD tools. And then I'm going to present the  way we solved it. Yes, I'm going to speak a little bit louder. And also how we open  source our solution, and how Fredrik and his company Axis started using the same solution.  So let's skip this part as promised, and instead go straight for the communication challenges  that we've seen at Ericsson. So first of all, why does Ericsson care about CI CD? Well,  any company that produces software should care about CI CD, obviously. And we have a  very complex environment. We have a lot of different products going out to our customers,  and configuration applied on top of them considering if it's going to be to customer A or if customer  B wants them. So we have many different pipelines and different pipeline tools used to produce  our software. Very often they're integrated into each other. They're called connected  pipelines, and that's what we like to call them. And I'll go through what I mean by that  later. We are a very large organization. We have hundreds of teams developing our products.  Each one of them uses different toolings. And the demands on traceability is very high.  We need to keep track of our software. We need to see which stages does it go through  in the pipeline, no matter if it stops at delivery or if it goes all the way into deployment  to the customer networks. We need to follow and see where does our software go. We also  need to be able to monitor it, independent of whichever technology we use in the background.  We need to be able to keep track of the important KPIs, and I guess also visualize it in a good  way. And one visualization tool would be nice. So I spoke about connected pipelines and what  I mean by that is we usually integrate our products into each other, sort of. So they're  hierarchies of products. So you've got product A and B to the left here, and they're produced  in their own separate pipelines before they're together integrated into the next level, or  the next connected pipeline, producing the product AB. This is a very common phenomenon  at Ericsson where we have multiple integration spots, and it's tough to keep track of all  of these dependencies to integrate upstream and see where does this software come from  and how it has been tested before we integrated. And again traceability is important here  all the way. So I've tried to summarize into a few questions of what are the exact challenges  we see? What is the problems that we need to solve? One thing is when we use different  pipeline environments, how do we communicate about when our product has been delivered  or released? Let's say our product A is done here. How does somebody downstream become aware  of this? How can we announce this to the world? And how can we, if we have another perspective,  how can we become aware when others release their software? Automatically the independent  of whichever technology stack we use would be nice to become aware of, oh hey, this  version X, Y, Z is now available, so we automatically integrate it into our pipeline and go for  it. And how can we visualize these connections? How can we see what kind of tests has been  done upstream but also downstream on our software later on? How can we see how many has downloaded  our products? It would be nice to know. Were their tests successful into their environment  or not? And why? So the solution for this was that we focused mostly on the communication.  How can we communicate across different tools? So we came up with a solution that's like  a common language, if you will, across these different pipelines, across different tools,  across different visualization tools that could be based off the same information. And we  called it AFL. Now AFL is created in 2012, so we've been using it for a lot of years  internally. A couple of years back we open sourced it and it's now available on GitHub.  But what it is, it's a message protocol. It's event based. It's defining a set of events  that you can use to communicate about different concepts used in your pipeline, be it artifacts,  be it tests, source code changes. You can communicate about this independent of whichever technology  you use. It doesn't matter if it's a Java application, it's uploaded somewhere to a  Maven repository or if it's a Docker image, you can still communicate about it in the  same thing, in the same way. And what I think is great about AFL, at least, you can cherry  pick whichever events suit your needs. You don't have to take all of the protocol, it's  a lot of events. You can cherry pick the ones that you specifically need in your use case.  So let's say you're only interested in the test case related events, then you only care  about those. And the events are also linked together. And that's very important for us  at least to maintain the traceability between them. So you can trace back from the latest  event, you can go backwards in time and see the whole pipeline chain. You can represent  the pipeline. And with AFL, you can answer some of the questions I posed previously. For  example, when we have a new artifact available, you can send out an event saying that I've  now created this artifact. You see it to the left of the, in the picture, it's called AFL  artifact created event. What comes after that is, could be, a published event, whenever  you upload it somewhere, you can send additional events linking back to the previously created  artifact. The linking here is important. The protocol itself defines how you can link your  events, if they must have or if it's optional links. These are all in JSON format, I guess  you're wondering. Next question could be, how can we visualize  our pipeline? It's a pipeline consists of many different steps, right? And depending  on how it looks, you want to maybe tell the world about what has happened, how do we visualize  this? This is a very simple CI pipeline. It's a GitLab syntax. So you build your product,  then you test it, you test it some more, and then you upload, you build binary somewhere.  Every standard case. And some of the AFL events that could be used for this are, for example,  these. You can have an artifact created event to signify the first built. This is where  you actually build your binary or whatever it might be. Then you start some tests. And  then you might want to send test case events. So you've got the trigger to start and then  finish events for this. And these can be sent, of course, these jobs are in parallel in this  pipeline. So you can send them in two sets. And then comes the artifact published event  as the last one to signify that, hey, now my artifact is available somewhere for downloading.  And I'm going to show you the exact same set of events only connected in an event graph  to make it clearer. So start to the left, got artifact created event. This is the first  one in chronological order. Then comes the test case events, which links back to the  artifact because this is what we're testing. And the link name in this case is called item  under test. Makes perfect sense to me, at least. Then comes the test case started event,  linking back to the triggering. And then we've got the finished event, test case finished  event. This one contains the result of the test, if it was successful or not. Now, of  course, if you have several test cases, you might want to group them under some sort of  label that saying, yes, I've tested it to this extent. And I want to label my artifact  now. This is ready for the next level of delivery somewhere. Then you can send a confidence  level modified event. This is very free formatted in a way that you can name the confidence  level whatever you want to as long as you standardize it, it shouldn't be a problem.  And then you point back to the artifact that you've created. So this is very useful for  us as listeners or consumers of this product. We know once it has been tested enough, we  know when it's reached the confidence level we want it to have before we actually start  using it. And once the last event comes in, it's the artifact published event, we know  where we can find it. Where is the artifact located? Where can I download it? And I have  one last wrap up here. As I said, we have open source this. And it's available on GitHub.  And we really hope that the community keeps growing and getting more contributions because  we see that a lot of different companies and organizations have the same problems that  we faced. And one thing I liked about AFL is that it provides one common language across  different technology stacks across different tools you might use. So you can base visualization  tools on the events in the state that for example. And what's also important is the  links between the events. It enables traceability so you can follow a complete set of pipeline.  You know what has happened to your software. And you know where it might go even further  downstream. And I'd like to invite Fredrik to this side of the room so we can switch  Mike because he's going to tell you about how access started contributing and using  AFL.  So if we can set this up here. Okay, so I'm from a company called Access Communications  and we do surveillance cameras, which is kind of a scary thing. But there's a lot of software  running these cameras and somebody said roughly around 20 million lines of code in the camera.  So there's a lot of software going into this small thing. It's a kind of a it's basic  the Linux box but still. And we have nowhere the scale that Ericsson has. We have about  1,000 developers in total and the team that does the teams that do the cameras are about  400 developers and about 150 testers. And we do we like to think we do continuous integration  at least because we commit to master a lot of times every day. And I'm going to try to  tell you our AFL story. So Emily gave you a good background on what AFL is and I'm  trying to go through some kind of a timeline how we found out about AFL and how we adopted  it and how we now very much would like to drive our all of our pipelines using it. That's  at least that's the ambition for for the future. And first I would like to give you some context  on what exactly it is that we do. So Access is quite an old company and we've done in  the history we've done network connected stuff and which it's basically just making  some kind of hardware putting in a box and selling it. That's the standard access way  that we've been doing it for for a lot of years and it's basically what other companies  do also they sell cardboard boxes with tech stuff in it. But more and more we've been  seeing a demand from our from our business and from our from our customers that they  want complete systems. They want cloud integration. They want they want a lot of stuff which which  in turn gives a lot more software that has to be tested has to work together. So that  that's where in there that comes the challenge of knowing how which which versions work together  with software stacks works best together and so on. So that's that's kind of our the similar  challenge that Ericsson has. But back to our story. The first recorded presence of the  AFL protocol at Access. As I've heard there were a couple of engineers attending some  kind of a conference or a meeting where where somebody from Ericsson was presenting some  kind of visualization tool how we could visualize your software. I don't think it was called.  I don't even think they was referred to as a pipeline but but some kind of software flow  and they found that this was something called AFL which was underneath here and they were  they got intrigued. So they they started doing some tinkering on their own and trying to  trying to make something with AFL in the access context. And and which was which was fine  but really didn't lead anywhere. And then it resurfaced in some kind of a in a research  academia industry project that access had where some of the researchers had found out  about AFL from Ericsson also. And and it resurfaced and they did some more tinkering  and some more engineers got interested but it was still kind of under the radar stuff.  Just engineers like technology enthusiasts that did stuff but nothing that really drove  the business or was that was a thing. And in like the next next big thing in my opinion  was was when Access actually released something that had to do with AFL. So these engineers  had been tinkering more and they wanted to to make something of it. They wanted to AFL  to produce something that was of value. So so they they we used Jenkins as almost everybody  else. So they made a Jenkins plugin which was very basic. It basically produced artifact  created event when we we did a build. That was all we did. So it was kind of worthless  on its own but still it's kind of a footprint that that now we're actually producing AFL  events. We have a message bus where there are events coming out. But anyone there was  really nobody listening which was which is kind of a problem because it's it's when  doing this type of thing. So the whole point is to somebody listening in and acting on  the events and there is like a chain created and a flow. So so our biggest problem here  was was nobody was listening in and if nobody's listening in a pub subculture it's kind of  hard to know if you're making sense if you if you in the beginning at least because because  you have to know that that's in the beginning you have to have some kind of acknowledgement  that your event it was useful. There's a value to it. So we had to to create some kind of  value in our minimal AFL implementation. And this is basically where I where I came in  in the in the AFL thing. I was a part of a team that does test automation and we were  struggling with our CI part of the testing part of our CI pipeline. It was kind of old  and we didn't it didn't really want to do what we wanted to do with our testing. So  we we started looking to rebuilding it basically or changing it and so on. And I had a coffee  discussion with a colleague of mine. He he was one of those AFL tinkering engineers. So  he said you should look at AFL if you're going to do things from scratch. You might as well  look at AFL because it's pretty cool. You can do cool stuff with it. And we looked into  AFL and we a colleague me and a colleague and we really liked it. We saw that this was  this is something that good that's good. Probably not solve our immediate problems but there  was a bigger issue of the system of systems talk and knowing information about the software  that goes in from from everywhere. So so we decided to make our new testing part of the  CI CD pipeline talk AFL and understand AFL. And in this we started listening to the Jenkins  events the lonely guy over here that was playing AFL music and nobody was listening. So we  actually started listening to that and we started triggering our tests based on those  those events which which gave us a much more understanding of what it is like to have an  event driven pipeline and what we would have us like to adhere to a protocol like AFL.  And we actually wound up being a great listener today for protocol. And during during the  development of our testing pipeline we had to learn AFL basically. So we we we wrote  a bunch of Python libraries and two of them which were kind of important at least we thought  we we said we have to we have to contribute back to since it was an open source if it  was an open source community and open source protocol we said we can contribute here we  can learn people how to teach people how to how to use AFL. And these two libraries the  AFL Python lib which is a kind of small library that lets you publish AFL messages and listen  to AFL messages on the message plus. It's it's very useful if you want to get started  with AFL. And then there is something called the AFL graph QL API which is basically an  API on top of the the event repository which is basically a storage for events because  if you want to look at events back in history you have to store them somewhere you can't  just keep them on the bus. So that's basically our graph QL which is kind of useful if you  want to start tinkering with with the storing events and looking back in history and so  on. And this this open sourcing of this got access more engaged in the community and we  actually there's a there's a conference called called AFL Summit. It's very small conference  where we actually access actually hosted it this fall. And in moving forward we're going  to try to to keep the backlogs working with an open backlog concept as we learned about  AFL how we are our journey we're trying to contribute directly to the to the community  rather than just doing stuff inside and then just dumping it outside which is kind of a  bad thing. And then moving forward as AFL is now a thing at Axis we actually have a  small team that work with AFL and we're we're going to really going to try to to drive all  the all of our pipelines we see them being event-driven in some way moving forward. And  also also we're going to try to to contribute more to the AFL community trying to to like  Emily said want to spread it want to make more people try it out and see if it can become  a thing. So we'll try to to contribute more of our experiences moving forward in 2020.  And that's that's that's my access AFL journey basically. And we'd like to thank you for  listening to us and here's some some contact information the AFL community on GitHub and  we have a Google group and a Slack channel and a YouTube channel also with some some  videos if you want to learn more or get a more basic tutorial of what AFL is. And now  I guess it's questions.  So the question was is there is some kind of a GitLab integration for for the  AFL. Not that I know of currently on GitHub I don't know if not not yet. So and I'm I'm  not quite sure how GitLab works. So I it should be possible there's no I don't see any like  big technical blocking it it's just that somebody has to to contribute it and do it.  Yeah. How would there any like this there any kind of stuff how to do it like manually  like triggering events.  They're awesome if you look at the Python library there is some there is some basic  basic tutorial how to set that up how to send events and and receive events. And then there's  also also something called I think it's called a full easy to use or which is basically a  set of Docker containers that you could just spin up you get a full like a small pipeline  which produces events but I don't it's it's that's something that Eric's been contributing  and I don't know if all of it is open source yet might be but it's basically something  you can try out to get some messages going and try to get a feel for how it how it works.  Yes absolutely so the question was since we've been sending events for several years now  with an Ericsson we have of course a lot of data accumulated so we the question was if  we can look at lead times and see where the bottlenecks in the pipelines and yes of course  we do have that possibility and that's one of the great benefits that we've seen internally  at least and that's why I think I don't know how many millions of events we have stored  but there's a lot of them.  So it's indeed possible if you have like a implementation of a storage somewhere and  then you can look it up and you can see because all the events are timestamped so you can  also see lead times between the events between different activities in your pipeline which  is very useful and yes Fredric mentioned and just like to add because aside from the protocol  we have some example implementations of services surrounding the protocol helping to send the  events helping to visualize them I think we also have open source and there are more coming  the more planned for that is just a matter of prioritization and so on.  Any more questions?  Yes in the back.  Have you also thought about using Eiffel or the outside of development like I know triggering  some team to rewrite the manual on new release or something?  So the question was have we tried to use Eiffel outside of development to trigger some manual  activities if I understood it correctly and I'm not aware that we listen and act upon  these events and then doing some sort of manual step of course it's at all possible but the  listeners usually are performing some sort of automatic task afterwards and as far as  I know we mostly focus on up until continuous delivery sort of deployment stages don't really  trigger on Eiffel events later on yet at least.  So there are like more generic Eiffel events like announcement events that say something  that something has happened and you could definitely use that for signaling other people  in the organization and we actually use that to sort of a debugging thing so we know that  we've gone a certain path through our testing CI system.  We do announcements here and there to see okay we've gone this far because it's a microservice  architecture and it's kind of hard to keep track of everything so it's a handy thing  but there are announcements events that are very generic so you could use them for almost  anything.  Any more questions?  Okay, thank you.  Thank you.  Thank you.  Thank you.
an_event_based_approach_for_cicd_pipelines_0_167.png an_event_based_approach_for_cicd_pipelines_168_359.png an_event_based_approach_for_cicd_pipelines_360_551.png an_event_based_approach_for_cicd_pipelines_552_1919.png an_event_based_approach_for_cicd_pipelines_1920_1967.png an_event_based_approach_for_cicd_pipelines_1968_2183.png an_event_based_approach_for_cicd_pipelines_2184_4547.png an_event_based_approach_for_cicd_pipelines_4548_6047.png an_event_based_approach_for_cicd_pipelines_6048_7415.png an_event_based_approach_for_cicd_pipelines_7416_8003.png an_event_based_approach_for_cicd_pipelines_8004_8339.png an_event_based_approach_for_cicd_pipelines_8340_10703.png an_event_based_approach_for_cicd_pipelines_10704_12167.png an_event_based_approach_for_cicd_pipelines_12168_12683.png an_event_based_approach_for_cicd_pipelines_12684_13763.png an_event_based_approach_for_cicd_pipelines_13764_14051.png an_event_based_approach_for_cicd_pipelines_14052_14411.png an_event_based_approach_for_cicd_pipelines_14412_14603.png an_event_based_approach_for_cicd_pipelines_14604_15203.png an_event_based_approach_for_cicd_pipelines_15204_16115.png an_event_based_approach_for_cicd_pipelines_16116_16415.png an_event_based_approach_for_cicd_pipelines_16416_18047.png an_event_based_approach_for_cicd_pipelines_18048_18107.png an_event_based_approach_for_cicd_pipelines_18108_18647.png an_event_based_approach_for_cicd_pipelines_18648_19859.png an_event_based_approach_for_cicd_pipelines_19860_20471.png an_event_based_approach_for_cicd_pipelines_20472_22055.png an_event_based_approach_for_cicd_pipelines_22056_23855.png an_event_based_approach_for_cicd_pipelines_23856_25091.png an_event_based_approach_for_cicd_pipelines_25092_26075.png an_event_based_approach_for_cicd_pipelines_26076_28043.png an_event_based_approach_for_cicd_pipelines_28044_28127.png an_event_based_approach_for_cicd_pipelines_28128_28787.png an_event_based_approach_for_cicd_pipelines_28788_31835.png an_event_based_approach_for_cicd_pipelines_31836_33083.png an_event_based_approach_for_cicd_pipelines_33084_35495.png an_event_based_approach_for_cicd_pipelines_35496_35543.png an_event_based_approach_for_cicd_pipelines_35544_41903.png an_event_based_approach_for_cicd_pipelines_41904_42025.png