Alyvix: Under the hood
CC Attribution 2.0 Belgium:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Melchiori, Francesco
The open-source software Alyvix is a synthetic monitoring system, based oncomputer vision. It automates any application, interacting with GUIs exactlyas a human would do. In this way, Alyvix can check the availability and canmeasure the responsiveness of applications. From a computer science point ofview, Alyvix is an open-source Python-based software, which can be deployed onWindows 64-bit machines as a part of an open-stack of libraries. Alyvix mainlyrelies on the following open-source software projects: RobotFramework fordesktop automation, OpenCV and Pillow for image processing, TesseractOCR fortext recognition and PyQt for GUI programming. Francesco Melchiori, as Alyvixproduct manager, will explain how Alyvix works leveraging on theaforementioned packages, what software design choices have been taken.Finally, he will show how this computer vision solution solves the need of aneffective visual synthetic monitoring of proprietary network applications.This aims to suggest other use cases with the Alyvix API.
 So, good afternoon everybody. Our next speaker for lightning talks is Francesco Melchiori and he'll be talking about Elifix. Please give him a big hand. Hello. My name is Francesco Melchiori. I'm a product manager in Budde, Phoenix, which is an IT Italian company. Our goal is to monitor the end user experience of general network applications. Here you've got the general idea. So, you've got the general network applications in cloud and the end user. In the middle, we've got the complexity of the network. So, at the end of the day, the end user suffers because of the performances of those applications. Our goal is to assess the perceived quality of service. So, here you've got three scenarios. The first one is when you have an open application with an open protocol and it's quite easy to, let's say, to assess the quality of service, you have to call the open APIs to an open protocol like HTTP. But the other two cases are definitely hard to solve because in the second case, we have to face a closed client with a closed protocol. So, custom socket programming. And the last scenario, we've got a virtualized application in cloud, even more severe. So, basically, you've got a video and audio stream directly on your web browser. So, that's difficult. We're developing a solution in order to solve all of these three scenarios. I'm talking about Alivix, but what is Alivix? Alivix is a visual-synthetic monitoring system. What does it mean? Visual, because Alivix looks like, looks at graphic user interfaces. It's synthetic because Alivix behaves like a human user using mouse and keyboard exactly as a human would do. And it's monitoring because Alivix is able to track transaction performances. So, first of all, Alivix can automate any application, why? It provides to you a graphical user interfaces tools in order to design those transactions. Well, it's made with Python, OpenCV, TAS, Prolacto, CR, and the robot framework. So, basically, the idea is that you have a transaction and you want to, let's say, to translate, to convert this transaction in one custom Alivix keyword. So, defining graphics on those transactions, planning interaction to it, and finally measuring the time the keyword takes to appear on the screen. We've got several tools. For example, the image finder. You can detect images, tiny images, really big images, doesn't matter. You have a rectangular finder. You can detect rectangles on your graphical user interface, like buttons, edit boxes, windows, and so on and so forth. And finally, you've got text finders. So, basically, Alivix is able to scrape, in some sense, your graphical user interfaces and match the text with regular expressions. It works also with a giant chunk of text. But, so, practically speaking, it's really easy to define a transaction through Alivix. It's a matter of selecting a finder type. It's a matter of pointing and clicking and dragging your main selection, your sub-components, and so on and so forth. Type in the name of the keyword, sorry, and then set up your recognition settings and then set up the interaction modes in order to click on those graphical elements, pushing the text string, and so on. And finally, we can also set up performance thresholds. So, but it's not the end of the story. We've got these three basic finders, but we can put everything together in a sort of really complex and abstract object. So, we can put an image finder, a rectangular finder, and a text finder, everything together, and you can link them together with the so-called regular region of interests. And everything, the object finder itself can inherit the interaction of the basic elements. So, finally, we've got a really complex object in order to solve complex transactions. So, now the thing is that you don't have just one single transaction, but a list of transactions in order to solve your end-user interaction flow. So, the idea is to convert every single transaction graphically. And what does it mean? It means list Alivix keywords in order to have what? The so-called test cases. And that's exactly what you have to do in the Alivix editor. So, if you run that test case, you can automate exactly that user interaction flow. Second, Alivix can measure application transaction. And here I've got to give you a grasp of the engine below, telling you something about the engine. We've got a thread. One thread is dedicated to the frame rubber. So, we are going to take frames from the screen at a certain pace. For example, here the frequency is 250 milliseconds. And in a point at a time, we've got our transaction. And there is another thread, the so-called object detector, that is going to take the first available frame from the frame rubber. And it has a lower frequency, for example, one second. And when it will detect the object on the screen, then a measurement system comes up and gives you the final transaction performance. So, that's going on and on for all your test cases, transactions after transaction, after transaction. The final achievement is that we wanted to, first of all, assess the availability of the transaction. So, we've got a transaction. The question is, is available or not? And then we've got to assess the responsiveness of the transaction. So, how many seconds it takes to appear? Finally, I'll report the test case transactions in several forms. First of all, to the command line. And then it provides you also a rich HTML interactive pages. And here you can find the screenshots of your transaction and the performance below. Finally, if a keyword breaks, you can find not a static screenshot, but an animation that gives you the reason why the keyword was broken. Alivix can report everything to a monitoring system. So, basically, here you've got your Alivix probe. And Alivix test cases can be automatized with a remote client like an S Client++ and it reports to your monitoring system. And this is the final time series. So, basically, it's really clear that our network application suffered down service down times and latency spikes. So, Alivix 2.3 is out right now. Just browse alivix.com to download your package. Alivix 2.4 is under development and we are going to release it this month. So, I wanted to show a short screencast of Alivix. This is a running test case on a complex graphical user interface. And as you can see, Alivix is able to recognize complex graphical objects and browse exactly as a human would do the user interface, filling complex form and so on and so forth. For example, here you can add a new address in this CMR, completing, completing, filling this complex form and so on and so forth. So, my final messages are, first of all, Alivix can really certify the ongoing quality of your IT services. And then, IT operations can really modulate the infrastructure resources, I mean the pool of hardware resources. And IT clients can really check their software level agreement with providers. This is my call for action. You can browse alivix.com in order to have a getting started documentation. And of course, you can contribute to GitHub. If you pull request something, we will be really, really happy. So, for us, Alivix is highs and ends like a human with a stopwatcher. Thanks a lot for your attention.
 So, good afternoon everybody. Our next speaker for lightning talks is Francesco Melchiori  and he'll be talking about Elifix. Please give him a big hand.  Hello. My name is Francesco Melchiori. I'm a product manager in Budde, Phoenix, which  is an IT Italian company. Our goal is to monitor the end user experience of general network  applications. Here you've got the general idea. So, you've got the general network applications  in cloud and the end user. In the middle, we've got the complexity of the network. So,  at the end of the day, the end user suffers because of the performances of those applications.  Our goal is to assess the perceived quality of service. So, here you've got three scenarios.  The first one is when you have an open application with an open protocol and it's quite easy  to, let's say, to assess the quality of service, you have to call the open APIs to an open  protocol like HTTP. But the other two cases are definitely hard to solve because in the  second case, we have to face a closed client with a closed protocol. So, custom socket  programming. And the last scenario, we've got a virtualized application in cloud, even  more severe. So, basically, you've got a video and audio stream directly on your web browser.  So, that's difficult. We're developing a solution in order to solve all of these three scenarios.  I'm talking about Alivix, but what is Alivix? Alivix is a visual-synthetic monitoring system.  What does it mean? Visual, because Alivix looks like, looks at graphic user interfaces.  It's synthetic because Alivix behaves like a human user using mouse and keyboard exactly  as a human would do. And it's monitoring because Alivix is able to track transaction performances.  So, first of all, Alivix can automate any application, why? It provides to you a graphical user  interfaces tools in order to design those transactions. Well, it's made with Python,  OpenCV, TAS, Prolacto, CR, and the robot framework. So, basically, the idea is that you have a  transaction and you want to, let's say, to translate, to convert this transaction in  one custom Alivix keyword. So, defining graphics on those transactions, planning interaction  to it, and finally measuring the time the keyword takes to appear on the screen.  We've got several tools. For example, the image finder. You can detect images, tiny  images, really big images, doesn't matter. You have a rectangular finder. You can detect  rectangles on your graphical user interface, like buttons, edit boxes, windows, and so  on and so forth. And finally, you've got text finders. So, basically, Alivix is able  to scrape, in some sense, your graphical user interfaces and match the text with regular  expressions. It works also with a giant chunk of text. But, so, practically speaking, it's  really easy to define a transaction through Alivix. It's a matter of selecting a finder  type. It's a matter of pointing and clicking and dragging your main selection, your sub-components,  and so on and so forth. Type in the name of the keyword, sorry, and then set up your recognition  settings and then set up the interaction modes in order to click on those graphical elements,  pushing the text string, and so on. And finally, we can also set up performance thresholds.  So, but it's not the end of the story. We've got these three basic finders, but we can  put everything together in a sort of really complex and abstract object. So, we can put  an image finder, a rectangular finder, and a text finder, everything together, and you  can link them together with the so-called regular region of interests. And everything,  the object finder itself can inherit the interaction of the basic elements. So, finally, we've got  a really complex object in order to solve complex transactions. So, now the thing is  that you don't have just one single transaction, but a list of transactions in order to solve  your end-user interaction flow. So, the idea is to convert every single transaction graphically.  And what does it mean? It means list Alivix keywords in order to have what? The so-called  test cases. And that's exactly what you have to do in the Alivix editor. So, if you run  that test case, you can automate exactly that user interaction flow. Second, Alivix can  measure application transaction. And here I've got to give you a grasp of the engine  below, telling you something about the engine. We've got a thread. One thread is dedicated  to the frame rubber. So, we are going to take frames from the screen at a certain pace.  For example, here the frequency is 250 milliseconds. And in a point at a time, we've got our transaction.  And there is another thread, the so-called object detector, that is going to take the  first available frame from the frame rubber. And it has a lower frequency, for example,  one second. And when it will detect the object on the screen, then a measurement system comes  up and gives you the final transaction performance. So, that's going on and on for all your test  cases, transactions after transaction, after transaction. The final achievement is that  we wanted to, first of all, assess the availability of the transaction. So, we've got a transaction.  The question is, is available or not? And then we've got to assess the responsiveness  of the transaction. So, how many seconds it takes to appear?  Finally, I'll report the test case transactions in several forms. First of all, to the command  line. And then it provides you also a rich HTML interactive pages. And here you can find  the screenshots of your transaction and the performance below.  Finally, if a keyword breaks, you can find not a static screenshot, but an animation  that gives you the reason why the keyword was broken. Alivix can report everything to  a monitoring system. So, basically, here you've got your Alivix probe. And Alivix test cases  can be automatized with a remote client like an S Client++ and it reports to your monitoring  system. And this is the final time series. So, basically,  it's really clear that our network application suffered down service down times and latency  spikes. So, Alivix 2.3 is out right now. Just browse  alivix.com to download your package. Alivix 2.4 is under development and we are going  to release it this month. So, I wanted to show a short screencast  of Alivix. This is a running test case on a complex graphical user interface. And as  you can see, Alivix is able to recognize complex graphical objects and browse exactly as a  human would do the user interface, filling complex form and so on and so forth.  For example, here you can add a new address in this CMR, completing, completing, filling  this complex form and so on and so forth. So, my final messages are, first of all, Alivix  can really certify the ongoing quality of your IT services. And then, IT operations  can really modulate the infrastructure resources, I mean the pool of hardware resources. And  IT clients can really check their software level agreement with providers.  This is my call for action. You can browse alivix.com in order to have a getting started  documentation. And of course, you can contribute to GitHub. If you pull request something,  we will be really, really happy. So, for us, Alivix is highs and ends like a human with  a stopwatcher. Thanks a lot for your attention.
Alyvix_Under_the_hood_Visual_synthetic_monitoring_of_network_applications_that_work_th_0_47.png Alyvix_Under_the_hood_Visual_synthetic_monitoring_of_network_applications_that_work_th_48_131.png Alyvix_Under_the_hood_Visual_synthetic_monitoring_of_network_applications_that_work_th_132_5543.png Alyvix_Under_the_hood_Visual_synthetic_monitoring_of_network_applications_that_work_th_5544_6203.png Alyvix_Under_the_hood_Visual_synthetic_monitoring_of_network_applications_that_work_th_6204_6323.png Alyvix_Under_the_hood_Visual_synthetic_monitoring_of_network_applications_that_work_th_6324_6407.png Alyvix_Under_the_hood_Visual_synthetic_monitoring_of_network_applications_that_work_th_6408_6575.png Alyvix_Under_the_hood_Visual_synthetic_monitoring_of_network_applications_that_work_th_6576_6887.png Alyvix_Under_the_hood_Visual_synthetic_monitoring_of_network_applications_that_work_th_6888_8651.png Alyvix_Under_the_hood_Visual_synthetic_monitoring_of_network_applications_that_work_th_8652_9671.png Alyvix_Under_the_hood_Visual_synthetic_monitoring_of_network_applications_that_work_th_9672_9755.png Alyvix_Under_the_hood_Visual_synthetic_monitoring_of_network_applications_that_work_th_9756_11411.png Alyvix_Under_the_hood_Visual_synthetic_monitoring_of_network_applications_that_work_th_11412_12251.png Alyvix_Under_the_hood_Visual_synthetic_monitoring_of_network_applications_that_work_th_12252_15155.png Alyvix_Under_the_hood_Visual_synthetic_monitoring_of_network_applications_that_work_th_15156_15299.png Alyvix_Under_the_hood_Visual_synthetic_monitoring_of_network_applications_that_work_th_15300_17819.png Alyvix_Under_the_hood_Visual_synthetic_monitoring_of_network_applications_that_work_th_17820_17934.png