Network visualization and automation
CC Attribution - NonCommercial - ShareAlike 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this
Fourmy, Antoine
Network visualization and automation Network automation and orchestration is the latest phase in the evolution of IP and optical networks. Over the last few years, network engineers have created a variety of libraries and softwares to help them with the management, configuration and automation of huge networks made of hundreds of thousands of network devices (e.g. routers, switches, antenna...). In this talk, I will introduce NetDim, a vendor-neutral software for network modeling and automation based on the standard Python library for GUI programming: tkinter. I will start the presentation with an introduction to network visualization, and show how tkinter can be used to implement graph drawing algorithms, as well as a full-on Geographic Information System (GIS), allowing one to place network devices on a world map at their GPS coordinates. I will then discuss the whys and wherefores of network automation, and show how to automatically generate and push scripts to a network device via SSH or Telnet.
 So, the plan is very simple. I will start with network visualization and I will end with network automation. But before I start, I want to make a short introduction into networking, like a crash course in networking. For those of you who don't know what it means. So when I say network, what I mean is a telecommunication network. It's a network which nodes are networking devices like routers, switches and antennas. And a network which links are transmission links, usually Ethernet cables and optical fibers. In the network of an internet service provider, we find three main types of nodes, router, optical switch and antenna. As you can see, these are very big devices, like with lots of cards and lots of interfaces. So this has nothing to do with the type of device that you would have at home. These are corporate devices that we would find in the network of a big company like Facebook or Google or in the network of an internet service provider like Vodafone, Telefonica and so on. Same for the transmission links. These are links like those ones on the left, an internet cable with 10G capacity, a 10G connector, and on the right, an optical fiber. So internet cables are used for short distance transmission and optical fibers can be used for both short distance transmission and long distance transmission. Well, there are a few approximations in this slide because it's like to make it simple for you to understand. But if we imagine that you are in Remini and you want to call someone in France, like with your mobile phone with a 4G, what would happen is that like your mobile phone will send a microwave signal to an antenna that should be located nearby like close to this Palakongresi and then the signal will travel through a bunch of antennas. It will be a microwave signal and this microwave part with antennas we call it the microwave back hole. Eventually, it will reach a router. A router is a device that will take the forwarding decision. What we can say about them is like the brain of the router. They know how to reach any part of the world and they will make the forwarding decision. They will know which path the traffic has to take. In order to do that, the router has what we call a routing table, which is a table that associates an outgoing interface to an IP address. And finally, since we want to go to France, which is very far, we need the signal to go very fast. So the router will send the signal to what we call an optical switch and this optical switch will perform an electrical to optical conversion, which means the signal is electrical at first and it will become a wavelength, which is a stream of light. And on top of that, in a single fiber, we can have usually up to 88 wavelengths, so 88 stream of light and this is a technology we call wavelength division multiplexing, which is used to have very high bandwidth capacity. When we buy network devices from a vendor like Cisco, Nokia, Huawei, they usually provide what we call a network management system. It's like a software or a web application that can be used for network visualization, inventory, provisioning, supervision, and sometimes automation as well. It's a network that has like an overview of the network of all devices. If we want to visualize our network, the input file that we have is usually a network topology technology. It could be like a CSV file, Excel file, text file. Basically it's a file that will contain the list of all nodes and the list of all links and the position of the nodes as well. So if we want to draw this network, as you can see, if we draw like nodes at random positions, this doesn't yield a very good display, so we have to find other ways. We have basically two options. The first one is to use the nodes at their physical location, where they're physically located by using GPS coordinates. To do that, we use a type of software that we call GIS software, which stands for Geographic Information System. That allows us to draw maps and draw our network devices on this map. The other option is to use a graph-during algorithm. I will make a demonstration of both if it keeps working. What I'm going to show now is how we can create a simple GIS software in Python, so a software to draw maps. The first thing that we need is a grid programming framework. We have a lot of such frameworks in Python. We have PyGDK, PyQT, WX, Python, PySide, but the one I'm going to use is Tecinter. It's Python interface to Tico. And it's the only grid framework that is part of the standard Python distributions, which means you don't have to pip install anything. It's by default. It's a built-in module. Inside that framework, we will need a widget for drawing. All widgets have one. In Tecinter, it's called a canvas. It's basically a widget that allows you to draw circles, rectangles, but most importantly, polygons. And we will use inside that widget a method to draw a polygon. The reason for that is that a map is actually just a set of polygons. If you look at Italy on this slide, Italy is just like a polygon. It can be considered a polygon. So to draw these polygons, we need to have coordinates. And we will use a special type of file, which is called a shape file. A shape file is a file that contains shapes. And the shape file that I'm going to use contains two types of shapes, polygons and multiple polygons. These shapes are described as a list of coordinates, as a list of points on the Earth, and a point on the Earth is defined as a longitude and latitude. So the first library that we will use is called PiSHP. It's simply a library that allows us to draw, to extract the shapes from the shape file. It contains a reader object that we can use to read the file. We can draw polygons with the create polygons method in Techinter. But as for multi polygons, we cannot draw them. We can only draw polygons. So what we will need to do is to convert all the multiple polygons. It's actually just a set of polygons. And we will need to convert a multi polygon into multiple polygons in order to be able to draw them. And to do that, we will use another library called shapely. When you use shapely and you convert a shape file multi polygon into a shapely multi polygon, it will actually make the multi polygon iterable, which means it will have this double underscore iter function so that you can loop over the multi polygon to yield all the polygons it contains. As I said, shape files contain coordinates defined as a longitude and latitude. And if you look on this slide, you will see that these are angles. But when we draw polygons in a software, in a canvas, we cannot use angles. We need pixel coordinates. So we will have to make some kind of conversion from angles to pixel. This is actually a conversion from 3D to 2D because we have a point on the Earth and we need to have a point on a plan. And this is what we call a projection. So there is a mathematical theorem by Gauss that basically says that if you make such a projection, like you have a sphere and you want to project this sphere on a map, it's called the remarkable theorem, then you will lose some information in the process. There will be what we call a distortion. It could be a distortion of distance, of angles, of shapes, any distortion, but we will lose some information. For instance, if we consider the Mercator projection, it preserves angles, but it doesn't preserve areas. In order to make that projection, to convert 3D coordinates to 2D coordinates, we will use another library called Pyproj. It's a library where you can choose a type of projection. It could be Mercator or it could be azimuthal orthographic like you see on this slide. And we will convert a longitude and a latitude into pixel coordinates. So if I sum up, we need a grid programming framework. We will use TakeInter. Inside this framework, we will need to use a widget to draw polygons. TakeInter scanVas. We need a file that contains the coordinates of the map, a shape file. We will use PySHP to read the shape file. We will use shapeLine to convert polygons into polygons. And we will use Pyproj to convert geographic coordinates into projected coordinates. So I will open a Jupyter notebook to show you how it looks like. So Okay, so this first code shows you how to initialize taking into our main window and how to initialize a canvas and finally how to use a create polygon method in order to create a polygons. If I start this code, I will get this, which is like a graphical interface with a polygon. Finally I have this snippet to show you how to use Pyproj. We need to initialize a projection. This EPSG code like 3395 stands for the Mercator projection. Then I have the longitude and latitude of Remini and I can use the Mercator projection to convert longitude and latitude into projected coordinates. I can use this inverse equal true to make the reverse operation which is convert projected coordinates into geographic coordinates. If I run this code, I print the coordinates. We can see that we have the xy and longitude latitude. Then I show you how to use the shape file reader and how to import shape lines. I have this local path to a shape file and I will start with the reader object. I will extract all the shapes from the shape file. If I loop over the shapes, I will convert all of them into shape lines shapes so that multiple ligands are iterable. Like this. You see that I have this azat shape magic eater and it returns true. All multiple ligands are iterable. Finally this is like the final code that shows you everything, how to start the kinter, how to give the local path to the shape file. I have this two projected coordinates function in order to convert longitude and latitude into projected coordinates. I will read the file, extract all its shapes. I will loop over the polygons, convert them into shape like polygons and finally I will use the create polygons function. If I execute this code, I get this. It's very simple but it's working. I can improve it a little bit by I can change the projection here to have an azimuthal autographic projection. I will create a circle which is the radius of the earth and I will use the take into file dialogue so that when you run the code, the kinter will ask you to choose a file and it will then draw it. If I run this code, it asks me for a file and if I draw it, I get this. This is very easy to do in Python. As you can see, it only takes 30 lines of Python to get this result. What I want to show now, if it works. No, it doesn't work. Okay. So I will actually use this method in order to import a real network. I will import project, this network file contains a real network. It's actually one of the main French backbone. You will see in a few minutes. Okay. So if I import this file. Okay. So this is one of the main French network. It is made of about 350 optical switches. In case you are interested, these optical switches are Alcatel, you send photonic service switch or PSS 1830. So basically if you are in France and you are trying to use internet with your mobile phone or with your computer. Well, there is a chance that the signal you send will actually be transported over this network. Of course if I wanted to do just visualization, I could use frameworks like I could use a Google map API, I could use Geo Django or I could use this tree also if I wanted to do it in JavaScript. But the reason why I want to use an actual software is that I can select some nodes, I can move them, I can delete them, I can create new nodes with a drag and drop system so I can actually create a network and I can look at the property of the node. Once I put the IP address here, okay. Well I can say like this node has this IP address and once this is done, I can hopefully start a new connection to the device like this. Okay, so the device I'm using, you don't see it from there but it's actually just here on the ground. It's a Cisco 1841. Well. It's back. Okay, so I'm connected to the router. There is a few things that I want to show you. The name of the router, it's called router 2, it's what you see on the left part right here. So this is what we call the host name. If I want to change it, what I need to do is to enter the configuration mode by writing configure terminal and then type host name and the host name I want, for instance, Remini. And when I do that, you see that the host name, the left part, is changed to Remini. So what I'm going to do now? The idea is that if you have like 5,000 devices and you want to change the host name on all 5,000 devices, you cannot possibly log into all the devices by hand and change the host name yourself. So you need to automate this process and we can do this by using Python. So I will come back to my slide. Wow. So this is a slide, this is another view of the network automation landscape, which is also a method you can use in order to automate the network. The first one is the one that network engineers have been using for many years. It's simply to connect to the device with a library such as Paramico. Paramico is a different library in Python to start SSH connections, but Paramico is very complicated to use. So network engineers have built other libraries on top of Paramico in order to make it easier for network engineers to automate. The two most famous libraries are NetMico and XCrypt. Finally you have also the NetConf and Yang solution. NetConf is a network configuration protocol and Yang is a modeling language that we use to shape the data that are sent over NetConf. This is standardized by the IETF. I don't want to spend too much time on this one, but just so you know there was a talk at the last Python U.S. in Portland about how to use NetConf and Yang. Finally you can use OSS protocols like TL1 and SNMP with Python library to SNMP, which is called PySNMP, but SNMP is usually very complicated to use. You can also use configuration management tools. These tools originate from the DevOps world, but they can actually be used for network devices as well. So you have Puppet, Chef, Saltstack, HPNA, but you have also Ansible, which is the one that is gaining momentum at the time, and it's an open source Python project. And well, if it works, I will make a demonstration of how to use NetMico, XCrypt, and Ansible in order to change the hostname. So I will come back to the code. I will just explain what I would have done. What I wanted to do was to use XCrypt, NetMico, and Ansible in order to show you how to do basic automation. I'm not sure I can still do it. Okay, I'm still connected to the Cisco device. You can see the hostname is Remini. And if I use XCrypt, so the idea is that I must create an Acunt object in order to give the credentials to connect to the device. I must create a host object to give him the IP address of the device. And finally, I will send all the commands required to make that automation. So if I say hostname friends and I run this code in a new shell, you see the hostname was changed to friends without having to log into the device. So if we had a lot of devices, what we would use is to multistread all the SSH connection in order to configure all the routers at the same time. XCrypt allows you to do this by if you loop over the IP address and you create host for each IP address, then you append them to a host list that will contain all the device that you want to run the script on. And XCrypt will actually use multiple threads so that all SSH connections are done in parallel. And you have this max thread parameter here when you can say, I want at most five threads. You can do exactly the same thing with NetMico. You have this connection parameters dictionary that contains the credentials like username password as well as the IP address of the device. You would use this connect handler to log into the device and you will send all the commands you need to send. So for instance, what I'm doing here is that I'm adding a description to the interface fastet.net 0 slash 0. And I'm also storing the result of show running config which is the configuration of the water inside a config variable. So if I run this script, it will take a few seconds. Okay. If I print config, I see that config contains the configuration of the water. And if I look at the interface, the configuration of the interface, I see that it has this description OSPF interface. That configuration was pushed with NetMico. I can also do the same thing with Ansible. I will start Siegwin. So the idea with Ansible is that you have a file called inventory. If I look at this file, you see it contains the IP address of the device you want to send the script to. And you have what we call playbooks. These are this file.yml files. You can look at one example here in order to do the same thing which is changing the host name. So I will first give the credential to Ansible and I will then use the iOS config module. And this line's host name, Paris, will change the host name to Paris. So I can do that using Siegwin by writing Ansible playbook. I tell him that the inventory file is inventory and I change the host name. Okay. As you can see, it worked. The Ansible script was sent to the device and the host name was changed. What I can also do is use the iOS command Ansible module in order to send the short running config command. And then I will use copy to store this configuration in a file output.text. I will also use Siegwin to send this one. So change host name, save running config. Okay. And if I look at the folder, you can see that I have this new output.text file. And if I look at what's inside, it contains the configuration of my device. So I had more things to show, but I think we can stop here. Thank you. Thanks, Antoine, for the talk. Questions? When you want to use commands that will require confirmation, like, I don't know, reload the router, probably will... Yes. Actually, with X-Crypt, you should look at the documentation. I don't know if I have internet. But if you look at X-Crypt documentation, you can tell when X-Crypt should expect a prompt that it does not recognize. For instance, if you connect to an Alcatel device, sometimes it will ask you for yes or no, say yes to continue the login part. And you can tell X-Crypt that if it says yes, then it should understand that the device is expecting something. And that works perfectly. Hi. Maybe you have told us where have you got the data for the geographic representation from? Where I took the shapefiles? Yeah. I mean, the raw data where it took it from. The shapefiles, you can find them online. I just Googled shapefiles countries or shapefile continents about the data of the network. That's because, well, I work at the company that runs this network. So I did an XML query to the SAM 5620, which is a software that supervises the network in order to retrieve the topology. Hi. Hi. What are the best practices to map like a lot of devices? Yeah. Well, actually, the tool I made, I would like to show you how to map the device. Well, actually, the tool I made, I wanted to show it, but because of the technical issues I couldn't, can do that. That was my next demonstration. If you have several devices like this, what you could do is what I did is like providing a graphical interface to XCrypt so that you don't have to cut anything. It's so graphical. I can go to script creation and I can make a script change host name. And then if I write configure terminal and then host name, you would like this part to be a variable, right? Is that your question? Yeah. Let's say if they have like a device is scripting this to the lab. So using this software, what you could do is like doing host name, which means replay this part with the host name value of the property of the device and you can save this script. Then if you look at the properties, it will look for the host name properties and it will replace the value. And actually the host name properties does not exist yet. So what I did is a way for network engineers to create properties on the fly, even if it does not exist. So I have this, I have a file called change host name and it contains the value of the host name. And if I import this file like this, import host name, you can see that the property, well, it doesn't work here, but you can see that you have new host name properties and it actually contains the value that is in the Excel file. What you would do is select all three devices like this and then you click on send a script, you will choose a script and for each devices, the host name part would be replaced with the actual value of host name for the node. This is using X script. X script on the back. X script but you could do the same using NetMico and you could do the same using NC build as well. And if you wanted to do this like in pure Python, you would just need to have a dictionary that maps, well, the IP address of the device to the value of host name or whatever you want and that would work too. The question was if you use some bolt or some secret storage, because actually it was credential in customer environment are very important. Yeah, no, I don't use any secret storage because it was just for a simple demonstration and well, it's not like, but yeah, you would need to encrypt the data in a real network. Thanks. More questions. So is there an Ansible interface for PyNMS? Is it what? Can you run Ansible scripts? Selecting a few hosts and right click run script? Actually I want to add that as a new feature but right now I'm just working on NC build trying to make a useful NC build script and then when I get something that's working, I want to make PyNMS like as a graphical interface for NC build so that you don't need to actually write the YAML file, you can do it graphically as well. More questions. Okay, no questions. So let's thank Antoine again. Thanks. Thank you.
 So, the plan is very simple.  I will start with network visualization and I will end with network automation.  But before I start, I want to make a short introduction into networking, like a crash  course in networking.  For those of you who don't know what it means.  So when I say network, what I mean is a telecommunication network.  It's a network which nodes are networking devices like routers, switches and antennas.  And a network which links are transmission links, usually Ethernet cables and optical  fibers.  In the network of an internet service provider, we find three main types of nodes, router,  optical switch and antenna.  As you can see, these are very big devices, like with lots of cards and lots of interfaces.  So this has nothing to do with the type of device that you would have at home.  These are corporate devices that we would find in the network of a big company like  Facebook or Google or in the network of an internet service provider like Vodafone, Telefonica  and so on.  Same for the transmission links.  These are links like those ones on the left, an internet cable with 10G capacity, a 10G  connector, and on the right, an optical fiber.  So internet cables are used for short distance transmission and optical fibers can be used  for both short distance transmission and long distance transmission.  Well, there are a few approximations in this slide because it's like to make it simple  for you to understand.  But if we imagine that you are in Remini and you want to call someone in France, like with  your mobile phone with a 4G, what would happen is that like your mobile phone will send a  microwave signal to an antenna that should be located nearby like close to this Palakongresi  and then the signal will travel through a bunch of antennas.  It will be a microwave signal and this microwave part with antennas we call it the microwave  back hole.  Eventually, it will reach a router.  A router is a device that will take the forwarding decision.  What we can say about them is like the brain of the router.  They know how to reach any part of the world and they will make the forwarding decision.  They will know which path the traffic has to take.  In order to do that, the router has what we call a routing table, which is a table that  associates an outgoing interface to an IP address.  And finally, since we want to go to France, which is very far, we need the signal to go  very fast.  So the router will send the signal to what we call an optical switch and this optical  switch will perform an electrical to optical conversion, which means the signal is electrical  at first and it will become a wavelength, which is a stream of light.  And on top of that, in a single fiber, we can have usually up to 88 wavelengths, so 88  stream of light and this is a technology we call wavelength division multiplexing, which  is used to have very high bandwidth capacity.  When we buy network devices from a vendor like Cisco, Nokia, Huawei, they usually provide  what we call a network management system.  It's like a software or a web application that can be used for network visualization,  inventory, provisioning, supervision, and sometimes automation as well.  It's a network that has like an overview of the network of all devices.  If we want to visualize our network, the input file that we have is usually a network topology  technology.  It could be like a CSV file, Excel file, text file.  Basically it's a file that will contain the list of all nodes and the list of all links  and the position of the nodes as well.  So if we want to draw this network, as you can see, if we draw like nodes at random positions,  this doesn't yield a very good display, so we have to find other ways.  We have basically two options.  The first one is to use the nodes at their physical location, where they're physically  located by using GPS coordinates.  To do that, we use a type of software that we call GIS software, which stands for Geographic  Information System.  That allows us to draw maps and draw our network devices on this map.  The other option is to use a graph-during algorithm.  I will make a demonstration of both if it keeps working.  What I'm going to show now is how we can create a simple GIS software in Python, so a software  to draw maps.  The first thing that we need is a grid programming framework.  We have a lot of such frameworks in Python.  We have PyGDK, PyQT, WX, Python, PySide, but the one I'm going to use is Tecinter.  It's Python interface to Tico.  And it's the only grid framework that is part of the standard Python distributions, which  means you don't have to pip install anything.  It's by default.  It's a built-in module.  Inside that framework, we will need a widget for drawing.  All widgets have one.  In Tecinter, it's called a canvas.  It's basically a widget that allows you to draw circles, rectangles, but most importantly,  polygons.  And we will use inside that widget a method to draw a polygon.  The reason for that is that a map is actually just a set of polygons.  If you look at Italy on this slide, Italy is just like a polygon.  It can be considered a polygon.  So to draw these polygons, we need to have coordinates.  And we will use a special type of file, which is called a shape file.  A shape file is a file that contains shapes.  And the shape file that I'm going to use contains two types of shapes, polygons and  multiple polygons.  These shapes are described as a list of coordinates, as a list of points on the Earth, and a point  on the Earth is defined as a longitude and latitude.  So the first library that we will use is called PiSHP.  It's simply a library that allows us to draw, to extract the shapes from the shape file.  It contains a reader object that we can use to read the file.  We can draw polygons with the create polygons method in Techinter.  But as for multi polygons, we cannot draw them.  We can only draw polygons.  So what we will need to do is to convert all the multiple polygons.  It's actually just a set of polygons.  And we will need to convert a multi polygon into multiple polygons in order to be able  to draw them.  And to do that, we will use another library called shapely.  When you use shapely and you convert a shape file multi polygon into a shapely multi polygon,  it will actually make the multi polygon iterable, which means it will have this double underscore  iter function so that you can loop over the multi polygon to yield all the polygons it  contains.  As I said, shape files contain coordinates defined as a longitude and latitude.  And if you look on this slide, you will see that these are angles.  But when we draw polygons in a software, in a canvas, we cannot use angles.  We need pixel coordinates.  So we will have to make some kind of conversion from angles to pixel.  This is actually a conversion from 3D to 2D because we have a point on the Earth and we  need to have a point on a plan.  And this is what we call a projection.  So there is a mathematical theorem by Gauss that basically says that if you make such  a projection, like you have a sphere and you want to project this sphere on a map, it's  called the remarkable theorem, then you will lose some information in the process.  There will be what we call a distortion.  It could be a distortion of distance, of angles, of shapes, any distortion, but we will lose  some information.  For instance, if we consider the Mercator projection, it preserves angles, but it doesn't  preserve areas.  In order to make that projection, to convert 3D coordinates to 2D coordinates, we will  use another library called Pyproj.  It's a library where you can choose a type of projection.  It could be Mercator or it could be azimuthal orthographic like you see on this slide.  And we will convert a longitude and a latitude into pixel coordinates.  So if I sum up, we need a grid programming framework.  We will use TakeInter.  Inside this framework, we will need to use a widget to draw polygons.  TakeInter scanVas.  We need a file that contains the coordinates of the map, a shape file.  We will use PySHP to read the shape file.  We will use shapeLine to convert polygons into polygons.  And we will use Pyproj to convert geographic coordinates into projected coordinates.  So I will open a Jupyter notebook to show you how it looks like.  So  Okay, so this first code shows you how to initialize taking into our main window and how to initialize  a canvas and finally how to use a create polygon method in order to create a polygons.  If I start this code, I will get this, which is like a graphical interface with a polygon.  Finally I have this snippet to show you how to use Pyproj.  We need to initialize a projection. This EPSG code like 3395 stands for the Mercator projection.  Then I have the longitude and latitude of Remini and I can use the Mercator projection  to convert longitude and latitude into projected coordinates.  I can use this inverse equal true to make the reverse operation which is convert projected  coordinates into geographic coordinates. If I run this code, I print the coordinates.  We can see that we have the xy and longitude latitude.  Then I show you how to use the shape file reader and how to import shape lines.  I have this local path to a shape file and I will start with the reader object.  I will extract all the shapes from the shape file.  If I loop over the shapes, I will convert all of them into shape lines shapes so that  multiple ligands are iterable.  Like this.  You see that I have this azat shape magic eater and it returns true.  All multiple ligands are iterable.  Finally this is like the final code that shows you everything, how to start the kinter, how  to give the local path to the shape file.  I have this two projected coordinates function in order to convert longitude and latitude  into projected coordinates.  I will read the file, extract all its shapes.  I will loop over the polygons, convert them into shape like polygons and finally I will  use the create polygons function.  If I execute this code, I get this.  It's very simple but it's working.  I can improve it a little bit by I can change the projection here to have an azimuthal autographic  projection.  I will create a circle which is the radius of the earth and I will use the take into  file dialogue so that when you run the code, the kinter will ask you to choose a file and  it will then draw it.  If I run this code, it asks me for a file and if I draw it, I get this.  This is very easy to do in Python.  As you can see, it only takes 30 lines of Python to get this result.  What I want to show now, if it works.  No, it doesn't work.  Okay.  So I will actually use this method in order to import a real network.  I will import project, this network file contains a real network.  It's actually one of the main French backbone.  You will see in a few minutes.  Okay.  So if I import this file.  Okay.  So this is one of the main French network.  It is made of about 350 optical switches.  In case you are interested, these optical switches are Alcatel, you send photonic service  switch or PSS 1830.  So basically if you are in France and you are trying to use internet with your mobile  phone or with your computer.  Well, there is a chance that the signal you send will actually be transported over this  network.  Of course if I wanted to do just visualization, I could use frameworks like I could use a  Google map API, I could use Geo Django or I could use this tree also if I wanted to  do it in JavaScript.  But the reason why I want to use an actual software is that I can select some nodes,  I can move them, I can delete them, I can create new nodes with a drag and drop system  so I can actually create a network and I can look at the property of the node.  Once I put the IP address here, okay.  Well I can say like this node has this IP address and once this is done, I can hopefully  start a new connection to the device like this.  Okay, so the device I'm using, you don't see it from there but it's actually just here  on the ground.  It's a Cisco 1841.  Well.  It's back.  Okay, so I'm connected to the router.  There is a few things that I want to show you.  The name of the router, it's called router 2, it's what you see on the left part right  here.  So this is what we call the host name.  If I want to change it, what I need to do is to enter the configuration mode by writing  configure terminal and then type host name and the host name I want, for instance, Remini.  And when I do that, you see that the host name, the left part, is changed to Remini.  So what I'm going to do now?  The idea is that if you have like 5,000 devices and you want to change the host name on all  5,000 devices, you cannot possibly log into all the devices by hand and change the host  name yourself.  So you need to automate this process and we can do this by using Python.  So I will come back to my slide.  Wow.  So this is a slide, this is another view of the network automation landscape, which is  also a method you can use in order to automate the network.  The first one is the one that network engineers have been using for many years.  It's simply to connect to the device with a library such as Paramico.  Paramico is a different library in Python to start SSH connections, but Paramico is very  complicated to use.  So network engineers have built other libraries on top of Paramico in order to make it easier  for network engineers to automate.  The two most famous libraries are NetMico and XCrypt.  Finally you have also the NetConf and Yang solution.  NetConf is a network configuration protocol and Yang is a modeling language that we use  to shape the data that are sent over NetConf.  This is standardized by the IETF.  I don't want to spend too much time on this one, but just so you know there was a talk  at the last Python U.S. in Portland about how to use NetConf and Yang.  Finally you can use OSS protocols like TL1 and SNMP with Python library to SNMP, which  is called PySNMP, but SNMP is usually very complicated to use.  You can also use configuration management tools.  These tools originate from the DevOps world, but they can actually be used for network  devices as well.  So you have Puppet, Chef, Saltstack, HPNA, but you have also Ansible, which is the one  that is gaining momentum at the time, and it's an open source Python project.  And well, if it works, I will make a demonstration of how to use NetMico, XCrypt, and Ansible  in order to change the hostname.  So I will come back to the code.  I will just explain what I would have done.  What I wanted to do was to use XCrypt, NetMico, and Ansible in order to show you how to do  basic automation.  I'm not sure I can still do it.  Okay, I'm still connected to the Cisco device.  You can see the hostname is Remini.  And if I use XCrypt, so the idea is that I must create an Acunt object in order to give  the credentials to connect to the device.  I must create a host object to give him the IP address of the device.  And finally, I will send all the commands required to make that automation.  So if I say hostname friends and I run this code in a new shell, you see the hostname  was changed to friends without having to log into the device.  So if we had a lot of devices, what we would use is to multistread all the SSH connection  in order to configure all the routers at the same time.  XCrypt allows you to do this by if you loop over the IP address and you create host for  each IP address, then you append them to a host list that will contain all the device  that you want to run the script on.  And XCrypt will actually use multiple threads so that all SSH connections are done in parallel.  And you have this max thread parameter here when you can say, I want at most five threads.  You can do exactly the same thing with NetMico.  You have this connection parameters dictionary that contains the credentials like username  password as well as the IP address of the device.  You would use this connect handler to log into the device and you will send all the commands  you need to send.  So for instance, what I'm doing here is that I'm adding a description to the interface  fastet.net 0 slash 0.  And I'm also storing the result of show running config which is the configuration of the  water inside a config variable.  So if I run this script, it will take a few seconds.  Okay.  If I print config, I see that config contains the configuration of the water.  And if I look at the interface, the configuration of the interface, I see that it has this description  OSPF interface.  That configuration was pushed with NetMico.  I can also do the same thing with Ansible.  I will start Siegwin.  So the idea with Ansible is that you have a file called inventory.  If I look at this file, you see it contains the IP address of the device you want to send  the script to.  And you have what we call playbooks.  These are this file.yml files.  You can look at one example here in order to do the same thing which is changing the  host name.  So I will first give the credential to Ansible and I will then use the iOS config module.  And this line's host name, Paris, will change the host name to Paris.  So I can do that using Siegwin by writing Ansible playbook.  I tell him that the inventory file is inventory and I change the host name.  Okay.  As you can see, it worked.  The Ansible script was sent to the device and the host name was changed.  What I can also do is use the iOS command Ansible module in order to send the short running  config command.  And then I will use copy to store this configuration in a file output.text.  I will also use Siegwin to send this one.  So change host name, save running config.  Okay.  And if I look at the folder, you can see that I have this new output.text file.  And if I look at what's inside, it contains the configuration of my device.  So I had more things to show, but I think we can stop here.  Thank you.  Thanks, Antoine, for the talk.  Questions?  When you want to use commands that will require confirmation, like, I don't know, reload the  router, probably will...  Yes.  Actually, with X-Crypt, you should look at the documentation.  I don't know if I have internet.  But if you look at X-Crypt documentation, you can tell when X-Crypt should expect a  prompt that it does not recognize.  For instance, if you connect to an Alcatel device, sometimes it will ask you for yes  or no, say yes to continue the login part.  And you can tell X-Crypt that if it says yes, then it should understand that the device  is expecting something.  And that works perfectly.  Hi.  Maybe you have told us where have you got the data for the geographic representation  from?  Where I took the shapefiles?  Yeah.  I mean, the raw data where it took it from.  The shapefiles, you can find them online.  I just Googled shapefiles countries or shapefile continents about the data of the network.  That's because, well, I work at the company that runs this network.  So I did an XML query to the SAM 5620, which is a software that supervises the network  in order to retrieve the topology.  Hi.  Hi.  What are the best practices to map like a lot of devices?  Yeah.  Well, actually, the tool I made, I would like to show you how to map the device.  Well, actually, the tool I made, I wanted to show it, but because of the technical issues  I couldn't, can do that.  That was my next demonstration.  If you have several devices like this, what you could do is what I did is like providing  a graphical interface to XCrypt so that you don't have to cut anything.  It's so graphical.  I can go to script creation and I can make a script change host name.  And then if I write configure terminal and then host name, you would like this part to  be a variable, right?  Is that your question?  Yeah.  Let's say if they have like a device is scripting this to the lab.  So using this software, what you could do is like doing host name, which means replay  this part with the host name value of the property of the device and you can save this  script.  Then if you look at the properties, it will look for the host name properties and it will  replace the value.  And actually the host name properties does not exist yet.  So what I did is a way for network engineers to create properties on the fly, even if it  does not exist.  So I have this, I have a file called change host name and it contains the value of the  host name.  And if I import this file like this, import host name, you can see that the property,  well, it doesn't work here, but you can see that you have new host name properties and  it actually contains the value that is in the Excel file.  What you would do is select all three devices like this and then you click on send a script,  you will choose a script and for each devices, the host name part would be replaced with  the actual value of host name for the node.  This is using X script.  X script on the back.  X script but you could do the same using NetMico and you could do the same using NC  build as well.  And if you wanted to do this like in pure Python, you would just need to have a dictionary  that maps, well, the IP address of the device to the value of host name or whatever you  want and that would work too.  The question was if you use some bolt or some secret storage, because actually it was credential  in customer environment are very important.  Yeah, no, I don't use any secret storage because it was just for a simple demonstration  and well, it's not like, but yeah, you would need to encrypt the data in a real network.  Thanks.  More questions.  So is there an Ansible interface for PyNMS?  Is it what?  Can you run Ansible scripts?  Selecting a few hosts and right click run script?  Actually I want to add that as a new feature but right now I'm just working on NC build  trying to make a useful NC build script and then when I get something that's working,  I want to make PyNMS like as a graphical interface for NC build so that you don't need to actually  write the YAML file, you can do it graphically as well.  More questions.  Okay, no questions.  So let's thank Antoine again.  Thanks.  Thank you.
Antoine_Fourmy_-_Network_visualization_and_automation_HUQTVbyGdHs_0_1115.png Antoine_Fourmy_-_Network_visualization_and_automation_HUQTVbyGdHs_1116_3767.png Antoine_Fourmy_-_Network_visualization_and_automation_HUQTVbyGdHs_3768_4163.png Antoine_Fourmy_-_Network_visualization_and_automation_HUQTVbyGdHs_4164_4211.png Antoine_Fourmy_-_Network_visualization_and_automation_HUQTVbyGdHs_4212_5843.png Antoine_Fourmy_-_Network_visualization_and_automation_HUQTVbyGdHs_5844_7463.png Antoine_Fourmy_-_Network_visualization_and_automation_HUQTVbyGdHs_7464_8507.png Antoine_Fourmy_-_Network_visualization_and_automation_HUQTVbyGdHs_8508_10415.png Antoine_Fourmy_-_Network_visualization_and_automation_HUQTVbyGdHs_10416_11351.png Antoine_Fourmy_-_Network_visualization_and_automation_HUQTVbyGdHs_11352_18107.png Antoine_Fourmy_-_Network_visualization_and_automation_HUQTVbyGdHs_18108_18155.png Antoine_Fourmy_-_Network_visualization_and_automation_HUQTVbyGdHs_18156_20147.png Antoine_Fourmy_-_Network_visualization_and_automation_HUQTVbyGdHs_20148_21251.png Antoine_Fourmy_-_Network_visualization_and_automation_HUQTVbyGdHs_21252_22487.png Antoine_Fourmy_-_Network_visualization_and_automation_HUQTVbyGdHs_22488_23555.png Antoine_Fourmy_-_Network_visualization_and_automation_HUQTVbyGdHs_23556_24011.png Antoine_Fourmy_-_Network_visualization_and_automation_HUQTVbyGdHs_24012_24755.png Antoine_Fourmy_-_Network_visualization_and_automation_HUQTVbyGdHs_24756_25175.png Antoine_Fourmy_-_Network_visualization_and_automation_HUQTVbyGdHs_25176_25223.png Antoine_Fourmy_-_Network_visualization_and_automation_HUQTVbyGdHs_25224_25271.png Antoine_Fourmy_-_Network_visualization_and_automation_HUQTVbyGdHs_25272_25499.png Antoine_Fourmy_-_Network_visualization_and_automation_HUQTVbyGdHs_25500_30719.png Antoine_Fourmy_-_Network_visualization_and_automation_HUQTVbyGdHs_30720_30971.png Antoine_Fourmy_-_Network_visualization_and_automation_HUQTVbyGdHs_30972_31127.png Antoine_Fourmy_-_Network_visualization_and_automation_HUQTVbyGdHs_31128_36275.png Antoine_Fourmy_-_Network_visualization_and_automation_HUQTVbyGdHs_36276_43367.png Antoine_Fourmy_-_Network_visualization_and_automation_HUQTVbyGdHs_43368_43415.png Antoine_Fourmy_-_Network_visualization_and_automation_HUQTVbyGdHs_43416_43463.png Antoine_Fourmy_-_Network_visualization_and_automation_HUQTVbyGdHs_43464_43511.png Antoine_Fourmy_-_Network_visualization_and_automation_HUQTVbyGdHs_43512_43571.png Antoine_Fourmy_-_Network_visualization_and_automation_HUQTVbyGdHs_43572_43619.png Antoine_Fourmy_-_Network_visualization_and_automation_HUQTVbyGdHs_43620_43715.png Antoine_Fourmy_-_Network_visualization_and_automation_HUQTVbyGdHs_43716_43763.png Antoine_Fourmy_-_Network_visualization_and_automation_HUQTVbyGdHs_43764_44663.png Antoine_Fourmy_-_Network_visualization_and_automation_HUQTVbyGdHs_44664_44735.png Antoine_Fourmy_-_Network_visualization_and_automation_HUQTVbyGdHs_44736_44807.png Antoine_Fourmy_-_Network_visualization_and_automation_HUQTVbyGdHs_44808_45095.png Antoine_Fourmy_-_Network_visualization_and_automation_HUQTVbyGdHs_45096_45155.png Antoine_Fourmy_-_Network_visualization_and_automation_HUQTVbyGdHs_45156_45203.png Antoine_Fourmy_-_Network_visualization_and_automation_HUQTVbyGdHs_45204_45875.png Antoine_Fourmy_-_Network_visualization_and_automation_HUQTVbyGdHs_45876_45935.png Antoine_Fourmy_-_Network_visualization_and_automation_HUQTVbyGdHs_45936_46019.png Antoine_Fourmy_-_Network_visualization_and_automation_HUQTVbyGdHs_46020_46103.png Antoine_Fourmy_-_Network_visualization_and_automation_HUQTVbyGdHs_46104_46187.png Antoine_Fourmy_-_Network_visualization_and_automation_HUQTVbyGdHs_46188_46235.png Antoine_Fourmy_-_Network_visualization_and_automation_HUQTVbyGdHs_46236_46283.png Antoine_Fourmy_-_Network_visualization_and_automation_HUQTVbyGdHs_46284_46343.png Antoine_Fourmy_-_Network_visualization_and_automation_HUQTVbyGdHs_46344_48875.png Antoine_Fourmy_-_Network_visualization_and_automation_HUQTVbyGdHs_48876_52655.png