Understanding Numba - the Python and Numpy compiler
CC Attribution - NonCommercial - ShareAlike 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this
Deil, Christoph
Do you have numerical code written in Python and Numpy? Do you wish it ran faster, using the full potential of your CPU? Then you should try Numba, a JIT compiler that translates a subset of Python and Numpy code into fast machine code. This talk will explain how Numba works, and when and how to use it for numerical algorithms, focusing on how to get very good performance on the CPU. To understand this talk, only a basic knowledge of Python and Numpy is needed. You will learn how Python compiles functions to bytecode and how Numba compiles bytecode to machine code. Why algorithms implemented using Numpy sometimes don't yield great performance, and how to do better using Numba. You will learn about the @numba.jit and @numba.vectorize decorators and how to create functions that use the CPU well by using e.g. multi-threading (several CPU cores), vector instructions (single instruction multiple data) and fast math (trade float accuracy for speed). You will also learn when it does and doesn't make sense to use Numba, by contrasting it briefly with some other options for high-performance computing from Python: PyPy, C, C++, Cython, Numexpr, Dask, PyTorch, Tensorflow and Google JAX
 Hi, thanks for coming. So I will talk about understanding number, the Python and NumPy compiler, and I want to be clear at the start, like I don't really understand number. So I'm a Gamma astronomer from Heidelberg. I'm not a number compiler or CPU expert. I recently started using it, and I think it's awesome, and I wanted to introduce it to you. So let me talk a little bit about how we use number and why we use it. So in Gamma astronomy, we have these telescopes like the HES telescopes in Namibia or the Charenkoff telescopes in Chile, and there we have to do lots of numerical computing for data calibration, reduction analysis, and we need both interactive data and method exploration and then production pipelines. And also the software is often written by astronomers, not professional programmers. And traditionally, the approach was, for example, for HES for the past decade to write everything in C++, also using the root software from CERN, and then maybe have a little bit of Python scripting on top. And now it's different. Now we're trying to build everything in Python and NumPy, and then when needed for performance, we write a little bit of numbers, Python or COC++. So it's a different approach. For the CTA software, we're prototyping it using this Python-first approach, and you see the stack, like it's based on Python and then the PyData ecosystem. So you have NumPy, SciPy, and so on. In astronomy, we also have AstroPy, which is the base library which contains the standard data formats and time handling coordinates, handling, and so on for astronomers. And then on top of this, we can implement the gamma ray analysis software. And this approach is not really unique to what we do in our project. It's kind of become the standard in astronomy. As you can see in this graph here, Python is really now the most popular language to use in astronomy. And as Perry Greenfield points out, like one reason is that Python is a language that is very powerful for developers, for professional developers, but it is also accessible to astronomers. And this makes for a good mix. And I have two other quotes from Jake VanderPlas from the Python keynote at 2017 in the same vein saying for scientific data exploration, speed of development is primary and speed of execution is secondary. And also in Python, we have these libraries for nearly everything. And Python is the glue that combines the scientific codes together. So we have these thousands of Python packages that implement all kinds of methods. Why do we need NumPy, actually? I mean, some algorithms are hard to write in Python and NumPy. Here's an example of a function computing one step in this game of life. I don't know if you're aware of this, but you have this update rule, whether cells live or die. And you can write it in NumPy, but then you have to write this double NumPy.roll call. And the code is really hard to write and hard to read and hard to maintain. So this would be a case where just writing a for loop is simpler. And then if you write it in Python, it's inefficient, and then writing in C and wrapping it from Python can be tedious. So this would be a good case where you would go to NumPy. And there's also a quote from Stan Sybert from the NumPy team, just saying, like, don't write such NumPy high-coos. If Python for loops are simpler, then write loops and use NumPy. Okay. So let me introduce NumPy. What is it? I mean to find out more good to the webpage, NumPy.org. The tagline is, NumPy makes Python code fast, and it's an open source JIT compiler that translates a subset of Python and NumPy code into fast machine code. And actually later in the talk, I'll try to unwrap that statement and explain what it means. But first, let's see a little bit. The name, NumPy, comes from the combination of NumPy and Mumba. So you do number crunching in Python, and it's then fast like Mumba. So Mumbas are apparently some of the fastest snakes in the world. This is where the name comes from. And then how do you use it? Let's assume you have some function doing math, like here, using this Monte Carlo method to compute the number Pi, where you just throw random points in the unit square and then check which ones are x squared plus y squared less than 1. So which ones are in the circle and you count. And that's one way to compute Pi. If you do this in Python, it's extremely slow for a million numbers. Because what Python does is that every number is represented by a Pi object. And then to do the actual computation, what happens in something like x squared or less than 1 and so on, always you take the Pi object and you figure out that it's an integer. Look at the integer inside. Then do the operation and the computation and then create another Pi object. So you have all of this overhead that makes math from Python really slow. To speed it up with number, it's very easy. You just import number and add number to your function. And then in this case, it's 30 times faster. So this is how you tell number to jit your function. Number also understands NumPi. So in this case, I have two arrays x and y, which are like not large NumPi arrays. And I do the same computation by passing them into a Monte Carlo Pi method and then uses a NumPi array expression to do the same computation. And this is faster than using Python. But it also uses much more memory. But again, you can add the number.jit decorator and it will run faster because what NumPi can do is it can avoid these temporary array copies. So when NumPi sees x squared, it creates a new NumPi array which is x squared. And creating these new arrays and allocating them and processing them is some overhead which can be avoided if you compile down to machine code. And that's why NumPi in this case is faster. If you want, you can also write the same function using a for loop. So it's really up to you. In this case, writing a NumPi array expression is more convenient. But sometimes writing a for loop is more convenient. And NumPi will compile your code either way into optimized machine code. So kind of the evolution of a scientific program that's coming to Python is that maybe you start out with C or Fortran and you write loops all the time to do your data processing. Then you come to Python and you figure out that loops are 100 times slower and you learn that you have to write everything as NumPi array expressions. But actually now that we have NumPi writing for loops, it's okay again. So it's a bit of a regression. What are the limitations of number? Number compiles individual functions. It does not compile whole programs like PyPy. Number only supports a subset of Python. It has some support for dictionary list and sets. But you cannot have mixed types of keys and values and all of the operations. Like some things, some Python data structures just cannot be efficiently processed and translated to machine code. And you see an example of this on the left side where I have a list with mixed data types of a string and an integer. And actually, number is not able to compile this to efficient machine code. And what you get is you get a typing error if you try this. Also number only supports a subset of NumPi. I mean, this is ever-growing, but not all functions and all arguments to those functions are available. So you have to see. And also, I mean, number does not support any kind of Python code. Like you cannot use pandas or scikit-learn or the request library or these kind of things from your Python function because number will not know how to translate that into machine code. It's really focused on math and numerics code. So a little bit more about these two JIT modes, the object mode and the no Python mode. If you add the number.jit decorator and you have some function which cannot be efficiently compiled to machine code, what will happen is that you get a number warning saying the compilation is falling back to object mode, but it will still run and give you the results. And what's happening under the hood is that all that happened was that number translated this function into something that's equivalent what the Cpyzen interpreter would execute, where you still have PyObjects and Python C API calls and you get the same performance as if you only use Python without number at all. So 99% of the time, this is not what you want. And to make it more obvious when you write functions which cannot be compiled efficiently, you can use this no Python equals true thing and then you will directly get a typing error saying fail to compile in no Python mode. And there's a shortcut because typing number.jit, no Python equals true is long. You can also type number.jit. If you actually do have a function where you need to go back and interact with the Python interpreter and Python objects, you can. And for this, you use the number.objectModeContextManager. This is rarely needed, but it can be useful if you have like a long-running function and you want to log the progress or update a Python progress bar, which is a Python object. Then you can say with number.objectMode and inside this with statement, you can again interact with your Python session. Okay. So now let's come to the part where we try to understand what number does at least a little bit. So the description kind of is number is a type specializing JIT compiler for Python byte code using LLVM. And this might make your head explode, but let's try to unwrap that a little bit. There's really three parts involved here, what's going on when you write a function and you JIT it and you call it. And the first part is Python itself, and then you have number, and then you have LLVM in the background. And I'll explain on the next slides how these interact. So what the Python compiler does, it starts with your source code, parses it into an abstract syntax tree, and then transforms it to byte code. And this happens on import of a module like when the Python interpreter sees the death statement, it creates this function object and attaches the byte code to the function object, which as you can see here on the left. And then what number can do is it can start with this byte code and compile and transform it to machine code, and that's what the number.jit decorator does. And actually when number.jit decorator is called, it does very little. All it does is it creates a CPU dispatcher proxy object because it cannot compile the function yet because it doesn't know what types come in and what kind of machine code it should generate. So only when the function is called, it will then JIT compile the byte code to LLVMIR exactly for those input types. And then it will also manage the LLVM compilation and execute the compiled function for you. So what is LLVM? LLVM is a compiler infrastructure project. There's many frontends for languages like C, C++, Fortran, Haskell, Rust, Julia, Swift and so on. There's also many backends for all kinds of hardware, so all the different CPU types. The vendors have added support for it and optimized it well. And you could consider number just to be the Python frontend for LLVM. So the way this works is that LLVM is shipped as a Python package called LLVM light that number depends on. And this is maintained by the number team at Anaconda that then ships number and LLVM light so that it's readily available. Concerning alternatives, I mean, at least for us, like the most obvious one and what people also use is Siphon and like number, Siphon is often used to speed up numeric Python code. Siphon is an ahead-of-time compiler where you have to type annotate your Python code and then it compiles to C and then you use a C compiler to compile this to machine code. It's more widely used at this point. It's easier to debug because C code, generated C code is easier to debug than LLVM IR code, which is lower level and looks more like a similar code and probably very few Python people can read and debug this. Yeah. And also, it's very good at interfacing with C and C++ code. I mean, number, on the other hand, is easier to use. You don't have to add type annotations. You don't have to need to have a C compiler, but it is harder to debug. Another advantage of number is that it optimizes just in time for your CPU and GPU so you don't need to build and distribute binaries for many architectures. Instead, it will always use all of the CPU features you have, like advanced processor instructions and so on. Other number alternatives, I mean, there are, Siphon is great and there are many other great tools that exist for high-performance computing in Python. So you have Siphon, C, C++ and PyBind 11 if you want to go the Python C extension way. There's PyPy, which is an alternative to Cpy in which JIT compiles the whole program. And then you have all of these modern things like TensorFlow, JAX, PyTorch, and Dask and so on that similarly to number also use Python and NumPy mainly as the language to specify what kind of computation should happen, but then they will do some kind of compilations and execute it in various ways. And I don't have time and also don't have the expertise to explain all of the differences, how JAX and TensorFlow and so on compares to number and how exactly they compile. But you have to make a choice there. And it's not easy because there's many great libraries now available in Python. Okay. So some more things about number. One thing you should know about is number.s or number minus minus sys info from the command line. Or you can also use it from Ipython and Jupyter if you just put the exclamation mark in front, you can execute shell commands so you can also get the info from there. And as you can see here on the left, this gives you all of the relevant information about the hardware on your computer like what CPU and GPU you have, which Python version, number version, and LLVM version you're using, whether you have the Intel short vector math library installed. I'll talk more about this later, whether you have the Intel threading building blocks library installed, which will make number faster in some cases also. And then which GPUs you have available and GPU drivers you have installed. One thing you can do if you want to, if you want to, if you have a multicore CPU and want to make your computation run fastest to add the parallel equals true option to the number to JIT call. And this will then do multi threading using one of the backends, either open and P threading building blocks or a custom one. If you want to use TBB, you have to do this content and start TBB in addition. And then as shown in this example, it will work automatically for NumPy array expressions. There's no code changes needed. So in this case, I got a 3.2 speed up on my four core CPU for this computation by parallelizing it. If you have four loops inside of your function, you should use the number.P range generator, I guess. And then what will happen is the same thing that number will parallelize this loop and you will get the speed up from using multiple cores or also vector instructions. And what I show here on the lower right is just always like when you have Python decorators, you can use them in two ways. You can either put them with the add sign on top of your function and the decorator will be automatically applied. Or you can just write your function without the decorator and then apply the decorator after by just passing the function into the decorator. So you can say number.JIT passing the compute function. Or you can say number.JIT parallel equals true. This is a decorator and you can pass in the compute function. And this can be convenient if you want to try out different options to compile the same function so you don't have to always copy and paste the code of the function also. Another option you have is fast math equals true and there you can trade accuracy for speed for some computations. So there is this IEEE floating point standard that in this case, for example, requires that the loop is must accumulate these numbers in order to get precisely defined results. But if you're willing to give this up and have a little bit of different accuracy, then the computation can go faster because the compiler can vectorize this reduction. There's another way you can speed up math functions. So if you have code that has square root, sign, exponential, and so on, then you can condo install the ICCRT package and this will make these Intel short vector math libraries functions available until number will tell elevator to use those. And these are just like much faster implementations of these math functions. So that's how you get fast math. You might ask like how fast is number? I would say number gives very good performance and there are many options to tweak the computation, but there is no simple answer to this question and there is also no simple answer to the question of like how number compares to Python, size, and numpy, C, Fortran, and so on. You can find many blog posts on the Internet and they all have kind of have different outcomes depending on what application they do and compiler flags they use and what hardware they have, they get different speeds for different tools. And we've already seen this for number on the previous slides. The speed up is not always the same. So what you have to do is if you care about performance, you start by defining a benchmark for your application that you really care about where you have a performance bottleneck and then you measure and then you try to improve it. Another thing I wanted to introduce are numpy U funds or universal functions. So these are functions like add sign and so on from numpy itself. And these all support array broadcasting which you can see here on the left side. So you can adjust multiply to numbers. You can also multiply a number with an array and it will broadcast these inputs and generate an output array and there's a more complex example here. And then they also have these special methods attached like accumulate which apply to one element at a time and accumulate it giving a different output array. And so far if you wanted to make one of these U funds, you had to write C code and use the numpy C API which was pretty hard. And now with number it's really easy. You use the number vectorize decorator and it will make the U fun for you. So the way you write your function is you don't put a for loop. You just write the operation for one element and then kind of the implicit for loop to loop over arrays and do broadcasting and so on. This will be generated by number for you. And there's two ways to do it. You can give the type signature in the vectorize call. Like I do here I say please make a U fun. Assuming the inputs are integer numbers and then number will generate one U fun on the vectorize call. If you don't give a signature then again such a dispatcher object is created and then dynamically when you call the U fun, when you call the function number will generate U funds for the input types you passed. So it will generate a different one if you put a float, pass a float input array or an integer input array. Okay. So I'm almost done already. So the, I mean, number is really a family of compilers. I've talked about two, number.jit for regular functions and number.vectorize for U funds and just given a quick introduction to those if you want to learn more then check out the number documentation. There is also the geo vectorize decorator which can make generalized U funds. There is the stencil for neighborhood computation. So if you want to do some kind of convolution or sliding window computation it's easy with the stencil decorator. There is C fun which can generate functions with well-defined C callback ABI. So this could be useful for example if you want to call a number function that you write in Python from C or C++ code. So if, like all of your application runs in C++ but then you want to extend it with Python then this would be one way to do it. And then there are these CudaJit and RockJit decorators to work with GPUs also. So just as a last point, I mean who uses number? So Jake Wanderplatz again like wrote a few blog posts in 2013 like when the project was still very young saying and becoming more and more convinced that number is the future of fast scientific computing in Python. This has not really happened so far. And then in 2018 Matthew Rocklin wrote an article advocating for the numeric Python community should consider adopting number more. So I think currently many people and applications use number for their work and projects like we do for gamma ray astronomy and many others do as well. But then the large libraries like NumPy, SciPy, Pandas and Scikit-learn have not adopted number yet. And there are some nice examples now of libraries or packages using number. For example, DataShader which is shown on the top left which is a large data visualization library is implementing their stuff in number. There's Librosa which is for digital signal processing and audio and music analysis is implemented using number. And Intel has written HPEG which is the performance toolkit and this can do big data processing and supports Pandas. So they've done something very similar to what NumBer did. They've defined a decorator, HPEG.jit which can take a function which does IO and then NumPy or Pandas computations and so on and it will parallelize this also to clusters using MPI. So my summary and conclusions are that I mean NumBer is a type specializing JIT compiler for Python bytecode to LLVNIR. The project started in 2012. The current version is 0.44 and I think they're well underway to version 1.0. It will use your CPU and GPU well and it's really easy to use. You just have to add this Python decorator. So use NumBer.jit for normal functions, NumBer.vectorize to make you funs and use NumBer minus s to check your machine and installation. If you don't find you get good performance or it's not working and then if you want to use multi-core CPU and get fast results, consider putting parallel equals true and fast mass equals true and also installing this SVML package. And I mean, yeah, I'm not a NumBer developer. I didn't do anything to make this work. I just wanted to thank the NumBer developers at Anaconda and also the other people and companies that have contributed like for example Intel. Very cool, thank you for the great talk. We have five minutes for questions. Again, we have microphones over there. Over there if you have a question, just line up behind the microphone and ask away. Hi. Thank you for the talk. How would you recommend packaging your package that uses NumBer? Would it depend on the LLVM chain on the target machines that where you install it? It's no. I mean, basically you don't have to do anything. Like you have a pure pricing package and you just put in your setup.py or into your dependencies you put number and that's all you do. And then PIP or Conda will automatically, I mean, number and LLVM. The LLVM lite automatically shipped with Conda with the base installation. But you can also PIP and Conda install them. So if you put NumBer as a dependencies, when someone installs your package they will always get it and you just kind of have a pure pricing package and you don't have to build wheels or binary distributions or do complex things. It's really easy. Thank you. All right. Any other questions? Excellent. Thanks for the talk. Yeah, that was really awesome. I just wanted to know, like, what do you think are some of the reasons that the adoption of NumBer is probably a bit slower than some may have expected? The, I mean, so I guess it took longer to reach 1.0 than they thought and projects wait for 1.0 and stability. I mean, NumBer has been amazingly stable for the past five years, I think, also, but it's still constantly improving. The, I mean, it is a bit scary if you have, if, like, for example, say NumPy itself or SciPy, they could get rid of a lot of code if they adopted NumBer and do things much more easily, like implement all of their U-funs and this kind of stuff. But then it would be all in, like, they would depend on LLV and Lite and NumBer and this would have to run and be stable for the next decade on all the supported platforms. So I know, for example, that on the NumPy mailing list, like, three years ago, there was a discussion whether to use NumBer as a dependency for NumPy and at the time people said no, mainly because, for example, ARM processor support was not there yet. And in the past years, the NumBer team has added this and is testing really on a large range of hardware and it's getting better and better and I think it will happen. I don't know how fast it will happen but I think projects will start to adopt NumBer more and more. Thank you. Cool. Let's go this side and then go back to that side. I have a question about the other case study for using C++ code. So could I use a just entire compiled function Python to call my C++ library using NumBer? Is there some support for that? I have to admit I don't know. So this is something where size and really shines. You can call any C and C++ code. I have not done this myself. I think there are ways to call into a C library. I mean, but I have to pass on that. Check the documentation, please. Thanks. Over there. I have a question. First of all, thank you for your talk. I started using NumBer a bit and discovered PyPy and I was wondering in a scenario where you have a, let's say you have multiple functions one after each other, what is the overhead if you put a NumBer at numbed.jit before each function versus compiling or using the whole code with PyPy. Is there a difference or? I'm not sure. I think it depends. I mean, for the cases where we are using NumBer, like this overhead of jit compiling the function does not matter at all. Like these milliseconds that are spent once to compile my functions, I really don't care about even if it would take a second and then my analysis runs for an hour and is really fast. I mean, if you have thousands of little functions that you need to jit compile, then, and you have very short running processes, then this might become relevant. And I don't know how PyPy compares in terms of like the jit speed compared to NumBer. So maybe what you're saying is that at the initialization of your module, it's not at that time where the compilation is happening. It's during the execution. Yeah, so you have the overhead of starting up LLVM and doing the compilation. And I mean, if you look at this, so basically this is what's here. I mean, you see this change of things that happen and some things are done by NumBer and some things are done by LLVM. But overall, this is very fast, like for a given function. But if you have thousands of functions and they're really long and complex and you use like high optimization options for LLVM, then this can take, I guess, seconds or minutes in extreme cases. I don't know. But I never was in the case where this matters. Oh, thank you. Thank you very much. We are running out of time. You can chat with the speaker during lunch and the conference. And thank you very much. Thanks, guys. Thanks. Thank you.
 Hi, thanks for coming.  So I will talk about understanding number, the Python and NumPy compiler, and I want  to be clear at the start, like I don't really understand number.  So I'm a Gamma astronomer from Heidelberg.  I'm not a number compiler or CPU expert.  I recently started using it, and I think it's awesome, and I wanted to introduce it to you.  So let me talk a little bit about how we use number and why we use it.  So in Gamma astronomy, we have these telescopes like the HES telescopes in Namibia or the  Charenkoff telescopes in Chile, and there we have to do lots of numerical computing for  data calibration, reduction analysis, and we need both interactive data and method exploration  and then production pipelines.  And also the software is often written by astronomers, not professional programmers.  And traditionally, the approach was, for example, for HES for the past decade to write everything  in C++, also using the root software from CERN, and then maybe have a little bit of  Python scripting on top.  And now it's different.  Now we're trying to build everything in Python and NumPy, and then when needed for performance,  we write a little bit of numbers, Python or COC++.  So it's a different approach.  For the CTA software, we're prototyping it using this Python-first approach, and you  see the stack, like it's based on Python and then the PyData ecosystem.  So you have NumPy, SciPy, and so on.  In astronomy, we also have AstroPy, which is the base library which contains the standard  data formats and time handling coordinates, handling, and so on for astronomers.  And then on top of this, we can implement the gamma ray analysis software.  And this approach is not really unique to what we do in our project.  It's kind of become the standard in astronomy.  As you can see in this graph here, Python is really now the most popular language to  use in astronomy.  And as Perry Greenfield points out, like one reason is that Python is a language that is  very powerful for developers, for professional developers, but it is also accessible to astronomers.  And this makes for a good mix.  And I have two other quotes from Jake VanderPlas from the Python keynote at 2017 in the same  vein saying for scientific data exploration, speed of development is primary and speed  of execution is secondary.  And also in Python, we have these libraries for nearly everything.  And Python is the glue that combines the scientific codes together.  So we have these thousands of Python packages that implement all kinds of methods.  Why do we need NumPy, actually?  I mean, some algorithms are hard to write in Python and NumPy.  Here's an example of a function computing one step in this game of life.  I don't know if you're aware of this, but you have this update rule, whether cells live  or die.  And you can write it in NumPy, but then you have to write this double NumPy.roll call.  And the code is really hard to write and hard to read and hard to maintain.  So this would be a case where just writing a for loop is simpler.  And then if you write it in Python, it's inefficient, and then writing in C and wrapping  it from Python can be tedious.  So this would be a good case where you would go to NumPy.  And there's also a quote from Stan Sybert from the NumPy team, just saying, like, don't  write such NumPy high-coos.  If Python for loops are simpler, then write loops and use NumPy.  Okay.  So let me introduce NumPy.  What is it?  I mean to find out more good to the webpage, NumPy.org.  The tagline is, NumPy makes Python code fast, and it's an open source JIT compiler that  translates a subset of Python and NumPy code into fast machine code.  And actually later in the talk, I'll try to unwrap that statement and explain what it  means.  But first, let's see a little bit.  The name, NumPy, comes from the combination of NumPy and Mumba.  So you do number crunching in Python, and it's then fast like Mumba.  So Mumbas are apparently some of the fastest snakes in the world.  This is where the name comes from.  And then how do you use it?  Let's assume you have some function doing math, like here, using this Monte Carlo method  to compute the number Pi, where you just throw random points in the unit square and then check  which ones are x squared plus y squared less than 1.  So which ones are in the circle and you count.  And that's one way to compute Pi.  If you do this in Python, it's extremely slow for a million numbers.  Because what Python does is that every number is represented by a Pi object.  And then to do the actual computation, what happens in something like x squared or less  than 1 and so on, always you take the Pi object and you figure out that it's an integer.  Look at the integer inside.  Then do the operation and the computation and then create another Pi object.  So you have all of this overhead that makes math from Python really slow.  To speed it up with number, it's very easy.  You just import number and add number to your function.  And then in this case, it's 30 times faster.  So this is how you tell number to jit your function.  Number also understands NumPi.  So in this case, I have two arrays x and y, which are like not large NumPi arrays.  And I do the same computation by passing them into a Monte Carlo Pi method and then uses  a NumPi array expression to do the same computation.  And this is faster than using Python.  But it also uses much more memory.  But again, you can add the number.jit decorator and it will run faster because what NumPi can  do is it can avoid these temporary array copies.  So when NumPi sees x squared, it creates a new NumPi array which is x squared.  And creating these new arrays and allocating them and processing them is some overhead  which can be avoided if you compile down to machine code.  And that's why NumPi in this case is faster.  If you want, you can also write the same function using a for loop.  So it's really up to you.  In this case, writing a NumPi array expression is more convenient.  But sometimes writing a for loop is more convenient.  And NumPi will compile your code either way into optimized machine code.  So kind of the evolution of a scientific program that's coming to Python is that maybe you  start out with C or Fortran and you write loops all the time to do your data processing.  Then you come to Python and you figure out that loops are 100 times slower and you learn  that you have to write everything as NumPi array expressions.  But actually now that we have NumPi writing for loops, it's okay again.  So it's a bit of a regression.  What are the limitations of number?  Number compiles individual functions.  It does not compile whole programs like PyPy.  Number only supports a subset of Python.  It has some support for dictionary list and sets.  But you cannot have mixed types of keys and values and all of the operations.  Like some things, some Python data structures just cannot be efficiently processed and translated  to machine code.  And you see an example of this on the left side where I have a list with mixed data types  of a string and an integer.  And actually, number is not able to compile this to efficient machine code.  And what you get is you get a typing error if you try this.  Also number only supports a subset of NumPi.  I mean, this is ever-growing, but not all functions and all arguments to those functions  are available.  So you have to see.  And also, I mean, number does not support any kind of Python code.  Like you cannot use pandas or scikit-learn or the request library or these kind of things  from your Python function because number will not know how to translate that into machine  code.  It's really focused on math and numerics code.  So a little bit more about these two JIT modes, the object mode and the no Python mode.  If you add the number.jit decorator and you have some function which cannot be efficiently  compiled to machine code, what will happen is that you get a number warning saying the  compilation is falling back to object mode, but it will still run and give you the results.  And what's happening under the hood is that all that happened was that number translated  this function into something that's equivalent what the Cpyzen interpreter would execute,  where you still have PyObjects and Python C API calls and you get the same performance  as if you only use Python without number at all.  So 99% of the time, this is not what you want.  And to make it more obvious when you write functions which cannot be compiled efficiently,  you can use this no Python equals true thing and then you will directly get a typing error  saying fail to compile in no Python mode.  And there's a shortcut because typing number.jit, no Python equals true is long.  You can also type number.jit.  If you actually do have a function where you need to go back and interact with the Python  interpreter and Python objects, you can.  And for this, you use the number.objectModeContextManager.  This is rarely needed, but it can be useful if you have like a long-running function and  you want to log the progress or update a Python progress bar, which is a Python object.  Then you can say with number.objectMode and inside this with statement, you can again  interact with your Python session.  Okay.  So now let's come to the part where we try to understand what number does at least a  little bit.  So the description kind of is number is a type specializing JIT compiler for Python byte  code using LLVM.  And this might make your head explode, but let's try to unwrap that a little bit.  There's really three parts involved here, what's going on when you write a function  and you JIT it and you call it.  And the first part is Python itself, and then you have number, and then you have LLVM in  the background.  And I'll explain on the next slides how these interact.  So what the Python compiler does, it starts with your source code, parses it into an abstract  syntax tree, and then transforms it to byte code.  And this happens on import of a module like when the Python interpreter sees the death  statement, it creates this function object and attaches the byte code to the function  object, which as you can see here on the left.  And then what number can do is it can start with this byte code and compile and transform  it to machine code, and that's what the number.jit decorator does.  And actually when number.jit decorator is called, it does very little.  All it does is it creates a CPU dispatcher proxy object because it cannot compile the  function yet because it doesn't know what types come in and what kind of machine code  it should generate.  So only when the function is called, it will then JIT compile the byte code to LLVMIR exactly  for those input types.  And then it will also manage the LLVM compilation and execute the compiled function for you.  So what is LLVM?  LLVM is a compiler infrastructure project.  There's many frontends for languages like C, C++, Fortran, Haskell, Rust, Julia, Swift  and so on.  There's also many backends for all kinds of hardware, so all the different CPU types.  The vendors have added support for it and optimized it well.  And you could consider number just to be the Python frontend for LLVM.  So the way this works is that LLVM is shipped as a Python package called LLVM light that  number depends on.  And this is maintained by the number team at Anaconda that then ships number and LLVM  light so that it's readily available.  Concerning alternatives, I mean, at least for us, like the most obvious one and what  people also use is Siphon and like number, Siphon is often used to speed up numeric Python  code.  Siphon is an ahead-of-time compiler where you have to type annotate your Python code  and then it compiles to C and then you use a C compiler to compile this to machine code.  It's more widely used at this point.  It's easier to debug because C code, generated C code is easier to debug than LLVM IR code,  which is lower level and looks more like a similar code and probably very few Python  people can read and debug this.  Yeah.  And also, it's very good at interfacing with C and C++ code.  I mean, number, on the other hand, is easier to use.  You don't have to add type annotations.  You don't have to need to have a C compiler, but it is harder to debug.  Another advantage of number is that it optimizes just in time for your CPU and GPU so you don't  need to build and distribute binaries for many architectures.  Instead, it will always use all of the CPU features you have, like advanced processor  instructions and so on.  Other number alternatives, I mean, there are, Siphon is great and there are many other great  tools that exist for high-performance computing in Python.  So you have Siphon, C, C++ and PyBind 11 if you want to go the Python C extension way.  There's PyPy, which is an alternative to Cpy in which JIT compiles the whole program.  And then you have all of these modern things like TensorFlow, JAX, PyTorch, and Dask and  so on that similarly to number also use Python and NumPy mainly as the language to specify  what kind of computation should happen, but then they will do some kind of compilations  and execute it in various ways.  And I don't have time and also don't have the expertise to explain all of the differences,  how JAX and TensorFlow and so on compares to number and how exactly they compile.  But you have to make a choice there.  And it's not easy because there's many great libraries now available in Python.  Okay.  So some more things about number.  One thing you should know about is number.s or number minus minus sys info from the command  line.  Or you can also use it from Ipython and Jupyter if you just put the exclamation mark in front,  you can execute shell commands so you can also get the info from there.  And as you can see here on the left, this gives you all of the relevant information about  the hardware on your computer like what CPU and GPU you have, which Python version, number  version, and LLVM version you're using, whether you have the Intel short vector math library  installed.  I'll talk more about this later, whether you have the Intel threading building blocks library  installed, which will make number faster in some cases also.  And then which GPUs you have available and GPU drivers you have installed.  One thing you can do if you want to, if you want to, if you have a multicore CPU and want  to make your computation run fastest to add the parallel equals true option to the number  to JIT call.  And this will then do multi threading using one of the backends, either open and P threading  building blocks or a custom one.  If you want to use TBB, you have to do this content and start TBB in addition.  And then as shown in this example, it will work automatically for NumPy array expressions.  There's no code changes needed.  So in this case, I got a 3.2 speed up on my four core CPU for this computation by parallelizing  it.  If you have four loops inside of your function, you should use the number.P range generator,  I guess.  And then what will happen is the same thing that number will parallelize this loop and  you will get the speed up from using multiple cores or also vector instructions.  And what I show here on the lower right is just always like when you have Python decorators,  you can use them in two ways.  You can either put them with the add sign on top of your function and the decorator  will be automatically applied.  Or you can just write your function without the decorator and then apply the decorator  after by just passing the function into the decorator.  So you can say number.JIT passing the compute function.  Or you can say number.JIT parallel equals true.  This is a decorator and you can pass in the compute function.  And this can be convenient if you want to try out different options to compile the same  function so you don't have to always copy and paste the code of the function also.  Another option you have is fast math equals true and there you can trade accuracy for  speed for some computations.  So there is this IEEE floating point standard that in this case, for example, requires that  the loop is must accumulate these numbers in order to get precisely defined results.  But if you're willing to give this up and have a little bit of different accuracy, then  the computation can go faster because the compiler can vectorize this reduction.  There's another way you can speed up math functions.  So if you have code that has square root, sign, exponential, and so on, then you can  condo install the ICCRT package and this will make these Intel short vector math libraries  functions available until number will tell elevator to use those.  And these are just like much faster implementations of these math functions.  So that's how you get fast math.  You might ask like how fast is number?  I would say number gives very good performance and there are many options to tweak the computation,  but there is no simple answer to this question and there is also no simple answer to the  question of like how number compares to Python, size, and numpy, C, Fortran, and so on.  You can find many blog posts on the Internet and they all have kind of have different outcomes  depending on what application they do and compiler flags they use and what hardware they  have, they get different speeds for different tools.  And we've already seen this for number on the previous slides.  The speed up is not always the same.  So what you have to do is if you care about performance, you start by defining a benchmark  for your application that you really care about where you have a performance bottleneck  and then you measure and then you try to improve it.  Another thing I wanted to introduce are numpy U funds or universal functions.  So these are functions like add sign and so on from numpy itself.  And these all support array broadcasting which you can see here on the left side.  So you can adjust multiply to numbers.  You can also multiply a number with an array and it will broadcast these inputs and generate  an output array and there's a more complex example here.  And then they also have these special methods attached like accumulate which apply to one  element at a time and accumulate it giving a different output array.  And so far if you wanted to make one of these U funds, you had to write C code and use the  numpy C API which was pretty hard.  And now with number it's really easy.  You use the number vectorize decorator and it will make the U fun for you.  So the way you write your function is you don't put a for loop.  You just write the operation for one element and then kind of the implicit for loop to  loop over arrays and do broadcasting and so on.  This will be generated by number for you.  And there's two ways to do it.  You can give the type signature in the vectorize call.  Like I do here I say please make a U fun.  Assuming the inputs are integer numbers and then number will generate one U fun on the  vectorize call.  If you don't give a signature then again such a dispatcher object is created and then dynamically  when you call the U fun, when you call the function number will generate U funds for  the input types you passed.  So it will generate a different one if you put a float, pass a float input array or an  integer input array.  Okay.  So I'm almost done already.  So the, I mean, number is really a family of compilers.  I've talked about two, number.jit for regular functions and number.vectorize for U funds  and just given a quick introduction to those if you want to learn more then check out the  number documentation.  There is also the geo vectorize decorator which can make generalized U funds.  There is the stencil for neighborhood computation.  So if you want to do some kind of convolution or sliding window computation it's easy with  the stencil decorator.  There is C fun which can generate functions with well-defined C callback ABI.  So this could be useful for example if you want to call a number function that you write  in Python from C or C++ code.  So if, like all of your application runs in C++ but then you want to extend it with Python  then this would be one way to do it.  And then there are these CudaJit and RockJit decorators to work with GPUs also.  So just as a last point, I mean who uses number?  So Jake Wanderplatz again like wrote a few blog posts in 2013 like when the project was  still very young saying and becoming more and more convinced that number is the future  of fast scientific computing in Python.  This has not really happened so far.  And then in 2018 Matthew Rocklin wrote an article advocating for the numeric Python community  should consider adopting number more.  So I think currently many people and applications use number for their work and projects like  we do for gamma ray astronomy and many others do as well.  But then the large libraries like NumPy, SciPy, Pandas and Scikit-learn have not adopted number  yet.  And there are some nice examples now of libraries or packages using number.  For example, DataShader which is shown on the top left which is a large data visualization  library is implementing their stuff in number.  There's Librosa which is for digital signal processing and audio and music analysis is  implemented using number.  And Intel has written HPEG which is the performance toolkit and this can do big data processing  and supports Pandas.  So they've done something very similar to what NumBer did.  They've defined a decorator, HPEG.jit which can take a function which does IO and then  NumPy or Pandas computations and so on and it will parallelize this also to clusters  using MPI.  So my summary and conclusions are that I mean NumBer is a type specializing JIT compiler  for Python bytecode to LLVNIR.  The project started in 2012.  The current version is 0.44 and I think they're well underway to version 1.0.  It will use your CPU and GPU well and it's really easy to use.  You just have to add this Python decorator.  So use NumBer.jit for normal functions, NumBer.vectorize to make you funs and use NumBer minus s to check  your machine and installation.  If you don't find you get good performance or it's not working and then if you want to  use multi-core CPU and get fast results, consider putting parallel equals true and fast mass  equals true and also installing this SVML package.  And I mean, yeah, I'm not a NumBer developer.  I didn't do anything to make this work.  I just wanted to thank the NumBer developers at Anaconda and also the other people and  companies that have contributed like for example Intel.  Very cool, thank you for the great talk.  We have five minutes for questions.  Again, we have microphones over there.  Over there if you have a question, just line up behind the microphone and ask away.  Hi.  Thank you for the talk.  How would you recommend packaging your package that uses NumBer?  Would it depend on the LLVM chain on the target machines that where you install it?  It's no.  I mean, basically you don't have to do anything.  Like you have a pure pricing package and you just put in your setup.py or into your dependencies  you put number and that's all you do.  And then PIP or Conda will automatically, I mean, number and LLVM.  The LLVM lite automatically shipped with Conda with the base installation.  But you can also PIP and Conda install them.  So if you put NumBer as a dependencies, when someone installs your package they will always  get it and you just kind of have a pure pricing package and you don't have to build wheels  or binary distributions or do complex things.  It's really easy.  Thank you.  All right.  Any other questions?  Excellent.  Thanks for the talk.  Yeah, that was really awesome.  I just wanted to know, like, what do you think are some of the reasons that the adoption  of NumBer is probably a bit slower than some may have expected?  The, I mean, so I guess it took longer to reach 1.0 than they thought and projects wait  for 1.0 and stability.  I mean, NumBer has been amazingly stable for the past five years, I think, also, but it's  still constantly improving.  The, I mean, it is a bit scary if you have, if, like, for example, say NumPy itself or  SciPy, they could get rid of a lot of code if they adopted NumBer and do things much  more easily, like implement all of their U-funs and this kind of stuff.  But then it would be all in, like, they would depend on LLV and Lite and NumBer and this  would have to run and be stable for the next decade on all the supported platforms.  So I know, for example, that on the NumPy mailing list, like, three years ago, there  was a discussion whether to use NumBer as a dependency for NumPy and at the time people  said no, mainly because, for example, ARM processor support was not there yet.  And in the past years, the NumBer team has added this and is testing really on a large  range of hardware and it's getting better and better and I think it will happen.  I don't know how fast it will happen but I think projects will start to adopt NumBer  more and more.  Thank you.  Cool.  Let's go this side and then go back to that side.  I have a question about the other case study for using C++ code.  So could I use a just entire compiled function Python to call my C++ library using NumBer?  Is there some support for that?  I have to admit I don't know.  So this is something where size and really shines.  You can call any C and C++ code.  I have not done this myself.  I think there are ways to call into a C library.  I mean, but I have to pass on that.  Check the documentation, please.  Thanks.  Over there.  I have a question.  First of all, thank you for your talk.  I started using NumBer a bit and discovered PyPy and I was wondering in a scenario where  you have a, let's say you have multiple functions one after each other, what is the overhead  if you put a NumBer at numbed.jit before each function versus compiling or using the whole  code with PyPy.  Is there a difference or?  I'm not sure.  I think it depends.  I mean, for the cases where we are using NumBer, like this overhead of jit compiling the function  does not matter at all.  Like these milliseconds that are spent once to compile my functions, I really don't care  about even if it would take a second and then my analysis runs for an hour and is really  fast.  I mean, if you have thousands of little functions that you need to jit compile, then, and you  have very short running processes, then this might become relevant.  And I don't know how PyPy compares in terms of like the jit speed compared to NumBer.  So maybe what you're saying is that at the initialization of your module, it's not at  that time where the compilation is happening.  It's during the execution.  Yeah, so you have the overhead of starting up LLVM and doing the compilation.  And I mean, if you look at this, so basically this is what's here.  I mean, you see this change of things that happen and some things are done by NumBer and  some things are done by LLVM.  But overall, this is very fast, like for a given function.  But if you have thousands of functions and they're really long and complex and you use  like high optimization options for LLVM, then this can take, I guess, seconds or minutes  in extreme cases.  I don't know.  But I never was in the case where this matters.  Oh, thank you.  Thank you very much.  We are running out of time.  You can chat with the speaker during lunch and the conference.  And thank you very much.  Thanks, guys.  Thanks.  Thank you.
Christoph_Deil_-_Understanding_Numba_-_the_Python_and_Numpy_compi_0_311.png Christoph_Deil_-_Understanding_Numba_-_the_Python_and_Numpy_compi_312_443.png Christoph_Deil_-_Understanding_Numba_-_the_Python_and_Numpy_compi_444_683.png Christoph_Deil_-_Understanding_Numba_-_the_Python_and_Numpy_compi_684_731.png Christoph_Deil_-_Understanding_Numba_-_the_Python_and_Numpy_compi_732_1307.png Christoph_Deil_-_Understanding_Numba_-_the_Python_and_Numpy_compi_1308_1955.png Christoph_Deil_-_Understanding_Numba_-_the_Python_and_Numpy_compi_1956_2651.png Christoph_Deil_-_Understanding_Numba_-_the_Python_and_Numpy_compi_2652_3299.png Christoph_Deil_-_Understanding_Numba_-_the_Python_and_Numpy_compi_3300_5387.png Christoph_Deil_-_Understanding_Numba_-_the_Python_and_Numpy_compi_5388_6395.png Christoph_Deil_-_Understanding_Numba_-_the_Python_and_Numpy_compi_6396_10331.png Christoph_Deil_-_Understanding_Numba_-_the_Python_and_Numpy_compi_10332_12227.png Christoph_Deil_-_Understanding_Numba_-_the_Python_and_Numpy_compi_12228_13787.png Christoph_Deil_-_Understanding_Numba_-_the_Python_and_Numpy_compi_13788_14711.png Christoph_Deil_-_Understanding_Numba_-_the_Python_and_Numpy_compi_14712_17099.png Christoph_Deil_-_Understanding_Numba_-_the_Python_and_Numpy_compi_17100_21551.png Christoph_Deil_-_Understanding_Numba_-_the_Python_and_Numpy_compi_21552_21647.png Christoph_Deil_-_Understanding_Numba_-_the_Python_and_Numpy_compi_21648_23807.png Christoph_Deil_-_Understanding_Numba_-_the_Python_and_Numpy_compi_23808_25535.png Christoph_Deil_-_Understanding_Numba_-_the_Python_and_Numpy_compi_25536_27011.png Christoph_Deil_-_Understanding_Numba_-_the_Python_and_Numpy_compi_27012_28247.png Christoph_Deil_-_Understanding_Numba_-_the_Python_and_Numpy_compi_28248_29399.png Christoph_Deil_-_Understanding_Numba_-_the_Python_and_Numpy_compi_29400_30911.png Christoph_Deil_-_Understanding_Numba_-_the_Python_and_Numpy_compi_30912_32555.png Christoph_Deil_-_Understanding_Numba_-_the_Python_and_Numpy_compi_32556_33347.png Christoph_Deil_-_Understanding_Numba_-_the_Python_and_Numpy_compi_33348_35075.png Christoph_Deil_-_Understanding_Numba_-_the_Python_and_Numpy_compi_35076_36707.png Christoph_Deil_-_Understanding_Numba_-_the_Python_and_Numpy_compi_36708_36959.png Christoph_Deil_-_Understanding_Numba_-_the_Python_and_Numpy_compi_36960_37031.png Christoph_Deil_-_Understanding_Numba_-_the_Python_and_Numpy_compi_37032_37115.png Christoph_Deil_-_Understanding_Numba_-_the_Python_and_Numpy_compi_37116_37175.png Christoph_Deil_-_Understanding_Numba_-_the_Python_and_Numpy_compi_37176_37223.png Christoph_Deil_-_Understanding_Numba_-_the_Python_and_Numpy_compi_37224_37271.png Christoph_Deil_-_Understanding_Numba_-_the_Python_and_Numpy_compi_37272_37319.png Christoph_Deil_-_Understanding_Numba_-_the_Python_and_Numpy_compi_37320_37811.png Christoph_Deil_-_Understanding_Numba_-_the_Python_and_Numpy_compi_37812_42947.png Christoph_Deil_-_Understanding_Numba_-_the_Python_and_Numpy_compi_42948_44747.png Christoph_Deil_-_Understanding_Numba_-_the_Python_and_Numpy_compi_44748_45683.png Christoph_Deil_-_Understanding_Numba_-_the_Python_and_Numpy_compi_45684_45714.png