Cython
CC Attribution 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Behnel, Stefan
You've probably heard of Cython as a tool for fast data processing. Well, it's more than that. Over the years, the Cython compiler has become a major pillar in the Python ecosystem, helping Python users all over the world to crunch their data, write fast and portable native code, and integrate native libraries into their Python workflow. Several excellent projects from the scientific Python world chose Cython to allow "normal people" to write and maintain efficient code for complex scientific algorithms, whenever Python itself comes to its limits. In this tutorial, I will present the Cython compiler and quickly show how to speed up code with it, how to talk to native code, and a couple of more things that you might want to try out with it.
 Okay, so let's get started. It's 1400, so everyone who's late is late. So welcome to the tutorial. This is the Scythe tutorial. I'm Stefan Bienel, one of the core developers of Scythe, so this is kind of a, you can actually ask me anything session. I'm also a Scythe trainer, so I give trainings, especially at the Python Academy in Leipzig. You can find more about that at the Python Academy or at my webpage there. I hope you've all downloaded the notebook that I'm going to present here. Anyone not prepared? Who dares? That's fine. Okay. Okay. So okay. First of all, what is Scythe? Scythe is, well, it's Python compiler, basically. First of all, it translates Python code to C code. Okay, so you can write native code in Python syntax. The cool twist about it is that because it's translated into C or C++, it allows you to directly interact with C code, with native code, C++ code, C code, with libraries. It allows you to write C and C++ code without actually writing C code. So that's the cool thing about it. It's an actual programming language, and many people use it to wrap libraries for Python or to speed up their code, into native code. But some people also just use it, you know, because it's a program language. They use it as a program language, like they would use Python. But as a kind of a cool programming language that makes many things easier than in other native languages. Okay. So I'm going to use Jupyter Notebook for the tutorial. And the first thing you would do in the notebook in order to enable Scythe support, because Scythe is actually one of languages that Jupyter supports. You would say load Scythe, and if you have Scythe installed, then this will enable Scythe compiled cells. So I'll do that. It tells me I should restart my kernel. Okay. So I'll just try again. It's not going to tell me anything, and it's imported Scythe. Okay. Cool. The next step in there is just a quick, you know, summary of what dependencies you have installed. So I'm using the latest Scythe master. Actually, you probably don't have that installed. So it's perfectly fine if you use Scythe 026 or even 25. That will be okay for this tutorial. I'm using Python 306. Is anyone using Python 2? Okay. You're excused. So that's kind of fine, too. Scythe supports Python 2.6 and later up to 307. So we continuously test against the latest development versions of C Python in order to stay compatible. And that means whatever code you write in Scythe will be adapted by us to newer Python versions for you. So you don't have to do much there. The kind of recent NumPy version. One thing to know about Scythe is it compiles code to C. So you need a C compiler or C++ compiler installed in order to make any actual use of it. Does anyone not have a C compiler installed? Okay. I'm first up there. Everyone else has it. That's fine. So you will need this for this tutorial. Because the way the compilation works is Python takes your Python or Scythe code, translates it into C, and then the C compiler translates that into a shared library into an extension module that you can then import and use in Python. Okay. That's the way it works. So you have a three-step process. The nice thing about the Jupyter notebook is that it removes most of that so you can just write Scythe code in the cell, run it, and it will be compiled in the back and you won't see anything about it and it will just get executed and imported. It's all there. You'll see that. Really nice. Okay. Regarding questions in this tutorial, you can ask questions at any time. Please just interrupt me. When there's anything you want to know, I'll try to answer anything. Well, not anything. You know what I mean. Okay. Here's a very simple example. This is just normal Python code. I'm using the Python math module, specifically the sine function from it. Just calculated in sine of five. It gives you minus 0.9. So that's simple. You would probably know that. And I can do the same thing in Scythe because Scythe can translate and compile arbitrary Python code. So I'll just do that there. And the first thing you notice, you don't get any output. Why is that? What Scythe does in the back is it compiles your module and the module gets imported as an extension module and actually gets executed. But the Jupyter notebook does not see what the kind of the last line in your code was. Okay. So you can't just say I got this result not presented to you. Instead the code in there gets executed as a module. Okay. And there's no standard output of modules. So what I can do instead is I can say print the result here and it's going to be printed for me. Okay. And that's the difference. So this is fairly simple. This is just still plain Python code. What I can do in Scythe now is instead of using the Python math module, I can actually use the math support in the C library. So libc math header. Scythe comes with all declarations that you need for that. So instead of having to declare anything anywhere, I can just say C import. So that's an extension to what you would see in Python. It's compile time import. So there's a couple of extensions to the Python language that you would use in the Python language. So one of them is C import. It gives you a static compile time import. And I'm using the libc math module here. And then I'm taking the sign function of that. And that's the actual low-level C sign function. Okay. And what I'm doing here is I'm taking that function and I'm assigning it to a variable called sign function. And this is a global module variable, meaning it's a Python variable. Okay. So what I end up with is a module that defines a name, sign function, which represents the C sign function and exports it to Python. Okay. So I can just take that and call it from my notebook. And here I'm using the C sign function directly from Python. Okay. This is kind of the quickest way to export a simple function, I would say, simple C functions into Python. Okay. What this does more or less is it defines a Python function, which internally calls the C sign function. And it takes a double and returns a C double, okay. And it returns a C double result. And that's hidden behind this assignment here. And here I've spelled it out. This actually is Python syntax. So Python allows me to define Python functions, but declare the signature in a C way. Okay. When I execute this, it defines my C sign function and I can run that from Python. Okay. So what you can see here is that this is actually a very basic property of the Python language. It mixes Python and C freely. So most of the code that you will see, most of the Python code that you will ever see, do some Python features here, use some Python objects here, then push something into C, do stuff there, call C function, get a return, and literally mix Python and C all over the place. And that makes Python such a cool language for many things because you get the simplicity of Python on one side and on the other side, you got all the native power of C, okay, in one program language. Okay. One nice debugging feature of Python is, well, here you see I've declared a Python cell in Jupyter. What I'm doing here is I'm passing the minus A parameter to that cell. And in addition to compiling my code now, what it will do is it will analyze and annotate, that's what the A is for, my code and then output it as an HTML fragment down here that tells me how the compiler saw my code and what it made of it. So for example, when I click on this line with a plus here of the signature definition, then you'll see there was a lot of code generated from Scython just to implement this Python function for me. And then down here, I can actually prove that it calls the C sign function because the C code says I take the variable X, call the C sign function on it directly, then take the result and turn it into a Python float. So that was what this function is doing, PyFloat from C double. And then return the result as a normal Python float object. And Scython generates these things automatically for you because it knows the result of the sign function here is declared as a C double, but it is used in a Python context, it's returned from Python function, so it must be an object and it automatically converts between the two value types for you behind the scenes. Okay. Now, this is very basic and as I said, this can be done just by assigning the C function to a Python variable. When it becomes a bit more interesting when you do more stuff on the C side, so what I'm doing here is I'm getting a double value in again, but then I'm taking the square of that and I'm taking the sign of the square. So this is implementing sign of square of the input and it's doing all that in C space again. Okay, so when I compile this and I actually compile this Python minus A, again, then you see that this line down here is really just plain C multiplication and then we take the result and call the sign function on it. Okay. And this is something you also often see in many wrapper generators that you find for Python, say, although there are many of them. In many of those, what you get is you get a plain wrapper around some C function, some C functionality. Okay. And then you have to deal with whatever that gives you in Python. And Python allows you to make that wrapper thin if you like, but thicker if you need it. Okay. And that's completely seamless. So whenever you need more functionality or you want to aggregate some C functionality behind some Python function call, for example, or hide it behind a Python class, you can easily do that by just implementing the thing in Python and then calling whatever C functionality you need straight from your code. Okay. So if you find that. Yeah, right. I didn't declare C def. Okay. So C def is another extension you've seen C import. C def is the other basic thing that you'll see all over the place in Python code. It just says this is a C definition. Okay. And what this does here is it declares the variable X square as having the C type double. Okay. And what this does, okay. In fact, I wouldn't even need that because Scythian is smart enough to do type inference here. I have two double values here to multiplying them. Obviously results in a C double and I wouldn't need to declare the result type of the variable, but I did. Yeah. Yeah, it should be. So there's a bit of type inference available in Scythian which helps to avoid having to declare too many things. Okay. So that was really a tiny quick intro. Let's come to a bigger example. So I owe this example to a guy called Caleb Hetting who presented a Scythian tutorial at PyCon Australia a couple of years ago. And a very nice example. Why? Because everyone likes Texas, right? So I borrowed that idea from him and what I did was I looked up the actual tax calculation function that we have here in Germany. And I found that they're currently also for the last year, there were something like 44 million taxpayers, so earnest, and the average income of them was €3703 per month, so times 12, that's about what you get as an average income per year. So I just run this and that's the average income. And then I started looking a bit for actual data that backed this number and didn't really find anything. So for the sake of this tutorial, I just create some alternative facts here and I'm sure all of you are happy with that. It's the right conference to do that. So I just assume that first of all, there are only 20 of the number of people, just to keep the calculations a bit faster, and a lot normal distribution for the income. So you can see that the average is about right and there's an alternative minimum income and maximum income here. And I plot this, you can see that, well, that's about what you would expect more or less from the income distribution. Well, more or less. So let's calculate everyone's taxes. I looked up the tax formula in Wikipedia and this is what it gave me. It actually gave me an Excel implementation of that formula. So I took that formulation and rephrased it in Python and this is essentially it. So if your income is above that number, then it's calculated like this. Otherwise, you know, it's a staged calculation. And that's also what makes it kind of interesting because this is difficult to express in simple terms. The best way to express this is actually with ifs. So this is how you would express it in Python and then you can calculate the average income down here and the average tax rate across the data that we have. You'll probably get slightly different results because you know, it's an alternative fact on your side too. But this is about the average income and the calculated tax, the average tax and then I'll just do the whole thing in Python and you can see that the average tax rate is about 24% for my data set. So when I asked time it how long that took, it's going to tell me some number after a while, a couple seconds or maybe a couple more seconds or some more seconds. Well you see, it actually takes quite a while in Python. And time it runs a couple of times in order to get accurate results and then it tells me that it's about a bit more than three seconds. So let's say that's 1,000,000 in milliseconds. I'll just remember that value and then in order to make things comparable while I'm on the way of optimizing stuff, I'm calculating the ratios along the way and I'm saying that Python is my base level and that's the slowest version I've probably come up with. Next thing I did was I found a way to express the whole thing in NumPy and this is what I got there. Basically it's selecting parts of the array that match these conditions and then doing the calculation for the income ranges based on that. So I can calculate the whole thing on NumPy segments and then sum them up separately and calculate the average from that and you can see that it gives me about the same average tax rate, 24%. And when I ask how long that took, it's going to be much faster. So that's 57 milliseconds and now I notice that I forgot one thing. Whenever you benchmark, one thing you should never forget is switch off the energy management. So I will rerun the Python version again and it's going to take a couple of seconds. So I'll get a proper result there now. Which probably shouldn't be very much different because it's running for so long so the processor will scale up during that time and I'm probably getting good results from that anyway. But the faster it runs, the less likely it becomes that my CPU is going to be at full speed because it's going to take a while until the power management notices that my CPU is actually being used and heavily used and only then it's going to scale up. And so I'm getting good results. But this is about what I had before. So I'll just keep the old value there. And then I'll rerun the NumPy version again. And then it's still about the same thing. So that is 58.1 milliseconds. And you can see that's already 54 times faster than the Python version. Okay. That is pretty good. Okay. The same thing can be done with a NumPy Ufunk by simply taking my Python function and applying it to all elements in the array. So I'll do that here. That's from PyFunk function in NumPy, to which I can pass my function in order to convert it into Ufunk. And then when I apply that to the whole array and sum up the results, I get the same text rate. And time it on that is going to be much slower than the slicing version, but still much faster than the pure Python version. Okay. So I'll remember that that took 818 milliseconds. And so now the NumPy version is way faster than that. And the Python version is only about four times slower. Okay. Let's get back to Python then. Up to now, this is pretty much what you would be able to achieve from Python as well, given the tools that we just had, so using plain NumPy. Now what Python allows you to do is you can take the Python version of your code and compile it into C. This is usually not going to give you a big effect simply because it's still going to use Python objects. It's still going to call Python functions along the way. So there's not much out of the boxes that Python can do for you. So when I just take the Python code that I implemented above, copy it straight in there. It's exactly the same thing. And now I use a Python cell instead of a plain Python cell. So I get the same thing. Compiled with Python. And I can now call that. On the list of incomes should get the same results, 24%. And when I call time it on that, it's going to take a while. And then we can see how it compares to the plain uncompiled Python version. And that's 275.0 milliseconds. And still about 15% faster than the plain Python version just by compiling it. Just by adding one line at the top of the Jupiter cell. Which is okay. I didn't really do anything. I didn't care. I just dropped code in and it just got 15% faster. If that's for free, that's fine. The cool thing about Python now is that I can start optimizing my code at a much deeper level at a native level rather than at a Python level. I have many things that I can do at a C level that can make it run much faster than what I can achieve in Python. And the first thing I would do is I add static typing. So you've already seen the Cdev declarations and the signature changes that I made. I can now start declaring variables as having C types instead of Python object types. So here's the same code copy again. And now what I'll do is I'll start changing the code and annotating it with static types with a goal of making it faster. So the first thing I'll obviously do is I know that the income is probably double. So that can be safely represented as a C double. And I know down here, well, this is a list. It's working on a list and it's taking the sum. And what I see here is down here, it's calling this function a lot of times in a generate expression. So it's running over the incomes in order to calculate the taxes. It's running over the incomes in order to calculate the sum. And when I see something like this, the first thing I would say is this can be done much faster by running over the list once and doing the calculation one step. So I'll change this into a loop. I'll just spell out the for loop that is hidden in this expression down here. And I'll say for income in incomes, my tax is... So you can follow this, but the actual implementation of this is right below. So you have the whole solution down there. So you can follow this, what I'm doing here, in order to see your own results. And if you're only interested in the results, you can just execute the cell. Like, call off cells below. Just so you know. Okay. So I'll spell out the expression here as a single loop. And that means my tax sum is going to be the sum of incomes, or the sum of calling the function on my income. So for each of the income values, I calculate the tax, sum up the taxes, and the total is the total of incomes. And then down here, I will say the final result is the tax sum by the total. Right? That's pretty much it. In order to make that fast, what I'll do is I'll declare all variables in here as having proper double values so that the whole calculation can actually be done in C-space. So all variables that I'm using here, the tax sum, the total, and the income itself, are all going to be C-variables. Okay. Which ones do you mean? These up here? Depends on the context. So what I did here is I already declared, so inside of my function, I already declared the income argument as a C double. And that means that all comparisons down here will be done in C-space using C-doubles, and Scython will infer the C double type for this variable, for the constant also, because that's what it's comparing. So the way the coercion then works is C-style. So if you compare an integer with a double, then the results, the values will be compared as doubles. Okay. If income was just a plain Python variable, then this would be a Python literal, and it would use object comparison. Okay. Okay. So what have I done so far? I'll drop the last line now. And what you can see now is all variables are doubles. The function is still called with summing up the values that we get back from the function call with summing up the totals, and we do the total calculation in the end. So just compile this as it is, and see if the output is still the same. It's still 24%. That looks right. And when I call time it on that, it's going to run for a while. And it should already be a bit faster, quite a bit faster. So we're down to 139 milliseconds now. And that compared to the plain compiled version is a factor of 20 faster. Okay. So what was it doing if those weren't initialized? I suppose I was lucky, and they were initialized to zero, but that means I was just lucky. Yeah, what the C compiler does internally. Yeah, so I should probably initialize them, and it should make a difference for me in this case. Just run it again. I mean, I get the same result, right? So it was still correct. Okay, there's one more thing I can do. When I run this through size minus a, and let's size and tell me what it did, I'm seeing a couple of things here, and you've already noticed the yellow lines, I guess. There are white lines, yellow lines, all this. And the level of yellowness in this output tells me how much Python interaction there is going on. And the white line tells me plain C code here, and everything that's kind of yellow to dark yellow means there's some kind of Python runtime interaction, some object operations going on. So apparently I'm not at the point yet where I've turned my code into native code, because there are still tons of yellow lines in there, and even just the calculations here seem yellow. Why is that? When I click on them, I can see that the actual calculation based on the income variable is done in C. So income times value minus something, but then the result is converted into a Python float. So what I forgot to do is this function is still Python function. It's still called as a Python function. It's using Python semantics for being called, so it's using argument tuples for input, and it's using object for output. And that is inefficient, since I'm only using this function internally inside of my program. I'm not really exporting it to anything. I don't care who else is going to use it. I just want this function to be called as fast as possible inside of my program. And for that, I can turn the whole function into a C function by changing the declaration. Now it's a Python function, and I can turn it into a C function, which makes it a C function. And now it's a static C function inside of my module. In case people were still using it from the outside, I can use a third type of function that's a CP function, which turns it into a C function with a Python wrapper. So it's still usable from Python. If it's just a C function, then it's only usable as a C function inside of my module. So here, I'm actually going to use a CP function. It doesn't make a big difference, but it still keeps the function visible from Python code. And now since it's a C function, I can declare the return type. I don't have to if I don't declare it, then still object. But since I know that the return type is actually a double, a C double, I can just spell it out and say this function is a C function and it gets a C double as input and returns a C double. Okay. If this is a CP depth, so it's Python that, does that mean like when you call it from Python, it will still return a C double, but then Python will convert that to a... Yes. So it's a C double. When is the CP depth, what does the Python wrapper actually do? And what it does is... Well, it's a complete wrapper, so it unwraps the input. It gets objects as input, or one object in this case, and packs it into a C double so you can pass in a Python float or Python integer. They'll just be converted into a C int. That's sorry, C double in this case. Then the calculation will be done and on the output. The result of that is a C double and since we are calling it from Python, the wrapper will convert the C double return value back into a Python float object. Okay. And if that function is called from other Python code, it doesn't commit... So if it's called from Python code, then that's exactly what the CP depth is for. Calls from inside of Python are fast. They use the C interface. They use the C function. Call it directly. Do not go through the wrapper. And the advantage of that is they actually see the signature. So if you pass a C double into that function, into a CP depth function, then the caller will see that the call function actually expects a C double. There's no conversion needed. It'll just pass it through at the C level as fast as it can. The C compiler will see that and maybe even inline the call or do whatever optimization it can. And the output is a C double. No conversion needed either. You just get a C double back all done. That's it. Okay. That's... That way you get the fastest possible call inside of Python without losing the ability to use the function from Python, from the outside Python world. Okay. So I'll compile that. And actually... And now you can see that the whole function dropped into C. So all the return values are now plain C double returns. And even down here, that was previously a bit yellow because the return value was a Python object. And it had to be converted back into a C double in order to sum it up. And now you can see that the function call is a C call that the... There's no input conversion going on, no output conversion. Any Python usage that's still left in this function is the iteration over the incomes list. And that's it. Okay. Okay. I've compiled this. It still gives me the same result. And I call time it on it. Actually we had 139 milliseconds. Now we're down to 30 milliseconds for the whole thing. Okay. And so the ratio changed to a factor of 90 compared to the previously compiled function, annotated function, and to a factor of 105 compared to the plain Python implementation. Okay. Any questions here? Well, the reason why the for loop is in yellow is that we're iterating over a Python list. So the input is a Python list and the iteration needs to step from value to value through the list. But this is already faster in Python than it is in Python. Because Python understands what a Python list is and it can unpack it at the C level and just iterate over the underlying C array of objects. Okay. So the conversion there, the iteration and the conversion there is very fast. So why is the last line still yellow? I'm going to go back up. This is a Python function. So I'm calling it from Python using it from my Jupyter notebook as a Python function. And so the return value here, you can see that the text sum divided by the total is been evaluated in C space. Another thing you can see from the C code is that the total is first checked for zero and you get a zero division error if the total happens to be zero. So if I pass in an empty list into this function, I will get an exception and not a crash in C. Wonderful. So there's a bit of exception handling going on here, which is very fast because this is just a C value comparison and the final conversion of the result for returning the result is also done in, is using a Python operation for creating a float object. And that's it. The whole calculation up to that point is happening in C now. The question was, can I declare a list of doubles for the incomes? No, I can't. So there's currently no way to express. So what I could say is this here is a list. I could even say this is a list and it's definitely not none. This is also something I can express in Syson syntax. What this does is it does a type check on the way in and if the input value is none, then it will raise, I think, a type error and if it's not a list, you get a value error. Something like that. But it will check the input and make sure it's definitely a Python list and nothing else. It has a tiny advantage because now Syson knows that it is a list and not maybe a tuple or maybe some iterable. Previously, the code supported any iterable. Now it only supports Python lists. So that will cut down the generated code a little because the loop code will be simpler, but it's probably not going to make a big difference. I can try it. So I just compile this and rerun the benchmark, see what kind of difference it makes. It's really tiny. It's a couple of percent. That's because the looping, the for loop in Syson is so fast and has special handling for tuples for lists. When you pass a list and it hits the fast path, this is a list. Then you end up with very fast code despite not knowing at compile time that it was a list. Because Syson is going to assume that most of the time what you're iterating over is probably a list. So we're optimizing for that. The cool thing if I leave this out, if I don't declare anything here for the incomes, it's still going to be about as fast, not a big difference, but it's going to support any input, any iterable input. Not just exactly Python lists. If I say does a list, then it would not even accept subtypes of lists. However unlikely they are, but it would reject anything that's not a list. Any more questions? We'll come into that in a minute. Let's write the next thing. So we've already achieved a factor of 105 faster code by now. And it's already faster than the NumPy version by a factor of two. So this is really nice and I think that's pretty much what I had implemented on here. Yeah, it looks exactly the same. So now question to you. We could now do an exercise here. I could let you do kind of the same thing that I did on another function. Optimize that. Do you want to do it? Or do you want me to continue my show and tell presentation? Does anyone want to do the exercise? The majority, sorry. Okay. So I'll just keep going. You can still do the same thing and we can talk about it during the conference. So I'll leave this to you and next topic then is memory use. And that's the thing you wanted to hear about. So if I'm actually using, yeah. Question is this the fastest version already? Well since my tutorial is not finished yet, probably not. So this is probably the fastest way to do it if your input is a Python list of numbers. Okay. Okay. Question was how does it compare to a fully native C implementation? You've seen all white lines. So the whole operation, the whole calculation was actually done in C space. So you get exactly the same performance in C as you get for this Python code. It's just plain C in the end. You can try it. You can write a C version and then wrap that in Python and call it. So Python is optimizing C compiler flags. Python actually does not configure your C compiler for you. What it does is it generates a C code. And then in this case, it's the Jupyter integration that calls the C compiler in the back. And that uses your C flags, LD flags, whatever you have defined in your environment. So it's your responsibility to configure your C compiler properly. And if you know better C flags, then the current user is just using them. Okay. So memory use. So the question, like two questions ago was what if the whole thing is stored in a NumPy array and not in a Python list? This is actually a very common case. How often does it happen to you that you have actual data in a Python list rather than like in a Pandas data frame or in a NumPy array somewhere? Probably quite likely that you'll hit the case that all these income values are already in NumPy array. And Python has special support for that. So there is a protocol in Python that was added in Python 2.6 and Python 3. Back in the days of, that was like 2007, 2008, around that. So it's already about 10 years old now. And that's the buffer protocol. And that is something that is supported natively by NumPy and some other libraries out there. And basically whenever you're pushing around big native data blobs in some way, it's probably going to support the buffer interface in one way or another. And so that's why Scythe has native support for buffers. So what it allows you to do is you can take a buffer, you can receive a buffer as input, unpack it into the C memory and just do some operation on the memory natively rather than going through some Python interface there. So it's one shot, unpack operation, and from that point on you have native C memory. This is with the interface here. What we previously had was we just had an object coming in, a Python list in this case. And what we're going to change now is we're not going to use the Python list of data. We're going to use the NumPy array of data that we have. And we pass it in and declare it to Scythe, to the Scythe compiled function as a memory view on that data. And that's a special syntax for that, which says it's a one-dimensional array of doubles. That's it. That's how you declare a memory view. And what Scythe knows from that is there's some object coming in which supports the buffer interface. It knows that the items in that object, in that memory, are C doubles, have the format of the C double. It's one-dimensional, and if whatever it gets in is not one-dimensional, it's going to reject it. It's going to say, fold your error, wrong input. But if it's one-dimensional, it's going to be happy about it and unpack it into a C buffer. And then you can operate on that. In this case, it's actually very simple. Since it's C space now, I'm no longer going to use the Python for loop. What I'll do instead is I'm going to use an integer loop for I in range len incomes. Is this actually going to work in Python 0.26? That's a good question. It might not. It does not, so in 0.27, this is going to work. In 0.26, I think you still have to say income shape 0, as you would know from number, probably, right, in size of the first dimension. So integer loop over the length of the array that we got in. And then in here, I'll just say we sum up incomes at I and run our function on the value at that point. And that's about it, I think. So I'll try to compile it. And looks next, Python is happy with that. Yep, still the same result. Now passing in a number array, so I changed the input from incomes, that's the list, to incomes number, which is the number array, gives me 24%, the expected value. Now when I run this, we have 11 milliseconds. So we'll use 11.1. So that is much faster. And that's already a factor of five compared to the NumPy version. Another factor of 2.7 compared to the last version we had, and a factor of 280 compared to the Python version. Okay. Is this clear to everyone? We got a memory use. The syntax is basically modeled after what you would have in NumPy, so in order to get a single dimensional slice, you would just put a code on in. And that's pretty much what would be used as a syntax here. Yep. Sorry again. Yeah. So question was, when I had the Python for loop for x in the incomes array now, why did that still work? Well, it still worked because NumPy arrays actually support iteration. I mean, we could actually optimize the same thing in Python as well. We now know that it's a memory view, and we can say you can actually iterate over a one-dimensional memory view. Why not? We can do that efficiently for you. So yes. I mean, that would be a nice feature. I'll note that down somewhere. Yeah. But, I mean, this is, it's kind of pretty enough. Okay. And it's about how much, three times faster? What did we have before? Yeah, 2.7 times. Okay. So using memory views here and using NumPy arrays as inputs made a big difference again. Okay. Now I'll show you one more thing here. And I compile this with size minus a again. There's one little thing down here. These two lines are yellow, which is a bit unexpected. But what it does internally is it plays it safe and tries to avoid crashes if you run over the size of the array. And since I actually know that I'm not running over the size of the array because that's exactly the range I'm iterating over, I can tell Scython, please don't do that. I can take care of myself. I know what I'm doing. So I can say C import Scython. That's the magic Scython module, which has some nice functionality. And amongst other things, it has a decorator bounce check, which allows me to disable bounce checks. And I can see that the bounce check is apparently gone. So whenever I is too large for the array, it's just going to crash for me, which is fine because I know it's not too large. That's me knowing it. And I'm telling the compiler, you know, I know better. And now when I run that again, same results here. And for the speed, what do I get? It's a bit faster. Okay. Tiny bit. Not a huge difference. But yeah. It's getting closer to the factor three compared to the list version. Okay. So these are little tweaks that I can do here and there. Once I know that my code is correct and it's actually running and does this right thing in all cases, I can start disabling safety measures that Scython provides for me. Okay. What else do we have? There's another exercise here, which you can also try at home and then talk to me afterwards. That's one more thing you can do. Now to speed this up and that's parallelizing the code. So far it's all running one thread. The whole calculation. And now what I can do is I can use openmp to use multithreading because my laptop has enough cores. It actually has just two cores in this case, but you know, multithreading and hyperthreading and yeah, it's a bit faster than just one thread. So I can actually use the power that my laptop has, the different cores and parallelize my code, which is trivial in this case because we're doing the same thing on each of the items. So this is trivial, the parallelizable. And all I have to do is to tell Scython to use parallel threads rather than one thread. So as I said, I'm using openmp for that. So you need a C compiler that supports it for a while. Clang has not supported it. I don't know if that changed along the way. If you have GCC, then this is going to be enough. I just have to tell the distu to build that the C compiler needs a compiler argument fopenmp and the linker needs that too. And then when I compile this, it's actually not going to change anything because I haven't changed my code yet. But that's the first thing I have to do, just checking that it works. So the openmp integration apparently worked. If it does not work for you, then it's probably a C compiler that's missing openmp support or something like that. Now what I can do here is I'm using an integer loop to run over my array. And all I have to change is, I'll just change this to import here, Scython has a parallel module. Scython has a spelt, yeah, and that comes with a prange function, which just changes one letter compared to range, but a lot in terms of functionality. And all I have to do now is change range into prange, which makes it a parallel range loop. Otherwise, I don't have to change anything, just still running over the whole length of the array. But now it's doing that in parallel. And openmp allows me to say how many threads I want to use here. There's an argument called numthreads. I have to look that up so you don't have to know it either. So I use for threats here. And another thing that I have to do is for this loop, I have to unlock the gil because otherwise I won't get parallelism. The gil is the lock in Cpython that guards basically everything, every access to the interpreter. So the gil is the thing that makes Cpython fast because it doesn't have to do lower level locking in all sorts of places. Just has one lock. And so it runs very fast. When a single thread it, but since my code is already using plain C code for the execution, I do not need the gil because it does not do any Python direction. It does not need the Python runtime. And since the gil is only there to safely guard the Python runtime, I can release the gil, let Python do other stuff in the meantime it wants to, and just start a couple of C threads that run my plain C code here efficiently in parallel. So one more argument, no gil, true. And that's it. It's all I have to change. So I changed the range loop into a p range, said I want four threads to run this loop, and I want to release the gil while it's running. And then when the loop is done, it's going to reacquire the gil and just lock things down to single thread again. So compile this. Got a little warning. So did something wrong? Ah, yes, obviously. So what it tells me is calling a function that requires the gil is not allowed without the gil. Why is that? Because I did not tell syson that it's actually safe to call this function without the gil. So there's a check in syson whenever I release the gil, and that's something really nice in comparison to writing actual C code. So if you've ever written a C extension in C and released the gil somewhere, it's probably going to suck fall at some point. This can not happen that easily in syson because syson does some checking. It tells you whenever you try to do Python operations without owning the gil. And so what I have to tell here is this is a function that can be called without holding the gil. And then what syson will do is it'll check that function and make sure that that function does not use any Python code either, into any Python interaction either. So it's going to check that function for me. It's going to check my loop for me. I'm going to see that I'm calling that function which allows me to call it without the gil. And so it's basically checking the whole execution chain, the whole call chain for no gil safety. And since it's all no gil safe, it's not using any Python interaction. It's going to say, oh, fine, and it's hopefully going to compile it for me. Still got a warning, which is fine. That's just an open AP warning. It's a warning, not an error. So it has now compiled my code for me. And I should get the same result. But it should be faster because I'm not using for threads to run it rather than just one. It's going to run it down to six milliseconds. So that's another factor of 1.6 or 1.7 faster than the previous memory view version. And the factor of 500 faster than the Python version. OK, so the question was, why did I have an auto pickle up here, auto pickle false? I should also have a solution down there. That is a new feature in in Python 026. Previously extension types, so native class implementations were not pickleable. Now they are. So there's automatic picking support. And that also applies to memory views. And so the memory view I'm using here would actually be capable of, you know, picking it, dumping it to disk and reloading it. And I didn't care. It just loads up my program without being useful. So that's why I'm disabling the auto pickle in order to avoid the code overhead. That's it. It's a really nice feature, but it's just not used here. OK, so that brings us down a factor of 500 and even compared to the NumPy version, we're now at a factor of nine. OK, and then I'm pretty much through with the main part of my tutorial. Any more questions regarding this? OK, question was about data dependencies when I'm paralyzing my loop. And I'm not just using incomes, I am using incomes somewhere else in the array. It's not a problem if you stick with that kind of scenario. If you're only reading from the array, that's fine. You can currently read from throughout the array. It's probably going to slow down your algorithm a bit if you do random access in the array rather than just running through it once. Simply because memory bandwidth becomes more important in that case. Actually, at that level, people have to care about memory bandwidth. You don't really have to care about that if you're dealing with Python code somewhere, but if you're going that deeply native, the memory bandwidth actually becomes important. And so as long as you're only doing read access to your array, that's not a problem. It can be done concurrently across all threads anywhere you like. But you obviously run into the thread concurrency problems when you're doing write access in the array. That was one more. OK. Yeah, here. How would you actually distribute packages that actually include Python? OK. So distribution, how does distribution of Python code work? In exactly the same way as it works for other native extensions. So I mean, we've been using NumPy here. NumPy is a native extension. So I mean, there are lots of native libraries, binary libraries in the Python world that you can get from PyPy. So we have integration with distutils. Basically what you would do from your setup high script is you start with your Python code, which is commonly written in PIX files PyX, which, well, I would say stands for Python extended, maybe. PIX. So you write your Python code in PIX files, and then we provide a Scythonize function where you can just say, Scythonize all my PIX files that you find in my project, for example. And Scythonize will generate extension objects. So distutils extension objects from that, that you can then pass into the setup function. And from that point on, distutils will take care of compiling your extension into binary modules. And then you probably end up building wheels from that and distributing them to PyPy. OK. Question was, when I'm currently passing in a NumPy array with the data, what if I had a C array coming from some C library, for example, spits out some C data, C memory that holds the data. What would I do with that? If I had that case, I would probably split up my function. And so I would unpack, I would provide a Python function that's still able to unpack the memory view and then drop the actual code into some C def function that I use internally, say, internal average tax rate. And it would receive, it would do the same thing with the expected C interface. So C double pointer data and say size t length. And then I would deal with that. So I would use the length here and just use the array. This is how you would do a C array indexing in Python, pretty much like in C. And then here, in order to do the conversion, I would call my function and delegate to it. So that would just be a plain C function. And I would do something like take the address of incomes zero and length of incomes or income shape zero and then just pass that into my function. So this would have exactly the same performance characteristics as before. Same speed, exactly same speed. The unpacking of the number array would be done in this wrapper function here. If internally I had some C function returning some buffer, then it would return as a double pointer and a length. That's the normal C interface that you would expect. And I would just pass that into my internal function. That way I could use both data sources, a C data source and a Python data source, both using exactly the same algorithm. Just I would then also move the bounce check down here. Using exactly the same implementation. But it's more versatile for me now because I can use it in more cases. There was another question up there. We can try. Can try if that makes a difference. You can do that. That's perfectly fine for memory use. Instead of saying this is an arbitrary memory view, I can say the previous version supported actually arbitrary memory use. You could do slicing and pass in a number slice rather than a plain array. When I say I need a stride of one here, then it's going to reject input that does not have a stride of one. I can no longer pass in a number of slices. Let's see if that makes a difference. I have to revert that back to incomes shape zero. And anything else? Data was incomes. I think that's it. Let's compile it. Let's see. Exactly the same speed. It is using strides internally. Strides for offset calculation. Apparently the C compiler sees the index calculation and is apparently capable of generating efficient code from that to avoid big overhead for the stride calculation. This is C and C compilers are surprisingly smart sometimes. That's also a nice thing about developing in Python. When we write the code that generates C code, we always know that there's a C compiler cleaning up after us. We don't have to do all optimizations ourselves. We just have to tell the C compiler well enough how the code works that we've generated to make it understand how it can optimize it. That's it. It's just going to do all the native adaptations and everything. Question was can you use, well essentially can you use OpenMP pragmas in Python code? You can't because they wouldn't be passed through into the C code. And you also probably wouldn't. There are some features in OpenMP that you could end up wanting but we don't support. But we actually support quite a bit of OpenMP of what it provides. We have one thing I didn't tell you is I'm doing the sum calculation here and this is one function global variable that I'm doing the sum over. And I still end up with one result despite having a parallel loop here. That's because plus equals is special in P range. When I do income equals income plus X, this is different. So what the plus equals does is it turns the income variable into a global, what's it called? Regression variable? Something like that. It's a variable that's global to all threads and the sum that I built in that variable has been accumulated back after the threads have run. That's what the plus equals. When I spell it like this, then the income will be thread local and every thread will build their own sum separately and I won't get the same result. And so this is the expected way to spell it and I get the expected output. So you can compare it to this rule in Python that's assigning to a function local variable, makes it local. That's because that's the expected behavior. When you assign to a variable inside of a function, in 99.9% of the cases, you want it to be a local variable. That's why you're assigning it to it. And that's why for the point something percent of the cases where you actually wanted to change global variable, that's the global keyword. And it's exactly the same thing. If you use some plus equals here, in almost all cases, you want a global sum and not a thread local something. That's why we just do it that way. Use case optimized. More questions? Yes? I'm glad you asked. So we still have about 15 minutes for the rest of the tutorial and I can show you an example. I'm actually wrapping an external C library. Because that is one of the... It's an extremely common use case for a Python. Different notebooks so you won't find that. I think I actually handed out the notebook too. That's the intro quick notebook. It should be in the zip file. So what I'm doing here is, well, you probably won't be able to compile this because you need Lua for it, which you might not have installed. So I've actually written a Lua wrapper, which is called lupa, which allows you to use Lua code from Python. So this is an actual example similar to what Lupa does. So this is a Python implementation of kind of a minimal wrapper for running Lua code from Python. And for that, I'm linking against the LuaJIT library, which is a JIT implementation of Lua. For that, I'm telling this utility where it can find the LuaJIT header files and the LuaJIT library. I would normally do that for my setuppy script. Since I'm using the Jupyter notebook here, I can do it right in my cell, which obviously is not portable. If I wanted to compile this in Windows, for example, then it would be able to find, well, use it and include what's that, right? Here it works for me because I'm on Linux. So this is where the LuaJIT header files are installed. I can configure the whole thing from my setuppy script as well. So I can pass in configuration into the Sitenize function that we had. I can tell it where to find external libraries, external header files, all that. So I can set that up in my setuppy script so that Siten knows how to configure the build for me. Okay. The example we had was, I was just saying, C import, libc math, and that's it. That works because the declarations of the math header file that comes with libc are shipped with as part of Siten. So Siten has declarations for a part of libc, for some parts of projects, for libcpp, so the stl and c++, for the C Python API, and I think a couple of more things, openmp, a couple of functions that you can call openmp. So this is all pretty clear. You can just say, C import, this, and you're done. So when you need a c++ vector, for example, you can just say, from libcpp vector, c import vector, and you can use this stl vector in your setup code. That's it. I can also show an example of that in a minute. If you're using your own C library, then you have to prepare the, or provide the declarations yourself. This is done using an expression, cdf extern from some header file. This is starting a block where you declare external functions. And as you can see here, this is probably not the complete content of the Lua header file. There's way more in that. But that's all I need for my code. So I don't need to repeat everything that's in the header. I just need to tell Scythian what's there that my program is going to use. Because the C compiler afterwards will actually see the complete header. My generated C code will say, include Lua header. So the C compiler will see the whole thing. And Scythian only has to know about the stuff that I'm using. So what I did was I looked through the Lua documentation and copied out everything that I needed. That includes struct, which is the interpreter state of the Lua runtime. Then a function to create a new runtime. Close and clean up the runtime. Load Lua code into a buffer and compile it. Then do some stack operations on the Lua stack and so on and so forth. A couple of enums, a couple of conversion functions. And that's all I need. So I literally copied those out of the header file. In here into my PIX file, I would normally copy those into an external declarations file. So rather than copying it in my use only here PIX file, my source file, I would use a PXD file. So PIX declarations file. Because that allows me to reuse the declarations in multiple Scythian modules. Sort of like a header file just for Scythian. And now, when Scythian knows that these C functions exist, I can write a Python function here which takes Lua code as an argument. In this case, I have to convert it to a byte string because that's what Lua expects. I can take Python text string, unicode string and pass it into Lua and the function needs to convert it here. Then I create a new runtime. The return value there is null on an error. So I just say if there was an error in Lua, then raise a memory error for me. Very common pattern in Scythian. So whenever you call malloc or some initialization function and it fails and the documentation says if null pointer returned here, then memory allocation failed. In Scythian, you would just say if not pointer, raise memory error done. And then since I have to clean up the whole thing afterwards, I say I open a try block and finally I say clear the Lua top and close the runtime, clean everything up. So I'm using try finally in order to make sure that I'm not leaking any memory. I'm not leaving the Lua runtime in some legal state here. And try finally is perfect for that. And that's also, again, that's the mix that you see in Scythian code between Python and C. Try finally if there's an exception raised from my code, then the finally block I do some C clean up. Safe thing to do. Okay, what do I do in this try block here? I load the data into Lua, compile it there. If that fails, I raise a syntax error. Then I call the compiled code. If that fails, I raise some runtime error. I could try to get some error message from Lua there and put it as an error message in. And then I look at the Lua stack and assert that what I expected was I expected a number back. That's the only thing I can convert here. So my code has to return some number. And then I convert that into a Python number. And when you look at the function here, Lua to number returns a float. So it looks up some value on the Lua stack and returns a C float. And when in my Python function, I just say return that C float, then it turns into a Python float. Okay. Okay. And here's some Lua code. So I'll just help that compile. Not initialized. Try that again. Okay. That's compiled with warnings. I guess that's fine. And it executes my Lua code. Okay. And returns 55 as a result. So that's just a recursive Fibonacci as you would expect from running stupid code as an example. And so this is basically how you would wrap an external library. The Python interface is really nice. Just Python function pass in code. And internally it does all the ugly setup, C management, something, stuff, create some C states, clean it up, do a save memory management, do failure handling, all at the C level. And you can't see any of that from the Python level. Okay. Anything else? Another question? You're creating non-Pi arrays from C arrays? Creating non-Pi arrays from C arrays. So basically you have some memory block there and you want to wrap it in a non-Pi array. There is a non-Pi API function for that. A C API function. That's another thing. We also have the non-Pi C API wrapped. So pretty clear. You can just say C import non-Pi and then do stuff from there. Call C API functions of non-Pi and that's one of those. Okay. Just last question. You showed the map of the syncing Python variable. Is there still a syncing Python with a Python object for it? That's the expected question. I hear that at all conferences lately. People always say, so that's this typing module in Python 3.5 and why don't you support that? Wouldn't that be the obvious thing to do? Sort of. It wouldn't be entirely obvious because it would still be limited to the typing system that you could probably use from Python. So for example, all the lower wrapping here would be impossible to do with any Python level typing. You simply wouldn't be able to declare an external C function and actually call it from Python code. So what this syntax would allow you to do is you could write Python code and add signature annotations in Python syntax and Python 3 syntax that would be interpreted by Python in order to basically optimize your code when it's compiled. And yes, that is possible. So recently someone came up and so, okay, there are a couple of things to that. We do have support for annotated Python code. So that's when you import this Python module, this magic module, in Python code, it actually exists so you can say import Python, then it provides some data types that you can use and some cast functions and stuff like that. For example, there's a Python.int or a Python.float or a Python.double or something like that. So they are all C types provided by that and someone converted that to a typing stop. So there's now a typing support for the typing order for PEP484 style typing which allows it to use Python types in type annotations. Thing is, Python itself does not pick them up yet, but that's probably doable. So there's work going on in that direction, but there are also limits to that because you can't get beyond the level of what Python allows you to do. You can optimize Python code with it, but you can't go native. Is it possible to interact with the Python debugger from the IPython? For example, when some code crashes in IPython, you can type the magic directive debug and then go into the tap trace and you can go up and down and the break point is up. Okay, question was, is there any integration with the IPython debugger? When you debug Python code from IPython, when it crashes and you say debug something, there is support for GDB. So you can do GDB debugging at the size and source level. There originally was support for Python debugging, so you could use GDB for Python source debugging and someone added size and support for that so you can now do a free source level debugging. You can debug at the C level, at the generated C code level, at the size and source code level and at the Python source code level, also using GDB, but that does not give you Python debugger support in that sense. That's actually work done, or the has been work done in PyCharm for supporting Python debugging. And that's at the source code line level using the actual Python debugger. So I don't know what the actual state there is, but there is the capability in that direction. Okay, I think that's pretty much it. I'm going to throw and I hope you learned something. Thank you.
 Okay, so let's get started.  It's 1400, so everyone who's late is late.  So welcome to the tutorial.  This is the Scythe tutorial.  I'm Stefan Bienel, one of the core developers of Scythe, so this is kind of a, you can actually  ask me anything session.  I'm also a Scythe trainer, so I give trainings, especially at the Python Academy in Leipzig.  You can find more about that at the Python Academy or at my webpage there.  I hope you've all downloaded the notebook that I'm going to present here.  Anyone not prepared?  Who dares?  That's fine.  Okay.  Okay.  So okay.  First of all, what is Scythe?  Scythe is, well, it's Python compiler, basically.  First of all, it translates Python code to C code.  Okay, so you can write native code in Python syntax.  The cool twist about it is that because it's translated into C or C++, it allows you to  directly interact with C code, with native code, C++ code, C code, with libraries.  It allows you to write C and C++ code without actually writing C code.  So that's the cool thing about it.  It's an actual programming language, and many people use it to wrap libraries for Python  or to speed up their code, into native code.  But some people also just use it, you know, because it's a program language.  They use it as a program language, like they would use Python.  But as a kind of a cool programming language that makes many things easier than in other  native languages.  Okay.  So I'm going to use Jupyter Notebook for the tutorial.  And the first thing you would do in the notebook in order to enable Scythe support, because  Scythe is actually one of languages that Jupyter supports.  You would say load Scythe, and if you have Scythe installed, then this will enable Scythe  compiled cells.  So I'll do that.  It tells me I should restart my kernel.  Okay.  So I'll just try again.  It's not going to tell me anything, and it's imported Scythe.  Okay.  Cool.  The next step in there is just a quick, you know, summary of what dependencies you have  installed.  So I'm using the latest Scythe master.  Actually, you probably don't have that installed.  So it's perfectly fine if you use Scythe 026 or even 25.  That will be okay for this tutorial.  I'm using Python 306.  Is anyone using Python 2?  Okay.  You're excused.  So that's kind of fine, too.  Scythe supports Python 2.6 and later up to 307.  So we continuously test against the latest development versions of C Python in order to  stay compatible.  And that means whatever code you write in Scythe will be adapted by us to newer Python  versions for you.  So you don't have to do much there.  The kind of recent NumPy version.  One thing to know about Scythe is it compiles code to C. So you need a C compiler or C++  compiler installed in order to make any actual use of it.  Does anyone not have a C compiler installed?  Okay.  I'm first up there.  Everyone else has it.  That's fine.  So you will need this for this tutorial.  Because the way the compilation works is Python takes your Python or Scythe code, translates  it into C, and then the C compiler translates that into a shared library into an extension  module that you can then import and use in Python.  Okay.  That's the way it works.  So you have a three-step process.  The nice thing about the Jupyter notebook is that it removes most of that so you can  just write Scythe code in the cell, run it, and it will be compiled in the back and you  won't see anything about it and it will just get executed and imported.  It's all there.  You'll see that.  Really nice.  Okay.  Regarding questions in this tutorial, you can ask questions at any time.  Please just interrupt me.  When there's anything you want to know, I'll try to answer anything.  Well, not anything.  You know what I mean.  Okay.  Here's a very simple example.  This is just normal Python code.  I'm using the Python math module, specifically the sine function from it.  Just calculated in sine of five.  It gives you minus 0.9.  So that's simple.  You would probably know that.  And I can do the same thing in Scythe because Scythe can translate and compile arbitrary  Python code.  So I'll just do that there.  And the first thing you notice, you don't get any output.  Why is that?  What Scythe does in the back is it compiles your module and the module gets imported as  an extension module and actually gets executed.  But the Jupyter notebook does not see what the kind of the last line in your code was.  Okay.  So you can't just say I got this result not presented to you.  Instead the code in there gets executed as a module.  Okay.  And there's no standard output of modules.  So what I can do instead is I can say print the result here and it's going to be printed  for me.  Okay.  And that's the difference.  So this is fairly simple.  This is just still plain Python code.  What I can do in Scythe now is instead of using the Python math module, I can actually use  the math support in the C library.  So libc math header.  Scythe comes with all declarations that you need for that.  So instead of having to declare anything anywhere, I can just say C import.  So that's an extension to what you would see in Python.  It's compile time import.  So there's a couple of extensions to the Python language that you would use in the Python  language.  So one of them is C import.  It gives you a static compile time import.  And I'm using the libc math module here.  And then I'm taking the sign function of that.  And that's the actual low-level C sign function.  Okay.  And what I'm doing here is I'm taking that function and I'm assigning it to a variable  called sign function.  And this is a global module variable, meaning it's a Python variable.  Okay.  So what I end up with is a module that defines a name, sign function, which represents the  C sign function and exports it to Python.  Okay.  So I can just take that and call it from my notebook.  And here I'm using the C sign function directly from Python.  Okay.  This is kind of the quickest way to export a simple function, I would say, simple C  functions into Python.  Okay.  What this does more or less is it defines a Python function, which internally calls the  C sign function.  And it takes a double and returns a C double, okay.  And it returns a C double result.  And that's hidden behind this assignment here.  And here I've spelled it out.  This actually is Python syntax.  So Python allows me to define Python functions, but declare the signature in a C way.  Okay.  When I execute this, it defines my C sign function and I can run that from Python.  Okay.  So what you can see here is that this is actually a very basic property of the Python language.  It mixes Python and C freely.  So most of the code that you will see, most of the Python code that you will ever see,  do some Python features here, use some Python objects here, then push something into C,  do stuff there, call C function, get a return, and literally mix Python and C all over the  place.  And that makes Python such a cool language for many things because you get the simplicity  of Python on one side and on the other side, you got all the native power of C, okay, in  one program language.  Okay.  One nice debugging feature of Python is, well, here you see I've declared a Python cell in  Jupyter.  What I'm doing here is I'm passing the minus A parameter to that cell.  And in addition to compiling my code now, what it will do is it will analyze and annotate,  that's what the A is for, my code and then output it as an HTML fragment down here that  tells me how the compiler saw my code and what it made of it.  So for example, when I click on this line with a plus here of the signature definition,  then you'll see there was a lot of code generated from Scython just to implement this Python  function for me.  And then down here, I can actually prove that it calls the C sign function because the C  code says I take the variable X, call the C sign function on it directly, then take  the result and turn it into a Python float.  So that was what this function is doing, PyFloat from C double.  And then return the result as a normal Python float object.  And Scython generates these things automatically for you because it knows the result of the  sign function here is declared as a C double, but it is used in a Python context, it's returned  from Python function, so it must be an object and it automatically converts between the  two value types for you behind the scenes.  Okay.  Now, this is very basic and as I said, this can be done just by assigning the C function  to a Python variable.  When it becomes a bit more interesting when you do more stuff on the C side, so what I'm  doing here is I'm getting a double value in again, but then I'm taking the square of that  and I'm taking the sign of the square.  So this is implementing sign of square of the input and it's doing all that in C space  again.  Okay, so when I compile this and I actually compile this Python minus A, again, then you  see that this line down here is really just plain C multiplication and then we take the  result and call the sign function on it.  Okay.  And this is something you also often see in many wrapper generators that you find for  Python, say, although there are many of them.  In many of those, what you get is you get a plain wrapper around some C function, some  C functionality.  Okay.  And then you have to deal with whatever that gives you in Python.  And Python allows you to make that wrapper thin if you like, but thicker if you need  it.  Okay.  And that's completely seamless.  So whenever you need more functionality or you want to aggregate some C functionality  behind some Python function call, for example, or hide it behind a Python class, you can  easily do that by just implementing the thing in Python and then calling whatever C functionality  you need straight from your code.  Okay.  So if you find that.  Yeah, right.  I didn't declare C def.  Okay.  So C def is another extension you've seen C import.  C def is the other basic thing that you'll see all over the place in Python code.  It just says this is a C definition.  Okay.  And what this does here is it declares the variable X square as having the C type double.  Okay.  And what this does, okay.  In fact, I wouldn't even need that because Scythian is smart enough to do type inference  here.  I have two double values here to multiplying them.  Obviously results in a C double and I wouldn't need to declare the result type of the variable,  but I did.  Yeah.  Yeah, it should be.  So there's a bit of type inference available in Scythian which helps to avoid having to  declare too many things.  Okay.  So that was really a tiny quick intro.  Let's come to a bigger example.  So I owe this example to a guy called Caleb Hetting who presented a Scythian tutorial  at PyCon Australia a couple of years ago.  And a very nice example.  Why?  Because everyone likes Texas, right?  So I borrowed that idea from him and what I did was I looked up the actual tax calculation  function that we have here in Germany.  And I found that they're currently also for the last year, there were something like 44  million taxpayers, so earnest, and the average income of them was €3703 per month, so times  12, that's about what you get as an average income per year.  So I just run this and that's the average income.  And then I started looking a bit for actual data that backed this number and didn't really  find anything.  So for the sake of this tutorial, I just create some alternative facts here and I'm sure all  of you are happy with that.  It's the right conference to do that.  So I just assume that first of all, there are only 20 of the number of people, just to  keep the calculations a bit faster, and a lot normal distribution for the income.  So you can see that the average is about right and there's an alternative minimum income  and maximum income here.  And I plot this, you can see that, well, that's about what you would expect more or less from  the income distribution.  Well, more or less.  So let's calculate everyone's taxes.  I looked up the tax formula in Wikipedia and this is what it gave me.  It actually gave me an Excel implementation of that formula.  So I took that formulation and rephrased it in Python and this is essentially it.  So if your income is above that number, then it's calculated like this.  Otherwise, you know, it's a staged calculation.  And that's also what makes it kind of interesting because this is difficult to express in simple  terms.  The best way to express this is actually with ifs.  So this is how you would express it in Python and then you can calculate the average income  down here and the average tax rate across the data that we have.  You'll probably get slightly different results because you know, it's an alternative fact  on your side too.  But this is about the average income and the calculated tax, the average tax and then  I'll just do the whole thing in Python and you can see that the average tax rate is about  24% for my data set.  So when I asked time it how long that took, it's going to tell me some number after a while,  a couple seconds or maybe a couple more seconds or some more seconds.  Well you see, it actually takes quite a while in Python.  And time it runs a couple of times in order to get accurate results and then it tells  me that it's about a bit more than three seconds.  So let's say that's 1,000,000 in milliseconds.  I'll just remember that value and then in order to make things comparable while I'm on  the way of optimizing stuff, I'm calculating the ratios along the way and I'm saying that  Python is my base level and that's the slowest version I've probably come up with.  Next thing I did was I found a way to express the whole thing in NumPy and this is what  I got there.  Basically it's selecting parts of the array that match these conditions and then doing  the calculation for the income ranges based on that.  So I can calculate the whole thing on NumPy segments and then sum them up separately and  calculate the average from that and you can see that it gives me about the same average  tax rate, 24%.  And when I ask how long that took, it's going to be much faster.  So that's 57 milliseconds and now I notice that I forgot one thing.  Whenever you benchmark, one thing you should never forget is switch off the energy management.  So I will rerun the Python version again and it's going to take a couple of seconds.  So I'll get a proper result there now.  Which probably shouldn't be very much different because it's running for so long so the processor  will scale up during that time and I'm probably getting good results from that anyway.  But the faster it runs, the less likely it becomes that my CPU is going to be at full  speed because it's going to take a while until the power management notices that my  CPU is actually being used and heavily used and only then it's going to scale up.  And so I'm getting good results.  But this is about what I had before.  So I'll just keep the old value there.  And then I'll rerun the NumPy version again.  And then it's still about the same thing.  So that is 58.1 milliseconds.  And you can see that's already 54 times faster than the Python version.  Okay.  That is pretty good.  Okay.  The same thing can be done with a NumPy Ufunk by simply taking my Python function and applying  it to all elements in the array.  So I'll do that here.  That's from PyFunk function in NumPy, to which I can pass my function in order to convert  it into Ufunk.  And then when I apply that to the whole array and sum up the results, I get the same text  rate.  And time it on that is going to be much slower than the slicing version, but still much faster  than the pure Python version.  Okay.  So I'll remember that that took 818 milliseconds.  And so now the NumPy version is way faster than that.  And the Python version is only about four times slower.  Okay.  Let's get back to Python then.  Up to now, this is pretty much what you would be able to achieve from Python as well, given  the tools that we just had, so using plain NumPy.  Now what Python allows you to do is you can take the Python version of your code and compile  it into C. This is usually not going to give you a big effect simply because it's still  going to use Python objects.  It's still going to call Python functions along the way.  So there's not much out of the boxes that Python can do for you.  So when I just take the Python code that I implemented above, copy it straight in there.  It's exactly the same thing.  And now I use a Python cell instead of a plain Python cell.  So I get the same thing.  Compiled with Python.  And I can now call that.  On the list of incomes should get the same results, 24%.  And when I call time it on that, it's going to take a while.  And then we can see how it compares to the plain uncompiled Python version.  And that's 275.0 milliseconds.  And still about 15% faster than the plain Python version just by compiling it.  Just by adding one line at the top of the Jupiter cell.  Which is okay.  I didn't really do anything.  I didn't care.  I just dropped code in and it just got 15% faster.  If that's for free, that's fine.  The cool thing about Python now is that I can start optimizing my code at a much deeper  level at a native level rather than at a Python level.  I have many things that I can do at a C level that can make it run much faster than what  I can achieve in Python.  And the first thing I would do is I add static typing.  So you've already seen the Cdev declarations and the signature changes that I made.  I can now start declaring variables as having C types instead of Python object types.  So here's the same code copy again.  And now what I'll do is I'll start changing the code and annotating it with static types  with a goal of making it faster.  So the first thing I'll obviously do is I know that the income is probably double.  So that can be safely represented as a C double.  And I know down here, well, this is a list.  It's working on a list and it's taking the sum.  And what I see here is down here, it's calling this function a lot of times in a generate  expression.  So it's running over the incomes in order to calculate the taxes.  It's running over the incomes in order to calculate the sum.  And when I see something like this, the first thing I would say is this can be done much  faster by running over the list once and doing the calculation one step.  So I'll change this into a loop.  I'll just spell out the for loop that is hidden in this expression down here.  And I'll say for income in incomes, my tax is...  So you can follow this, but the actual implementation of this is right below.  So you have the whole solution down there.  So you can follow this, what I'm doing here, in order to see your own results.  And if you're only interested in the results, you can just execute the cell.  Like, call off cells below.  Just so you know.  Okay.  So I'll spell out the expression here as a single loop.  And that means my tax sum is going to be the sum of incomes, or the sum of calling the  function on my income.  So for each of the income values, I calculate the tax, sum up the taxes, and the total is  the total of incomes.  And then down here, I will say the final result is the tax sum by the total.  Right?  That's pretty much it.  In order to make that fast, what I'll do is I'll declare all variables in here as having  proper double values so that the whole calculation can actually be done in C-space.  So all variables that I'm using here, the tax sum, the total, and the income itself,  are all going to be C-variables.  Okay.  Which ones do you mean?  These up here?  Depends on the context.  So what I did here is I already declared, so inside of my function, I already declared  the income argument as a C double.  And that means that all comparisons down here will be done in C-space using C-doubles, and  Scython will infer the C double type for this variable, for the constant also, because that's  what it's comparing.  So the way the coercion then works is C-style.  So if you compare an integer with a double, then the results, the values will be compared  as doubles.  Okay.  If income was just a plain Python variable, then this would be a Python literal, and it  would use object comparison.  Okay.  Okay.  So what have I done so far?  I'll drop the last line now.  And what you can see now is all variables are doubles.  The function is still called with summing up the values that we get back from the function  call with summing up the totals, and we do the total calculation in the end.  So just compile this as it is, and see if the output is still the same.  It's still 24%.  That looks right.  And when I call time it on that, it's going to run for a while.  And it should already be a bit faster, quite a bit faster.  So we're down to 139 milliseconds now.  And that compared to the plain compiled version is a factor of 20 faster.  Okay.  So what was it doing if those weren't initialized?  I suppose I was lucky, and they were initialized to zero, but that means I was just lucky.  Yeah, what the C compiler does internally.  Yeah, so I should probably initialize them, and it should make a difference for me in  this case.  Just run it again.  I mean, I get the same result, right?  So it was still correct.  Okay, there's one more thing I can do.  When I run this through size minus a, and let's size and tell me what it did, I'm seeing  a couple of things here, and you've already noticed the yellow lines, I guess.  There are white lines, yellow lines, all this.  And the level of yellowness in this output tells me how much Python interaction there  is going on.  And the white line tells me plain C code here, and everything that's kind of yellow to dark  yellow means there's some kind of Python runtime interaction, some object operations going on.  So apparently I'm not at the point yet where I've turned my code into native code, because  there are still tons of yellow lines in there, and even just the calculations here seem yellow.  Why is that?  When I click on them, I can see that the actual calculation based on the income variable is  done in C.  So income times value minus something, but then the result is converted into a Python  float.  So what I forgot to do is this function is still Python function.  It's still called as a Python function.  It's using Python semantics for being called, so it's using argument tuples for input, and  it's using object for output.  And that is inefficient, since I'm only using this function internally inside of my program.  I'm not really exporting it to anything.  I don't care who else is going to use it.  I just want this function to be called as fast as possible inside of my program.  And for that, I can turn the whole function into a C function by changing the declaration.  Now it's a Python function, and I can turn it into a C function, which makes it a C function.  And now it's a static C function inside of my module.  In case people were still using it from the outside, I can use a third type of function  that's a CP function, which turns it into a C function with a Python wrapper.  So it's still usable from Python.  If it's just a C function, then it's only usable as a C function inside of my module.  So here, I'm actually going to use a CP function.  It doesn't make a big difference, but it still keeps the function visible from Python  code.  And now since it's a C function, I can declare the return type.  I don't have to if I don't declare it, then still object.  But since I know that the return type is actually a double, a C double, I can just spell it out  and say this function is a C function and it gets a C double as input and returns a  C double.  Okay.  If this is a CP depth, so it's Python that, does that mean like when you call it from  Python, it will still return a C double, but then Python will convert that to a...  Yes.  So it's a C double.  When is the CP depth, what does the Python wrapper actually do?  And what it does is...  Well, it's a complete wrapper, so it unwraps the input.  It gets objects as input, or one object in this case, and packs it into a C double so  you can pass in a Python float or Python integer.  They'll just be converted into a C int.  That's sorry, C double in this case.  Then the calculation will be done and on the output.  The result of that is a C double and since we are calling it from Python, the wrapper  will convert the C double return value back into a Python float object.  Okay.  And if that function is called from other Python code, it doesn't commit...  So if it's called from Python code, then that's exactly what the CP depth is for.  Calls from inside of Python are fast.  They use the C interface.  They use the C function.  Call it directly.  Do not go through the wrapper.  And the advantage of that is they actually see the signature.  So if you pass a C double into that function, into a CP depth function, then the caller  will see that the call function actually expects a C double.  There's no conversion needed.  It'll just pass it through at the C level as fast as it can.  The C compiler will see that and maybe even inline the call or do whatever optimization  it can.  And the output is a C double.  No conversion needed either.  You just get a C double back all done.  That's it.  Okay.  That's...  That way you get the fastest possible call inside of Python without losing the ability  to use the function from Python, from the outside Python world.  Okay.  So I'll compile that.  And actually...  And now you can see that the whole function dropped into C.  So all the return values are now plain C double returns.  And even down here, that was previously a bit yellow because the return value was a Python  object.  And it had to be converted back into a C double in order to sum it up.  And now you can see that the function call is a C call that the...  There's no input conversion going on, no output conversion.  Any Python usage that's still left in this function is the iteration over the incomes  list.  And that's it.  Okay.  Okay.  I've compiled this.  It still gives me the same result.  And I call time it on it.  Actually we had 139 milliseconds.  Now we're down to 30 milliseconds for the whole thing.  Okay.  And so the ratio changed to a factor of 90 compared to the previously compiled function,  annotated function, and to a factor of 105 compared to the plain Python implementation.  Okay.  Any questions here?  Well, the reason why the for loop is in yellow is that we're iterating over a Python list.  So the input is a Python list and the iteration needs to step from value to value through  the list.  But this is already faster in Python than it is in Python.  Because Python understands what a Python list is and it can unpack it at the C level and  just iterate over the underlying C array of objects.  Okay.  So the conversion there, the iteration and the conversion there is very fast.  So why is the last line still yellow?  I'm going to go back up.  This is a Python function.  So I'm calling it from Python using it from my Jupyter notebook as a Python function.  And so the return value here, you can see that the text sum divided by the total is  been evaluated in C space.  Another thing you can see from the C code is that the total is first checked for zero  and you get a zero division error if the total happens to be zero.  So if I pass in an empty list into this function, I will get an exception and not a crash in  C. Wonderful.  So there's a bit of exception handling going on here, which is very fast because this is  just a C value comparison and the final conversion of the result for returning the result is  also done in, is using a Python operation for creating a float object.  And that's it.  The whole calculation up to that point is happening in C now.  The question was, can I declare a list of doubles for the incomes?  No, I can't.  So there's currently no way to express.  So what I could say is this here is a list.  I could even say this is a list and it's definitely not none.  This is also something I can express in Syson syntax.  What this does is it does a type check on the way in and if the input value is none, then  it will raise, I think, a type error and if it's not a list, you get a value error.  Something like that.  But it will check the input and make sure it's definitely a Python list and nothing  else.  It has a tiny advantage because now Syson knows that it is a list and not maybe a tuple  or maybe some iterable.  Previously, the code supported any iterable.  Now it only supports Python lists.  So that will cut down the generated code a little because the loop code will be simpler,  but it's probably not going to make a big difference.  I can try it.  So I just compile this and rerun the benchmark, see what kind of difference it makes.  It's really tiny.  It's a couple of percent.  That's because the looping, the for loop in Syson is so fast and has special handling  for tuples for lists.  When you pass a list and it hits the fast path, this is a list.  Then you end up with very fast code despite not knowing at compile time that it was a  list.  Because Syson is going to assume that most of the time what you're iterating over is  probably a list.  So we're optimizing for that.  The cool thing if I leave this out, if I don't declare anything here for the incomes, it's  still going to be about as fast, not a big difference, but it's going to support any  input, any iterable input.  Not just exactly Python lists.  If I say does a list, then it would not even accept subtypes of lists.  However unlikely they are, but it would reject anything that's not a list.  Any more questions?  We'll come into that in a minute.  Let's write the next thing.  So we've already achieved a factor of 105 faster code by now.  And it's already faster than the NumPy version by a factor of two.  So this is really nice and I think that's pretty much what I had implemented on here.  Yeah, it looks exactly the same.  So now question to you.  We could now do an exercise here.  I could let you do kind of the same thing that I did on another function.  Optimize that.  Do you want to do it?  Or do you want me to continue my show and tell presentation?  Does anyone want to do the exercise?  The majority, sorry.  Okay.  So I'll just keep going.  You can still do the same thing and we can talk about it during the conference.  So I'll leave this to you and next topic then is memory use.  And that's the thing you wanted to hear about.  So if I'm actually using, yeah.  Question is this the fastest version already?  Well since my tutorial is not finished yet, probably not.  So this is probably the fastest way to do it if your input is a Python list of numbers.  Okay.  Okay.  Question was how does it compare to a fully native C implementation?  You've seen all white lines.  So the whole operation, the whole calculation was actually done in C space.  So you get exactly the same performance in C as you get for this Python code.  It's just plain C in the end.  You can try it.  You can write a C version and then wrap that in Python and call it.  So Python is optimizing C compiler flags.  Python actually does not configure your C compiler for you.  What it does is it generates a C code.  And then in this case, it's the Jupyter integration that calls the C compiler in the back.  And that uses your C flags, LD flags, whatever you have defined in your environment.  So it's your responsibility to configure your C compiler properly.  And if you know better C flags, then the current user is just using them.  Okay.  So memory use.  So the question, like two questions ago was what if the whole thing is stored in a NumPy  array and not in a Python list?  This is actually a very common case.  How often does it happen to you that you have actual data in a Python list rather than like  in a Pandas data frame or in a NumPy array somewhere?  Probably quite likely that you'll hit the case that all these income values are already  in NumPy array.  And Python has special support for that.  So there is a protocol in Python that was added in Python 2.6 and Python 3.  Back in the days of, that was like 2007, 2008, around that.  So it's already about 10 years old now.  And that's the buffer protocol.  And that is something that is supported natively by NumPy and some other libraries out there.  And basically whenever you're pushing around big native data blobs in some way, it's probably  going to support the buffer interface in one way or another.  And so that's why Scythe has native support for buffers.  So what it allows you to do is you can take a buffer, you can receive a buffer as input,  unpack it into the C memory and just do some operation on the memory natively rather than  going through some Python interface there.  So it's one shot, unpack operation, and from that point on you have native C memory.  This is with the interface here.  What we previously had was we just had an object coming in, a Python list in this case.  And what we're going to change now is we're not going to use the Python list of data.  We're going to use the NumPy array of data that we have.  And we pass it in and declare it to Scythe, to the Scythe compiled function as a memory  view on that data.  And that's a special syntax for that, which says it's a one-dimensional array of doubles.  That's it.  That's how you declare a memory view.  And what Scythe knows from that is there's some object coming in which supports the buffer  interface.  It knows that the items in that object, in that memory, are C doubles, have the format  of the C double.  It's one-dimensional, and if whatever it gets in is not one-dimensional, it's going to reject  it.  It's going to say, fold your error, wrong input.  But if it's one-dimensional, it's going to be happy about it and unpack it into a C  buffer.  And then you can operate on that.  In this case, it's actually very simple.  Since it's C space now, I'm no longer going to use the Python for loop.  What I'll do instead is I'm going to use an integer loop for I in range len incomes.  Is this actually going to work in Python 0.26?  That's a good question.  It might not.  It does not, so in 0.27, this is going to work.  In 0.26, I think you still have to say income shape 0, as you would know from number, probably,  right, in size of the first dimension.  So integer loop over the length of the array that we got in.  And then in here, I'll just say we sum up incomes at I and run our function on the value  at that point.  And that's about it, I think.  So I'll try to compile it.  And looks next, Python is happy with that.  Yep, still the same result.  Now passing in a number array, so I changed the input from incomes, that's the list, to  incomes number, which is the number array, gives me 24%, the expected value.  Now when I run this, we have 11 milliseconds.  So we'll use 11.1.  So that is much faster.  And that's already a factor of five compared to the NumPy version.  Another factor of 2.7 compared to the last version we had, and a factor of 280 compared  to the Python version.  Okay.  Is this clear to everyone?  We got a memory use.  The syntax is basically modeled after what you would have in NumPy, so in order to get  a single dimensional slice, you would just put a code on in.  And that's pretty much what would be used as a syntax here.  Yep.  Sorry again.  Yeah.  So question was, when I had the Python for loop for x in the incomes array now, why did  that still work?  Well, it still worked because NumPy arrays actually support iteration.  I mean, we could actually optimize the same thing in Python as well.  We now know that it's a memory view, and we can say you can actually iterate over a one-dimensional  memory view.  Why not?  We can do that efficiently for you.  So yes.  I mean, that would be a nice feature.  I'll note that down somewhere.  Yeah.  But, I mean, this is, it's kind of pretty enough.  Okay.  And it's about how much, three times faster?  What did we have before?  Yeah, 2.7 times.  Okay.  So using memory views here and using NumPy arrays as inputs made a big difference again.  Okay.  Now I'll show you one more thing here.  And I compile this with size minus a again.  There's one little thing down here.  These two lines are yellow, which is a bit unexpected.  But what it does internally is it plays it safe and tries to avoid crashes if you run  over the size of the array.  And since I actually know that I'm not running over the size of the array because that's  exactly the range I'm iterating over, I can tell Scython, please don't do that.  I can take care of myself.  I know what I'm doing.  So I can say C import Scython.  That's the magic Scython module, which has some nice functionality.  And amongst other things, it has a decorator bounce check, which allows me to disable bounce  checks.  And I can see that the bounce check is apparently gone.  So whenever I is too large for the array, it's just going to crash for me, which is fine  because I know it's not too large.  That's me knowing it.  And I'm telling the compiler, you know, I know better.  And now when I run that again, same results here.  And for the speed, what do I get?  It's a bit faster.  Okay.  Tiny bit.  Not a huge difference.  But yeah.  It's getting closer to the factor three compared to the list version.  Okay.  So these are little tweaks that I can do here and there.  Once I know that my code is correct and it's actually running and does this right thing  in all cases, I can start disabling safety measures that Scython provides for me.  Okay.  What else do we have?  There's another exercise here, which you can also try at home and then talk to me afterwards.  That's one more thing you can do.  Now to speed this up and that's parallelizing the code.  So far it's all running one thread.  The whole calculation.  And now what I can do is I can use openmp to use multithreading because my laptop has  enough cores.  It actually has just two cores in this case, but you know, multithreading and hyperthreading  and yeah, it's a bit faster than just one thread.  So I can actually use the power that my laptop has, the different cores and parallelize my  code, which is trivial in this case because we're doing the same thing on each of the  items.  So this is trivial, the parallelizable.  And all I have to do is to tell Scython to use parallel threads rather than one thread.  So as I said, I'm using openmp for that.  So you need a C compiler that supports it for a while.  Clang has not supported it.  I don't know if that changed along the way.  If you have GCC, then this is going to be enough.  I just have to tell the distu to build that the C compiler needs a compiler argument fopenmp  and the linker needs that too.  And then when I compile this, it's actually not going to change anything because I haven't  changed my code yet.  But that's the first thing I have to do, just checking that it works.  So the openmp integration apparently worked.  If it does not work for you, then it's probably a C compiler that's missing openmp support  or something like that.  Now what I can do here is I'm using an integer loop to run over my array.  And all I have to change is, I'll just change this to import here, Scython has a parallel  module.  Scython has a spelt, yeah, and that comes with a prange function, which just changes  one letter compared to range, but a lot in terms of functionality.  And all I have to do now is change range into prange, which makes it a parallel range loop.  Otherwise, I don't have to change anything, just still running over the whole length of  the array.  But now it's doing that in parallel.  And openmp allows me to say how many threads I want to use here.  There's an argument called numthreads.  I have to look that up so you don't have to know it either.  So I use for threats here.  And another thing that I have to do is for this loop, I have to unlock the gil because  otherwise I won't get parallelism.  The gil is the lock in Cpython that guards basically everything, every access to the  interpreter.  So the gil is the thing that makes Cpython fast because it doesn't have to do lower level  locking in all sorts of places.  Just has one lock.  And so it runs very fast.  When a single thread it, but since my code is already using plain C code for the execution,  I do not need the gil because it does not do any Python direction.  It does not need the Python runtime.  And since the gil is only there to safely guard the Python runtime, I can release the  gil, let Python do other stuff in the meantime it wants to, and just start a couple of C  threads that run my plain C code here efficiently in parallel.  So one more argument, no gil, true.  And that's it.  It's all I have to change.  So I changed the range loop into a p range, said I want four threads to run this loop,  and I want to release the gil while it's running.  And then when the loop is done, it's going to reacquire the gil and just lock things  down to single thread again.  So compile this.  Got a little warning.  So did something wrong?  Ah, yes, obviously.  So what it tells me is calling a function that requires the gil is not allowed without  the gil.  Why is that?  Because I did not tell syson that it's actually safe to call this function without the gil.  So there's a check in syson whenever I release the gil, and that's something really nice  in comparison to writing actual C code.  So if you've ever written a C extension in C and released the gil somewhere, it's probably  going to suck fall at some point.  This can not happen that easily in syson because syson does some checking.  It tells you whenever you try to do Python operations without owning the gil.  And so what I have to tell here is this is a function that can be called without holding  the gil.  And then what syson will do is it'll check that function and make sure that that function  does not use any Python code either, into any Python interaction either.  So it's going to check that function for me.  It's going to check my loop for me.  I'm going to see that I'm calling that function which allows me to call it without the gil.  And so it's basically checking the whole execution chain, the whole call chain for no gil safety.  And since it's all no gil safe, it's not using any Python interaction.  It's going to say, oh, fine, and it's hopefully going to compile it for me.  Still got a warning, which is fine.  That's just an open AP warning.  It's a warning, not an error.  So it has now compiled my code for me.  And I should get the same result.  But it should be faster because I'm not using for threads to run it rather than just one.  It's going to run it down to six milliseconds.  So that's another factor of 1.6 or 1.7 faster than the previous memory view version.  And the factor of 500 faster than the Python version.  OK, so the question was, why did I have an auto pickle up here, auto pickle false?  I should also have a solution down there.  That is a new feature in in Python 026.  Previously extension types, so native class implementations were not pickleable.  Now they are.  So there's automatic picking support.  And that also applies to memory views.  And so the memory view I'm using here would actually be capable of, you know, picking  it, dumping it to disk and reloading it.  And I didn't care.  It just loads up my program without being useful.  So that's why I'm disabling the auto pickle in order to avoid the code overhead.  That's it.  It's a really nice feature, but it's just not used here.  OK, so that brings us down a factor of 500 and even compared to the NumPy version, we're  now at a factor of nine.  OK, and then I'm pretty much through with the main part of my tutorial.  Any more questions regarding this?  OK, question was about data dependencies when I'm paralyzing my loop.  And I'm not just using incomes, I am using incomes somewhere else in the array.  It's not a problem if you stick with that kind of scenario.  If you're only reading from the array, that's fine.  You can currently read from throughout the array.  It's probably going to slow down your algorithm a bit if you do random access in the array  rather than just running through it once.  Simply because memory bandwidth becomes more important in that case.  Actually, at that level, people have to care about memory bandwidth.  You don't really have to care about that if you're dealing with Python code somewhere,  but if you're going that deeply native, the memory bandwidth actually becomes important.  And so as long as you're only doing read access to your array, that's not a problem.  It can be done concurrently across all threads anywhere you like.  But you obviously run into the thread concurrency problems when you're doing write access in  the array.  That was one more.  OK.  Yeah, here.  How would you actually distribute packages that actually include Python?  OK.  So distribution, how does distribution of Python code work?  In exactly the same way as it works for other native extensions.  So I mean, we've been using NumPy here.  NumPy is a native extension.  So I mean, there are lots of native libraries, binary libraries in the Python world that  you can get from PyPy.  So we have integration with distutils.  Basically what you would do from your setup high script is you start with your Python code,  which is commonly written in PIX files PyX, which, well, I would say stands for Python  extended, maybe.  PIX.  So you write your Python code in PIX files, and then we provide a Scythonize function  where you can just say, Scythonize all my PIX files that you find in my project, for  example.  And Scythonize will generate extension objects.  So distutils extension objects from that, that you can then pass into the setup function.  And from that point on, distutils will take care of compiling your extension into binary  modules.  And then you probably end up building wheels from that and distributing them to PyPy.  OK.  Question was, when I'm currently passing in a NumPy array with the data, what if I had  a C array coming from some C library, for example, spits out some C data, C memory that  holds the data.  What would I do with that?  If I had that case, I would probably split up my function.  And so I would unpack, I would provide a Python function that's still able to unpack the memory  view and then drop the actual code into some C def function that I use internally, say,  internal average tax rate.  And it would receive, it would do the same thing with the expected C interface.  So C double pointer data and say size t length.  And then I would deal with that.  So I would use the length here and just use the array.  This is how you would do a C array indexing in Python, pretty much like in C.  And then here, in order to do the conversion, I would call my function and delegate to it.  So that would just be a plain C function.  And I would do something like take the address of incomes zero and length of incomes or income  shape zero and then just pass that into my function.  So this would have exactly the same performance characteristics as before.  Same speed, exactly same speed.  The unpacking of the number array would be done in this wrapper function here.  If internally I had some C function returning some buffer, then it would return as a double  pointer and a length.  That's the normal C interface that you would expect.  And I would just pass that into my internal function.  That way I could use both data sources, a C data source and a Python data source, both  using exactly the same algorithm.  Just I would then also move the bounce check down here.  Using exactly the same implementation.  But it's more versatile for me now because I can use it in more cases.  There was another question up there.  We can try.  Can try if that makes a difference.  You can do that.  That's perfectly fine for memory use.  Instead of saying this is an arbitrary memory view, I can say the previous version supported  actually arbitrary memory use.  You could do slicing and pass in a number slice rather than a plain array.  When I say I need a stride of one here, then it's going to reject input that does not have  a stride of one.  I can no longer pass in a number of slices.  Let's see if that makes a difference.  I have to revert that back to incomes shape zero.  And anything else?  Data was incomes.  I think that's it.  Let's compile it.  Let's see.  Exactly the same speed.  It is using strides internally.  Strides for offset calculation.  Apparently the C compiler sees the index calculation and is apparently capable of generating efficient  code from that to avoid big overhead for the stride calculation.  This is C and C compilers are surprisingly smart sometimes.  That's also a nice thing about developing in Python.  When we write the code that generates C code, we always know that there's a C compiler cleaning  up after us.  We don't have to do all optimizations ourselves.  We just have to tell the C compiler well enough how the code works that we've generated to  make it understand how it can optimize it.  That's it.  It's just going to do all the native adaptations and everything.  Question was can you use, well essentially can you use OpenMP pragmas in Python code?  You can't because they wouldn't be passed through into the C code.  And you also probably wouldn't.  There are some features in OpenMP that you could end up wanting but we don't support.  But we actually support quite a bit of OpenMP of what it provides.  We have one thing I didn't tell you is I'm doing the sum calculation here and this is  one function global variable that I'm doing the sum over.  And I still end up with one result despite having a parallel loop here.  That's because plus equals is special in P range.  When I do income equals income plus X, this is different.  So what the plus equals does is it turns the income variable into a global, what's it called?  Regression variable?  Something like that.  It's a variable that's global to all threads and the sum that I built in that variable  has been accumulated back after the threads have run.  That's what the plus equals.  When I spell it like this, then the income will be thread local and every thread will  build their own sum separately and I won't get the same result.  And so this is the expected way to spell it and I get the expected output.  So you can compare it to this rule in Python that's assigning to a function local variable,  makes it local.  That's because that's the expected behavior.  When you assign to a variable inside of a function, in 99.9% of the cases, you want it  to be a local variable.  That's why you're assigning it to it.  And that's why for the point something percent of the cases where you actually wanted to  change global variable, that's the global keyword.  And it's exactly the same thing.  If you use some plus equals here, in almost all cases, you want a global sum and not a  thread local something.  That's why we just do it that way.  Use case optimized.  More questions?  Yes?  I'm glad you asked.  So we still have about 15 minutes for the rest of the tutorial and I can show you an example.  I'm actually wrapping an external C library.  Because that is one of the...  It's an extremely common use case for a Python.  Different notebooks so you won't find that.  I think I actually handed out the notebook too.  That's the intro quick notebook.  It should be in the zip file.  So what I'm doing here is, well, you probably won't be able to compile this because you  need Lua for it, which you might not have installed.  So I've actually written a Lua wrapper, which is called lupa, which allows you to use Lua  code from Python.  So this is an actual example similar to what Lupa does.  So this is a Python implementation of kind of a minimal wrapper for running Lua code from  Python.  And for that, I'm linking against the LuaJIT library, which is a JIT implementation of  Lua.  For that, I'm telling this utility where it can find the LuaJIT header files and the LuaJIT  library.  I would normally do that for my setuppy script.  Since I'm using the Jupyter notebook here, I can do it right in my cell, which obviously  is not portable.  If I wanted to compile this in Windows, for example, then it would be able to find, well,  use it and include what's that, right?  Here it works for me because I'm on Linux.  So this is where the LuaJIT header files are installed.  I can configure the whole thing from my setuppy script as well.  So I can pass in configuration into the Sitenize function that we had.  I can tell it where to find external libraries, external header files, all that.  So I can set that up in my setuppy script so that Siten knows how to configure the build  for me.  Okay.  The example we had was, I was just saying, C import, libc math, and that's it.  That works because the declarations of the math header file that comes with libc are shipped  with as part of Siten.  So Siten has declarations for a part of libc, for some parts of projects, for libcpp, so  the stl and c++, for the C Python API, and I think a couple of more things, openmp, a  couple of functions that you can call openmp.  So this is all pretty clear.  You can just say, C import, this, and you're done.  So when you need a c++ vector, for example, you can just say, from libcpp vector, c import  vector, and you can use this stl vector in your setup code.  That's it.  I can also show an example of that in a minute.  If you're using your own C library, then you have to prepare the, or provide the declarations  yourself.  This is done using an expression, cdf extern from some header file.  This is starting a block where you declare external functions.  And as you can see here, this is probably not the complete content of the Lua header  file.  There's way more in that.  But that's all I need for my code.  So I don't need to repeat everything that's in the header.  I just need to tell Scythian what's there that my program is going to use.  Because the C compiler afterwards will actually see the complete header.  My generated C code will say, include Lua header.  So the C compiler will see the whole thing.  And Scythian only has to know about the stuff that I'm using.  So what I did was I looked through the Lua documentation and copied out everything that  I needed.  That includes struct, which is the interpreter state of the Lua runtime.  Then a function to create a new runtime.  Close and clean up the runtime.  Load Lua code into a buffer and compile it.  Then do some stack operations on the Lua stack and so on and so forth.  A couple of enums, a couple of conversion functions.  And that's all I need.  So I literally copied those out of the header file.  In here into my PIX file, I would normally copy those into an external declarations file.  So rather than copying it in my use only here PIX file, my source file, I would use a PXD  file.  So PIX declarations file.  Because that allows me to reuse the declarations in multiple Scythian modules.  Sort of like a header file just for Scythian.  And now, when Scythian knows that these C functions exist, I can write a Python function  here which takes Lua code as an argument.  In this case, I have to convert it to a byte string because that's what Lua expects.  I can take Python text string, unicode string and pass it into Lua and the function needs  to convert it here.  Then I create a new runtime.  The return value there is null on an error.  So I just say if there was an error in Lua, then raise a memory error for me.  Very common pattern in Scythian.  So whenever you call malloc or some initialization function and it fails and the documentation  says if null pointer returned here, then memory allocation failed.  In Scythian, you would just say if not pointer, raise memory error done.  And then since I have to clean up the whole thing afterwards, I say I open a try block  and finally I say clear the Lua top and close the runtime, clean everything up.  So I'm using try finally in order to make sure that I'm not leaking any memory.  I'm not leaving the Lua runtime in some legal state here.  And try finally is perfect for that.  And that's also, again, that's the mix that you see in Scythian code between Python and  C. Try finally if there's an exception raised from my code, then the finally block I do  some C clean up.  Safe thing to do.  Okay, what do I do in this try block here?  I load the data into Lua, compile it there.  If that fails, I raise a syntax error.  Then I call the compiled code.  If that fails, I raise some runtime error.  I could try to get some error message from Lua there and put it as an error message in.  And then I look at the Lua stack and assert that what I expected was I expected a number  back.  That's the only thing I can convert here.  So my code has to return some number.  And then I convert that into a Python number.  And when you look at the function here, Lua to number returns a float.  So it looks up some value on the Lua stack and returns a C float.  And when in my Python function, I just say return that C float, then it turns into a  Python float.  Okay.  Okay.  And here's some Lua code.  So I'll just help that compile.  Not initialized.  Try that again.  Okay.  That's compiled with warnings.  I guess that's fine.  And it executes my Lua code.  Okay.  And returns 55 as a result.  So that's just a recursive Fibonacci as you would expect from running stupid code as an  example.  And so this is basically how you would wrap an external library.  The Python interface is really nice.  Just Python function pass in code.  And internally it does all the ugly setup, C management, something, stuff, create some  C states, clean it up, do a save memory management, do failure handling, all at the C level.  And you can't see any of that from the Python level.  Okay.  Anything else?  Another question?  You're creating non-Pi arrays from C arrays?  Creating non-Pi arrays from C arrays.  So basically you have some memory block there and you want to wrap it in a non-Pi array.  There is a non-Pi API function for that.  A C API function.  That's another thing.  We also have the non-Pi C API wrapped.  So pretty clear.  You can just say C import non-Pi and then do stuff from there.  Call C API functions of non-Pi and that's one of those.  Okay.  Just last question.  You showed the map of the syncing Python variable.  Is there still a syncing Python with a Python object for it?  That's the expected question.  I hear that at all conferences lately.  People always say, so that's this typing module in Python 3.5 and why don't you support that?  Wouldn't that be the obvious thing to do?  Sort of.  It wouldn't be entirely obvious because it would still be limited to the typing system  that you could probably use from Python.  So for example, all the lower wrapping here would be impossible to do with any Python  level typing.  You simply wouldn't be able to declare an external C function and actually call it from  Python code.  So what this syntax would allow you to do is you could write Python code and add signature  annotations in Python syntax and Python 3 syntax that would be interpreted by Python  in order to basically optimize your code when it's compiled.  And yes, that is possible.  So recently someone came up and so, okay, there are a couple of things to that.  We do have support for annotated Python code.  So that's when you import this Python module, this magic module, in Python code, it actually  exists so you can say import Python, then it provides some data types that you can use  and some cast functions and stuff like that.  For example, there's a Python.int or a Python.float or a Python.double or something like that.  So they are all C types provided by that and someone converted that to a typing stop.  So there's now a typing support for the typing order for PEP484 style typing which allows  it to use Python types in type annotations.  Thing is, Python itself does not pick them up yet, but that's probably doable.  So there's work going on in that direction, but there are also limits to that because  you can't get beyond the level of what Python allows you to do.  You can optimize Python code with it, but you can't go native.  Is it possible to interact with the Python debugger from the IPython?  For example, when some code crashes in IPython, you can type the magic directive debug and  then go into the tap trace and you can go up and down and the break point is up.  Okay, question was, is there any integration with the IPython debugger?  When you debug Python code from IPython, when it crashes and you say debug something, there  is support for GDB.  So you can do GDB debugging at the size and source level.  There originally was support for Python debugging, so you could use GDB for Python source debugging  and someone added size and support for that so you can now do a free source level debugging.  You can debug at the C level, at the generated C code level, at the size and source code  level and at the Python source code level, also using GDB, but that does not give you  Python debugger support in that sense.  That's actually work done, or the has been work done in PyCharm for supporting Python  debugging.  And that's at the source code line level using the actual Python debugger.  So I don't know what the actual state there is, but there is the capability in that direction.  Okay, I think that's pretty much it.  I'm going to throw and I hope you learned something.  Thank you.
EuroSciPy_2017__Cython_vl9La7wH7QI_0_131.png EuroSciPy_2017__Cython_vl9La7wH7QI_132_1535.png EuroSciPy_2017__Cython_vl9La7wH7QI_1536_3551.png EuroSciPy_2017__Cython_vl9La7wH7QI_3552_4547.png EuroSciPy_2017__Cython_vl9La7wH7QI_4548_10031.png EuroSciPy_2017__Cython_vl9La7wH7QI_10032_10151.png EuroSciPy_2017__Cython_vl9La7wH7QI_10152_12923.png EuroSciPy_2017__Cython_vl9La7wH7QI_12924_13811.png EuroSciPy_2017__Cython_vl9La7wH7QI_13812_15503.png EuroSciPy_2017__Cython_vl9La7wH7QI_15504_16031.png EuroSciPy_2017__Cython_vl9La7wH7QI_16032_16415.png EuroSciPy_2017__Cython_vl9La7wH7QI_16416_16475.png EuroSciPy_2017__Cython_vl9La7wH7QI_16476_16571.png EuroSciPy_2017__Cython_vl9La7wH7QI_16572_19199.png EuroSciPy_2017__Cython_vl9La7wH7QI_19200_20639.png EuroSciPy_2017__Cython_vl9La7wH7QI_20640_21455.png EuroSciPy_2017__Cython_vl9La7wH7QI_21456_22043.png EuroSciPy_2017__Cython_vl9La7wH7QI_22044_22139.png EuroSciPy_2017__Cython_vl9La7wH7QI_22140_22883.png EuroSciPy_2017__Cython_vl9La7wH7QI_22884_23663.png EuroSciPy_2017__Cython_vl9La7wH7QI_23664_25223.png EuroSciPy_2017__Cython_vl9La7wH7QI_25224_26759.png EuroSciPy_2017__Cython_vl9La7wH7QI_26760_26807.png EuroSciPy_2017__Cython_vl9La7wH7QI_26808_27419.png EuroSciPy_2017__Cython_vl9La7wH7QI_27420_28667.png EuroSciPy_2017__Cython_vl9La7wH7QI_28668_28931.png EuroSciPy_2017__Cython_vl9La7wH7QI_28932_29579.png EuroSciPy_2017__Cython_vl9La7wH7QI_29580_29651.png EuroSciPy_2017__Cython_vl9La7wH7QI_29652_30179.png EuroSciPy_2017__Cython_vl9La7wH7QI_30180_30239.png EuroSciPy_2017__Cython_vl9La7wH7QI_30240_30383.png EuroSciPy_2017__Cython_vl9La7wH7QI_30384_30467.png EuroSciPy_2017__Cython_vl9La7wH7QI_30468_30515.png EuroSciPy_2017__Cython_vl9La7wH7QI_30516_31139.png EuroSciPy_2017__Cython_vl9La7wH7QI_31140_31595.png EuroSciPy_2017__Cython_vl9La7wH7QI_31596_31655.png EuroSciPy_2017__Cython_vl9La7wH7QI_31656_31703.png EuroSciPy_2017__Cython_vl9La7wH7QI_31704_33467.png EuroSciPy_2017__Cython_vl9La7wH7QI_33468_34343.png EuroSciPy_2017__Cython_vl9La7wH7QI_34344_35291.png EuroSciPy_2017__Cython_vl9La7wH7QI_35292_35915.png EuroSciPy_2017__Cython_vl9La7wH7QI_35916_36995.png EuroSciPy_2017__Cython_vl9La7wH7QI_36996_37043.png EuroSciPy_2017__Cython_vl9La7wH7QI_37044_38435.png EuroSciPy_2017__Cython_vl9La7wH7QI_38436_39587.png EuroSciPy_2017__Cython_vl9La7wH7QI_39588_39719.png EuroSciPy_2017__Cython_vl9La7wH7QI_39720_42047.png EuroSciPy_2017__Cython_vl9La7wH7QI_42048_42575.png EuroSciPy_2017__Cython_vl9La7wH7QI_42576_45371.png EuroSciPy_2017__Cython_vl9La7wH7QI_45372_47111.png EuroSciPy_2017__Cython_vl9La7wH7QI_47112_47231.png EuroSciPy_2017__Cython_vl9La7wH7QI_47232_47279.png EuroSciPy_2017__Cython_vl9La7wH7QI_47280_48383.png EuroSciPy_2017__Cython_vl9La7wH7QI_48384_48791.png EuroSciPy_2017__Cython_vl9La7wH7QI_48792_49595.png EuroSciPy_2017__Cython_vl9La7wH7QI_49596_49691.png EuroSciPy_2017__Cython_vl9La7wH7QI_49692_56195.png EuroSciPy_2017__Cython_vl9La7wH7QI_56196_57887.png EuroSciPy_2017__Cython_vl9La7wH7QI_57888_60035.png EuroSciPy_2017__Cython_vl9La7wH7QI_60036_61595.png EuroSciPy_2017__Cython_vl9La7wH7QI_61596_62603.png EuroSciPy_2017__Cython_vl9La7wH7QI_62604_63863.png EuroSciPy_2017__Cython_vl9La7wH7QI_63864_63911.png EuroSciPy_2017__Cython_vl9La7wH7QI_63912_64439.png EuroSciPy_2017__Cython_vl9La7wH7QI_64440_64535.png EuroSciPy_2017__Cython_vl9La7wH7QI_64536_65483.png EuroSciPy_2017__Cython_vl9La7wH7QI_65484_67343.png EuroSciPy_2017__Cython_vl9La7wH7QI_67344_74759.png EuroSciPy_2017__Cython_vl9La7wH7QI_74760_74819.png EuroSciPy_2017__Cython_vl9La7wH7QI_74820_74879.png EuroSciPy_2017__Cython_vl9La7wH7QI_74880_78731.png EuroSciPy_2017__Cython_vl9La7wH7QI_78732_78779.png EuroSciPy_2017__Cython_vl9La7wH7QI_78780_78827.png EuroSciPy_2017__Cython_vl9La7wH7QI_78828_78899.png EuroSciPy_2017__Cython_vl9La7wH7QI_78900_79091.png EuroSciPy_2017__Cython_vl9La7wH7QI_79092_79343.png EuroSciPy_2017__Cython_vl9La7wH7QI_79344_79487.png EuroSciPy_2017__Cython_vl9La7wH7QI_79488_79595.png EuroSciPy_2017__Cython_vl9La7wH7QI_79596_80171.png EuroSciPy_2017__Cython_vl9La7wH7QI_80172_80375.png EuroSciPy_2017__Cython_vl9La7wH7QI_80376_81383.png EuroSciPy_2017__Cython_vl9La7wH7QI_81384_81431.png EuroSciPy_2017__Cython_vl9La7wH7QI_81432_81911.png EuroSciPy_2017__Cython_vl9La7wH7QI_81912_82319.png EuroSciPy_2017__Cython_vl9La7wH7QI_82320_82391.png EuroSciPy_2017__Cython_vl9La7wH7QI_82392_82439.png EuroSciPy_2017__Cython_vl9La7wH7QI_82440_82547.png EuroSciPy_2017__Cython_vl9La7wH7QI_82548_82643.png EuroSciPy_2017__Cython_vl9La7wH7QI_82644_85115.png EuroSciPy_2017__Cython_vl9La7wH7QI_85116_85523.png EuroSciPy_2017__Cython_vl9La7wH7QI_85524_86039.png EuroSciPy_2017__Cython_vl9La7wH7QI_86040_86111.png EuroSciPy_2017__Cython_vl9La7wH7QI_86112_87047.png EuroSciPy_2017__Cython_vl9La7wH7QI_87048_87119.png EuroSciPy_2017__Cython_vl9La7wH7QI_87120_89891.png EuroSciPy_2017__Cython_vl9La7wH7QI_89892_89939.png EuroSciPy_2017__Cython_vl9La7wH7QI_89940_90023.png EuroSciPy_2017__Cython_vl9La7wH7QI_90024_92111.png EuroSciPy_2017__Cython_vl9La7wH7QI_92112_92315.png EuroSciPy_2017__Cython_vl9La7wH7QI_92316_92615.png EuroSciPy_2017__Cython_vl9La7wH7QI_92616_94847.png EuroSciPy_2017__Cython_vl9La7wH7QI_94848_95075.png EuroSciPy_2017__Cython_vl9La7wH7QI_95076_95159.png EuroSciPy_2017__Cython_vl9La7wH7QI_95160_95363.png EuroSciPy_2017__Cython_vl9La7wH7QI_95364_101411.png EuroSciPy_2017__Cython_vl9La7wH7QI_101412_101747.png EuroSciPy_2017__Cython_vl9La7wH7QI_101748_107519.png EuroSciPy_2017__Cython_vl9La7wH7QI_107520_107615.png EuroSciPy_2017__Cython_vl9La7wH7QI_107616_107915.png EuroSciPy_2017__Cython_vl9La7wH7QI_107916_115355.png EuroSciPy_2017__Cython_vl9La7wH7QI_115356_115571.png EuroSciPy_2017__Cython_vl9La7wH7QI_115572_122471.png EuroSciPy_2017__Cython_vl9La7wH7QI_122472_123779.png EuroSciPy_2017__Cython_vl9La7wH7QI_123780_126779.png EuroSciPy_2017__Cython_vl9La7wH7QI_126780_127031.png EuroSciPy_2017__Cython_vl9La7wH7QI_127032_127427.png EuroSciPy_2017__Cython_vl9La7wH7QI_127428_127499.png EuroSciPy_2017__Cython_vl9La7wH7QI_127500_127619.png EuroSciPy_2017__Cython_vl9La7wH7QI_127620_127679.png EuroSciPy_2017__Cython_vl9La7wH7QI_127680_127739.png EuroSciPy_2017__Cython_vl9La7wH7QI_127740_127823.png EuroSciPy_2017__Cython_vl9La7wH7QI_127824_127895.png EuroSciPy_2017__Cython_vl9La7wH7QI_127896_128387.png EuroSciPy_2017__Cython_vl9La7wH7QI_128388_128435.png EuroSciPy_2017__Cython_vl9La7wH7QI_128436_136895.png EuroSciPy_2017__Cython_vl9La7wH7QI_136896_137041.png