LLVM and Python
CC Attribution 2.0 Belgium:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Taşkaya, Batuhan
Python with LLVM has at least one decade of history. This session will be going to cover-up how python implementations tried to use LLVM such as CPython's Unladen Swallow branch (PEP 3146) or attempts from PyPy and why they failed. After that it will show what are the current python projects that use LLVM for speed, such as numba and python libraries for working with LLVM IR. In the end, it will mention about new ideas that would unite the powers of both LLVM and Python. This talk is about LLVM's influence over Python's ecosystem. It is targeted an audience of language developers who want to integrate LLVM and developers who are curious about why dont dynamic languages can unite their power with LLVM to speed-up. It will start with python's implementations and the approaches they take. The Unladen Swallow branch which basically tries to integrate LLVM to CPython (under google sponsored) is a good example of these approaches. There are attempts from the PyPy side but they are also failed because of the dynamic nature of Python. After this fails, we'll swap back to our current time and show projects that are benefiting from LLVM to speed up python especially on the scientific side such as numba (which offers JITting via LLVM). Besides these projects, there are also a few projects that offer an interface to LLVM. Such as llvmpy and llvmlite. I've been using llvmlite about 1 year in my side projects and toy languages so these projects has the potential to inspire developers to work with LLVM and build languages a-top on it. In the end, it will show what is the future of these two big projects (LLVM & Python) and how we can participate.
 Okay, hello everyone, welcome to the discussion. My name is Batran, I'm a high school student from Turkey, and in this session I'm going to talk about LRVM and Python. This talk is specific to Python and the approaches taken by Python's JIT compiler using LLVM, but there are some good points that can be used in compiling dynamic languages to LLVM. So we are going to start with Cpython's workflow and pipeline. You may see this relevant, but most of the applications use these workflow's pipes to generate LLVM and JIT compile. Then we are going to see dynamic nature of Cpython, like why you shouldn't trust user code and why can't we can easily convert Python to LLVM or other representations. Then we are going to see some projects that are not currently under development and dead because of some reasons. The first one is unladen swallow. It's a Google sponsored fork of Cpython, which is, which tried to bring LLVM into Python's core, but then Google lost its interest, Google's clients lost its interest. So it's that. Then there is Py to LLVM and Py to LLVM, Py to LLVM was an old project by a developer and it is retrieved as Py to LLVM and the both are dead right now. It's a JIT compiler for Python too. And Piston, Piston is relatively new project. It is sponsored by Dropbox, but then Dropbox switched performance critical code to other languages such as Go. And it is also dead. There are some current projects like Numva. Numva is a JIT compiler for NumPy, Python, mathematical oriented code, which uses LLVM light at the back end. LLVM light is a Python binding to LLVM, which offers a great API and it offers a way to access JIT execution engine of LLVM. And in the future part, we are going to see how we can implement and develop a JIT compiler for Python in two ways. One is converting Python's AST into the LLVM IR and one is converting Python's bytecode to LLVM IR. We are going to start to see Python's workflow. See Python takes your source and generates tokens from it. Tokens are individual parts and they have no relations. Then the see Python's parsers takes the tokens and constructs a concrete syntax tree which has relationships, but it is hard to work on. And then it strips all of the irrelevant information from that concrete syntax tree and constructs an AST. And then through the AST, it constructs a control flow graph like structure. There are no real control flow graphs in see Python. But there's some structure that like control flow graph and it generates bytecode from the AST and that structure. And then it executes that bytecode in Python's virtual machine. The AST and bytecode is suitable for coordination, but the other parts aren't. And let's see them in details. The tokenization starts with reading your Python file like it should be a.py file or the standard input that doesn't matter. And reads the file because it's contents in the encoding you have specified or the UTF-8. And it tokenizes it with certain patterns. Like that pattern depends on the context. It knows how to split A dot B. A has an individual meaning by its own. B has an individual meaning by its own. But it doesn't split 1 dot 1 because 1 dot 1 is a float. But you can use tokens to generate machine code because there are no relationships. No blocks, no nothing. It's pretty rough format. Like this is a... Okay. Can I highlight this? Yeah. This is some tokens. Like 1 is number, operator is plus. I know you can't see it because of highlight. Yeah. The number, this is 1 plus 1. But you can generate code from this because there are no relationships. Then there's a parser. Parser is a program that takes that generate tokens and uses rules to construct a tree like structure concrete and concrete symmetry. It's pretty complex because there's irrelevant information such as white space or parentheses. There are relationships but the blocks aren't suitable for direct code generation. And Python's parser interface is deprecated and will be removed. So Python advocates to work on in ASD level. ASD is the abstract version of that concrete syntax tree. There are notes that are defined in ASDL format, Zephyr's ASDL format, which are easy to work on. Like when you parse a function, it says this is the return, this is the function's name, this is the annotations of the function's code, et cetera, et cetera. Python says you can manipulate ASD or generate your code from ASD. I give you some interfaces, some helper functions, but don't use the parser because it's so raw. And it keeps only relevant information for generating code. Like it doesn't keep any white space or comments. It keeps line-know information for syntax errors for highlighting where this syntax error happened. This is an example of Python's not visitor interface. Like you can, whenever Python sees a constant note, it will call the visit constant handler. Whenever it sees a name note, it will call the name handler so you can generate intermediate representation here. And after that ASD step, Python will construct code objects that contain byte code and some information. This code object is going to execute under C Python's virtual machine, which is a stack-based virtual machine that works like you push something, you pop something with the instructions. The instructions are formatted like there's an opcode and an oparc operation argument. And there are some extra sequences. Like if you use a lot constant opcode, you specify argument as an index of code const data structure. So it will make the Python faster. Unfortunately, there are no direct interfaces to assemble byte code in Python itself. The interface are assembled in Python, but there are interfaces for disassembling byte code objects which we can use in JIT compiling the function's body. This is an example function that takes X as an input and compares if X is greater than 4, returns power of 2, et cetera, et cetera. And when we disassemble this, we see instructions like this. The first one is a load test. It loads the local variable from the argument spec. And they are format like this. This is jump target thing. Python's stack-based virtual machine operates like this. It checks if X is greater than 4. If not, it pops directly here. If it is true, it runs this place. So this is a jump target from that return power of 3 line. Yeah. This was the workflow of C Python. Let's see why we can't directly trust the user code because we won't give the types of the arguments or any kind of name. They are all dynamic and it will make our job harder. User can even change built-ins like user can change the function or they can even put handlers inside of class definition. When you define a class, Python loads something called build class, which is a helper function. And when you change a build class from the built-ins module, whenever someone defines a class in Python, your function gets called. And this makes things even more harder. And compared to other languages, in Python, everything is missable unless you say otherwise. Yeah. Let's see some attempts from the past that tried to integrate LLVM with Python itself. On latest wallow, it's a Google-sponsored fork of C Python that tried to make a JIT compiler for C Python itself. Its name comes from Monty Python show. It tries to make Python faster. It starts with fork of C Python. And they plan to eventually merge this into the Python score, but then there are some side effects which prevent this. It was sponsored by Google and then Google lost its interest because this was a fork and deployment was hard. Google hasn't any performance critical code in Python and Google's clients don't want to allocate money and time for this. Its features, it made Python 2.5 times faster on single thread Python code. It's single thread. They didn't solve the global interpreter log problem. In Python, if you have a CPU bound code, it can run, it runs one thread at a one time. It doesn't matter if you have four threads. It runs one thread because of some reference counting issues. On latest wallow, didn't solve it, but it makes Python faster 2.5 times in single thread code. Their aim was five times, but there are some problems about reaching that aim. They didn't break any Python code because they were already based on C Python itself. So every C extension run as usual. They analyzed Python code before running it. So they know if you change the LAN function or not. So that they make some assumptions, perform it on this and some data collected at runtime. But if X function gets always integers, they said, okay, generate an intermediate representation based on that fact. But if it gets a string, there's a guard which prevents these assumptions from falling. It converts bytecode. It opales in the bytecode level instead of ASD, which is their choice. There is a hot threshold in on latest wallow. So there's a cost of making a function compared to LLVM. So if a function called so many times and they know so many about it, they compile it to LLVM. And before that compilation, they collect data about that function. They know if branch never executed, they just cut out it. Or if they know all arguments are integer, they optimize for that. But ensuring that everything works as same, even the arguments can change. They put guards which has cost if mispron<|id|><|translate|>. they worked with linking LLVM into the Python binary. There are some cons, but besides of this, one is memory usage. The memory usage increased up to eight times. It's not always eight times, but the most part of it eight times. This is one of the costs of LLVM and other libraries, they increase memory usage. If you keep different versions of same code in the memory, it will increase by default. So they keep the native code, they keep the byte code, they keep the ASD. So memory is increased. There are some optimizer structures. They tried to optimize Python byte code before sending to LLVM. So they create some structure which keeps Python's internal knowledge, but they cost some cons about startup time, static linking again. And when you want to compile a Python function to LLVM, you need to initialize LLVM and C++ initialize, take some time. And there are some over-retops here on time routines. Binary size increased as usual because you link it to LLVM. Why don't they reach 2.5 times, they reached 2.5 times, but not five times? Because when they was working, this project was from 2009, LLVM isn't that good at just time compilation and they had some issues with that. And LLVM's G2 code doesn't work well with GDB. If you are developing a C extension in Python, you probably want to debug it with GDB or sec files and use all profile. And LLVM's G2 code didn't work well, so they allocate some time to fix this, but they lost some gain in performance. This project lasted one year and then Google lost its interest. If the project continues, in my opinion, they can reach the five times goal easily on some codes. Some LLVM points about this, they initially selected LLVM because they don't want to generate assembly code on every machine target. LLVM was easy. The optimization of code generation was great, but in 2009, it isn't directly suitable for G-Tink. And cost of operations compiling was so heavy. And without collecting runtime data, it is hard to optimize. They was collecting after learning this, so they spent some time to do this without collecting runtime data. The next project is PyLLVM, which is a compiler for subset of Python that specialize in machine learning, algorithms and computations. In Python, mathematical code gets executed slowly because of there's a over-route of dynamic things. If you compile this to LLVM, you get so much gain and they try to do this. They are free-shoes. They are as fast as C code in some cases that compiles with clang. They only work for subset of Python. If your function doesn't fit that subset, they directly return your Python function. If it fits, they turn a wrapper around the github code. They don't collect runtime data, but they do static type inference and symbol table generation. Python has a symbol table which you can use that comes with standard library, but it doesn't give you anything about types. So you need to create your own symbol table which records types and infers the types, which could be cool. They use something called LLVM Py, which is a library, a wrapper library around LLVM's API, C++ API. They use Python ASD model instead of bytecode, which is more suitable in my opinions. They try to convert the function body, and if they see something, they can convert, they raise an error, and if they catch that error, they return normal function. If they don't see that error, they just return the github function, and they infer the types. Let's see Piston. Piston is a github compiler, and the runtime for Python 2. It's runtime most based on C-Pythons. So they have some gain on that. They use something called tracing giths, and they specialize on web applications, especially dropbox tests. They manage to run Django's full test suite, as far as I remember. They feature as they are a 95% test of the C-Python in the latest relays in 2017. They have C-API support because of C-Python runtime, and in a big codebase, you can run Piston with little changes over the general codebase. They have more than three executors. Aluminum is the highest level, and there's VGIT, and there's ASD interpreter. So if your function gets called less, they interpret the function's ASD. If it is called more than 25 times, they get with VGIT, and if it's called more than 1000 times, they get it with LLVM. They have ASD-like control flow graphs, and ASD interpreter to giths tracing, and they use something called inline caching, and rewriting of functions body, and C-Python-based runtime. So there are some projects from current dates, Numba. Numba is up-to-date project that uses LLVM to GIT, it's a GIT compiler. It's specialized in NumPy and Python functions. It knows how to boot-shot these to assembly level, and gets best performance, so it's a subset, and it works only with type inference. If it knows what types it gets, it directly GITs. If it doesn't know, it records runtime data. It's used with simple Python decorator. You just add something at the top of the function called the GIT, and Numba automatically handles everything else. It supports NVIDIA, CUDA, and AMD-Rug experimentally. It's almost as fast as C in some cases, which is pretty good. It works like this. It takes Python source, it uses bytecode to generate IR, and then it converts that bytecode to their own specialized intermincer representation. They made some assumptions based on type inference, and they generate LLVM IR and GIT afterwards. There's some different modes in LLVM and Numba. One is no-python mode. No-python mode means this object never gets Python objects. It always gets integers and other atomic types. So if you use it, you get more performance. It generates LLVM IR, and it optimizes it. LLVM light has an interface to LLVM, and you can optimize the bytecode. It finds the functions. In the future, we are going to see how we can write our LLVM compiler. This is our target function. It takes A and B, and it uses Python's annotation feature, which we don't handle with type inference. You need to specify the types. We are going to use AST, LLVM light, the same library that Numba uses, and annotations. This is the function. We will have a compiler. If we can convert the functions by the, we will convert. If we can't, we will return the normal function, so it will work in a subset of Python. This is a basic compiler. It will try to get function source code. If it can't, it will return normal function. And if it can, it will parse the source code into the AST, and it will try to traverse that AST using not visitor class, Python's AST return. If any error happens during that traversing, it will return normal function. So if something doesn't fit our subset, we will raise the compiler, and if this part catches that error, it will return the function. This is the AST of that function. It starts with a function definition. There is an argument part, there is a body part, the collector part, and return annotation. The module is the highest level of any AST, and we are going to generate an LLVM module from the Python's AST, and set to south mode. And generic visit is a function that traverses the body of given node. So we will give the module, and it will traverse every node that lives under the module. This is the function, if you want to recall. The functions, first of all, we will remove the decorator that is used to git our code. So it won't be infinite loop. Then we are going to generate an argument spec from that. There's a cast LLVM function, which is just checks the annotation and converts an LLVM error. And then we are going to get the return type. So we can convert an LLVM function type from that. It takes the return type and the values of arguments. Then we are going to use that function type to generate an LLVM function, which will live under the module we have created. It will take the function type, and it will have the name of the node. Then we are going to append the basic block, which we can use that afterwards to get a builder in that basic block. So we are going to traverse the node's body one by one, and use that builder to build our expressions. The first expression is constants. LLVM types doesn't directly fit to Python ones, so we need to convert it. Pythons integers doesn't have a specific bit length, so we are going to use 32 for default, and we are going to use floats for our float type, not double. This is the cast LLVM function. If the given type is integer, we are going to return LLVM's int type. If the float, we are going to return float type. And if the type is something we don't know, we will raise the compiler. This is the part that constructs intermediate representation, LLVM's constant, with the type of the IR and the value of the nodes. There's argument access part. We will visit the name nodes, and if we can fetch the argument from the given functions type spec, we will fetch it and return the value. If we can't, we will raise the compiler because it's something global or we don't know. There's binary operations. We will try to get the operator from the builder object. If we can, we will raise the compiler because some operator we don't know. And then we will return the left side and right side under that operator. The return operation is pretty simple. We will just call it method of builder and return the given value, which is going to be binary operation. So we wrote a basic compiler with a compile class method, a compiler error, if it doesn't fit subset and visitors for module function return binary operation name and constant. The funny part is creating an execution engine. We will create a default triple target that's configured to LLVM and create a target machine. In the creating target machine, you can specify the optimization level or anything you want. Then we will create the execution engine which takes bootstrapping assembly. We don't have anything to bootstrap and target machine and we will return the machine, engine. This is the giting part. We will create the engine with this method and then parse the R module and verify it if it's a valid, intermittent representation or not. Then we will add that module to our engine, the finalize and run the static constructors. And then we will get the function as a C types function pointer. So at the end, we will fetch that functions pointer and cast it to C types C fun time. Python offers a way to access C functions with C types. So there's a C type C fun type which takes a type set and the type spec is given here and we will cast the function pointer to that. We did this with ASD. So can we do this same thing with bytecode? Yes, we will have something like this to decorate our function and we will have that same part. We will keep a stack of the functions bytecode because Python's visual machine is bytecode, it's stack based. So we will, this is the opcode by one by one. If the opcode is something we know, we will add the stack. If we don't know, we will break the for loop and just turn to normal function. Loading names is simple. If we see a load fast bytecode, we will return the given arguments to stack. If we see a load const, we will cast it to LLVM and append to stack. If we see a binary operation, we will pop two things from the stack and create an operation from it. And if we see a return value, we will return. This is the whole code to construct that IR. Yeah, this is it. Thank you for listening. Is there any questions? Is there any questions? Anyone has a question, please raise your hand. Yeah, okay. Okay. So we will present things for projects that pay to the extent to compile, open code into native code. And can you make explicit what makes your position different? What are the things that are different in your approach than the one that's where in Python? Okay, you are asking what are things different in my JIT compilers other than the projects I presented? Okay, I need to go to the outline to list other projects. Okay, this will... Okay, there was four projects. Unladen Swallow was using bytecode. The bytecode I am using in the second approach. So Unladen Swallow's approach and my approach is almost same. Pyaladame uses ASD and in my first approach, I used ASD to use that same approach. Piston and Numba also uses bytecode. So mine isn't different than others, but I tried to simplify the process of there in my examples. In both ways. I have several questions. And so, typing issue is a really issue for Python. And it seems to me that in your approach, you're delegating that to the user. Yeah. To the type annotation. So that's actually a very big difference compared to the other approach. You are not taking care of type inference, which makes things much easier. But when you annotate a variable with it, it means Python int, which is actually big int. And you are casting them to 32 bits with the girl, which is a totally different world. Yeah, I mentioned that. LLVM types doesn't directly fit the Python type. So for the sake of easily, I casted it to 32 bit integer. But in a real world application, it would be nonsense to cast 32 bit int. Yeah, or you could, for instance, using C types can exhibit 32 bits integer. Yeah. And you can ask the user to specify that it's okay. But hardware integers instead. Well, Python's typing doesn't allow to specify integers. There's literals, but there are nothing to specify the bit length of the integer in type annotations. So it would be hard to use or to define that. How much of a speed up is your compiler? How much of a speed up is your compiler? My compiler gets around to 20% speed because I don't do any optimizations, any, except I think I just convert the direct Python to LLVM IR and it gets 20% speed up. What stops you from doing optimizations? Why are you not doing running the optimizer? Because for the sake of presentation, I want to keep everything simple. As you're using type annotations, you don't need a jit at all. You could do that statically. You just take the function and perform your static compilation and then run the binary into your Python life. And that's okay. So why would you use a jit in your situation where you only use static information? You are asking why I'm using jit when there is already defined type annotations. I can just convert to functions by the ahead of time population. Because in the approach taken by like unladen swallow, they record runtime data. I want to keep that in the slides, but it doesn't fit. So I have a project called Saving Inhead which takes the exact thing like unladen swallow and it records runtime data. So you are right. In this way, it would make sense to not running jit, but if you are going to record the runtime data or do other things, you need to use jit. Any more questions? When are you using the byte code one versus the AST based? To me, AST is much more simpler to work on because you don't need the extra bootstrap and code Python offers an interface to access it. But I want to show how unladen swallow and Pstone worked. So I'd like an example to byte code. You say the AST one is the future. You say using the AST by this one is the future. Yeah. One of the shortcomings presented by the Python Pstone worker was that most of the Python code is actually just glue calling native functions. That's true for system library, but that's also true for the package lightning by. And it presents a very strong barrier to the optimizer. And so did you take that into account? Or when you see a call to a native function, you're just calling it the Python way and getting back the results? You're asking if I am optimizing native functions with bootstrapping some AST? Numba does that, but my jit compiler doesn't do that. OK. Thank you. OK, we thank the speaker again. Thank you.
 Okay, hello everyone, welcome to the discussion.  My name is Batran, I'm a high school student from Turkey, and in this session I'm going  to talk about LRVM and Python.  This talk is specific to Python and the approaches taken by Python's JIT compiler using LLVM,  but there are some good points that can be used in compiling dynamic languages to LLVM.  So we are going to start with Cpython's workflow and pipeline.  You may see this relevant, but most of the applications use these workflow's pipes to  generate LLVM and JIT compile.  Then we are going to see dynamic nature of Cpython, like why you shouldn't trust user  code and why can't we can easily convert Python to LLVM or other representations.  Then we are going to see some projects that are not currently under development and dead  because of some reasons.  The first one is unladen swallow.  It's a Google sponsored fork of Cpython, which is, which tried to bring LLVM into Python's  core, but then Google lost its interest, Google's clients lost its interest.  So it's that.  Then there is Py to LLVM and Py to LLVM, Py to LLVM was an old project by a developer  and it is retrieved as Py to LLVM and the both are dead right now.  It's a JIT compiler for Python too.  And Piston, Piston is relatively new project.  It is sponsored by Dropbox, but then Dropbox switched performance critical code to other  languages such as Go.  And it is also dead.  There are some current projects like Numva.  Numva is a JIT compiler for NumPy, Python, mathematical oriented code, which uses LLVM  light at the back end.  LLVM light is a Python binding to LLVM, which offers a great API and it offers a way to  access JIT execution engine of LLVM.  And in the future part, we are going to see how we can implement and develop a JIT compiler  for Python in two ways.  One is converting Python's AST into the LLVM IR and one is converting Python's bytecode  to LLVM IR.  We are going to start to see Python's workflow.  See Python takes your source and generates tokens from it.  Tokens are individual parts and they have no relations.  Then the see Python's parsers takes the tokens and constructs a concrete syntax tree which  has relationships, but it is hard to work on.  And then it strips all of the irrelevant information from that concrete syntax tree and constructs  an AST.  And then through the AST, it constructs a control flow graph like structure.  There are no real control flow graphs in see Python.  But there's some structure that like control flow graph and it generates bytecode from the  AST and that structure.  And then it executes that bytecode in Python's virtual machine.  The AST and bytecode is suitable for coordination, but the other parts aren't.  And let's see them in details.  The tokenization starts with reading your Python file like it should be a.py file or  the standard input that doesn't matter.  And reads the file because it's contents in the encoding you have specified or the UTF-8.  And it tokenizes it with certain patterns.  Like that pattern depends on the context.  It knows how to split A dot B. A has an individual meaning by its own.  B has an individual meaning by its own.  But it doesn't split 1 dot 1 because 1 dot 1 is a float.  But you can use tokens to generate machine code because there are no relationships.  No blocks, no nothing.  It's pretty rough format.  Like this is a...  Okay.  Can I highlight this?  Yeah.  This is some tokens.  Like 1 is number, operator is plus.  I know you can't see it because of highlight.  Yeah.  The number, this is 1 plus 1.  But you can generate code from this because there are no relationships.  Then there's a parser.  Parser is a program that takes that generate tokens and uses rules to construct a tree like  structure concrete and concrete symmetry.  It's pretty complex because there's irrelevant information such as white space or parentheses.  There are relationships but the blocks aren't suitable for direct code generation.  And Python's parser interface is deprecated and will be removed.  So Python advocates to work on in ASD level.  ASD is the abstract version of that concrete syntax tree.  There are notes that are defined in ASDL format, Zephyr's ASDL format, which are easy to work  on.  Like when you parse a function, it says this is the return, this is the function's name,  this is the annotations of the function's code, et cetera, et cetera.  Python says you can manipulate ASD or generate your code from ASD.  I give you some interfaces, some helper functions, but don't use the parser because it's so  raw.  And it keeps only relevant information for generating code.  Like it doesn't keep any white space or comments.  It keeps line-know information for syntax errors for highlighting where this syntax error  happened.  This is an example of Python's not visitor interface.  Like you can, whenever Python sees a constant note, it will call the visit constant handler.  Whenever it sees a name note, it will call the name handler so you can generate intermediate  representation here.  And after that ASD step, Python will construct code objects that contain byte code and some  information.  This code object is going to execute under C Python's virtual machine, which is a stack-based  virtual machine that works like you push something, you pop something with the instructions.  The instructions are formatted like there's an opcode and an oparc operation argument.  And there are some extra sequences.  Like if you use a lot constant opcode, you specify argument as an index of code const  data structure.  So it will make the Python faster.  Unfortunately, there are no direct interfaces to assemble byte code in Python itself.  The interface are assembled in Python, but there are interfaces for disassembling byte  code objects which we can use in JIT compiling the function's body.  This is an example function that takes X as an input and compares if X is greater than  4, returns power of 2, et cetera, et cetera.  And when we disassemble this, we see instructions like this.  The first one is a load test.  It loads the local variable from the argument spec.  And they are format like this.  This is jump target thing.  Python's stack-based virtual machine operates like this.  It checks if X is greater than 4.  If not, it pops directly here.  If it is true, it runs this place.  So this is a jump target from that return power of 3 line.  Yeah.  This was the workflow of C Python.  Let's see why we can't directly trust the user code because we won't give the types of  the arguments or any kind of name.  They are all dynamic and it will make our job harder.  User can even change built-ins like user can change the function or they can even put  handlers inside of class definition.  When you define a class, Python loads something called build class, which is a helper function.  And when you change a build class from the built-ins module, whenever someone defines  a class in Python, your function gets called.  And this makes things even more harder.  And compared to other languages, in Python, everything is missable unless you say otherwise.  Yeah.  Let's see some attempts from the past that tried to integrate LLVM with Python itself.  On latest wallow, it's a Google-sponsored fork of C Python that tried to make a JIT compiler  for C Python itself.  Its name comes from Monty Python show.  It tries to make Python faster.  It starts with fork of C Python.  And they plan to eventually merge this into the Python score, but then there are some  side effects which prevent this.  It was sponsored by Google and then Google lost its interest because this was a fork and  deployment was hard.  Google hasn't any performance critical code in Python and Google's clients don't want  to allocate money and time for this.  Its features, it made Python 2.5 times faster on single thread Python code.  It's single thread.  They didn't solve the global interpreter log problem.  In Python, if you have a CPU bound code, it can run, it runs one thread at a one time.  It doesn't matter if you have four threads.  It runs one thread because of some reference counting issues.  On latest wallow, didn't solve it, but it makes Python faster 2.5 times in single thread  code.  Their aim was five times, but there are some problems about reaching that aim.  They didn't break any Python code because they were already based on C Python itself.  So every C extension run as usual.  They analyzed Python code before running it.  So they know if you change the LAN function or not.  So that they make some assumptions, perform it on this and some data collected at runtime.  But if X function gets always integers, they said, okay, generate an intermediate representation  based on that fact.  But if it gets a string, there's a guard which prevents these assumptions from falling.  It converts bytecode.  It opales in the bytecode level instead of ASD, which is their choice.  There is a hot threshold in on latest wallow.  So there's a cost of making a function compared to LLVM.  So if a function called so many times and they know so many about it, they compile it  to LLVM.  And before that compilation, they collect data about that function.  They know if branch never executed, they just cut out it.  Or if they know all arguments are integer, they optimize for that.  But ensuring that everything works as same, even the arguments can change.  They put guards which has cost if mispron .  they worked with linking LLVM into the Python binary.  There are some cons, but besides of this,  one is memory usage.  The memory usage increased up to eight times.  It's not always eight times,  but the most part of it eight times.  This is one of the costs of LLVM and other libraries,  they increase memory usage.  If you keep different versions of same code in the memory,  it will increase by default.  So they keep the native code,  they keep the byte code, they keep the ASD.  So memory is increased.  There are some optimizer structures.  They tried to optimize Python byte code before sending to LLVM.  So they create some structure which keeps  Python's internal knowledge, but they cost some cons  about startup time, static linking again.  And when you want to compile a Python function to LLVM,  you need to initialize LLVM and C++ initialize,  take some time.  And there are some over-retops here on time routines.  Binary size increased as usual  because you link it to LLVM.  Why don't they reach 2.5 times,  they reached 2.5 times, but not five times?  Because when they was working, this project was from 2009,  LLVM isn't that good at just time compilation  and they had some issues with that.  And LLVM's G2 code doesn't work well with GDB.  If you are developing a C extension in Python,  you probably want to debug it with GDB or sec files  and use all profile.  And LLVM's G2 code didn't work well,  so they allocate some time to fix this,  but they lost some gain in performance.  This project lasted one year  and then Google lost its interest.  If the project continues, in my opinion,  they can reach the five times goal easily on some codes.  Some LLVM points about this,  they initially selected LLVM  because they don't want to generate assembly code  on every machine target.  LLVM was easy.  The optimization of code generation was great,  but in 2009, it isn't directly suitable for G-Tink.  And cost of operations compiling was so heavy.  And without collecting runtime data,  it is hard to optimize.  They was collecting after learning this,  so they spent some time to do this  without collecting runtime data.  The next project is PyLLVM,  which is a compiler for subset of Python  that specialize in machine learning,  algorithms and computations.  In Python, mathematical code gets executed slowly  because of there's a over-route of dynamic things.  If you compile this to LLVM,  you get so much gain and they try to do this.  They are free-shoes.  They are as fast as C code in some cases  that compiles with clang.  They only work for subset of Python.  If your function doesn't fit that subset,  they directly return your Python function.  If it fits, they turn a wrapper around the github code.  They don't collect runtime data,  but they do static type inference  and symbol table generation.  Python has a symbol table which you can use  that comes with standard library,  but it doesn't give you anything about types.  So you need to create your own symbol table  which records types and infers the types,  which could be cool.  They use something called LLVM Py,  which is a library, a wrapper library  around LLVM's API, C++ API.  They use Python ASD model instead of bytecode,  which is more suitable in my opinions.  They try to convert the function body,  and if they see something, they can convert,  they raise an error, and if they catch that error,  they return normal function.  If they don't see that error,  they just return the github function,  and they infer the types.  Let's see Piston.  Piston is a github compiler,  and the runtime for Python 2.  It's runtime most based on C-Pythons.  So they have some gain on that.  They use something called tracing giths,  and they specialize on web applications,  especially dropbox tests.  They manage to run Django's full test suite,  as far as I remember.  They feature as they are a 95% test of the C-Python  in the latest relays in 2017.  They have C-API support because of C-Python runtime,  and in a big codebase, you can run Piston  with little changes over the general codebase.  They have more than three executors.  Aluminum is the highest level, and there's VGIT,  and there's ASD interpreter.  So if your function gets called less,  they interpret the function's ASD.  If it is called more than 25 times,  they get with VGIT,  and if it's called more than 1000 times,  they get it with LLVM.  They have ASD-like control flow graphs,  and ASD interpreter to giths tracing,  and they use something called inline caching,  and rewriting of functions body,  and C-Python-based runtime.  So there are some projects from current dates,  Numba.  Numba is up-to-date project that uses LLVM to GIT,  it's a GIT compiler.  It's specialized in NumPy and Python functions.  It knows how to boot-shot these to assembly level,  and gets best performance, so it's a subset,  and it works only with type inference.  If it knows what types it gets, it directly GITs.  If it doesn't know, it records runtime data.  It's used with simple Python decorator.  You just add something at the top of the function  called the GIT, and Numba automatically handles  everything else.  It supports NVIDIA, CUDA, and AMD-Rug experimentally.  It's almost as fast as C in some cases,  which is pretty good.  It works like this.  It takes Python source, it uses bytecode to generate IR,  and then it converts that bytecode to their own  specialized intermincer representation.  They made some assumptions based on type inference,  and they generate LLVM IR and GIT afterwards. There's some  different modes in LLVM and Numba.  One is no-python mode.  No-python mode means this object never gets Python objects.  It always gets integers and other atomic types.  So if you use it, you get more performance.  It generates LLVM IR, and it optimizes it.  LLVM light has an interface to LLVM,  and you can optimize the bytecode.  It finds the functions.  In the future, we are going to see how we can write  our LLVM compiler.  This is our target function.  It takes A and B, and it uses Python's  annotation feature, which we don't handle with type inference.  You need to specify the types.  We are going to use AST, LLVM light,  the same library that Numba uses, and annotations.  This is the function.  We will have a compiler.  If we can convert the functions by the, we will convert.  If we can't, we will return the normal function,  so it will work in a subset of Python.  This is a basic compiler.  It will try to get function source code.  If it can't, it will return normal function.  And if it can, it will parse the source code into the AST,  and it will try to traverse that AST  using not visitor class, Python's AST return.  If any error happens during that traversing,  it will return normal function.  So if something doesn't fit our subset,  we will raise the compiler,  and if this part catches that error,  it will return the function.  This is the AST of that function.  It starts with a function definition.  There is an argument part, there is a body part,  the collector part, and return annotation.  The module is the highest level of any AST,  and we are going to generate an LLVM module  from the Python's AST, and set to south mode.  And generic visit is a function  that traverses the body of given node.  So we will give the module,  and it will traverse every node that lives under the module.  This is the function, if you want to recall.  The functions, first of all, we will remove the decorator  that is used to git our code.  So it won't be infinite loop.  Then we are going to generate an argument spec from that.  There's a cast LLVM function,  which is just checks the annotation  and converts an LLVM error.  And then we are going to get the return type.  So we can convert an LLVM function type from that.  It takes the return type and the values of arguments.  Then we are going to use that function type  to generate an LLVM function,  which will live under the module we have created.  It will take the function type,  and it will have the name of the node.  Then we are going to append the basic block,  which we can use that afterwards to get a builder  in that basic block.  So we are going to traverse the node's body one by one,  and use that builder to build our expressions.  The first expression is constants.  LLVM types doesn't directly fit to Python ones,  so we need to convert it.  Pythons integers doesn't have a specific bit length,  so we are going to use 32 for default,  and we are going to use floats for our float type, not double.  This is the cast LLVM function.  If the given type is integer,  we are going to return LLVM's int type.  If the float, we are going to return float type.  And if the type is something we don't know,  we will raise the compiler.  This is the part that constructs  intermediate representation, LLVM's constant,  with the type of the IR and the value of the nodes.  There's argument access part.  We will visit the name nodes,  and if we can fetch the argument  from the given functions type spec,  we will fetch it and return the value.  If we can't, we will raise the compiler  because it's something global or we don't know.  There's binary operations.  We will try to get the operator from the builder object.  If we can, we will raise the compiler  because some operator we don't know.  And then we will return the left side  and right side under that operator.  The return operation is pretty simple.  We will just call it method of builder  and return the given value,  which is going to be binary operation.  So we wrote a basic compiler with a compile class method,  a compiler error, if it doesn't fit subset  and visitors for module function return  binary operation name and constant.  The funny part is creating an execution engine.  We will create a default triple target  that's configured to LLVM and create a target machine.  In the creating target machine,  you can specify the optimization level  or anything you want.  Then we will create the execution engine  which takes bootstrapping assembly.  We don't have anything to bootstrap and target machine  and we will return the machine, engine.  This is the giting part.  We will create the engine with this method  and then parse the R module and verify it  if it's a valid, intermittent representation or not.  Then we will add that module to our engine,  the finalize and run the static constructors.  And then we will get the function  as a C types function pointer.  So at the end, we will fetch that functions pointer  and cast it to C types C fun time.  Python offers a way to access C functions with C types.  So there's a C type C fun type which takes a type set  and the type spec is given here  and we will cast the function pointer to that.  We did this with ASD.  So can we do this same thing with bytecode?  Yes, we will have something like this  to decorate our function and we will have that same part.  We will keep a stack of the functions bytecode  because Python's visual machine is bytecode,  it's stack based.  So we will, this is the opcode by one by one.  If the opcode is something we know,  we will add the stack.  If we don't know, we will break the for loop  and just turn to normal function.  Loading names is simple.  If we see a load fast bytecode,  we will return the given arguments to stack.  If we see a load const,  we will cast it to LLVM and append to stack.  If we see a binary operation,  we will pop two things from the stack  and create an operation from it.  And if we see a return value, we will return.  This is the whole code to construct that IR.  Yeah, this is it.  Thank you for listening.  Is there any questions?  Is there any questions?  Anyone has a question, please raise your hand.  Yeah, okay.  Okay.  So we will present things for projects  that pay to the extent to compile,  open code into native code.  And can you make explicit what makes your position different?  What are the things that are different in your approach  than the one that's where in Python?  Okay, you are asking what are things different  in my JIT compilers other than the projects I presented?  Okay, I need to go to the outline  to list other projects.  Okay, this will...  Okay, there was four projects.  Unladen Swallow was using bytecode.  The bytecode I am using in the second approach.  So Unladen Swallow's approach and my approach is almost same.  Pyaladame uses ASD and in my first approach,  I used ASD to use that same approach.  Piston and Numba also uses bytecode.  So mine isn't different than others,  but I tried to simplify the process of there in my examples.  In both ways.  I have several questions.  And so, typing issue is a really issue for Python.  And it seems to me that in your approach,  you're delegating that to the user.  Yeah.  To the type annotation.  So that's actually a very big difference  compared to the other approach.  You are not taking care of type inference,  which makes things much easier.  But when you annotate a variable with it,  it means Python int, which is actually big int.  And you are casting them to 32 bits with the girl,  which is a totally different world.  Yeah, I mentioned that.  LLVM types doesn't directly fit the Python type.  So for the sake of easily,  I casted it to 32 bit integer.  But in a real world application,  it would be nonsense to cast 32 bit int.  Yeah, or you could, for instance,  using C types can exhibit 32 bits integer.  Yeah.  And you can ask the user to specify that it's okay.  But hardware integers instead.  Well, Python's typing doesn't allow to specify integers.  There's literals, but there are nothing  to specify the bit length of the integer  in type annotations.  So it would be hard to use or to define that.  How much of a speed up is your compiler?  How much of a speed up is your compiler?  My compiler gets around to 20% speed  because I don't do any optimizations,  any, except I think I just convert the direct Python  to LLVM IR and it gets 20% speed up.  What stops you from doing optimizations?  Why are you not doing running the optimizer?  Because for the sake of presentation,  I want to keep everything simple.  As you're using type annotations,  you don't need a jit at all.  You could do that statically.  You just take the function and perform your static compilation  and then run the binary into your Python life.  And that's okay.  So why would you use a jit in your situation  where you only use static information?  You are asking why I'm using jit  when there is already defined type annotations.  I can just convert to functions  by the ahead of time population.  Because in the approach taken by like unladen swallow,  they record runtime data.  I want to keep that in the slides, but it doesn't fit.  So I have a project called Saving Inhead  which takes the exact thing like unladen swallow  and it records runtime data.  So you are right.  In this way, it would make sense to not running jit,  but if you are going to record the runtime data  or do other things, you need to use jit.  Any more questions?  When are you using the byte code one versus the AST based?  To me, AST is much more simpler to work on  because you don't need the extra bootstrap and code  Python offers an interface to access it.  But I want to show how unladen swallow and Pstone worked.  So I'd like an example to byte code.  You say the AST one is the future.  You say using the AST by this one is the future.  Yeah.  One of the shortcomings presented by the Python Pstone  worker was that most of the Python code is actually  just glue calling native functions.  That's true for system library, but that's also true  for the package lightning by.  And it presents a very strong barrier to the optimizer.  And so did you take that into account?  Or when you see a call to a native function,  you're just calling it the Python way  and getting back the results?  You're asking if I am optimizing native functions  with bootstrapping some AST?  Numba does that, but my jit compiler  doesn't do that.  OK.  Thank you.  OK, we thank the speaker again.  Thank you.
llvm_python_0_143.png llvm_python_144_2051.png llvm_python_2052_2711.png llvm_python_2712_3359.png llvm_python_3360_4211.png llvm_python_4212_5819.png llvm_python_5820_7487.png llvm_python_7488_8387.png llvm_python_8388_10235.png llvm_python_10236_12827.png llvm_python_12828_13019.png llvm_python_13020_14603.png llvm_python_14604_17843.png llvm_python_17844_19655.png llvm_python_19656_24695.png llvm_python_24696_25823.png llvm_python_25824_28271.png llvm_python_28272_29231.png llvm_python_29232_30599.png llvm_python_30600_31739.png llvm_python_31740_32555.png llvm_python_32556_33011.png llvm_python_33012_33167.png llvm_python_33168_33719.png llvm_python_33720_34055.png llvm_python_34056_34679.png llvm_python_34680_34787.png llvm_python_34788_36515.png llvm_python_36516_38183.png llvm_python_38184_38627.png llvm_python_38628_39203.png llvm_python_39204_40607.png llvm_python_40608_41291.png llvm_python_41292_41483.png llvm_python_41484_41699.png llvm_python_41700_42419.png llvm_python_42420_42683.png llvm_python_42684_42947.png llvm_python_42948_44483.png llvm_python_44484_44531.png llvm_python_44532_44591.png llvm_python_44592_44639.png llvm_python_44640_44699.png llvm_python_44700_52871.png llvm_python_52872_52993.png