DjangoCon US 2018 - Keynote
CC Attribution - ShareAlike 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this
Wijaya, Mariatta
Managing a large open source project like CPython is no easy task. Learn how the Python core team automated their GitHub workflow with bots, making it easier for maintainers and contributors to collaborate together. Even if you’re not managing a large project, you can still build your own bot! Hear some ideas on what you can automate on GitHub and personalize your bot based on your own workflow. All you need is Python. Don’t be a robot; build the bot.
 Thank you so much for having me here. It's really an honor to be keynoting at DjangoCon. The Django community here is really special to me. My first conference talk ever was DjangoCon in Philadelphia only two years ago. The year after that I spoke at seven conferences, five countries, three continents. It's awesome, so I really feel excited to be back here. Thank you so much for inviting me as a keynote speaker. Thank you. Before I start, I have a few brief announcements. I'm from Japan. I'm a Japanese hiring. Go there. My Cascades is a conference happening in Seattle in February and CFP is still open. Please submit a talk. Feel free to follow me on Twitter, but my next tweet probably about my ice cream selfie. Today I'm here to talk to you about Core Python Development. I'm a Python Core developer. There is this one question that I receive again and again, and I think every Core developer will receive it too. That is, when can you merge my pool requests? I'm sorry. There's just today there are close to 1,000 open pool requests to see Python. The problem is we're outnumbered. There are only so few of us who can merge your pool requests. Meanwhile, there are like hundreds, thousands of you contributing, and I really, really appreciate all your contributions. They are welcome, but I'm sorry. We just can't possibly go through all of them quickly. We don't have enough time. I'm sorry. I'm Canadian. The other problem is our workflow is pretty complicated. For every single one pool request that comes, we need to ask all of these questions, bug or feature. It's really important. Did you sign the CLA? Did you write tests? All the tests passed? Do you, is there an issue created? Somebody actually asked for this change? You write a news entry, blah, blah, blah, backport. It's very complicated, and we do this for every single pool request. Today, I just want to focus on one of these issues, the issue with backporting. Perhaps you don't even know what is backport, and why is it even important for Python? Backporting is the act of applying changes from the newer version software to the older versions of the same software. Perhaps it will help if you realize that Python still maintains these different versions. The master branch will become Python 38. The alpha release is due January. 371 and 367 still maintained. We still accept bug fixes to those, and it's due to be out October 20th. Backport is how you will receive 371, 367. Without backporting, you will not get that. It's a very important process in our development. Let's think about this scenario. Say you're using Python, you're using fstrings, Python 36, and then you discover bug, maybe there's a problem with the documentation or whatnot. You come to the bug tracker and tell me, hey, there's a problem with fstrings, and because it's fstrings, you thought, all right, this applies to Python 36. Then you decided to help and fix the bug, making a pull request to the 36 branch. But that's not how it works. This happens all the time. People making pull requests to the wrong branch, and we have to tell them, no, close that. Please read the death guide. This is how it really works. You discover a bug in fstrings, that's fine. Please do tell us about it. But then we're going to tell you that, all right, this applies to all these branches. But first, please make the pull request to the master branch. We're going to review it, criticize it, make sure you sign the CLA, make sure you have tests, write the docs, and everything. Once we're happy with that, we'll merge it. Only after we merge it, we'll backport it. Okay, sounds simple, right? Backport it. Now, let's go to figure out how this happens. This is how we used to do it, to backport a single commit. We need to type out all of these lines. Get fetch, create a branch, try to pick, push it to master, push it to origin, check out another branch, delete it, stop, leave the terminal, go to the GitHub, create the pull request, and go back and do all of it again. It's tedious. It's a lot of work. You're typing a lot of things, and it's a boring, repetitive task. So thankfully, we figured out a way to make it a little bit better. So I wrote this utility script called cherry picker, and it's on PyPI. So all we need to do is install this. Instead of typing out all those really long lines, we type one line. Great. And it takes care of all those boring details, the leading branch, creating branch. It even opens up the web browser, pointing to the GitHub pull request creation. So it's much simpler. Can you think that we'll be happy with this? I was not. I have problems. Problem is, I have to be on a computer. That's a blocker for me. If I was able to review pull request on a phone, why do I have to move and wait until I go to my computer to finish the backboard? I want to be able to do it from, like, if I'm vacationing on a beach, on my phone, reviewing pull request, let's merge it back for that. I don't want to be on the computer. That's a problem. And because sometimes I have to wait until I'm back on the computer, I forgot to do it. The backboards were missed and people started asking me to do it. The other problem is I started receiving bug reports. One people complained that it was a pain having to install Cherry Pickle because they didn't have Python 3.6 yet. It's a pain for them to install Python 3.6. They're missing out on f-strings. And the other problem is some people use it on Windows and it's actually great. Python on Windows is awesome. The problem is me. I don't have Windows machines, so I don't know how to help them, how to support them. So it is a problem for me. But even though it's simple, it's still just boring. I do not get smarter running Cherry Pickle. I don't earn anything. It's not intelligent activity. It's boring. And I don't feel it's worth my time doing. But it's important. We need it. And in fact, it was during Python in Portland last year, I spoke at another contributor and he asked the same thing. Like, when will you review my full request and merge it? And I'm sorry. I think I made up excuses. I mumbled a bit about backporting. And it was him who suggested that, okay, how about calling me? How about, don't worry about this. Let contributors, hundreds of contributors take care of this for you. Let us do the backport. And I feel like it sounds like a good idea at first. But I don't think your time is any less valuable than mine. If I'm not willing to do this, I don't think it's worth my time. Why should I ask you to do it? So it's time to build the bot. Automate this. The sooner we have the bot, the better. Problem is I didn't actually know how to build the bot at that time. I actually was waiting for another core developer to build the bot. And we needed it. So I thought maybe it's a good opportunity for me to learn how to build a bot. So I started learning. I started reading GitHub developer guides. And I learned about webhooks. So basically you can configure your repository. You can add webhooks to your GitHub repo and let GitHub notify you whenever there's something happen inside your repo. It can send you events like, hey, there's a new pull request. Somebody open it, somebody close it, or it can tell you about the changing status. If the commit status change to pending success, failure, all that, it can tell you about pull request review was submitted, dismissed. There's a lot of different events that you can subscribe to. So when GitHub sends you all these webhook events, it will actually make a post request to a certain URL. You just need to tell GitHub where to send that post request. So that's where your bot will be. You will be creating a web service, anything you want, plus Django, but it's a web service with a URL for GitHub to send that request. Now for Python, of course, we're going to build it with Python. And pretty much in PSF's infrastructure, I run on Heroku. So that's what I started learning, like how to build things, web services with Python on Heroku. So this is how the webhooks work. That's one direction. The other direction is how do you want to make action to GitHub from your web service? Maybe you want to be able to create pull requests. You want to be able to merge pull requests, leave comments, apply labels, and all that. Turns out there are a lot of things you can do. Whatever you can do on GitHub, on the web, by yourself, there is an API for it. So it got me really excited. So I thought, okay, now I think I understand how to build a bot. And I started thinking like this is how it will work. I will subscribe to the PR Merch event, have my bot run Cherry Pickle, and then I'm going to open a pull request with the REST API. Sounds simple. But remember, Cherry Pickle was really running Git Cherry Pick. So it needs a copy. It needs to first clone CPython repo, right? Remember CPython is 27 years old. It's big. Cloning CPython takes two minutes at least. It's long. And then I discovered something. In Heroku, request signs out after 30 seconds. So what do I do here? I didn't know. I've never built a bot. I never do anything with Heroku. So I checked with Ernest, PSF infrastructure. He works for infrastructure for PSF. He's the director of infrastructure. And I thought maybe we need to set up our own server, not Heroku, something that's not limited by 30 seconds of time or so or anything. And he said, yes, totally, we can do this in Heroku. You will use salary, do it in the background dyno, in a worker dyno as a background task. And there is no time to sell it on it. All right. Okay. I'm learning Heroku. I'm learning about GitHub API. So now I don't know how salary works. I guess I will learn about it too. So he sent me documentation to salary. It was straight forward. I was able to figure out quickly. So this is how it works now. This is the bot. I will have a web service. All it does is it receives the web hook event. And then it's going to just start a background task on salary. That is so the background test is we'll do all those long running things like cloning C Python, running cherry pickers, opening full request happens in the background. This bot is called Ms. Islington. So Ms. Islington is open source. You can check out the code. It is 536. It uses AIO HTTP, which is an async web server and client. The real magic that let building bot working with GitHub API easy is GitHub. GitHub is a library maintained by Brad Cannon, who is another Python core developer. This is the library you want to use if you want to build your own integration with GitHub. You want to build bot with Python, 3.6 and above. Use GitHub. Using the diagram, it seems straightforward. This is the actual code from Ms. Islington. From top to bottom, I will tell it when I receive the pull request close event. That's the first line. I'm going to check if the pull request has been merged. That's the if statement. From there, I will receive the commit hash. I will know which branches. If we tell it to backport, it will check the labels. And then basically in the end, that's the backport task delay. That's the command to start a salary task. It's simple. It's easy. This is possible because of GitHub. And when we built the bot, we deployed it. It actually worked. And it was exciting. This is the very first time ever three of our bots talked to each other. Automatically. I didn't realize this to work, actually. I didn't expect it to work because I didn't wrote any unit tests. I tested in production. I didn't know how to write tests at the time. But it worked. And it's like magic. And now like all the problems I used to have with Cherry Pickle, it's gone. I can do this from my phone. I no longer forget about it. I don't receive back reports anymore because I only need to make sure it works with the bot. And it's no longer boring because it's fun. But now there's another problem. It's still taking time. The time is related to part of our other workflow, waiting for tests to pass. C Python runs lots of tests and status checks like this. We use Travis, AppPierre, Azure Pipelines. Waiting for all of this to finish on good days. Takes at least 30 minutes for one peer. It's long. I'm not going to wait for this. So I need to do something else, try to come back to it later, check if it's finished. If not, okay, wait again. Come back again. Is it done? And so on. It's boring. It's long. That's a person waiting for time to pass. Spider web starts to form. Would it be nice if I have a bot that waits for this and just tell me when it's done? Like that, right? So yes, we can totally do this. We have this. Spoiler. It works. This is how I got inspired and I started figuring out how to build a bot that does this. So basically there is the status event where you can subscribe to. And then it will give you the commit hash. And you can make another API call to get the combined status, all the status of the single commit and then you can post a comment saying that whether it's finished or not. Now here comes one problem as I was trying to build this. I didn't know which pull request that commit comes from that web hook event in the first that didn't tell me which pull request is from. Just the commit hash. And it took me time to figure it out. I searched in the pull request documentation. I searched in issues. Documentation could not figure out which pull request contains this commit hash. The answer was buried hidden in the search issues API. So it actually says this. You can use. If you know the commit you can use the search issues API to search pull request. So I wrote out this utility function. It's pretty straightforward. That's the URL. Basically just pass in the commit hash, make a request, use engage your hub and I get it. So it looks more like this now. I received the web hook event. I'm going to get combined status. Now I know it. If it's finished I'm going to figure out the pull request containing that and then post a comment in that pull request. So that's how it works. And now all my problems really, really gone. Not only I figured out how to take care of the backboard waiting for test to pass is no longer boring. I feel so accomplished. My life is complete. Yay. Let's try it. Let's figure out how to fix this other issues. And this email came from another code developer. Victor. He said, hello. What if we let Mrs. Lington merge the pull request on her cell? Sure, you want a bot that merge pull request to see Python? You're asking can we give Python commit bit to a bot? We don't even do this easily to people, real intelligent human. Doing this to a bot. I guess as long as it's written Python with F strings maybe it's okay. So we did it. We built a bot that merge code into see Python and it is written with Python. This is the code with using GitHub, async.io. This is how we merge code to see Python. You just need to, you call that API, ghput. You're making a put request to GitHub, pass the commit title, commit message, squash, commit hash. That's all. Simple. That's how we did it. So now our workflow looks like this. If you realize just how little core developers are doing now. Let the bot does all the boring things. And this is awesome. Like now core developers get to spend more time doing the important things. Like actually reviewing pull request. They're not bothered by waiting for your task to pass. They don't even merge it. Let the bot do it. And this time I realized something. First I didn't expect it. We would ever want to let a bot do this. But now we have it. We can't live without it. If we are back to this, everybody is going to be mad and like telling me enable miss Euclid and again. We thought we were happy with just this. This is what we wanted in the first place. But this once we learn more of how to build more automation, more bots. We really can't live without it. We let it. I know some other people now asking me to do more bots for C Python. We're spoiled. But now that I know how to build bots, I started asking myself. All right. I realized how simple it was. I started asking what other bots can I build? I'm sure I can find more boring things in my life and let the bot do it for me. In fact, I now get grumpy when I know that bots can do this. Why am I still doing this? So one of the bots I'm still working is called blackout. This is a project I want to finish soon. Black is an uncompromising code for matter for Python. It first came out. I got excited. I love it. I want this. I want black everywhere I can. I want integrated in my life. Then I read the instruction on how to integrate black into my life. There is this long list of instructions. If you use this ID, download this ID, if you use that ID, download this plugin, and I'm pre-commit, configure it, and I'm like just the mere thought of, God, I need to download something. No. What if I have more than one computer? I have to download the same thing again? I don't want to do it. So I wrote a bot that runs black. So basically I don't even need to install black again or update it on my computer. I just tell the bot, okay, I apply the label blackout. The bot formats it for me. That's it. Nobody needs to argue. Nobody needs to install plugin or whatever. Let the bot do it. I have this working. I'm still not fully satisfied with how it works. So this is my next project. If you're interested, that's the source code, blackout, Marriota blackout. I use this other bot called PyApp. And it's a service that helps keep my Python dependencies up to date. And it's really useful. I recommend you all use it. It's free for open source projects. So when one of your dependencies has an update, they made a new release to PyPI. You get a pull request updating your requirements. It's great. The problem is I have to wait for the test to pass before I can merge it. And then I have to merge it and delete the branch. If a bot can create the pull request, why can't it merge it itself? Why do I have to wait for this? In my mind, throughout this whole workflow, the only thing that requires human intelligence that requires my involvement is reviewing the change log. And to be honest, I don't even review change log anymore. As long as it passes, my coverage didn't drop. Merge it, right? I know a bot can merge this. So this is an actual open pull request in Mrs. LinkedIn. And purposely, I don't want to merge this because I refuse to press that green button because I know a bot can do it. So if PyApp doesn't provide auto-merge ability soon enough, I'm going to build my own bot that just merge and delete the branch. Build the bot. This is an idea that I have. I've been working on adding the PSS code of conduct to some of our GitHub repositories. So I added the code of conduct to Cpython, to the DevGuide, to the core workflow. By working on this, I mean I copy paste things. And you know, it's boring. Basically, I create the same pull request again and again. I think a bot could totally do this. There should be a bot that can go through every public repost under Python organization and add the code of conduct. I think it's important to have. So I wish there's such a bot that can just create this for me, merge it itself. Delete the branch. I don't need to be bothered because a bot can totally do this. Since last month, I've been taking a break from any volunteering activities, not contributing to open source since September, yes, September, and also this month. So I figured maybe I should have a similar to out of office auto reply. I will have an out of office, out of open source auto reply on GitHub. So if you all mentioned me on GitHub this month, my bot will tell you that I'm sorry, Mary was not going to look at this until November. I didn't want to include this here just because it may be seen as product pitch. A lot of people asking me how they can set this up for themselves. This is done by Zapier. This is a multi-step by Zapier. It's triggered by if there's a new mention of me, I still have to filter it like if the mention is outside of Zapier. So if my co-worker mentioned me, it's fine, but everybody else, they'll get this message. I still get to run Python. There's a step for me that I need to extract the pull request number where the mention happened and then I tell Zapier to create the comment. So if you're interested, talk to Zapier, try out Zapier. Dave Bisley needs a bot and he didn't even realize it. Earlier this year he asked on Twitter, is there an easy way to allow issues but not pull request? And he had good reason for doing this. I think one of the reply was moved to something else that doesn't sound really easy. You can totally do a bot that takes care of this. It's as simple as this, receive the PR event, close it. Much simpler than this is LinkedIn. I know this can be done. I wrote it. Those are the lines of code you need to write in order to accomplish this. It's easy. So I hope that I've given you ideas to automate your life. I mean, I've used GitHub examples, GitHub bot examples throughout this talk just because that's where I spent my life most of the time. But don't be limited to just GitHub bots. Automate your life. Don't be a robot, build a bot. And if it is GitHub bot you're after. I did wrote a tutorial telling you how to do it. Go there and GitHub bot tutorial. But yeah, automate. Don't do the boring things. You know, look at your own life. I'm sure you will find even more boring things. So thank you so much. My name is Mariera. Thanks for coming to my talk. I hope you enjoyed it. Follow me on Twitter or if you would like to get in touch, can email me mariera at python.org. Thank you so much. Thank you so much and see you next time.
 Thank you so much for having me here.  It's really an honor to be keynoting at DjangoCon.  The Django community here is really special to me.  My first conference talk ever was DjangoCon in Philadelphia only two years ago.  The year after that I spoke at seven conferences, five countries, three continents.  It's awesome, so I really feel excited to be back here.  Thank you so much for inviting me as a keynote speaker.  Thank you.  Before I start, I have a few brief announcements.  I'm from Japan.  I'm a Japanese hiring.  Go there.  My Cascades is a conference happening in Seattle in February and CFP is still open.  Please submit a talk.  Feel free to follow me on Twitter, but my next tweet probably about my ice cream selfie.  Today I'm here to talk to you about Core Python Development.  I'm a Python Core developer.  There is this one question that I receive again and again, and I think every Core developer  will receive it too.  That is, when can you merge my pool requests?  I'm sorry.  There's just today there are close to 1,000 open pool requests to see Python.  The problem is we're outnumbered.  There are only so few of us who can merge your pool requests.  Meanwhile, there are like hundreds, thousands of you contributing, and I really, really  appreciate all your contributions.  They are welcome, but I'm sorry.  We just can't possibly go through all of them quickly.  We don't have enough time.  I'm sorry.  I'm Canadian.  The other problem is our workflow is pretty complicated.  For every single one pool request that comes, we need to ask all of these questions, bug  or feature.  It's really important.  Did you sign the CLA?  Did you write tests?  All the tests passed?  Do you, is there an issue created?  Somebody actually asked for this change?  You write a news entry, blah, blah, blah, backport.  It's very complicated, and we do this for every single pool request.  Today, I just want to focus on one of these issues, the issue with backporting.  Perhaps you don't even know what is backport, and why is it even important for Python?  Backporting is the act of applying changes from the newer version software to the older  versions of the same software.  Perhaps it will help if you realize that Python still maintains these different versions.  The master branch will become Python 38.  The alpha release is due January.  371 and 367 still maintained.  We still accept bug fixes to those, and it's due to be out October 20th.  Backport is how you will receive 371, 367.  Without backporting, you will not get that.  It's a very important process in our development.  Let's think about this scenario.  Say you're using Python, you're using fstrings, Python 36, and then you discover bug, maybe  there's a problem with the documentation or whatnot.  You come to the bug tracker and tell me, hey, there's a problem with fstrings, and because  it's fstrings, you thought, all right, this applies to Python 36.  Then you decided to help and fix the bug, making a pull request to the 36 branch.  But that's not how it works.  This happens all the time.  People making pull requests to the wrong branch, and we have to tell them, no, close that.  Please read the death guide.  This is how it really works.  You discover a bug in fstrings, that's fine.  Please do tell us about it.  But then we're going to tell you that, all right, this applies to all these branches.  But first, please make the pull request to the master branch.  We're going to review it, criticize it, make sure you sign the CLA, make sure you have  tests, write the docs, and everything.  Once we're happy with that, we'll merge it.  Only after we merge it, we'll backport it.  Okay, sounds simple, right?  Backport it.  Now, let's go to figure out how this happens.  This is how we used to do it, to backport a single commit.  We need to type out all of these lines.  Get fetch, create a branch, try to pick, push it to master, push it to origin, check out  another branch, delete it, stop, leave the terminal, go to the GitHub, create the pull  request, and go back and do all of it again.  It's tedious.  It's a lot of work.  You're typing a lot of things, and it's a boring, repetitive task.  So thankfully, we figured out a way to make it a little bit better.  So I wrote this utility script called cherry picker, and it's on PyPI.  So all we need to do is install this.  Instead of typing out all those really long lines, we type one line.  Great.  And it takes care of all those boring details, the leading branch, creating branch.  It even opens up the web browser, pointing to the GitHub pull request creation.  So it's much simpler.  Can you think that we'll be happy with this?  I was not.  I have problems.  Problem is, I have to be on a computer.  That's a blocker for me.  If I was able to review pull request on a phone, why do I have to move and wait until  I go to my computer to finish the backboard?  I want to be able to do it from, like, if I'm vacationing on a beach, on my phone, reviewing  pull request, let's merge it back for that.  I don't want to be on the computer.  That's a problem.  And because sometimes I have to wait until I'm back on the computer, I forgot to do it.  The backboards were missed and people started asking me to do it.  The other problem is I started receiving bug reports.  One people complained that it was a pain having to install Cherry Pickle because they didn't  have Python 3.6 yet.  It's a pain for them to install Python 3.6.  They're missing out on f-strings.  And the other problem is some people use it on Windows and it's actually great.  Python on Windows is awesome.  The problem is me.  I don't have Windows machines, so I don't know how to help them, how to support them.  So it is a problem for me.  But even though it's simple, it's still just boring.  I do not get smarter running Cherry Pickle.  I don't earn anything.  It's not intelligent activity.  It's boring.  And I don't feel it's worth my time doing.  But it's important.  We need it.  And in fact, it was during Python in Portland last year, I spoke at another contributor  and he asked the same thing.  Like, when will you review my full request and merge it?  And I'm sorry.  I think I made up excuses.  I mumbled a bit about backporting.  And it was him who suggested that, okay, how about calling me?  How about, don't worry about this.  Let contributors, hundreds of contributors take care of this for you.  Let us do the backport.  And I feel like it sounds like a good idea at first.  But I don't think your time is any less valuable than mine.  If I'm not willing to do this, I don't think it's worth my time.  Why should I ask you to do it?  So it's time to build the bot.  Automate this.  The sooner we have the bot, the better.  Problem is I didn't actually know how to build the bot at that time.  I actually was waiting for another core developer to build the bot.  And we needed it.  So I thought maybe it's a good opportunity for me to learn how to build a bot.  So I started learning.  I started reading GitHub developer guides.  And I learned about webhooks.  So basically you can configure your repository.  You can add webhooks to your GitHub repo and let GitHub notify you whenever there's  something happen inside your repo.  It can send you events like, hey, there's a new pull request.  Somebody open it, somebody close it, or it can tell you about the changing status.  If the commit status change to pending success, failure, all that, it can tell you about pull  request review was submitted, dismissed.  There's a lot of different events that you can subscribe to.  So when GitHub sends you all these webhook events, it will actually make a post request  to a certain URL.  You just need to tell GitHub where to send that post request.  So that's where your bot will be.  You will be creating a web service, anything you want, plus Django, but it's a web service  with a URL for GitHub to send that request.  Now for Python, of course, we're going to build it with Python.  And pretty much in PSF's infrastructure, I run on Heroku.  So that's what I started learning, like how to build things, web services with Python on  Heroku.  So this is how the webhooks work.  That's one direction.  The other direction is how do you want to make action to GitHub from your web service?  Maybe you want to be able to create pull requests.  You want to be able to merge pull requests, leave comments, apply labels, and all that.  Turns out there are a lot of things you can do.  Whatever you can do on GitHub, on the web, by yourself, there is an API for it.  So it got me really excited.  So I thought, okay, now I think I understand how to build a bot.  And I started thinking like this is how it will work.  I will subscribe to the PR Merch event, have my bot run Cherry Pickle, and then I'm going  to open a pull request with the REST API.  Sounds simple.  But remember, Cherry Pickle was really running Git Cherry Pick.  So it needs a copy.  It needs to first clone CPython repo, right?  Remember CPython is 27 years old.  It's big.  Cloning CPython takes two minutes at least.  It's long.  And then I discovered something.  In Heroku, request signs out after 30 seconds.  So what do I do here?  I didn't know.  I've never built a bot.  I never do anything with Heroku.  So I checked with Ernest, PSF infrastructure.  He works for infrastructure for PSF.  He's the director of infrastructure.  And I thought maybe we need to set up our own server, not Heroku, something that's not  limited by 30 seconds of time or so or anything.  And he said, yes, totally, we can do this in Heroku.  You will use salary, do it in the background dyno, in a worker dyno as a background task.  And there is no time to sell it on it.  All right.  Okay.  I'm learning Heroku.  I'm learning about GitHub API.  So now I don't know how salary works.  I guess I will learn about it too.  So he sent me documentation to salary.  It was straight forward.  I was able to figure out quickly.  So this is how it works now.  This is the bot.  I will have a web service.  All it does is it receives the web hook event.  And then it's going to just start a background task on salary.  That is so the background test is we'll do all those long running things like cloning  C Python, running cherry pickers, opening full request happens in the background.  This bot is called Ms. Islington.  So Ms. Islington is open source.  You can check out the code.  It is 536.  It uses AIO HTTP, which is an async web server and client.  The real magic that let building bot working with GitHub API easy is GitHub.  GitHub is a library maintained by Brad Cannon, who is another Python core developer.  This is the library you want to use if you want to build your own integration with GitHub.  You want to build bot with Python, 3.6 and above.  Use GitHub.  Using the diagram, it seems straightforward.  This is the actual code from Ms. Islington.  From top to bottom, I will tell it when I receive the pull request close event.  That's the first line.  I'm going to check if the pull request has been merged.  That's the if statement.  From there, I will receive the commit hash.  I will know which branches.  If we tell it to backport, it will check the labels.  And then basically in the end, that's the backport task delay.  That's the command to start a salary task.  It's simple.  It's easy.  This is possible because of GitHub.  And when we built the bot, we deployed it.  It actually worked.  And it was exciting.  This is the very first time ever three of our bots talked to each other.  Automatically.  I didn't realize this to work, actually.  I didn't expect it to work because I didn't wrote any unit tests.  I tested in production.  I didn't know how to write tests at the time.  But it worked.  And it's like magic.  And now like all the problems I used to have with Cherry Pickle, it's gone.  I can do this from my phone.  I no longer forget about it.  I don't receive back reports anymore because I only need to make sure it works with the  bot.  And it's no longer boring because it's fun.  But now there's another problem.  It's still taking time.  The time is related to part of our other workflow, waiting for tests to pass.  C Python runs lots of tests and status checks like this.  We use Travis, AppPierre, Azure Pipelines.  Waiting for all of this to finish on good days.  Takes at least 30 minutes for one peer.  It's long.  I'm not going to wait for this.  So I need to do something else, try to come back to it later, check if it's finished.  If not, okay, wait again.  Come back again.  Is it done?  And so on.  It's boring.  It's long.  That's a person waiting for time to pass.  Spider web starts to form.  Would it be nice if I have a bot that waits for this and just tell me when it's done?  Like that, right?  So yes, we can totally do this.  We have this.  Spoiler.  It works.  This is how I got inspired and I started figuring out how to build a bot that does this.  So basically there is the status event where you can subscribe to.  And then it will give you the commit hash.  And you can make another API call to get the combined status, all the status of the single  commit and then you can post a comment saying that whether it's finished or not.  Now here comes one problem as I was trying to build this.  I didn't know which pull request that commit comes from that web hook event in the first  that didn't tell me which pull request is from.  Just the commit hash.  And it took me time to figure it out.  I searched in the pull request documentation.  I searched in issues.  Documentation could not figure out which pull request contains this commit hash.  The answer was buried hidden in the search issues API.  So it actually says this.  You can use.  If you know the commit you can use the search issues API to search pull request.  So I wrote out this utility function.  It's pretty straightforward.  That's the URL.  Basically just pass in the commit hash, make a request, use engage your hub and I get it.  So it looks more like this now.  I received the web hook event.  I'm going to get combined status.  Now I know it.  If it's finished I'm going to figure out the pull request containing that and then post  a comment in that pull request.  So that's how it works.  And now all my problems really, really gone.  Not only I figured out how to take care of the backboard waiting for test to pass is no  longer boring.  I feel so accomplished.  My life is complete.  Yay.  Let's try it.  Let's figure out how to fix this other issues.  And this email came from another code developer.  Victor.  He said, hello.  What if we let Mrs. Lington merge the pull request on her cell?  Sure, you want a bot that merge pull request to see Python?  You're asking can we give Python commit bit to a bot?  We don't even do this easily to people, real intelligent human.  Doing this to a bot.  I guess as long as it's written Python with F strings maybe it's okay.  So we did it.  We built a bot that merge code into see Python and it is written with Python.  This is the code with using GitHub, async.io.  This is how we merge code to see Python.  You just need to, you call that API, ghput.  You're making a put request to GitHub, pass the commit title, commit message, squash, commit  hash.  That's all.  Simple.  That's how we did it.  So now our workflow looks like this.  If you realize just how little core developers are doing now.  Let the bot does all the boring things.  And this is awesome.  Like now core developers get to spend more time doing the important things.  Like actually reviewing pull request.  They're not bothered by waiting for your task to pass.  They don't even merge it.  Let the bot do it.  And this time I realized something.  First I didn't expect it.  We would ever want to let a bot do this.  But now we have it.  We can't live without it.  If we are back to this, everybody is going to be mad and like telling me enable miss  Euclid and again.  We thought we were happy with just this.  This is what we wanted in the first place.  But this once we learn more of how to build more automation, more bots.  We really can't live without it.  We let it.  I know some other people now asking me to do more bots for C Python.  We're spoiled.  But now that I know how to build bots, I started asking myself.  All right.  I realized how simple it was.  I started asking what other bots can I build?  I'm sure I can find more boring things in my life and let the bot do it for me.  In fact, I now get grumpy when I know that bots can do this.  Why am I still doing this?  So one of the bots I'm still working is called blackout.  This is a project I want to finish soon.  Black is an uncompromising code for matter for Python.  It first came out.  I got excited.  I love it.  I want this.  I want black everywhere I can.  I want integrated in my life.  Then I read the instruction on how to integrate black into my life.  There is this long list of instructions.  If you use this ID, download this ID, if you use that ID, download this plugin, and I'm  pre-commit, configure it, and I'm like just the mere thought of, God, I need to download  something.  No.  What if I have more than one computer?  I have to download the same thing again?  I don't want to do it.  So I wrote a bot that runs black.  So basically I don't even need to install black again or update it on my computer.  I just tell the bot, okay, I apply the label blackout.  The bot formats it for me.  That's it.  Nobody needs to argue.  Nobody needs to install plugin or whatever.  Let the bot do it.  I have this working.  I'm still not fully satisfied with how it works.  So this is my next project.  If you're interested, that's the source code, blackout, Marriota blackout.  I use this other bot called PyApp.  And it's a service that helps keep my Python dependencies up to date.  And it's really useful.  I recommend you all use it.  It's free for open source projects.  So when one of your dependencies has an update, they made a new release to PyPI.  You get a pull request updating your requirements.  It's great.  The problem is I have to wait for the test to pass before I can merge it.  And then I have to merge it and delete the branch.  If a bot can create the pull request, why can't it merge it itself?  Why do I have to wait for this?  In my mind, throughout this whole workflow, the only thing that requires human intelligence  that requires my involvement is reviewing the change log.  And to be honest, I don't even review change log anymore.  As long as it passes, my coverage didn't drop.  Merge it, right?  I know a bot can merge this.  So this is an actual open pull request in Mrs. LinkedIn.  And purposely, I don't want to merge this because I refuse to press that green button  because I know a bot can do it.  So if PyApp doesn't provide auto-merge ability soon enough, I'm going to build my own bot  that just merge and delete the branch.  Build the bot.  This is an idea that I have.  I've been working on adding the PSS code of conduct to some of our GitHub repositories.  So I added the code of conduct to Cpython, to the DevGuide, to the core workflow.  By working on this, I mean I copy paste things.  And you know, it's boring.  Basically, I create the same pull request again and again.  I think a bot could totally do this.  There should be a bot that can go through every public repost under Python organization  and add the code of conduct.  I think it's important to have.  So I wish there's such a bot that can just create this for me, merge it itself.  Delete the branch.  I don't need to be bothered because a bot can totally do this.  Since last month, I've been taking a break from any volunteering activities, not contributing  to open source since September, yes, September, and also this month.  So I figured maybe I should have a similar to out of office auto reply.  I will have an out of office, out of open source auto reply on GitHub.  So if you all mentioned me on GitHub this month, my bot will tell you that I'm sorry,  Mary was not going to look at this until November.  I didn't want to include this here just because it may be seen as product pitch.  A lot of people asking me how they can set this up for themselves.  This is done by Zapier.  This is a multi-step by Zapier.  It's triggered by if there's a new mention of me, I still have to filter it like if the  mention is outside of Zapier.  So if my co-worker mentioned me, it's fine, but everybody else, they'll get this message.  I still get to run Python.  There's a step for me that I need to extract the pull request number where the mention  happened and then I tell Zapier to create the comment.  So if you're interested, talk to Zapier, try out Zapier.  Dave Bisley needs a bot and he didn't even realize it.  Earlier this year he asked on Twitter, is there an easy way to allow issues but not  pull request?  And he had good reason for doing this.  I think one of the reply was moved to something else that doesn't sound really easy.  You can totally do a bot that takes care of this.  It's as simple as this, receive the PR event, close it.  Much simpler than this is LinkedIn.  I know this can be done.  I wrote it.  Those are the lines of code you need to write in order to accomplish this.  It's easy.  So I hope that I've given you ideas to automate your life.  I mean, I've used GitHub examples, GitHub bot examples throughout this talk just because  that's where I spent my life most of the time.  But don't be limited to just GitHub bots.  Automate your life.  Don't be a robot, build a bot.  And if it is GitHub bot you're after.  I did wrote a tutorial telling you how to do it.  Go there and GitHub bot tutorial.  But yeah, automate.  Don't do the boring things.  You know, look at your own life.  I'm sure you will find even more boring things.  So thank you so much.  My name is Mariera.  Thanks for coming to my talk.  I hope you enjoyed it.  Follow me on Twitter or if you would like to get in touch, can email me mariera at python.org.  Thank you so much.  Thank you so much and see you next time.
DjangoCon_US_2018_-_Keynote_with_Mariatta_Wijaya_uOLs3QeZy7M_0_119.png DjangoCon_US_2018_-_Keynote_with_Mariatta_Wijaya_uOLs3QeZy7M_120_251.png DjangoCon_US_2018_-_Keynote_with_Mariatta_Wijaya_uOLs3QeZy7M_252_335.png DjangoCon_US_2018_-_Keynote_with_Mariatta_Wijaya_uOLs3QeZy7M_336_395.png DjangoCon_US_2018_-_Keynote_with_Mariatta_Wijaya_uOLs3QeZy7M_396_1259.png DjangoCon_US_2018_-_Keynote_with_Mariatta_Wijaya_uOLs3QeZy7M_1260_1847.png DjangoCon_US_2018_-_Keynote_with_Mariatta_Wijaya_uOLs3QeZy7M_1848_2603.png DjangoCon_US_2018_-_Keynote_with_Mariatta_Wijaya_uOLs3QeZy7M_2604_3407.png DjangoCon_US_2018_-_Keynote_with_Mariatta_Wijaya_uOLs3QeZy7M_3408_4799.png DjangoCon_US_2018_-_Keynote_with_Mariatta_Wijaya_uOLs3QeZy7M_4800_5267.png DjangoCon_US_2018_-_Keynote_with_Mariatta_Wijaya_uOLs3QeZy7M_5268_6239.png DjangoCon_US_2018_-_Keynote_with_Mariatta_Wijaya_uOLs3QeZy7M_6240_7019.png DjangoCon_US_2018_-_Keynote_with_Mariatta_Wijaya_uOLs3QeZy7M_7020_7307.png DjangoCon_US_2018_-_Keynote_with_Mariatta_Wijaya_uOLs3QeZy7M_7308_7487.png DjangoCon_US_2018_-_Keynote_with_Mariatta_Wijaya_uOLs3QeZy7M_7488_8231.png DjangoCon_US_2018_-_Keynote_with_Mariatta_Wijaya_uOLs3QeZy7M_8232_8363.png DjangoCon_US_2018_-_Keynote_with_Mariatta_Wijaya_uOLs3QeZy7M_8364_9287.png DjangoCon_US_2018_-_Keynote_with_Mariatta_Wijaya_uOLs3QeZy7M_9288_9767.png DjangoCon_US_2018_-_Keynote_with_Mariatta_Wijaya_uOLs3QeZy7M_9768_10283.png DjangoCon_US_2018_-_Keynote_with_Mariatta_Wijaya_uOLs3QeZy7M_10284_10331.png DjangoCon_US_2018_-_Keynote_with_Mariatta_Wijaya_uOLs3QeZy7M_10332_10379.png DjangoCon_US_2018_-_Keynote_with_Mariatta_Wijaya_uOLs3QeZy7M_10380_11207.png DjangoCon_US_2018_-_Keynote_with_Mariatta_Wijaya_uOLs3QeZy7M_11208_15083.png DjangoCon_US_2018_-_Keynote_with_Mariatta_Wijaya_uOLs3QeZy7M_15084_15887.png DjangoCon_US_2018_-_Keynote_with_Mariatta_Wijaya_uOLs3QeZy7M_15888_18287.png DjangoCon_US_2018_-_Keynote_with_Mariatta_Wijaya_uOLs3QeZy7M_18288_19259.png DjangoCon_US_2018_-_Keynote_with_Mariatta_Wijaya_uOLs3QeZy7M_19260_21935.png DjangoCon_US_2018_-_Keynote_with_Mariatta_Wijaya_uOLs3QeZy7M_21936_21983.png DjangoCon_US_2018_-_Keynote_with_Mariatta_Wijaya_uOLs3QeZy7M_21984_22811.png DjangoCon_US_2018_-_Keynote_with_Mariatta_Wijaya_uOLs3QeZy7M_22812_23855.png DjangoCon_US_2018_-_Keynote_with_Mariatta_Wijaya_uOLs3QeZy7M_23856_25235.png DjangoCon_US_2018_-_Keynote_with_Mariatta_Wijaya_uOLs3QeZy7M_25236_26135.png DjangoCon_US_2018_-_Keynote_with_Mariatta_Wijaya_uOLs3QeZy7M_26136_26975.png DjangoCon_US_2018_-_Keynote_with_Mariatta_Wijaya_uOLs3QeZy7M_26976_27179.png DjangoCon_US_2018_-_Keynote_with_Mariatta_Wijaya_uOLs3QeZy7M_27180_28163.png DjangoCon_US_2018_-_Keynote_with_Mariatta_Wijaya_uOLs3QeZy7M_28164_28763.png DjangoCon_US_2018_-_Keynote_with_Mariatta_Wijaya_uOLs3QeZy7M_28764_30467.png DjangoCon_US_2018_-_Keynote_with_Mariatta_Wijaya_uOLs3QeZy7M_30468_30875.png DjangoCon_US_2018_-_Keynote_with_Mariatta_Wijaya_uOLs3QeZy7M_30876_31211.png DjangoCon_US_2018_-_Keynote_with_Mariatta_Wijaya_uOLs3QeZy7M_31212_31631.png DjangoCon_US_2018_-_Keynote_with_Mariatta_Wijaya_uOLs3QeZy7M_31632_31739.png DjangoCon_US_2018_-_Keynote_with_Mariatta_Wijaya_uOLs3QeZy7M_31740_31847.png DjangoCon_US_2018_-_Keynote_with_Mariatta_Wijaya_uOLs3QeZy7M_31848_32267.png DjangoCon_US_2018_-_Keynote_with_Mariatta_Wijaya_uOLs3QeZy7M_32268_33551.png DjangoCon_US_2018_-_Keynote_with_Mariatta_Wijaya_uOLs3QeZy7M_33552_34415.png DjangoCon_US_2018_-_Keynote_with_Mariatta_Wijaya_uOLs3QeZy7M_34416_34511.png DjangoCon_US_2018_-_Keynote_with_Mariatta_Wijaya_uOLs3QeZy7M_34512_35855.png DjangoCon_US_2018_-_Keynote_with_Mariatta_Wijaya_uOLs3QeZy7M_35856_36275.png DjangoCon_US_2018_-_Keynote_with_Mariatta_Wijaya_uOLs3QeZy7M_36276_36743.png DjangoCon_US_2018_-_Keynote_with_Mariatta_Wijaya_uOLs3QeZy7M_36744_37475.png DjangoCon_US_2018_-_Keynote_with_Mariatta_Wijaya_uOLs3QeZy7M_37476_39095.png DjangoCon_US_2018_-_Keynote_with_Mariatta_Wijaya_uOLs3QeZy7M_39096_40151.png DjangoCon_US_2018_-_Keynote_with_Mariatta_Wijaya_uOLs3QeZy7M_40152_40943.png DjangoCon_US_2018_-_Keynote_with_Mariatta_Wijaya_uOLs3QeZy7M_40944_42887.png DjangoCon_US_2018_-_Keynote_with_Mariatta_Wijaya_uOLs3QeZy7M_42888_44651.png DjangoCon_US_2018_-_Keynote_with_Mariatta_Wijaya_uOLs3QeZy7M_44652_46007.png DjangoCon_US_2018_-_Keynote_with_Mariatta_Wijaya_uOLs3QeZy7M_46008_47147.png DjangoCon_US_2018_-_Keynote_with_Mariatta_Wijaya_uOLs3QeZy7M_47148_47867.png DjangoCon_US_2018_-_Keynote_with_Mariatta_Wijaya_uOLs3QeZy7M_47868_48155.png DjangoCon_US_2018_-_Keynote_with_Mariatta_Wijaya_uOLs3QeZy7M_48156_48431.png DjangoCon_US_2018_-_Keynote_with_Mariatta_Wijaya_uOLs3QeZy7M_48432_49847.png DjangoCon_US_2018_-_Keynote_with_Mariatta_Wijaya_uOLs3QeZy7M_49848_50243.png DjangoCon_US_2018_-_Keynote_with_Mariatta_Wijaya_uOLs3QeZy7M_50244_50291.png DjangoCon_US_2018_-_Keynote_with_Mariatta_Wijaya_uOLs3QeZy7M_50292_50399.png DjangoCon_US_2018_-_Keynote_with_Mariatta_Wijaya_uOLs3QeZy7M_50400_50447.png DjangoCon_US_2018_-_Keynote_with_Mariatta_Wijaya_uOLs3QeZy7M_50448_50521.png