Thursday, July 17, 2014

Applying patches from mutt onto a git tree easily

This post is for project maintainers using git who wish to merge patches easily into a project directly from mutt. Projects using git vary in size and there many different ways to merge patches from contributors. What strategy you use can depend on whether or not you are expecting to merge hundreds of patches, or just a few. If you happen to be very unfortunate and are forced to use Gerrit a mechanism was chosen for you for review and how patches will get merged / pushed. If you're just using raw git directly you can do whatever you like. For big projects git pull requests are commonly used. Small projects can instead live with manual patch application from an mailbox inbox. Even large projects can't realistically expect folks to be submitting every patch with pull requests, and so manual patch application also applies to large projects. How you get your patch out of your inbox and get it merged will vary depending on what software you are using to read your mailbox. Tons of folks are using gmail these days and even there it's not that easy: you'd have to go to the right pane, go to drop down menu and select "Show original", then save that page as a text file, edit it to remove the top junk right before the From: and finally you can then git am that file.

This doesn't scale well. A plugin can surely help but bleh, the command line is so much better. For that you can use Mutt. The typical approach on mutt is to use the default hooks to save a file onto disk and then go and 'git am' it. It'd be much easier if we just had hooks to apply patches directly into a git tree though. The following are configuration options you can use and a bit of shell that will allow that. Ben Hutchings's blog post on git and mutt in 2011 described a way to extract patches into a directory and then you'd just git am them. Those instructions no longer work on newer versions of mutt, I'll provide updated settings and also extended these hooks to allow you to apply patches without even having to drop down to another shell, while also giving you the option to inspect them manually if you wish.

Here's what I have on my .muttrc :

macro index (t '~/mailtogit/mail-to-mbox^M'  "Dumps tagged patches into ~/incoming/*.mbox"
macro index (a '~/mailtogit/git-apply-incomming^M'  "git am ~/incoming/*.mbox"
macro index (g '~/mailtogit/git-apply^M'  "git am tagged patches"
macro index (r 'rm -f ~/incoming/*.mbox^M'  "Nukes all ~/incoming/"
macro index (l 'ls -ltr ~/incoming/^M'  "ls -l ~/incoming/"        
macro index ,t '~/mailtogit/mail-to-mbox^M'  "Dumps currently viewed patch into ~/incoming/*.mbox"
macro index ,g '~/mailtogit/git-apply^M' "git am currently viewed patch"
macro index ,a '~/mailtogit/git-abort^M' "git am --abort"          
macro index ,r '~/mailtogit/git-reset^M' "git-reset --hard origin" 

The first hook (t allows you to dump patches you tag into an ~/incoming/ directory, mutt will show you what those are. The (a will apply all the patches that you just took out into that directory. The (g hook will merge the two steps into one and just dump the tagged patches and apply them immediately. If you have to clear the ~/incoming/ directory just use the (r hook. If you'd like to review what's in that directory you can use the (l hook. With ,t you can dump the currently viewed patch into ~/incoming/, this lets you extract a patch without tagging it. The ,g hook will also skip having to tag a patch and just apply it. If you want to abort a 'git am' operation you can use ,a. Finally to reset your tree to origin, just use the ,r hook.

This all depends on 5 small scripts, the ones that change directory obviously are making these scripts depend on one single projects so the question arises as to how to generalize this so that mutt is aware of the project a patch was sent for and you can apply it to that right tree so that we don't have to stuff mutt with tons of different project specific hooks. There are two approaches that come to mind, one is to have the shell script read the List-ID tag, for example List-ID: , and have a mapping of those to git trees. The other is to trust rather the directory the e-mail went in under mutt, which assumes you already had filters for each List-ID. The issue with both of these approaches is that at times a patch may go to multiple lists but in Linux' case, where this does apply, it should be specific to at least one git tree you do care, unless I guess you are maintaining multiple subsystems. Another possibility that comes to mind is to have git format-patch add yet-another-tag into the e-mail that it spits out the e-mails used for submission, perhaps Gid-ID: and the tree? This also has some issues though for many reasons, so for now this is what I have and use. Let me know if you come up with something more generic.


formail -cds ~/mailtogit/procmail -
ls -l ~/incoming/


cd ~/backports
git am ~/incoming/*.mbox


rm -f ~/incoming/*
cd ~/backports
git am -s ~/incoming/*.mbox


cd ~/backports/
git am --abort
rm -f ~/incoming/*


cd ~/backports/
git reset --hard origin
rm -f ~/incoming/*