title: Switching a Git repository from ISO-8859-1 to UTF-8 encoding for source code files
body: <p>I'm going to convert a large Mercurial project to Git this weekend using <a href="https://github.com/frej/fast-export" rel="noreferrer">fast-export</a>.  I've tested that several times, and the results are good.</p>
  
  <p>We'd also like to turn our source code encoding (lots of German comments/string literals with Umlauts) from ISO-8859-1 to UTF-8 (all other non-java files in the repo should stay as-is), and the Git migration delivers us a chance to do it now since everybody needs to clone again anyway.  However, I don't find a good approach for it.</p>
  
  <ol>
  <li>I've tried the <code>git filter-tree --tree-filter ...</code> approach from <a href="https://stackoverflow.com/questions/11052199/convert-git-repository-file-encoding#11053818">this comment on SO</a>.  However while this seems ideal, due to the size of the repository (about 200000 commits, 18000 code files) it would take much more time than just the weekend I have.  I've tried running it (in a heavily optimized version where the list of files is chunked and the sublists are converted in parallel (using <a href="https://www.gnu.org/software/parallel/" rel="noreferrer">GNU parallel</a>)) straight from a 64GB tmpfs volume on a linux VM with 72 cores, and still it would take several days...</li>
  <li>Alternatively, I've tried the simple approach where I perform the conversion simply on any active branch individually and commit the changes.  However, the result is not satisfying because then I almost always get conflicts when merging or cherry-picking pre-conversion commits.</li>
  <li>Now I'm running approach 1 again but not trying to rewrite the complete history of all branches (<code>--all</code> as <code>&lt;rev-list&gt;</code>) but just all commits reachable from the current active branches' and not reachable by some past commit which is (hopefully) a predecessor of all current branches (<code>branch-a branch-b branch-c --not old-tag-before-branch-a-b-c-forked-off</code> as <code>&lt;rev-list&gt;</code>).  It's still running but I fear that I can't really trust the results as this seems like a very bad idea.</li>
  <li>We could just switch the encoding in the master branch with a normal commit as in approach 2, but again this would make cherry-picking fixes from/to master a disaster.  And it would introduce lots of encoding problems because developers would surely forget to change their IDE settings when switching between master and non-converted branches.</li>
  </ol>
  
  <p>So right now, I somehow feel the best solution could be to just stick to ISO-8859-1.</p>
  
  <p>Does anyone have an idea?  Someone mentioned that maybe <a href="http://www.catb.org/~esr/reposurgeon/" rel="noreferrer">reposurgeon</a> can do basically approach 1 using its <code>transcode</code> operation with a performance much better than <code>git filter-tree --tree-filter ...</code> but I have no clue how that works.</p>