Take the 2-minute tour ×

For the sake of argument let's say I am working on a project, X, which has a dependency, Y. Now Y is a stand alone open source project regularly maintained and updated by third parties. I check out the latest revision of Y, commit it to the repository that hosts X, and as time goes by I potentially make changes to Y in my local repo. Two months later, I decide I want to merge the latest changes from the open source repo back to mine to get the latest bug fixes, features, etc. If these two branches were part of the same repo, this would have been a no brainer. How can I [relatively] painlessly do this cross-repo merge in Git, Mercurial, and Subversion?


share|improve this question
add comment

2 Answers

My take on that is that you could look at what Debian does with its Git packaging workflow employing the git-buildpackage tool.

The workflow provided by this tool is agnostic with regard to VCS used by upstream vendor, and is organized (roughly) like this:

  • You have (at least) two branches: upstream and master.
  • upstream holds snapshots of (unmodified) upstream sources usually taken from release tarballs provided by the upstream vendor. That is, each commit on this branch results from these steps:

    1. The upstream branch is checked out.
    2. All existing files are deleted (git rm -rf .).
    3. The new version of upstream sources is unwrapped and copied over to the work tree, and then added (git add .).
    4. A new commit is then recorded (and a tag upstream/vX.Y.Z is created pointing to this commit).
  • master contains what's on upstream plus a set of files providing the infrastructure to build the Debian package (actually, this is just a single directory named "debian").

    Each time a new version of upstream sources is imported to the upstream branch, that branch is merged into master, and the package maintainer then works on master tailoring their "debianization" to match the changes introduced by upstream.

I think this approach might well be used in your case using plain Git:

  • Maintain such an "upstream" branch (you might call it "vendor" or "that_framework" etc). It should receive only new versions of upstream sources (and may be also occasional upstream patches etc).
  • After importing the new versions of upstream sources to that branch merge it to your master (or whatever branch suits better in your workflow).

Working with Mercurial and Subversion could also be done using their respective Git shims, but I suspect (while not being exactly sure) that this would rather complicate matters, not simplify.

share|improve this answer
Interesting. Even though the old version of upstream libraries are continuously deleted and replaced by new ones, I am not sure that this would actually prevent the repo size from bloating over time. Also I am not certain how a lack of a history of changesets would affect merges. Both of these concerns may be completely invalid, and I am interested in knowing the answer if anyone with more version management knowledge cares to share... –  Ash Apr 30 '13 at 18:00
@Ash, 1) Git is only ever cares about the contents of the files it tracks, so if one does git rm -rf . + git add . of the new release as I've described, only the files which actually changed will constitute to the size increase. Such a "strange" way to deal with the new upstream release is only needed to ensure all the files deleted by upstream between the old and the new releases are deleted from our three as well. –  kostix Apr 30 '13 at 18:49
@Ash, 2) While I'll not swear by it, I think with Git, the history of changesets won't affect merges: while performing a typical three-way merge, Git does only use the history records to locate the so-called "merge base" -- the last common commit between the both sides of the merge. If this merge base is found, the three-way merge proceeds as is, using these three commits only, otherwise it's degraded to a simple merging using only two commits. No complex "patch calculation" (a-la Darcs) takes place. –  kostix Apr 30 '13 at 18:55
add comment

Are you saying you'd put a copy of Y in your repository for X? If so that's not the right way to do it for any of the VCSs you mentioned. You want to "fork" Y's repository into Y-mine, into which you commit your changes. Then you have your build system's configuration file in project X contain as a dependency a pointer to a specific revision in the repository Y-mine -- any modern build system does this.

That gives you the best of both worlds -- you can merge into Y-mine from Y whenever you want, and you have the exact version of Y-mine stored in X for 100% reproducible builds.

Git and Mercurial both have subrepo systems that allow you to say "Y-mine at version Z is part of repo X", but they're klunkier than letting pip or maven or sbt or gem or visual studio or ivy2 or whatever... handle the dependency management.

share|improve this answer
You again and again ignore Occam Razor. No new entities, no additional layers will be better starting point –  Lazy Badger Apr 30 '13 at 9:42
Great point. Hadn't thought of that. Thanks! –  Ash Apr 30 '13 at 17:42
I wasn't suggesting a "new entity" just picking the right one. Python, Ruby, Go, Node, and Perl all already have repos-as-dependencies built-in to the package manager one is already using. In a Java or C# environment one is using maven or an IDE that handles this already. My point was not to try to do with the VCS what the package manager already does. –  Ry4an May 1 '13 at 13:07
add comment

Your Answer


