We all know sometimes a e.g. merge can go wrong.

If our [unit / integration] tests are in the same repository as the merge, is there then a weakness there that if the tests have merged incorrectly and do not appear in the final code then "we don't know what we don't know".

Should the unit test project then instead be a complete separate item?

In many ways I'd say no as it could litter a master branch with feature tests which are not yet relevant.

In compiled languages then the compilation will fail if the test's reliant code is missing, so will help to a certain degree. But perhaps a non-compiled language is better as it would highly specific missing items when running the test.

Does anyone have a workflow which it is possible to know which features may of merged incorrectly?

  • Has this actually happened? How did it happen? Or are you just worrying about a hypothetical problem? – Winston Ewert Sep 17 '14 at 3:17
  • Good query Winston.. I have seen bad merges cause problems few times (that is an issue to solve in it's own right). But in regards to tests it's just an academic question really. – Alex KeySmith Sep 17 '14 at 7:58

Unit tests are code as everything else. So if you merge your code, and your unit tests depend on the version or revision of the non-testing code (which is normally the case), you should always merge your tests together with everything else. This includes the resolving of any merge conflicts, if they occur.

If the merge conflicts of your test suite are not resolved, you loose the ability to compile and run it (or in a non-compiled language: the test suite may bring wrong results or crash at run time) - is that really worth the hassle? Delaying the resolving of a merge conflict is never helpful, the longer you wait, the harder it gets. Test code is therefore not different from any other kind of code. And if resolving the merge conflicts of your tests needs so much effort that you consider omitting the resolve as a serious option, there is something very, very wrong with your workflow.

  • Thanks, I suppose I'm thinking of situations like when a merge has seemed to be very clean, but it's gone astray for some reason. – Alex KeySmith Jun 19 '14 at 10:15
  • @AlexKey: so if a merge goes wrong without an SCM conflict, what else except automatic tests (and the compiler) can detect this? Though I have really seldom seen that situation practice, maybe because I work primarily with compiled languages. – Doc Brown Jun 19 '14 at 10:20
  • Exactly, a very tricky one to pickup automatically. I suppose I'm thinking that perhaps all unit tests go into a separate repository with a simpler git workflow without feature merges? But I suppose that could cause problems on it's own (no merges!). Or perhaps keeping UI or smoke tests in a separate repository? Just thinking out loud. – Alex KeySmith Jun 19 '14 at 10:26
  • @AlexKey: I am still not sure if I got your question right. So when you wrote "merge can go wrong" - you had not merge conflicts in mind, but only merges with "semantic" errors? And if that is correct, I don't understand why putting tests into a separate repository would change anything to the situation? – Doc Brown Jun 19 '14 at 19:57
  • True I agree. I suppose I was wondering if anyone had workflows to help mitigate semantic errors. I suppose I'm picturing that a unit test project may have a simpler branching strategy with less to go wrong. Bit of an open question I realise. – Alex KeySmith Jun 20 '14 at 8:04

Tests and code should merge at the same time.

Consider the opposite case. What happens if a merge conflict loses your new lines of code? If you still have the old test, so you would never know.

Modifying the regression becomes your record that you expected it the behavior to change, and should follow along with the code as it merges.

Your Answer

