title: docker-compose up --exit-code-from exits without error when system under test crashes fast
body: <p>I am creating some integration tests for my docker images using <code>docker-compose</code>. <code>docker-compose</code> has a neat flag <code>--exit-code-from</code> to allow stopping all dockers and returning exit code from the test docker once the tests are finished.</p>
  
  <p>Trouble is however that <code>--exit-code-from</code> includes automatically the flag <code>--abort-on-container-exit</code>. This is quite logical, but creates the following problem:</p>
  
  <p><strong>Normal case</strong></p>
  
  <ol>
  <li>system under test starts</li>
  <li>integration-tests run tests and exit with error code X</li>
  <li>all containers are stopped</li>
  <li>exit code X is returned.</li>
  </ol>
  
  <p><strong>Problem case</strong></p>
  
  <ol>
  <li>system under test starts</li>
  <li>before integration-tests has the time to finish, system under test exits with an error</li>
  <li>all containers are stopped</li>
  <li>exit code 0 is returned! So it is as if the tests succeeded.</li>
  </ol>
  
  <p>You can reproduce this with the following files:</p>
  
  <p><strong>Dockerfile</strong></p>
  
  <pre><code>FROM alpine as development
  ENTRYPOINT [ "/bin/false" ]
  
  FROM alpine as test
  ENTRYPOINT [ "/bin/sleep", "3" ]
  </code></pre>
  
  <p><strong>docker-compose.yml</strong></p>
  
  <pre><code>version: "3.5"
  services:
    sut:
      build:
        context: .
        target: development
    test:
      build:
        context: .
        target: test
  </code></pre>
  
  <p>and then running</p>
  
  <pre><code>docker-compose up --build --exit-code-from test
  echo $? # print exit code from previous command
  </code></pre>
  
  <p>which will return exit code 0.</p>
  
  <p><strong>I would like exit code 0 to mean: integration tests really ran and were successful.</strong></p>
  
  <p><strong>EDIT:</strong>
  after seeing this issue <a href="https://github.com/docker/compose/pull/6077/commits" rel="nofollow noreferrer">https://github.com/docker/compose/pull/6077/commits</a> merged into <code>docker-compose</code> 1.22.0; I ran again with the latest version of <code>docker-compose</code> (1.24.1 as of now), but I run into my problem still.</p>