title: Why won't these chai tests fail?
body: <p>We have some simple "is this really working" chai tests of an electron app using spectron and WebdriverIO. The example code we started with is from </p>
  
  <p><a href="https://github.com/jwood803/ElectronSpectronDemo" rel="nofollow noreferrer">https://github.com/jwood803/ElectronSpectronDemo</a> as reported in <a href="https://github.com/jwood803/ElectronSpectronDemo/issues/2" rel="nofollow noreferrer">https://github.com/jwood803/ElectronSpectronDemo/issues/2</a>, the chai-as-promised tests are not catching mismatches, so I thought I would add some additional tests to find out why Chai is not failing tests where the electron app has text that doesn't match the expected unit test text.</p>
  
  <p>Let's start with something really simple, the rest of the code is at <a href="https://github.com/drjasonharrison/ElectronSpectronDemo" rel="nofollow noreferrer">https://github.com/drjasonharrison/ElectronSpectronDemo</a></p>
  
  <pre><code>describe('Test Example', function () {
      beforeEach(function (done) {
          app.start().then(function() { done(); } );
      });
  
      afterEach(function (done) {
          app.stop().then(function() { done(); });
      });
  
      it('yes == no should fail', function () {
          chai.expect("yes").to.equal("no");
      });
  
      it('yes == yes should succeed', function () {
          chai.expect("yes").to.equal("yes");
      });
  </code></pre>
  
  <p>The first unit test fails, the second succeeds.</p>
  
  <p>And when we put the assertion into a function this still detects the failure:</p>
  
  <pre><code>it('should fail, but succeeds!?', function () {
      function fn() {
          var yes = 'yes';
          yes.should.equal('no');
      };
      fn();
  });
  </code></pre>
  
  <p>So now into the world of electron, webdriverio, and spectron, the application title is supposed to be "Hello World!", so this should fail, but it passes:</p>
  
  <pre><code>it('tests the page title', function () {
      page.getApplicationTitle().should.eventually.equal("NO WAY");
  });
  </code></pre>
  
  <p>Hmm, let's try a more familiar test:</p>
  
  <pre><code>it('should fail, waitUntilWindowLoaded, yes != no', function () {
      app.client.waitUntilWindowLoaded().getTitle().then(
          function (txt) {
              console.log('txt = ' + txt);
              var yes = 'yes';
              yes.should.equal('no');
          }
      );
  });
  </code></pre>
  
  <p>Output:</p>
  
  <pre><code>    ✓ should fail, waitUntilWindowLoaded, yes != no
  txt = Hello World!
  </code></pre>
  
  <p>It succeeds? What? Why? How?</p>