title: Various TimeOut Methods on TestNG are not working
body: <p>A bit of a broad problem so bear with me.</p>
  
  <p>I've recently inherited a testing suite (java, selenium, testng) that is completely external to the product it is testing.  The suite has a major issue with tests running over time (one case hit 9 hours recently on what should be a 90 second test case).</p>
  
  <p>To remedy this issue, I have been trying to enforce time-outs on the tests, but I have had no luck.  A quick overview of the build:</p>
  
  <p>We build and test using maven, </p>
  
  <pre><code>mvn clean test -Dsuite=debug -Dbrowser=firefox -Dmakescreen=NO
  </code></pre>
  
  <p>and designate the test suite(s) to be run using an xml doc like so:</p>
  
  <pre><code>&lt;?xml version="1.0" encoding="UTF-8"?&gt;
  &lt;!DOCTYPE suite SYSTEM 'http://testng.org/testng-1.0.dtd'&gt;
  &lt;suite name="Test Debugging" verbose="1"&gt;
      &lt;suite-files&gt;
               &lt;suite-file path="suites/DebugTest.xml"/&gt;
      &lt;/suite-files&gt;
  &lt;/suite&gt;
  </code></pre>
  
  <p>DebugTest.xml looks something like:</p>
  
  <pre><code>&lt;?xml version="1.0" encoding="UTF-8"?&gt;
  &lt;!DOCTYPE suite SYSTEM 'http://testng.org/testng-1.0.dtd'&gt;
  &lt;suite name="SUITENAME" verbose="1"&gt;
      &lt;test name="TESTNAME"&gt;
          &lt;parameter name="FileName" value="filename.csv" /&gt;
          &lt;classes&gt;
              &lt;class name="tests.testpath" /&gt;
          &lt;/classes&gt;
      &lt;/test&gt;
  &lt;/suite&gt;
  </code></pre>
  
  <p>Before changes, I've confirmed the tests I am working with run in roughly 10 seconds and pass.</p>
  
  <p>Each test follows the following format:</p>
  
  <pre><code>public class TestName extends BaseTest{
  
  @Override
  public void runTest(){
      MainPage mainPage = new MainPage();
      //test does various things
      mainPage.assertCoreElements();
  }
  </code></pre>
  
  <p>You'll see I am extending a Basetest Class and overriding the runtest method.  The BaseTest Looks like:</p>
  
  <pre><code>public abstract class BaseTest extends BaseEntity implements IAnalys {
  
  
  /**
   * To override.
   */
  public abstract void runTest();
  
  /**
   * Test
   *
   * @throws Throwable Throwable
   */
  @Test
  public void xTest(ITestContext ctx) throws Throwable {
      System.setProperty("location", "");
      Class&lt;? extends BaseTest&gt; currentClass = this.getClass();
      String testRailTestId = TestRailOperations.getTestIdByTitleInRun(testRailHostAddress,  testRailLogin,  testRailPassword,  testRailRunId,  ctx.getName());
      checkJiraIssue(currentClass); //not an jira extended method
      checkJiraIssue(currentClass.getMethod("runTest", null)); //not an jira extended method
  
      try {
          processBugAnnotation();
          logger.logTestName(currentClass.getName());
          zephyrExecution();
          runTest();
          logger.logTestEnd(currentClass.getName());
          TestRailOperations.setResultToTest(lastStep, testRailHostAddress,  testRailLogin,  testRailPassword, testRailTestId, TestRailOperations.TestResult.PASSED, "");
          zephyrTestPass();
      } catch (Exception | AssertionError e) {
          logger.debug(this, e);
          if (shouldAnalys()) {
              invokeAnalys(e, getBrowser().getDriver().getPageSource());
          } else {
              TestRailOperations.setResultToTest(lastStep, testRailHostAddress,  testRailLogin,  testRailPassword, testRailTestId, TestRailOperations.TestResult.FAILED, Logger.fatalMessage);
              Logger.fatalMessage = "No info";
              zephyrTestFail();
              logger.warn("");
              logger.warnRed(getLoc("loc.test.failed"));
              logger.warn("");
              throw e;
          }
      } finally {
          if (Browser.getAnalyzeTraffic()) {
              getBrowser().assertAnalyzedTrafficResponseCode();
              getBrowser().assertAnalyzedTrafficLoadTime();
              ProxyServ.saveToFile(this.getClass().getName());
          }
          makeScreen(currentClass);
          printAllFormAppearingTimes();
          clearFormAppearingTimes();
          setIsFirstTimeLoadMainPage(true);
      }
  }
  </code></pre>
  
  <p>I also have a before method in the parent class that gets the browser and moves us to the starting url, and an after class that quits the browser.</p>
  
  <p><strong>My Goal</strong></p>
  
  <p>I want to configure these tests to timeout at a given limit (initially something like 10ms to prove it works).</p>
  
  <p><strong>What I've Tried</strong></p>
  
  <ol>
  <li>@Test(timeout=10000) at the xTest method declaration:  </li>
  </ol>
  
  <p>This was the first thing I tried and it worked...sort of.  The test in question did time out at the right point, but it would also spawn a separate thread running independently of the original test.  This thread would open a browser, then terminate without closing the browser, leaving whatever machine I was working on with hundreds of open windows after a given run.  Not ideal!  I've checked my after class method, and it clearly tries to close all browsers (unsuccessfully here).</p>
  
  <p>Some of my reading indicates that a second thread of control is being spawned independent of the before and after methods, which is responsible for my hanging windows.</p>
  
  <ol start="2">
  <li>@Test(timeout=10000) on runtest method in the test class</li>
  </ol>
  
  <p>this is not ideal because I'd have to copy paste to every single test in the suite, but gave it a try anyway.  This does work, but again seems to have issues with a second thread.  If the original test completes in time a second browser window opens, but nothing happens in it because it doesn't seem to be hitting any code in the before method, then fails when I assert the page loaded.</p>
  
  <ol start="3">
  <li>@Rule Timeout at both Basetest and test level</li>
  </ol>
  
  <p>To get around the issues mentioned above I followed the documentation here: <a href="https://github.com/junit-team/junit/wiki/Timeout-for-tests" rel="nofollow">https://github.com/junit-team/junit/wiki/Timeout-for-tests</a></p>
  
  <p>This was completely ineffective, using both</p>
  
  <pre><code>`@Rule public Timeout globalTimeout = Timeout.seconds(10);`
  </code></pre>
  
  <p>and</p>
  
  <pre><code>`@ClassRule public Timeout globalTimeout = Timeout.seconds(10);`
  </code></pre>
  
  <p>Would not cause tests to timeout, event when I set the time to 1 second on a 30 second run.</p>
  
  <ol start="4">
  <li><p>I've also tried setting the timeout value in the xml suite:</p>
  
  <p><code>&lt;suite name="Test Debugging" verbose="1" time-out="200"&gt;</code></p></li>
  </ol>
  
  <p>Again with no measurable impact even at smallest possible time slice.</p>
  
  <p>At this point I am truly at a loss -- I either need a way to handle the extra threads that are spawning with the @test annotations, or I need to figure out why instructions from the other methods are not being followed.</p>
  
  <p>Thanks for the help,</p>
  
  <p>Joe</p>
  
  <p>EDIT:  Supplying Code for before and after functions</p>
  
  <pre><code>@BeforeClass
  public void before() {
      Browser browser = getBrowser();
      browser.windowMaximise();
      long before = new Date().getTime();
      browser.navigate(browser.getStartBrowserURL());
      long openTimeFirstPage = new Date().getTime() - before;
      listFormAppearingTime.add("Main Page=" + String.valueOf(openTimeFirstPage));
      SikuliElement allowGeolocationChrome = new SikuliElement("ALLOW_CHROME.png", 5);
      SikuliElement allowGeolocationIE = new SikuliElement("ALLOW_IE.png", 5);
      if (Browser.currentBrowser.equals(Browsers.CHROME)) {
          if (allowGeolocationChrome.isPresent()) {
              allowGeolocationChrome.click();
          }
      }
      if (Browser.currentBrowser.equals(Browsers.IEXPLORE)) {
          if (allowGeolocationIE.isPresent()) {
              allowGeolocationIE.click();
          }
      }
      stepNumber = 1;
      if (Browser.getAnalyzeTraffic()) {
          browser.startAnalyzeTraffic();
      }
      if ("true".equalsIgnoreCase(System.getProperty(ZEPHYR_CONNECTION_KEY, "false"))) {
          try {
              reporter = new ZephyrJiraReporter();
              newClass = true;
              if (reporter.report()) {
                  reporter.createTestCycle();
              }
          } catch (Exception e) {
              logger.debug(this, e);
              logger.error(e.getMessage());
          }
      }
  }
  
  
  /**
   * Close browser after each test Class
   * And update Jira-Zephyr test cycle
   */
  @AfterClass
  public void after() {
      ProxyServ.stopProxyServer();
      if (getBrowser().isBrowserAlive()) {
          if ("true".equalsIgnoreCase(System.getProperty(ZEPHYR_CONNECTION_KEY, "false"))) {
              updateTestCycle();
          }
          getBrowser().exit();
      }
  }
  </code></pre>