title: How to order TestNG Factory execution given a single method test and sortable test data?
body: <p>First of all you should know that I am using TestNG v6.8.8 and Java JDK 6.  I experience this problem running on different flavors of Linux and Mac OS 10.9.4.  On to the code listing.</p>
  
  <h1>Factory class</h1>
  
  <pre><code>import org.testng.annotations.Factory;
  
  import java.util.ArrayList;
  import java.util.Collections;
  import java.util.List;
  
  /**
   * @author rcourtright
   *         Date: 8/5/14
   *         Time: 4:50 PM
   */
  public class ErsatzFactory {
  
      private final List&lt;String&gt; testData;
  
      public ErsatzFactory() {
          testData = new ArrayList&lt;String&gt;();
          int order = 0;
          for (int i = 0 ; i &lt; 9; i++) {
              testData.add(order++ + "-Test");
          }
          Collections.sort(testData);
      }
  
      @Factory
      public Object[] setUp() {
          List&lt;ErsatzTest&gt; objects = new ArrayList&lt;ErsatzTest&gt;();
          int order = 0;
          for (String testDatum : testData) {
              objects.add(
                      order,
                      new ErsatzTest(testDatum)
              );
              order++;
          }
          return objects.toArray();
      }
  
  }
  </code></pre>
  
  <h2>Test class</h2>
  
  <pre><code>import org.testng.ITest;
  import org.testng.annotations.Listeners;
  import org.testng.annotations.Test;
  
  /**
   * @author rcourtright
   *         Date: 8/5/14
   *         Time: 4:45 PM
   */
  //@Listeners(ErsatzListener.class)
  public class ErsatzTest implements ITest {
  
      private String order;
  
      public ErsatzTest(String order) {
          this.order = order;
      }
  
      @Test
      public void justDoIt() {
          System.out.println(order);
      }
  
      @Override
      public String getTestName() {
          return order;
      }
  }
  </code></pre>
  
  <h3>TestNG XML</h3>
  
  <pre><code>&lt;!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd"&gt;
  
  &lt;suite name="ersatz-testing" verbose="1" &gt;
      &lt;test name="ersatz-test-factory" preserve-order="true"&gt;
          &lt;classes&gt;
              &lt;class name="ErsatzFactory"/&gt;
          &lt;/classes&gt;
      &lt;/test&gt;
  &lt;/suite&gt;
  </code></pre>
  
  <p>So you can see in my construction I have a list which has a natural order and I sort that collection.  It is the job of the Factory to execute the single test method in that order.  By debugging I have found that the collection returned by the Factory is in the desired order.  But that is not how they are executed.</p>
  
  <p>Using TestNG supported ant target, this is the order of execution:</p>
  
  Test results
  
  <blockquote>
  <pre><code>ersatz-test-executor:
     [testng] 5-Test
     [testng] 2-Test
     [testng] 7-Test
     [testng] 6-Test
     [testng] 4-Test
     [testng] 8-Test
     [testng] 1-Test
     [testng] 0-Test
     [testng] 3-Test
     [testng]
     [testng] ===============================================
     [testng] ersatz-testing
     [testng] Total tests run: 9, Failures: 0, Skips: 0
     [testng] ===============================================
     [testng]
  </code></pre>
  </blockquote>
  
  <p>This is both unexpected and undesirable.  Clearly this test is trivial, but it is a proxy for a rather complex system test for which I must get the order of execution correct.</p>
  
  <p>Although not listed here, I must use a TestNG Listener for post test result processing and it appears that I must use a Factory as opposed to a stand-alone DataProvider.  (I have commented out the Listener annotation in the test class declaration.)  I should note that the Listener works without incident.</p>
  
  <p>Because there is one and only one method, I cannot use the Priority annotation or method dependencies.  The test is completely data driven.  The data is sortable and, if memory serves, when I used a DataProvider from within the test class I got that order.  But I also need the Listener for decorating the test results reports and that has driven me to using the factory.  I should note that the execution problem exists whether I use the listener or not.  If you compile and run this code I expect you will see the results listed in what appears to be a random order.</p>
  
  <p>Thanks in advance for your consideration of this problem.</p>