title: What are the correct commands to run the CTS and VTS test plans?
body: <p>I am about to setup the VTS and CTS tests for our AOSP. Both of the test suites are using the Trade Federation test framework. What is confusing me is how to run the different test plans. </p>
  
  <p>According to the documentation (<a href="https://source.android.com/compatibility/vts/systems" rel="nofollow noreferrer">https://source.android.com/compatibility/vts/systems</a>) for the VTS one has to decide which test plan to run. And then use the run command to test it.</p>
  
  <p>E.g. If I want to run the default VTS test plan I run it with.</p>
  
  <pre><code>me@computer&gt; vts-tradefed
  vts-tf &gt; run vts
  </code></pre>
  
  <p>This will launch a number of tests to the connected device.</p>
  
  <p>Next, when launching the CTS tests, I expected to call the corresponding functions, or what appears to be. With the following instructions I expected to run CTS tests with the test plan called "cts".</p>
  
  <pre><code>me@computer&gt; cts-tradefed
  cts-tf &gt; run cts
  </code></pre>
  
  <p>This seems to work fine and the tests appears to start. But then I read in the manual for the CTS (<a href="https://source.android.com/compatibility/cts/run" rel="nofollow noreferrer">https://source.android.com/compatibility/cts/run</a>) that the cts shall be executed as <code>run cts --plan &lt;test-plan&gt;</code>. And they give the example <code>run cts --plan CTS</code> below to run the default cts plan.</p>
  
  <blockquote>
    <p>Start the default test plan (contains all test packages) by appending: run cts --plan CTS . This kicks off all CTS tests required
    for compatibility.</p>
    
    <p>For CTS v1 (Android 6.0 and earlier), enter list plans to view a list of test plans in the repository or list packages to view a list
    of test packages in the repository.
            For CTS v2 (Android 7.0 and later), enter list modules to see a list of test modules. </p>
    
    <p>Alternately, run the CTS plan of your choosing from the command line using: cts-tradefed run cts --plan</p>
  </blockquote>
  
  <p>When testing it it seems to work as well. Two thinks makes me wonder. First of all the test plan in the example are refered to with capital letters, i.e., "CTS" instead of "cts". Secondly the run-command seems to work completely different here. For me it makes sense that the <code>run</code>-command is a built in tradefed command, and that its argument should be the name of the test plan. This is also confirmed by tradefed itself.</p>
  
  <p>VTS help:</p>
  
  <pre><code>vts-tf &gt; help run
  r(?:un)? help:
      command &lt;config&gt; [options]        Run the specified command
      &lt;config&gt; [options]                Shortcut for the above: run specified command
      cmdfile &lt;cmdfile.txt&gt;             Run the specified commandfile
      commandAndExit &lt;config&gt; [options] Run the specified command, and run 'exit -c' immediately afterward
      cmdfileAndExit &lt;cmdfile.txt&gt;      Run the specified commandfile, and run 'exit -c' immediately afterward
  
      ----- Vendor Test Suite specific options ----- 
      &lt;plan&gt; --module/-m &lt;module&gt;       Run a test module
      &lt;plan&gt; --module/-m &lt;module&gt; --test/-t &lt;test_name&gt;    Run a specific test from the module. Test name can be &lt;package&gt;.&lt;class&gt;, &lt;package&gt;.&lt;class&gt;#&lt;method&gt; or &lt;native_binary_name&gt;
          Available Options:
              --serial/-s &lt;device_id&gt;: The device to run the test on
              --abi/-a &lt;abi&gt;         : The ABI to run the test against
              --logcat-on-failure    : Capture logcat when a test fails
              --bugreport-on-failure : Capture a bugreport when a test fails
              --screenshot-on-failure: Capture a screenshot when a test fails
              --shard-count &lt;shards&gt;: Shards a run into the given number of independent chunks, to run on multiple devices in parallel.
       ----- In order to retry a previous run -----
      retry --retry &lt;session id to retry&gt; [--retry-type &lt;FAILED | NOT_EXECUTED&gt;]
          Without --retry-type, retry will run both FAIL and NOT_EXECUTED tests
  </code></pre>
  
  <p>CTS help:</p>
  
  <pre><code>cts-tf &gt; help run
  r(?:un)? help:
      command &lt;config&gt; [options]        Run the specified command
      &lt;config&gt; [options]                Shortcut for the above: run specified command
      cmdfile &lt;cmdfile.txt&gt;             Run the specified commandfile
      commandAndExit &lt;config&gt; [options] Run the specified command, and run 'exit -c' immediately afterward
      cmdfileAndExit &lt;cmdfile.txt&gt;      Run the specified commandfile, and run 'exit -c' immediately afterward
  
      ----- Compatibility Test Suite specific options ----- 
      &lt;plan&gt; --module/-m &lt;module&gt;       Run a test module
      &lt;plan&gt; --module/-m &lt;module&gt; --test/-t &lt;test_name&gt;    Run a specific test from the module. Test name can be &lt;package&gt;.&lt;class&gt;, &lt;package&gt;.&lt;class&gt;#&lt;method&gt; or &lt;native_binary_name&gt;
          Available Options:
              --serial/-s &lt;device_id&gt;: The device to run the test on
              --abi/-a &lt;abi&gt;         : The ABI to run the test against
              --logcat-on-failure    : Capture logcat when a test fails
              --bugreport-on-failure : Capture a bugreport when a test fails
              --screenshot-on-failure: Capture a screenshot when a test fails
              --shard-count &lt;shards&gt;: Shards a run into the given number of independent chunks, to run on multiple devices in parallel.
       ----- In order to retry a previous run -----
      retry --retry &lt;session id to retry&gt; [--retry-type &lt;FAILED | NOT_EXECUTED&gt;]
          Without --retry-type, retry will run both FAIL and NOT_EXECUTED tests
  </code></pre>
  
  <p>The explanations are pretty much identical. So it actually makes sense to run with <code>run cts</code> and <code>run vts</code> repectively. Is this just a silly question and am I completely wrong? Since these tests are important for our compability I want to be certain to have them run in the correct way.</p>