title: How to add a reporter to testthat?
body: <p>I'm trying to add a JUnit reporter to testthat. While I can do it <strong>inside</strong> the package (<a href="https://github.com/hadley/testthat/pull/481" rel="nofollow noreferrer">https://github.com/hadley/testthat/pull/481</a>) I cannot make it pass the same set of tests when in a separate package (<a href="https://github.com/lbartnik/testthatJUnit" rel="nofollow noreferrer">https://github.com/lbartnik/testthatJUnit</a>).</p>
  
  <p>My main problem is that tests designed for reporters and present in <code>testthat</code> do not pass if I pull JUnit reporter code to a separate package. I'm guessing that there is something in those tests that "sets the scene" for testing reporters but I'm unable to identify this "something".</p>
  
  <p>Any help much appreciated.</p>
  
  <p>Details follow.</p>
  
  <pre><code>&gt; library(devtools)
  &gt; test("testthatJUnit")
  Loading testthatJUnit
  Testing testthatJUnit
  JUnitReporter: 
  
  DONE ==========================================================================
  &gt; test("testthat")     
  Loading testthat
  unloadNamespace("testthat") not successful, probably because another loaded package depends on it.Forcing unload. If you encounter problems, please restart R.
  Testing testthat
  Bare expectations: 
  Colours: ..
  compare.character: .................
  compare.numeric: .....................
  compare.time: .......
  # ... lines cut ...
  
  DONE ===========================================================================
  &gt; test("testthatJUnit")
  Loading testthatJUnit
  Testing testthatJUnit
  JUnitReporter: .............
  </code></pre>
  
  <p>If I modify the test to see why the first <code>test("testthatJUnit")</code> is silent I see this:</p>
  
  <pre><code>&gt; test("testthatJUnit")
  Loading testthatJUnit
  Testing testthatJUnit
  JUnitReporter: Bare: .1
  Failed -------------------------------------------------------------------------
  1. Error: Junit reporter regression (@test-reporter-junit.R#13) ----------------
  attempt to apply non-function
  1: test_dir("test_dir") at /mnt/storage-a/lukaszb/src/third-party/testthatJUnit/tests/testthat/test-reporter-junit.R:13
  2: test_files(paths, reporter = reporter, env = env, ...)
  3: with_reporter(reporter = current_reporter, results &lt;- lapply(paths, test_file, env = env, 
         reporter = current_reporter, start_end_reporter = FALSE, load_helpers = FALSE))
  4: force(code)
  5: lapply(paths, test_file, env = env, reporter = current_reporter, start_end_reporter = FALSE, 
         load_helpers = FALSE)
  6: FUN(X[[i]], ...)
  7: with_reporter(reporter = reporter, start_end_reporter = start_end_reporter, {
         lister$start_file(basename(path))
         source_file(path, new.env(parent = env), chdir = TRUE)
         end_context()
     })  
  8: force(code)
  9: source_file(path, new.env(parent = env), chdir = TRUE)
  10: eval(exprs, env)
  11: eval(expr, envir, enclos)
  12: expect_that(1, equals(1)) at test_dir/test-bare-expectations.R:3
  13: condition(object)
  14: expect_equal(x, expected, ..., expected.label = label)
  15: expect(comp$equal, sprintf("%s not equal to %s.\n%s", lab_act, lab_exp, comp$message), 
         info = info)
  16: withRestarts(if (expectation_broken(exp)) {
         stop(exp)
     } else {
         signalCondition(exp)
     }, continue_test = function(e) NULL)
  17: withOneRestart(expr, restarts[[1L]])
  18: doWithOneRestart(return(expr), restart)
  19: signalCondition(exp)
  20: (function (e) 
     {   
         register_expectation(e, frame + 11, sys.nframe() - 6)
         invokeRestart("continue_test")
     })(structure(list(message = "1 not equal to 1.\nEqual\n", srcref = NULL), .Names = c("message", 
     "srcref"), class = c("expectation_success", "expectation", "condition")))
  21: register_expectation(e, frame + 11, sys.nframe() - 6)
  22: get_reporter()$add_result(context = get_reporter()$.context, test = test, result = e)
  23: o_apply(self$reporters, "add_result", context = context, test = test, result = result)
  24: lapply(objects, function(x) x[[method]](...))
  25: FUN(X[[i]], ...)
  26: x[[method]](...)
  </code></pre>
  
  <p>But if I run <code>test("testthat")</code> and again <code>test("testthatJUnit")</code> everything comes back to normal:</p>
  
  <pre><code>&gt; test("testthatJUnit")
  Loading testthatJUnit
  Testing testthatJUnit
  JUnitReporter: Bare: ..
  Basic: .......
  empty: 1
  error: 2.3456.
  failures: 7.8..
  helper: .
  skip: S
  </code></pre>
  
  <p>My guess is that there is something in <code>testthat</code>'s tests that sets up the scene for testing reporters but so far I haven't been able to identify it. Any help?</p>