{"action":"created","issue":{"url":"https://api.github.com/repos/sherpa/sherpa/issues/210","repository_url":"https://api.github.com/repos/sherpa/sherpa","labels_url":"https://api.github.com/repos/sherpa/sherpa/issues/210/labels{/name}","comments_url":"https://api.github.com/repos/sherpa/sherpa/issues/210/comments","events_url":"https://api.github.com/repos/sherpa/sherpa/issues/210/events","html_url":"https://github.com/sherpa/sherpa/pull/210","id":151982550,"number":210,"title":"New smoke test","user":{"login":"olaurino","id":5692321,"avatar_url":"https://avatars.githubusercontent.com/u/5692321?v=3","gravatar_id":"","url":"https://api.github.com/users/olaurino","html_url":"https://github.com/olaurino","followers_url":"https://api.github.com/users/olaurino/followers","following_url":"https://api.github.com/users/olaurino/following{/other_user}","gists_url":"https://api.github.com/users/olaurino/gists{/gist_id}","starred_url":"https://api.github.com/users/olaurino/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/olaurino/subscriptions","organizations_url":"https://api.github.com/users/olaurino/orgs","repos_url":"https://api.github.com/users/olaurino/repos","events_url":"https://api.github.com/users/olaurino/events{/privacy}","received_events_url":"https://api.github.com/users/olaurino/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/sherpa/sherpa/labels/3%20%20-%20Working","name":"3  - Working","color":"ededed"},{"url":"https://api.github.com/repos/sherpa/sherpa/labels/area:tests","name":"area:tests","color":"d4c5f9"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":2,"created_at":"2016-04-29T22:38:21Z","updated_at":"2016-05-02T16:01:01Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/sherpa/sherpa/pulls/210","html_url":"https://github.com/sherpa/sherpa/pull/210","diff_url":"https://github.com/sherpa/sherpa/pull/210.diff","patch_url":"https://github.com/sherpa/sherpa/pull/210.patch"},"body":"# Description\r\nThis PR introduces some changes to the way tests work, especially in CIAO. Not much changes for Standalone Sherpa, except that we are officially bringing in `py.test` and `mock` as dependencies for the tests, so we can start using features from these packages.\r\n\r\nThis PR introduces a new smoke test as a single unittest `TestSuite`, with a convenience high level function in the main namespace. I included some simple tests, but we can beef it up at will, knowing that this is now simply a smoke test. It might be worth adding ds9 tests with the same enforcement mechanism I added for xspec and pycrates/astropy.\r\n\r\nThe new function `smoke` replaces the old `test` and provides some convenient flags to improve the smoke test effectiveness. In particular, one can request a specific fits backend to be present as well as xspec. Failure to load these modules results in a failure of the smoke test.\r\n\r\nA new `sherpa_smoke` console script will be available to both standalone and CIAO users (as we now have `setuptools` in CIAO) as an entry point to the `smoke` function, with options available to make the tests require xspec and/or a specific fits backend. Note that the smoke test will run and skip some tests if xspec and fits backends are not available.\r\n\r\nThe console script and `smoke` function also accept an option to always make the smoke test fail, which can be used to test the new setup, making sure that when the smoke test fails the appropriate responses are given to the user.\r\n\r\nI also removed the `SherpaTest` class and all references to it, as well as the main methods in the tests file, which would make no sense at this point.\r\n\r\nFinally, the PR changes the travis builds. So far we had a special build to run the \"smoke test\", which was really just a subset of unit and regression tests that did not require any test data. I removed the special smoke test build, and I added smoke test runs to the most complex (xspec enabled) and the most basic (no xspec, no matplotlib, no astropy, no setuptools) build.\r\n\r\nAs we are going to include setuptools in CIAO as well, we can probably get rid of the \"nosetuptools\" build.\r\n\r\n# Testing\r\nThe smoke test has its own unit tests which make sure it properly outputs error messages and tries to exit with an error status if backends are not available (even in builds that have the backends or xspec), and that it runs successfully when they are indeed present.\r\n\r\nThe test classes in the smoke test suite (so far only one class) are also run as part of the unit and regression tests. Test cases requiring backends are skipped accordingly."},"comment":{"url":"https://api.github.com/repos/sherpa/sherpa/issues/comments/216276920","html_url":"https://github.com/sherpa/sherpa/pull/210#issuecomment-216276920","issue_url":"https://api.github.com/repos/sherpa/sherpa/issues/210","id":216276920,"user":{"login":"DougBurke","id":224638,"avatar_url":"https://avatars.githubusercontent.com/u/224638?v=3","gravatar_id":"","url":"https://api.github.com/users/DougBurke","html_url":"https://github.com/DougBurke","followers_url":"https://api.github.com/users/DougBurke/followers","following_url":"https://api.github.com/users/DougBurke/following{/other_user}","gists_url":"https://api.github.com/users/DougBurke/gists{/gist_id}","starred_url":"https://api.github.com/users/DougBurke/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/DougBurke/subscriptions","organizations_url":"https://api.github.com/users/DougBurke/orgs","repos_url":"https://api.github.com/users/DougBurke/repos","events_url":"https://api.github.com/users/DougBurke/events{/privacy}","received_events_url":"https://api.github.com/users/DougBurke/received_events","type":"User","site_admin":false},"created_at":"2016-05-02T16:01:01Z","updated_at":"2016-05-02T16:01:01Z","body":"It wasn't clear to me whether this replaces `sherpa_test` in the standalone build or do we now have both?"}}