{"action":"created","issue":{"url":"https://api.github.com/repos/ARMmbed/htrun/issues/60","labels_url":"https://api.github.com/repos/ARMmbed/htrun/issues/60/labels{/name}","comments_url":"https://api.github.com/repos/ARMmbed/htrun/issues/60/comments","events_url":"https://api.github.com/repos/ARMmbed/htrun/issues/60/events","html_url":"https://github.com/ARMmbed/htrun/issues/60","id":127637172,"number":60,"title":"Missing setup() / teardown() call in main htrun flow","user":{"login":"PrzemekWirkus","id":6360720,"avatar_url":"https://avatars.githubusercontent.com/u/6360720?v=3","gravatar_id":"","url":"https://api.github.com/users/PrzemekWirkus","html_url":"https://github.com/PrzemekWirkus","followers_url":"https://api.github.com/users/PrzemekWirkus/followers","following_url":"https://api.github.com/users/PrzemekWirkus/following{/other_user}","gists_url":"https://api.github.com/users/PrzemekWirkus/gists{/gist_id}","starred_url":"https://api.github.com/users/PrzemekWirkus/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/PrzemekWirkus/subscriptions","organizations_url":"https://api.github.com/users/PrzemekWirkus/orgs","repos_url":"https://api.github.com/users/PrzemekWirkus/repos","events_url":"https://api.github.com/users/PrzemekWirkus/events{/privacy}","received_events_url":"https://api.github.com/users/PrzemekWirkus/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/ARMmbed/htrun/labels/bug","name":"bug","color":"fc2929"}],"state":"open","locked":false,"assignee":{"login":"stegut01","id":14980131,"avatar_url":"https://avatars.githubusercontent.com/u/14980131?v=3","gravatar_id":"","url":"https://api.github.com/users/stegut01","html_url":"https://github.com/stegut01","followers_url":"https://api.github.com/users/stegut01/followers","following_url":"https://api.github.com/users/stegut01/following{/other_user}","gists_url":"https://api.github.com/users/stegut01/gists{/gist_id}","starred_url":"https://api.github.com/users/stegut01/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/stegut01/subscriptions","organizations_url":"https://api.github.com/users/stegut01/orgs","repos_url":"https://api.github.com/users/stegut01/repos","events_url":"https://api.github.com/users/stegut01/events{/privacy}","received_events_url":"https://api.github.com/users/stegut01/received_events","type":"User","site_admin":false},"milestone":null,"comments":2,"created_at":"2016-01-20T09:14:56Z","updated_at":"2016-01-20T12:59:27Z","closed_at":null,"body":"# Description\r\nWe are [only calling ```setup()``` / ```teardown()```](https://github.com/ARMmbed/htrun/blob/master/mbed_host_tests/__init__.py#L413) when we handle ```---run``` switch (not used in CI). We are not calling setup/teardown in normal workflow:\r\n```python\r\ndef execute_run(self):\r\n...\r\ntry:\r\n    self.test_supervisor = get_host_test(\"run_binary_auto\")\r\n    # Call to setup if function is implemented\r\n    if hasattr(self.test_supervisor, 'setup') and callable(getattr(self.test_supervisor, 'setup')):\r\n        self.test_supervisor.setup()\r\n\r\n    # Call to test function\r\n    result = self.test_supervisor.test(self)    # This is blocking, waits for {end}\r\n\r\n    # Call to teardown if function is implemented\r\n    if hasattr(self.test_supervisor, 'teardown') and callable(getattr(self.test_supervisor, 'teardown')):\r\n        self.test_supervisor.teardown()\r\nexcept Exception, e:\r\n    print str(e)\r\n    self.print_result(self.RESULT_ERROR)\r\n```\r\n\r\nWe should be calling it also [here](https://github.com/ARMmbed/htrun/blob/master/mbed_host_tests/__init__.py#L480):\r\n```python\r\ndef execute(self):\r\n...\r\ntry:\r\n    CONFIG = self.detect_test_config(verbose=True) # print CONFIG\r\n\r\n    result = None\r\n    if \"host_test_name\" in CONFIG:\r\n        if is_host_test(CONFIG[\"host_test_name\"]):\r\n            #self.notify(\"HOST: CONFIG['host_test_name'] is '%s'\" % CONFIG[\"host_test_name\"])\r\n            self.test_supervisor = get_host_test(CONFIG[\"host_test_name\"])\r\n            result = self.test_supervisor.test(self)    #result = self.test()\r\n        else:\r\n            self.notify(\"HOST: Error! Unknown host test name '%s' (use 'mbedhtrun --list' to verify)!\"% CONFIG[\"host_test_name\"])\r\n            self.notify(\"HOST: Error! You can use switch '-e <dir>' to specify local directory with host tests to load\")\r\n            self.print_result(self.RESULT_ERROR)\r\n    else:\r\n        self.notify(\"HOST: Error! No host test name defined in preamble\")\r\n        self.print_result(self.RESULT_ERROR)\r\n```\r\n\r\nprobably missing code in above snippet:\r\n```python\r\nself.test_supervisor.setup()\r\nresult = self.test_supervisor.test(self)    #result = self.test()\r\nself.test_supervisor.teardown()\r\n```\r\n\r\n# Reproduce\r\n* In [mbed_host_tests/host_tests/default_auto.py](https://github.com/ARMmbed/htrun/blob/master/mbed_host_tests/host_tests/default_auto.py#L21) add setup/teardown with prints:\r\n```python\r\nclass DefaultAuto(BaseHostTest):\r\n    \"\"\" Simple, basic host test's test runner waiting for serial port\r\n        output from MUT, no supervision over test running in MUT is executed.\r\n    \"\"\"\r\n    def setup(self):\r\n        stdout.write(\"!!!\")\r\n        stdout.flush()\r\n        \r\n    def teardown(self):\r\n        stdout.write(\"@@@\")\r\n        stdout.flush()\r\n    \r\n    def test(self, selftest):\r\n        stdout.write(\"%%%\")\r\n        stdout.flush()\r\n        return \"XXX\"\r\n```\r\n\r\n* Add calls to setup/teardown [here](https://github.com/ARMmbed/htrun/blob/master/mbed_host_tests/__init__.py#L488):\r\n```python\r\n                    self.test_supervisor = get_host_test(CONFIG[\"host_test_name\"])\r\n                    self.test_supervisor.setup()\r\n                    result = self.test_supervisor.test(self)    #result = self.test()\r\n                    self.test_supervisor.teardown()\r\n```\r\n\r\n* Call ```mbedgt```:\r\n```\r\n$ mbedgt -V -n mbed-drivers-test-basic\r\n...\r\nmbedgt: selecting test case observer...\r\n        calling mbedhtrun: mbedhtrun -d F: -p COM184:9600 -f \".\\build\\frdm-k64f-gcc\\test\\mbed-drivers-test-basic.bin\" -C 4 -c shell -m K64F -e \"./test/host_tests\"\r\nmbedgt: mbed-host-test-runner: started\r\nHOST: Instrumentation: \"COM184\" and disk: \"F:\"\r\nHOST: My PID is 7516\r\nHOST: Copy image onto target...\r\n        1 file(s) copied.\r\nHOST: Initialize serial port...\r\n...port ready!\r\nHOST: Reset target...\r\nHOST: Detecting test case properties...\r\nHOST: Property 'timeout' = '20'\r\nHOST: Property 'host_test_name' = 'default_auto'\r\nHOST: Property 'description' = 'Basic'\r\nHOST: Property 'test_id' = 'MBED_A1'\r\nHOST: Start test...\r\n!!!%%%@@@\r\n{{XXX}}\r\n{{end}}\r\n```"},"comment":{"url":"https://api.github.com/repos/ARMmbed/htrun/issues/comments/173196538","html_url":"https://github.com/ARMmbed/htrun/issues/60#issuecomment-173196538","issue_url":"https://api.github.com/repos/ARMmbed/htrun/issues/60","id":173196538,"user":{"login":"PrzemekWirkus","id":6360720,"avatar_url":"https://avatars.githubusercontent.com/u/6360720?v=3","gravatar_id":"","url":"https://api.github.com/users/PrzemekWirkus","html_url":"https://github.com/PrzemekWirkus","followers_url":"https://api.github.com/users/PrzemekWirkus/followers","following_url":"https://api.github.com/users/PrzemekWirkus/following{/other_user}","gists_url":"https://api.github.com/users/PrzemekWirkus/gists{/gist_id}","starred_url":"https://api.github.com/users/PrzemekWirkus/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/PrzemekWirkus/subscriptions","organizations_url":"https://api.github.com/users/PrzemekWirkus/orgs","repos_url":"https://api.github.com/users/PrzemekWirkus/repos","events_url":"https://api.github.com/users/PrzemekWirkus/events{/privacy}","received_events_url":"https://api.github.com/users/PrzemekWirkus/received_events","type":"User","site_admin":false},"created_at":"2016-01-20T12:59:27Z","updated_at":"2016-01-20T12:59:27Z","body":"@mazimkhan Do you think that there is a chance for us to call teardown twice?\r\n* Once we try to kill a process and we already called a teardown in a setup-test-teardown flow?"}}