{"action":"created","issue":{"url":"https://api.github.com/repos/napalm-automation/napalm-base/issues/42","repository_url":"https://api.github.com/repos/napalm-automation/napalm-base","labels_url":"https://api.github.com/repos/napalm-automation/napalm-base/issues/42/labels{/name}","comments_url":"https://api.github.com/repos/napalm-automation/napalm-base/issues/42/comments","events_url":"https://api.github.com/repos/napalm-automation/napalm-base/issues/42/events","html_url":"https://github.com/napalm-automation/napalm-base/pull/42","id":161665787,"number":42,"title":"New testing framework","user":{"login":"dbarrosop","id":6246622,"avatar_url":"https://avatars.githubusercontent.com/u/6246622?v=3","gravatar_id":"","url":"https://api.github.com/users/dbarrosop","html_url":"https://github.com/dbarrosop","followers_url":"https://api.github.com/users/dbarrosop/followers","following_url":"https://api.github.com/users/dbarrosop/following{/other_user}","gists_url":"https://api.github.com/users/dbarrosop/gists{/gist_id}","starred_url":"https://api.github.com/users/dbarrosop/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dbarrosop/subscriptions","organizations_url":"https://api.github.com/users/dbarrosop/orgs","repos_url":"https://api.github.com/users/dbarrosop/repos","events_url":"https://api.github.com/users/dbarrosop/events{/privacy}","received_events_url":"https://api.github.com/users/dbarrosop/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":6,"created_at":"2016-06-22T12:13:32Z","updated_at":"2016-07-02T13:22:19Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/napalm-automation/napalm-base/pulls/42","html_url":"https://github.com/napalm-automation/napalm-base/pull/42","diff_url":"https://github.com/napalm-automation/napalm-base/pull/42.diff","patch_url":"https://github.com/napalm-automation/napalm-base/pull/42.patch"},"body":"This is the new testing framework. It's based on py.test this time because I think it's way more interesting and powerful than the rest. This is how it works:\r\n\r\n1. To mock tests create a folder with the test name. Inside create as many folders as you want with different name, each one containing different mocked data. The result will be that for each subfolder a different test case will be generated. See https://github.com/dbarrosop/napalm-eos/tree/test_framework/test/unit/mocked_data\r\n1. If there is no test folder that means is used by the `open` function.\r\n1. I have grouped functions that we were rewriting all the time. For example, how to read a json or a text file. See https://github.com/dbarrosop/napalm-base/blob/test_framework/napalm_base/test/double.py\r\n1. Instead of editing files, to control wether to run the test with mocked data or what is the username, password, etc., we can use env variables. See https://github.com/dbarrosop/napalm-base/blob/test_framework/napalm_base/test/conftest.py#L6\r\n1. The `NotImplemented` check for the tests is done with a decorator. The decorator does more stuff as well, like setting some parameters to figure out the test case. See https://github.com/dbarrosop/napalm-base/blob/test_framework/napalm_base/test/getters.py#L13\r\n\r\nThis is the result of an execution:\r\n\r\n```\r\npy.test test/unit/test_getters.py -v\r\n==================================================================== test session starts =====================================================================\r\nplatform darwin -- Python 2.7.11, pytest-2.9.1, py-1.4.31, pluggy-0.3.1 -- /Users/dbarroso/.virtualenvs/napalm/bin/python2.7\r\ncachedir: .cache\r\nrootdir: /Users/dbarroso/workspace/napalm/napalm-eos, inifile: setup.cfg\r\nplugins: pylama-7.0.7, pythonpath-0.7\r\ncollecting 0 items\r\n[NAPALM] We failed to find test cases for test_get_bgp_neighbors. Please, add test cases in /Users/dbarroso/workspace/napalm/napalm-eos/test/unit/mocked_data/test_get_bgp_neighbors\r\ncollected 5 items\r\n\r\ntest/unit/test_getters.py::TestGetter::test_get_bgp_neighbors_detail[empty_case] <- ../napalm-base/napalm_base/test/getters.py SKIPPED\r\ntest/unit/test_getters.py::TestGetter::test_get_interfaces[normal] <- ../napalm-base/napalm_base/test/getters.py PASSED\r\ntest/unit/test_getters.py::TestGetter::test_get_bgp_neighbors[test_case0] <- ../napalm-base/napalm_base/test/getters.py SKIPPED\r\ntest/unit/test_getters.py::TestGetter::test_get_facts[something] <- ../napalm-base/napalm_base/test/getters.py PASSED\r\ntest/unit/test_getters.py::TestGetter::test_get_facts[something_else] <- ../napalm-base/napalm_base/test/getters.py PASSED\r\n\r\n============================================================ 3 passed, 2 skipped in 0.01 seconds =============================================================\r\n```\r\n\r\nTest `test/unit/test_getters.py::TestGetter::test_get_bgp_neighbors_detail[empty_case]` is skipped because there the method returns a `NotImplemented` exception and `test/unit/test_getters.py::TestGetter::test_get_bgp_neighbors` because there is not test case defined. Not that there is a log entry notifying about that:\r\n\r\n`[NAPALM] We failed to find test cases for test_get_bgp_neighbors. Please, add test cases in /Users/dbarroso/workspace/napalm/napalm-eos/test/unit/mocked_data/test_get_bgp_neighbors`\r\n\r\nSee napalm-automation/napalm-eos#25 for details on how to implement this on a driver."},"comment":{"url":"https://api.github.com/repos/napalm-automation/napalm-base/issues/comments/230101485","html_url":"https://github.com/napalm-automation/napalm-base/pull/42#issuecomment-230101485","issue_url":"https://api.github.com/repos/napalm-automation/napalm-base/issues/42","id":230101485,"user":{"login":"jzohrab","id":1637133,"avatar_url":"https://avatars.githubusercontent.com/u/1637133?v=3","gravatar_id":"","url":"https://api.github.com/users/jzohrab","html_url":"https://github.com/jzohrab","followers_url":"https://api.github.com/users/jzohrab/followers","following_url":"https://api.github.com/users/jzohrab/following{/other_user}","gists_url":"https://api.github.com/users/jzohrab/gists{/gist_id}","starred_url":"https://api.github.com/users/jzohrab/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jzohrab/subscriptions","organizations_url":"https://api.github.com/users/jzohrab/orgs","repos_url":"https://api.github.com/users/jzohrab/repos","events_url":"https://api.github.com/users/jzohrab/events{/privacy}","received_events_url":"https://api.github.com/users/jzohrab/received_events","type":"User","site_admin":false},"created_at":"2016-07-02T13:22:19Z","updated_at":"2016-07-02T13:22:19Z","body":"Re the json diffs: I created a small method in another project to help me diff with testing with large strings which might be useful for this.  Can submit a PR later.  https://github.com/jzohrab/capirca/blob/add_yml_parser/tests/integration/filediff.py"}}
{"actor":{"display_login":"jzohrab"}}