{"action":"created","issue":{"url":"https://api.github.com/repos/JoinMarket-Org/joinmarket/issues/477","repository_url":"https://api.github.com/repos/JoinMarket-Org/joinmarket","labels_url":"https://api.github.com/repos/JoinMarket-Org/joinmarket/issues/477/labels{/name}","comments_url":"https://api.github.com/repos/JoinMarket-Org/joinmarket/issues/477/comments","events_url":"https://api.github.com/repos/JoinMarket-Org/joinmarket/issues/477/events","html_url":"https://github.com/JoinMarket-Org/joinmarket/pull/477","id":142436374,"number":477,"title":"Test improvements","user":{"login":"AdamISZ","id":4278257,"avatar_url":"https://avatars.githubusercontent.com/u/4278257?v=3","gravatar_id":"","url":"https://api.github.com/users/AdamISZ","html_url":"https://github.com/AdamISZ","followers_url":"https://api.github.com/users/AdamISZ/followers","following_url":"https://api.github.com/users/AdamISZ/following{/other_user}","gists_url":"https://api.github.com/users/AdamISZ/gists{/gist_id}","starred_url":"https://api.github.com/users/AdamISZ/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/AdamISZ/subscriptions","organizations_url":"https://api.github.com/users/AdamISZ/orgs","repos_url":"https://api.github.com/users/AdamISZ/repos","events_url":"https://api.github.com/users/AdamISZ/events{/privacy}","received_events_url":"https://api.github.com/users/AdamISZ/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":3,"created_at":"2016-03-21T18:45:47Z","updated_at":"2016-03-23T13:26:03Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/JoinMarket-Org/joinmarket/pulls/477","html_url":"https://github.com/JoinMarket-Org/joinmarket/pull/477","diff_url":"https://github.com/JoinMarket-Org/joinmarket/pull/477.diff","patch_url":"https://github.com/JoinMarket-Org/joinmarket/pull/477.patch"},"body":"This is a combination of a few things with the goal of improving our tests:\r\n\r\n#378 from @WyseNynja (the commit is included and a merge) to use pytest, see his explanation there.\r\n\r\n@chris-belcher 's set of transaction serialize/deserialize tests, with a bunch more added from Bitcoin Core [here](https://github.com/bitcoin/bitcoin/blob/master/src/test/data/tx_valid.json) (on the basis that they represent various interesting edge cases, but more for validation, not for serialization; in any case it doesn't hurt). I might add more.\r\n\r\nI have already folded into the pytest setup also the bip32 test vectors, and some valid/invalid bitcoin address tests (already threw up an error, see PR #476).\r\n\r\nNext up would be some more tests for the other pybitcointools functions we currently use and don't currently have an upgrade path for (i.e. tests for EC and ECDSA will *not* be done until we fold in secp256k1, but testing e.g. WIF private keys will).\r\n\r\nProbably the most important practical additional step would be to break down the regtest and tumbler-test and wallet-test scripts to run without use of subprocess, so that they can be included in the pytest setup. miniircd and regtest setup can be automated, but then call the functions in scripts like sendpayment and yieldgenerator directly, so that coverage can be measured. I hope this won't be too hard, but haven't tried yet. Once it's done, the travis PR-tester will be simplified and a lot more useful.\r\n\r\nSo this isn't really for merging until at least that last point is addressed. Apart from that, don't have to do everything at once of course."},"comment":{"url":"https://api.github.com/repos/JoinMarket-Org/joinmarket/issues/comments/200342436","html_url":"https://github.com/JoinMarket-Org/joinmarket/pull/477#issuecomment-200342436","issue_url":"https://api.github.com/repos/JoinMarket-Org/joinmarket/issues/477","id":200342436,"user":{"login":"AdamISZ","id":4278257,"avatar_url":"https://avatars.githubusercontent.com/u/4278257?v=3","gravatar_id":"","url":"https://api.github.com/users/AdamISZ","html_url":"https://github.com/AdamISZ","followers_url":"https://api.github.com/users/AdamISZ/followers","following_url":"https://api.github.com/users/AdamISZ/following{/other_user}","gists_url":"https://api.github.com/users/AdamISZ/gists{/gist_id}","starred_url":"https://api.github.com/users/AdamISZ/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/AdamISZ/subscriptions","organizations_url":"https://api.github.com/users/AdamISZ/orgs","repos_url":"https://api.github.com/users/AdamISZ/repos","events_url":"https://api.github.com/users/AdamISZ/events{/privacy}","received_events_url":"https://api.github.com/users/AdamISZ/received_events","type":"User","site_admin":false},"created_at":"2016-03-23T13:26:03Z","updated_at":"2016-03-23T13:26:03Z","body":"(I will continue to add commits and squash at the end when ready)\r\n\r\nA big step forward now:\r\n\r\nFully automated end to end test is working. The bitcoin(regtest) and irc(miniircd, snapshot included) will automatically start up and shut down within the test framework. Pre-configuration of bitcoin.conf and rpc creds aside, there is nothing else needed to do a test run. The command to run tests is like this:\r\n\r\n    PYTHONPATH=.:$PYTHONPATH py.test -s --btcroot=/path/to/bitcoin/bin --btcconf=/path/to/bitcoin.conf --btcpwd=password\r\n\r\nTakes around 1 minute at the moment, but will get longer in future.\r\n\r\nTests currently included are for: bitcoin addresses, libnacl encryption, transaction serialization, and sendpayment.\r\n\r\nHow I've addressed the above discussed issue \"how to do tests without subprocess\": sendpayment is run as a script here, and the yield generators are subprocesses (so not within the code coverage). The idea is to make several tests, in each of which only one bot is being covered by the tests, while all the others run in the background. In a way it's clunkier, but it's probably more correct anyway: isolating a test to see if we can trigger errors in a specific bot instance. I'd envisage having 3 or 4 \"main tests\" of standard running conditions that can be included in travis, with more, deeper, more complex tests being possible to run outside travis in a bigger test suite that takes longer. Pytest makes it easy to do that by passing command line arguments that filter which of the various test functions are to be included.\r\n\r\nThe coverage is now ~ 50% and will increase further with some trivial additions/changes, but note the last 10-20% or so of coverage for bitcoin code will of course not be addressed (EC stuff) deliberately until secp256k1."}}