{"action":"created","issue":{"url":"https://api.github.com/repos/threatgrid/ctia/issues/269","repository_url":"https://api.github.com/repos/threatgrid/ctia","labels_url":"https://api.github.com/repos/threatgrid/ctia/issues/269/labels{/name}","comments_url":"https://api.github.com/repos/threatgrid/ctia/issues/269/comments","events_url":"https://api.github.com/repos/threatgrid/ctia/issues/269/events","html_url":"https://github.com/threatgrid/ctia/issues/269","id":151988659,"number":269,"title":"Generative tests should shrink","user":{"login":"polygloton","id":368005,"avatar_url":"https://avatars.githubusercontent.com/u/368005?v=3","gravatar_id":"","url":"https://api.github.com/users/polygloton","html_url":"https://github.com/polygloton","followers_url":"https://api.github.com/users/polygloton/followers","following_url":"https://api.github.com/users/polygloton/following{/other_user}","gists_url":"https://api.github.com/users/polygloton/gists{/gist_id}","starred_url":"https://api.github.com/users/polygloton/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/polygloton/subscriptions","organizations_url":"https://api.github.com/users/polygloton/orgs","repos_url":"https://api.github.com/users/polygloton/repos","events_url":"https://api.github.com/users/polygloton/events{/privacy}","received_events_url":"https://api.github.com/users/polygloton/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2016-04-29T23:37:07Z","updated_at":"2016-05-03T14:04:06Z","closed_at":null,"body":"We have generative tests that take, for example, 100 samples and then applies them to an expectation.  When there is test failure, however, that means that we get up to 100 test failure prints for each test (and there are several).  That sort of misses the point of test.check.  We need to take advantage of the features for shrinking tests failures to a single failure message.\r\n\r\nSee for example:\r\nhttps://github.com/polygloton/ctia/blob/issue-169-spike/test/ctia/domain/id_test.clj"},"comment":{"url":"https://api.github.com/repos/threatgrid/ctia/issues/comments/216537641","html_url":"https://github.com/threatgrid/ctia/issues/269#issuecomment-216537641","issue_url":"https://api.github.com/repos/threatgrid/ctia/issues/269","id":216537641,"user":{"login":"yogsototh","id":93899,"avatar_url":"https://avatars.githubusercontent.com/u/93899?v=3","gravatar_id":"","url":"https://api.github.com/users/yogsototh","html_url":"https://github.com/yogsototh","followers_url":"https://api.github.com/users/yogsototh/followers","following_url":"https://api.github.com/users/yogsototh/following{/other_user}","gists_url":"https://api.github.com/users/yogsototh/gists{/gist_id}","starred_url":"https://api.github.com/users/yogsototh/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/yogsototh/subscriptions","organizations_url":"https://api.github.com/users/yogsototh/orgs","repos_url":"https://api.github.com/users/yogsototh/repos","events_url":"https://api.github.com/users/yogsototh/events{/privacy}","received_events_url":"https://api.github.com/users/yogsototh/received_events","type":"User","site_admin":false},"created_at":"2016-05-03T14:04:06Z","updated_at":"2016-05-03T14:04:06Z","body":"After taking a look, I believe that it might be difficult to code some tests using `defspec`.\n\nFor example,  in my generative tests I have often this structure:\n\n1. generate a sequence of entities,\n2. generate another sequence of entities depending on the first sequence,\n3. generate another sequence depending of the two previous sequences...\n\nFurthermore we use the `def-test-for-each-store` macro.\nWe would then have to make another `defspec-for-each-store` macro.\n\nIt might be a bit hard to rewrite this kind of tests directly.\n\nSo, I believe in a first time we could just use `reduced` to short circuit tests.\nStopping on the first failure will be an improvement.\n\n~~~clojure\n(deftest dummy-short-circuited-test\n  (reduce\n    (fn [_ i]\n      (if (= i (+ i 1))\n        (is (= i (+ i 1)))\n        (reduced (is (= i (+ i 1))))))\n    (range 100)))\n~~~\n\nThis is far from perfect, but that could help to make test output clearer."}}