{"action":"opened","issue":{"url":"https://api.github.com/repos/blue-yonder/tsfresh/issues/413","repository_url":"https://api.github.com/repos/blue-yonder/tsfresh","labels_url":"https://api.github.com/repos/blue-yonder/tsfresh/issues/413/labels{/name}","comments_url":"https://api.github.com/repos/blue-yonder/tsfresh/issues/413/comments","events_url":"https://api.github.com/repos/blue-yonder/tsfresh/issues/413/events","html_url":"https://github.com/blue-yonder/tsfresh/issues/413","id":352229502,"node_id":"MDU6SXNzdWUzNTIyMjk1MDI=","number":413,"title":"SampleTauTestCase.test_ratio failing inconsistently","user":{"login":"earthgecko","id":96679,"node_id":"MDQ6VXNlcjk2Njc5","avatar_url":"https://avatars1.githubusercontent.com/u/96679?v=4","gravatar_id":"","url":"https://api.github.com/users/earthgecko","html_url":"https://github.com/earthgecko","followers_url":"https://api.github.com/users/earthgecko/followers","following_url":"https://api.github.com/users/earthgecko/following{/other_user}","gists_url":"https://api.github.com/users/earthgecko/gists{/gist_id}","starred_url":"https://api.github.com/users/earthgecko/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/earthgecko/subscriptions","organizations_url":"https://api.github.com/users/earthgecko/orgs","repos_url":"https://api.github.com/users/earthgecko/repos","events_url":"https://api.github.com/users/earthgecko/events{/privacy}","received_events_url":"https://api.github.com/users/earthgecko/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2018-08-20T17:35:25Z","updated_at":"2018-08-20T17:35:25Z","closed_at":null,"author_association":"CONTRIBUTOR","body":"The SampleTauTestCase.test_ratio test is failing inconsistently.  The self.assertTrue\r\nwas set at the new value of 0.002 introduced at https://github.com/blue-yonder/tsfresh/commit/a57a09fe62a62fe0d2564a056f7fd99f58822312 and this fails between 14 and 24\r\ntimes in 100 in my testing.\r\n\r\nTesting this at the original 0.001 value pre the above commit results in failure\r\nbetween 46 and 61 times in 100 in my testing.\r\nFurther testing at 0.003, 0.004 and 0.005 all generate a decreasing amount of\r\nerrors.\r\n\r\nUsing the value of 0.006 seems to generate 0 errors on repeated tests of 100\r\niterations.\r\n\r\nThis test can be easily tested and reproduced with the following code:\r\n\r\n```python\r\nimport numpy as np\r\nfrom tsfresh.examples.driftbif_simulation import sample_tau\r\n\r\nless_than_value = 0.002\r\n# less_than_value = 0.003\r\n# less_than_value = 0.004\r\n# less_than_value = 0.005\r\n# less_than_value = 0.006\r\n\r\ncount = 0\r\nerrors = 0\r\nfor i in range(100):\r\n    count += 1\r\n    tau = sample_tau(100000, ratio=0.4)\r\n    sample = np.array(tau)\r\n    before = np.sum(sample <= 1/0.3)\r\n    beyond = np.sum(sample > 1/0.3)\r\n    abs_value = abs(0.4 - float(before)/(before+beyond))\r\n    if abs_value < less_than_value:\r\n        print('%s - before - %s, beyond - %s, abs_value - %s' % (str(count), str(before), str(beyond), str(abs_value)))\r\n    else:\r\n        print('ERROR :: %s - before - %s, beyond - %s, abs_value - %s' % (str(count), str(before), str(beyond), str(abs_value)))\r\n        errors += 1\r\n\r\nprint('%s errors in %s tests' % (str(errors), str(count)))\r\n```\r\n\r\n@nils-braun any thoughts here, as it stands now this test is going to fail ~14-22% of the time it seems.\r\n\r\n"}}
{"actor":{"display_login":"earthgecko"}}