{"action":"created","issue":{"url":"https://api.github.com/repos/Axelrod-Python/Axelrod/issues/672","repository_url":"https://api.github.com/repos/Axelrod-Python/Axelrod","labels_url":"https://api.github.com/repos/Axelrod-Python/Axelrod/issues/672/labels{/name}","comments_url":"https://api.github.com/repos/Axelrod-Python/Axelrod/issues/672/comments","events_url":"https://api.github.com/repos/Axelrod-Python/Axelrod/issues/672/events","html_url":"https://github.com/Axelrod-Python/Axelrod/pull/672","id":167897365,"number":672,"title":"Re write of the ResultsSetFromFile","user":{"login":"drvinceknight","id":2131546,"avatar_url":"https://avatars.githubusercontent.com/u/2131546?v=3","gravatar_id":"","url":"https://api.github.com/users/drvinceknight","html_url":"https://github.com/drvinceknight","followers_url":"https://api.github.com/users/drvinceknight/followers","following_url":"https://api.github.com/users/drvinceknight/following{/other_user}","gists_url":"https://api.github.com/users/drvinceknight/gists{/gist_id}","starred_url":"https://api.github.com/users/drvinceknight/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/drvinceknight/subscriptions","organizations_url":"https://api.github.com/users/drvinceknight/orgs","repos_url":"https://api.github.com/users/drvinceknight/repos","events_url":"https://api.github.com/users/drvinceknight/events{/privacy}","received_events_url":"https://api.github.com/users/drvinceknight/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/Axelrod-Python/Axelrod/labels/1-positive-review","name":"1-positive-review","color":"009800"},{"url":"https://api.github.com/repos/Axelrod-Python/Axelrod/labels/ready-for-review","name":"ready-for-review","color":"eb6420"}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":12,"created_at":"2016-07-27T16:24:43Z","updated_at":"2016-07-28T15:14:06Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/Axelrod-Python/Axelrod/pulls/672","html_url":"https://github.com/Axelrod-Python/Axelrod/pull/672","diff_url":"https://github.com/Axelrod-Python/Axelrod/pull/672.diff","patch_url":"https://github.com/Axelrod-Python/Axelrod/pull/672.patch"},"body":"Closes #573 \r\n\r\nThis changes the `ResultsSetFromFile` to read from disk (potentially just once) and do all the calculations as that happens.\r\n\r\nOn #573 the suggestion was originally to have this class as a separate `BigResultClass` but it's not only faster it's also (**much**) quicker so I just suggest we make this the default.\r\n\r\n- If creating the results set from `tournament.play` then the data will only be read once. If creating from a file and the number of interactions, list of players and number of repetitions is not known then you will read through the file 3 times (but it's still fast).\r\n- There is an option to `keep_interactions` which can be passed to the `tournament.play` method. \r\n- The progress bar is now **much** more granular.\r\n\r\nOne negative thing: running the tests spits out a lot of gunk. This seems to have something to do with the two hypothesis tests (that basically check that the results line up with the base results set) and the files not being closed properly. Not sure what can be done here. Not sure if this is a real problem (all errors/failures are reported after the gunk, it's just a bit more messy).\r\n\r\nHere's the memory profile (just running the new class). Note that analysing these 5 tournaments took a total of 8 minutes on a small 2 core machine (this is very fast relative to before: I think this points out how bad the previous way, that I wrote, of doing things was):\r\n\r\n```\r\n\r\nLine #    Mem usage    Increment   Line Contents\r\n================================================\r\n\r\n    19                                 # Demo strategies\r\n    20                                 # Turns: 200\r\n    21                                 # Repetitions: 100\r\n\r\n    23  25.78906 MiB   0.57422 MiB       read(filename)\r\n\r\n    27                                 # Basic strategies\r\n    28                                 # Turns: 200\r\n    29                                 # Repetitions: 100\r\n\r\n    31  26.42188 MiB   0.63281 MiB       read(filename)                         \r\n\r\n    35                                 # Ordinary strategies\r\n    36                                 # Turns: 50\r\n    37                                 # Repetitions: 5\r\n\r\n    39  29.92188 MiB   3.50000 MiB       read(filename)\r\n\r\n    43                                 # Ordinary strategies\r\n    44                                 # Turns: 50\r\n    45                                 # Repetitions: 10\r\n\r\n    47  33.43359 MiB   3.51172 MiB       read(filename)                         \r\n\r\n    51                                 # Ordinary strategies\r\n    52                                 # Turns: 50\r\n    53                                 # Repetitions: 20\r\n\r\n    55  35.88281 MiB   2.44922 MiB       read(filename)\r\n```\r\n\r\n**TLDR: the largest tournament (all strategies, 50 tournaments, 20 repetitions) takes 0.003 GB of ram)** <- That assumes I'm reading this correctly (?). I'm rerunning that now with the master branch just to compare one last time (this will take a couple of hours).\r\n"},"comment":{"url":"https://api.github.com/repos/Axelrod-Python/Axelrod/issues/comments/235925972","html_url":"https://github.com/Axelrod-Python/Axelrod/pull/672#issuecomment-235925972","issue_url":"https://api.github.com/repos/Axelrod-Python/Axelrod/issues/672","id":235925972,"user":{"login":"drvinceknight","id":2131546,"avatar_url":"https://avatars.githubusercontent.com/u/2131546?v=3","gravatar_id":"","url":"https://api.github.com/users/drvinceknight","html_url":"https://github.com/drvinceknight","followers_url":"https://api.github.com/users/drvinceknight/followers","following_url":"https://api.github.com/users/drvinceknight/following{/other_user}","gists_url":"https://api.github.com/users/drvinceknight/gists{/gist_id}","starred_url":"https://api.github.com/users/drvinceknight/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/drvinceknight/subscriptions","organizations_url":"https://api.github.com/users/drvinceknight/orgs","repos_url":"https://api.github.com/users/drvinceknight/repos","events_url":"https://api.github.com/users/drvinceknight/events{/privacy}","received_events_url":"https://api.github.com/users/drvinceknight/received_events","type":"User","site_admin":false},"created_at":"2016-07-28T15:14:06Z","updated_at":"2016-07-28T15:14:06Z","body":"👍  (I've just reverted the garbage collector.)"}}
{"actor":{"display_login":"drvinceknight"}}