{"action":"created","issue":{"url":"https://api.github.com/repos/HPCE/hpce-2016-cw1/issues/15","repository_url":"https://api.github.com/repos/HPCE/hpce-2016-cw1","labels_url":"https://api.github.com/repos/HPCE/hpce-2016-cw1/issues/15/labels{/name}","comments_url":"https://api.github.com/repos/HPCE/hpce-2016-cw1/issues/15/comments","events_url":"https://api.github.com/repos/HPCE/hpce-2016-cw1/issues/15/events","html_url":"https://github.com/HPCE/hpce-2016-cw1/issues/15","id":183221798,"number":15,"title":"Evaluating the speed-up graphs","user":{"login":"majpoynton","id":9308721,"avatar_url":"https://avatars.githubusercontent.com/u/9308721?v=3","gravatar_id":"","url":"https://api.github.com/users/majpoynton","html_url":"https://github.com/majpoynton","followers_url":"https://api.github.com/users/majpoynton/followers","following_url":"https://api.github.com/users/majpoynton/following{/other_user}","gists_url":"https://api.github.com/users/majpoynton/gists{/gist_id}","starred_url":"https://api.github.com/users/majpoynton/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/majpoynton/subscriptions","organizations_url":"https://api.github.com/users/majpoynton/orgs","repos_url":"https://api.github.com/users/majpoynton/repos","events_url":"https://api.github.com/users/majpoynton/events{/privacy}","received_events_url":"https://api.github.com/users/majpoynton/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":6,"created_at":"2016-10-15T17:26:51Z","updated_at":"2016-10-16T11:08:03Z","closed_at":null,"body":"The spec says\r\n\r\n> For each of the implementations you will produce a graph\r\n\r\nwhich must\r\n\r\n> Compare all four implementations in some meaningful way\r\n\r\nand then goes on to say \r\n\r\n> Task: Evaluate the scaling against max-iter (x-axis) with the baseline width, height, max-frames, and save it as results/scaling_max_iter.pdf.\r\nTask: Evaluate the scaling against dimension (x-axis), where dimension (N) is defined the same way as the earlier experiment (image size is NxN) and save it as results/scaling_dimension.pdf.\r\nTask: Evaluate the scaling with width, and save as results/scaling_width.pdf.\r\nTask: Evaluate the scaling with height, and save as results/scaling_height.pdf.\r\nTask: Evaluate the scaling with max-frames, and save as results/scaling_max_frames.pdf.\r\n\r\nI might be a little confused here, but if each graph compares all four implementations wouldn't that make 3 of the graphs redundant?\r\n\r\nSo are we supposed to make 4 graphs (one for each implementation), then another 5 graphs (scaling_max_iter.pdf etc.) where the 5 scaling experiments are only done on parallel_both, to make 9 in total, or are we supposed to make 20 graphs, i.e. do the experiments scaling each of the 5 parameters for each of the 4 implementations? Or what are we supposed to do?\r\nSorry if I've misunderstood.\r\n\r\nThanks\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/HPCE/hpce-2016-cw1/issues/comments/254040512","html_url":"https://github.com/HPCE/hpce-2016-cw1/issues/15#issuecomment-254040512","issue_url":"https://api.github.com/repos/HPCE/hpce-2016-cw1/issues/15","id":254040512,"user":{"login":"m8pple","id":3024705,"avatar_url":"https://avatars.githubusercontent.com/u/3024705?v=3","gravatar_id":"","url":"https://api.github.com/users/m8pple","html_url":"https://github.com/m8pple","followers_url":"https://api.github.com/users/m8pple/followers","following_url":"https://api.github.com/users/m8pple/following{/other_user}","gists_url":"https://api.github.com/users/m8pple/gists{/gist_id}","starred_url":"https://api.github.com/users/m8pple/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/m8pple/subscriptions","organizations_url":"https://api.github.com/users/m8pple/orgs","repos_url":"https://api.github.com/users/m8pple/repos","events_url":"https://api.github.com/users/m8pple/events{/privacy}","received_events_url":"https://api.github.com/users/m8pple/received_events","type":"User","site_admin":false},"created_at":"2016-10-16T11:08:03Z","updated_at":"2016-10-16T11:08:03Z","body":"@nar213 and @jeremych1000 have the right idea.\r\n\r\nPart of what I'm trying to get people used to is the idea of performance\r\nas being relative, which means it has to be relative to something. As you\r\nincrementally change a program the performance will change in different\r\nways for different parameters, so you need to be able to look at them\r\nside-by-side on a single graph.\r\n\r\nThis is motivated by experiences where people will produce independent\r\ngraphs for different implementations, with different y axis scales. The\r\ndata is there, but not in an actionable format (what can I learn from it?)\r\n\r\nA comment on @nar213 's graph - it shows the right data, and is exactly\r\nthe kind of comparison I'm looking for, but some possible tweaks:\r\n- You are varying an independent parameter, so in principle it is a curve. Connecting\r\n   the points with a line makes sense in that context (though with straight\r\n   lines rather than curves (curve interpolation is usually bad). However, keep\r\n   the points as well, so we can see where the sample points are.\r\n- The key thing we are investigating is how this is scaling with a parallel implementation\r\n   on multiple cores, and looking at how it scales compared to a sequential implementation.\r\n   So a key piece of information (e.g. in the title) becomes...\r\n- Don't forget labels on the x and y axis. Is time in seconds or minutes? What is the parameter\r\n\r\n(I realise you might have cut those out before putting the graph up here).\r\n"}}
{"actor":{"display_login":"m8pple"}}