{"action":"created","issue":{"url":"https://api.github.com/repos/miguelmarco/bleachermark/issues/1","repository_url":"https://api.github.com/repos/miguelmarco/bleachermark","labels_url":"https://api.github.com/repos/miguelmarco/bleachermark/issues/1/labels{/name}","comments_url":"https://api.github.com/repos/miguelmarco/bleachermark/issues/1/comments","events_url":"https://api.github.com/repos/miguelmarco/bleachermark/issues/1/events","html_url":"https://github.com/miguelmarco/bleachermark/issues/1","id":176321721,"number":1,"title":"Constructing Bleachermarks with dynamic Benchmarks","user":{"login":"jsrndk","id":3774274,"avatar_url":"https://avatars.githubusercontent.com/u/3774274?v=3","gravatar_id":"","url":"https://api.github.com/users/jsrndk","html_url":"https://github.com/jsrndk","followers_url":"https://api.github.com/users/jsrndk/followers","following_url":"https://api.github.com/users/jsrndk/following{/other_user}","gists_url":"https://api.github.com/users/jsrndk/gists{/gist_id}","starred_url":"https://api.github.com/users/jsrndk/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jsrndk/subscriptions","organizations_url":"https://api.github.com/users/jsrndk/orgs","repos_url":"https://api.github.com/users/jsrndk/repos","events_url":"https://api.github.com/users/jsrndk/events{/privacy}","received_events_url":"https://api.github.com/users/jsrndk/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":13,"created_at":"2016-09-12T08:37:56Z","updated_at":"2016-09-13T12:24:09Z","closed_at":null,"body":"It can be beneficial to specify an infinite list as benchmarks. Use cases include\r\n1. Adaptatively run benchmarks for X seconds\r\n1. Sequentially run benchmarks for X seconds, each benchmark run for Y seconds.\r\n\r\nThis issue is to discuss what a reasonable syntax for supporting this is, in the construct of `Bleachermark` and possibly elsewhere.\r\n\r\nA related discussion is whether to (attempt to) protect the user from combining this with benchmarking strategies in a non-sensical way (e.g. interleaved benchmarking on an infinite list makes no sense)."},"comment":{"url":"https://api.github.com/repos/miguelmarco/bleachermark/issues/comments/246664457","html_url":"https://github.com/miguelmarco/bleachermark/issues/1#issuecomment-246664457","issue_url":"https://api.github.com/repos/miguelmarco/bleachermark/issues/1","id":246664457,"user":{"login":"nthiery","id":545328,"avatar_url":"https://avatars.githubusercontent.com/u/545328?v=3","gravatar_id":"","url":"https://api.github.com/users/nthiery","html_url":"https://github.com/nthiery","followers_url":"https://api.github.com/users/nthiery/followers","following_url":"https://api.github.com/users/nthiery/following{/other_user}","gists_url":"https://api.github.com/users/nthiery/gists{/gist_id}","starred_url":"https://api.github.com/users/nthiery/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/nthiery/subscriptions","organizations_url":"https://api.github.com/users/nthiery/orgs","repos_url":"https://api.github.com/users/nthiery/repos","events_url":"https://api.github.com/users/nthiery/events{/privacy}","received_events_url":"https://api.github.com/users/nthiery/received_events","type":"User","site_admin":false},"created_at":"2016-09-13T12:24:09Z","updated_at":"2016-09-13T12:24:09Z","body":"On Tue, Sep 13, 2016 at 12:28:59AM -0700, Johan Rosenkilde wrote:\n>    The way we are considering it, two pipelines that differ only in the\n>    size of the \"data\" that is generated by the first pipeline element,\n>    they should be two Benchmarks, run in the same Bleachermark. A\n>    \"benchmark\" is really \"a single instance of problem, algorithm\n>    (pipeline) and input size\" which is called many times only to average\n>    over random inputs (of the given size).\n\n+1\n\n>    Every time we call the first function of a benchmark's pipeline we are\n>    serving it a run_id. But this is only supposed to seed a random\n>    generator, look up a line number in a data input file, or similar. In\n>    particular, don't use it for encoding size of problem. That's abuse.\n\n+1\n\nLuckily I did not mean to abuse that run_id :-) Let me try to clarify\nthis with a documentation. Note that this is about the semantic, not\nthe specific syntax:\n\n    BlearcherMark(pipeline, range=iterable)\n\n        pipeline -- a pipeline of functions\n        iterable -- an iterable of data\n\n    Return a collection of benchmarks sharing the same pipeline and\n    differing only by the data fed to the first item of the pipeline.\n\n    The first item `f` of the pipeline shall take a mandatory\n    argument. It can also accept a ``run_id`` optional argument.\n    Typically `f` takes as mandatory input a size (e.g represented by a\n    non negative integer) and generates some random data for that size.\n\n    EXAMPLES::\n\n        sage: B = BleacherMark([random_list, sort], range=NN)\n        sage: B.run(adaptative=True, hint=\"polynomial\")\n\n        sage: def data_gen(n, run_id=None):\n        ....:     random.seed(run_id)\n        ....:     return ...(n)\n        sage: B = BleacherMark([data_gen, f], range=...)\n\nCheers,\n"}}
{"actor":{"display_login":"nthiery"}}