{"action":"created","issue":{"url":"https://api.github.com/repos/PerfDotNet/BenchmarkDotNet/issues/200","repository_url":"https://api.github.com/repos/PerfDotNet/BenchmarkDotNet","labels_url":"https://api.github.com/repos/PerfDotNet/BenchmarkDotNet/issues/200/labels{/name}","comments_url":"https://api.github.com/repos/PerfDotNet/BenchmarkDotNet/issues/200/comments","events_url":"https://api.github.com/repos/PerfDotNet/BenchmarkDotNet/issues/200/events","html_url":"https://github.com/PerfDotNet/BenchmarkDotNet/issues/200","id":158572295,"number":200,"title":"[BUG] MemoryDiagnoser is broken.","user":{"login":"ig-sinicyn","id":11225241,"avatar_url":"https://avatars.githubusercontent.com/u/11225241?v=3","gravatar_id":"","url":"https://api.github.com/users/ig-sinicyn","html_url":"https://github.com/ig-sinicyn","followers_url":"https://api.github.com/users/ig-sinicyn/followers","following_url":"https://api.github.com/users/ig-sinicyn/following{/other_user}","gists_url":"https://api.github.com/users/ig-sinicyn/gists{/gist_id}","starred_url":"https://api.github.com/users/ig-sinicyn/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ig-sinicyn/subscriptions","organizations_url":"https://api.github.com/users/ig-sinicyn/orgs","repos_url":"https://api.github.com/users/ig-sinicyn/repos","events_url":"https://api.github.com/users/ig-sinicyn/events{/privacy}","received_events_url":"https://api.github.com/users/ig-sinicyn/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":{"login":"mattwarren","id":157298,"avatar_url":"https://avatars.githubusercontent.com/u/157298?v=3","gravatar_id":"","url":"https://api.github.com/users/mattwarren","html_url":"https://github.com/mattwarren","followers_url":"https://api.github.com/users/mattwarren/followers","following_url":"https://api.github.com/users/mattwarren/following{/other_user}","gists_url":"https://api.github.com/users/mattwarren/gists{/gist_id}","starred_url":"https://api.github.com/users/mattwarren/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mattwarren/subscriptions","organizations_url":"https://api.github.com/users/mattwarren/orgs","repos_url":"https://api.github.com/users/mattwarren/repos","events_url":"https://api.github.com/users/mattwarren/events{/privacy}","received_events_url":"https://api.github.com/users/mattwarren/received_events","type":"User","site_admin":false},"milestone":null,"comments":18,"created_at":"2016-06-05T20:19:31Z","updated_at":"2016-06-06T12:35:16Z","closed_at":null,"body":"Bench.Net 0.9.7\r\n\r\nCode:\r\n```cs\r\n\tclass Program\r\n\t{\r\n\t\tstatic void Main(string[] args)\r\n\t\t{\r\n\t\t\tnew SimpleBenchmark().BaselineOutput();\r\n\t\t\tvar config = new ManualConfig();\r\n\t\t\tconfig.Add(DefaultConfig.Instance);\r\n\t\t\tconfig.Add(new MemoryDiagnoser());\r\n\r\n\t\t\tBenchmarkRunner.Run<SimpleBenchmark>(config);\r\n\t\t}\r\n\t}\r\n\r\n\tpublic class SimpleBenchmark\r\n\t{\r\n\t\tprivate volatile int x;\r\n\r\n\t\t[Benchmark]\r\n\t\tpublic int Baseline()\r\n\t\t{\r\n\t\t\tx = 0;\r\n\t\t\tfor (int i = 0; i < 10000000; i++)\r\n\t\t\t{\r\n\t\t\t\tx++;\r\n\t\t\t}\r\n\t\t\treturn x;\r\n\t\t}\r\n\r\n\t\tpublic void BaselineOutput()\r\n\t\t{\r\n\t\t\tvar m = GC.GetTotalMemory(false);\r\n\t\t\tvar sw = Stopwatch.StartNew();\r\n\r\n\t\t\tBaseline();\r\n\r\n\t\t\tsw.Stop();\r\n\t\t\tvar m2 = GC.GetTotalMemory(false);\r\n\t\t\tConsole.WriteLine($\"Allocated: {m2 - m}, elapsed {sw.Elapsed.TotalMilliseconds}ms\");\r\n\t\t}\r\n\t}\r\n```\r\n\r\nOutput:\r\n```\r\nAllocated: 0, elapsed 25,7643ms\r\n...\r\n   Method |     Median |    StdDev | Gen 0 | Gen 1 | Gen 2 | Bytes Allocated/Op |\r\n--------- |----------- |---------- |------ |------ |------ |------------------- |\r\n Baseline | 24.9992 ms | 0.1676 ms |     - |     - |     - |           2 696,37 |\r\n```\r\n\r\nThe code obviously runs without allocations. The issue is blocker for us as it means we cannot trust output from the diagnoser. If you had any idea or workaround, write a comment please:)\r\n\r\nAs far as I can see this can be fixed only with introduction of two-way notifications before and after each of benchmark run.\r\n\r\nThere's alternate solution: run the diagnosers in-process with the benchmark. Sadly, it fires a new issue: there should be a way to report results from the diagnoser back to the main process. Ouch:)"},"comment":{"url":"https://api.github.com/repos/PerfDotNet/BenchmarkDotNet/issues/comments/223946010","html_url":"https://github.com/PerfDotNet/BenchmarkDotNet/issues/200#issuecomment-223946010","issue_url":"https://api.github.com/repos/PerfDotNet/BenchmarkDotNet/issues/200","id":223946010,"user":{"login":"goldshtn","id":1685437,"avatar_url":"https://avatars.githubusercontent.com/u/1685437?v=3","gravatar_id":"","url":"https://api.github.com/users/goldshtn","html_url":"https://github.com/goldshtn","followers_url":"https://api.github.com/users/goldshtn/followers","following_url":"https://api.github.com/users/goldshtn/following{/other_user}","gists_url":"https://api.github.com/users/goldshtn/gists{/gist_id}","starred_url":"https://api.github.com/users/goldshtn/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/goldshtn/subscriptions","organizations_url":"https://api.github.com/users/goldshtn/orgs","repos_url":"https://api.github.com/users/goldshtn/repos","events_url":"https://api.github.com/users/goldshtn/events{/privacy}","received_events_url":"https://api.github.com/users/goldshtn/received_events","type":"User","site_admin":false},"created_at":"2016-06-06T12:35:16Z","updated_at":"2016-06-06T12:35:16Z","body":"I think your description is accurate, but indeed with a lot of repetitions the samples should be good enough. That's also what PerfView does, and from my experience it's pretty good when you alloc >100MB or so. \n\n> On Jun 6, 2016, at 14:14, Matt Warren <notifications@github.com> wrote:\n> \n> Btw while we are at it, maybe you could also use the type info provided by the allocation event to give extra detail on what kind of allocations were performed. Or just refer folks to PerfView -- there's a point where BDN turns into a profiler... But you could e.g. say something like \"most objects you allocated were strings, try StringBuilder\" or whatever.\n> \n> I looked into that and from my understanding (I might be wrong!) the type info is only from the last allocation that triggered the 100K threshold. So you might have 99K of String allocations, but if a byte [] tips it over 100K, it's the byte [] and it's stack-trace that is reported.\n> \n> However, because BenchmarkDotNet run's lots of iterations of your benchmark this might not be such an issue\n> \n> But yes it would be really nice to have this, especially with Stack Traces.\n> \n> —\n> You are receiving this because you were mentioned.\n> Reply to this email directly, view it on GitHub, or mute the thread.\n> \n"}}
{"actor":{"display_login":"goldshtn"}}