{"action":"created","issue":{"url":"https://api.github.com/repos/PerfDotNet/BenchmarkDotNet/issues/177","repository_url":"https://api.github.com/repos/PerfDotNet/BenchmarkDotNet","labels_url":"https://api.github.com/repos/PerfDotNet/BenchmarkDotNet/issues/177/labels{/name}","comments_url":"https://api.github.com/repos/PerfDotNet/BenchmarkDotNet/issues/177/comments","events_url":"https://api.github.com/repos/PerfDotNet/BenchmarkDotNet/issues/177/events","html_url":"https://github.com/PerfDotNet/BenchmarkDotNet/issues/177","id":154677925,"number":177,"title":"[Suggestion] common messaging API","user":{"login":"ig-sinicyn","id":11225241,"avatar_url":"https://avatars.githubusercontent.com/u/11225241?v=3","gravatar_id":"","url":"https://api.github.com/users/ig-sinicyn","html_url":"https://github.com/ig-sinicyn","followers_url":"https://api.github.com/users/ig-sinicyn/followers","following_url":"https://api.github.com/users/ig-sinicyn/following{/other_user}","gists_url":"https://api.github.com/users/ig-sinicyn/gists{/gist_id}","starred_url":"https://api.github.com/users/ig-sinicyn/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ig-sinicyn/subscriptions","organizations_url":"https://api.github.com/users/ig-sinicyn/orgs","repos_url":"https://api.github.com/users/ig-sinicyn/repos","events_url":"https://api.github.com/users/ig-sinicyn/events{/privacy}","received_events_url":"https://api.github.com/users/ig-sinicyn/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":2,"created_at":"2016-05-13T10:08:11Z","updated_at":"2016-05-19T08:56:39Z","closed_at":null,"body":"For now benchmark.net has multiple points of failure / informational messages that are passed as a plain text in the log:\r\n* validator warnings | errors\r\n* analyser warnings (no errors?),\r\n* toolchain warnings / errors (not reported at all?)\r\n* exceptions thrown.\r\n\r\nI suggest to wrap all these into common interface, IMessage:\r\n```cs\r\nenum MessageSource\r\n{\r\n\tAnalyser, Validator, Etc\r\n}\r\nenum MessageKind\r\n{\r\n\tExecutionFailure, InvalidSetup, Error, Warning, Notification, Informational, Debug, Etc\r\n}\r\ninterface IMessage\r\n{\r\n\tMessageSource Source { get; }\r\n\r\n\tMessageKind MessageKind { get; }\r\n\r\n\t// use case: to identify specific messages that can be parsed.\r\n\tstring MessageCategory { get; }\r\n\r\n\tstring Message { get; }\r\n}\r\n```\r\nand to report all messages using that interface. This will be especially useful for out-of process analysers, they will be able to consume notification messages instead of doing weird tricks like this:\r\n(in `SynchronousProcessOutputLoggerWithDiagnoser.cs`)\r\n```\r\nif (diagnosticsAlreadyRun || !line.StartsWith(IterationMode.MainWarmup.ToString()))\r\n{\r\n\tcontinue;\r\n}\r\n\r\ntry\r\n{\r\n\tdiagnoser?.AfterBenchmarkHasRun(benchmark, process);\r\n}\r\nfinally\r\n{\r\n\t// Always set this, even if something went wrong, otherwise we will try on every run of a benchmark batch\r\n\tdiagnosticsAlreadyRun = true;\r\n}\r\n```\r\n\r\nThese messages can be passed via pipes, WCF via memorymappedfile + global mutex or via ConcurrentQueue (for in-process messaging). Concrete implementation should be chosen by the toolchain.\r\n\r\nAlso, the idea can be extended up to two-way messaging (of course, there should be a new interface for the reply messages).\r\n\r\nIt will allow to fix the #154 as there will be an API to pass arbitrary message into the child process and it'll be possible to move the benchmark dispatching logic into the child process. \r\n\r\nMoreover, we can remove the compilation/code generation part at all and replace it with something [like this](https://github.com/rsdn/CodeJam/blob/master/Main/src-perfomance/[L1_InProcess]/Toolchains/RunnableBenchmarkFactory.cs#L11) (prototype code, the API is not stabilized yet)."},"comment":{"url":"https://api.github.com/repos/PerfDotNet/BenchmarkDotNet/issues/comments/220266144","html_url":"https://github.com/PerfDotNet/BenchmarkDotNet/issues/177#issuecomment-220266144","issue_url":"https://api.github.com/repos/PerfDotNet/BenchmarkDotNet/issues/177","id":220266144,"user":{"login":"ig-sinicyn","id":11225241,"avatar_url":"https://avatars.githubusercontent.com/u/11225241?v=3","gravatar_id":"","url":"https://api.github.com/users/ig-sinicyn","html_url":"https://github.com/ig-sinicyn","followers_url":"https://api.github.com/users/ig-sinicyn/followers","following_url":"https://api.github.com/users/ig-sinicyn/following{/other_user}","gists_url":"https://api.github.com/users/ig-sinicyn/gists{/gist_id}","starred_url":"https://api.github.com/users/ig-sinicyn/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ig-sinicyn/subscriptions","organizations_url":"https://api.github.com/users/ig-sinicyn/orgs","repos_url":"https://api.github.com/users/ig-sinicyn/repos","events_url":"https://api.github.com/users/ig-sinicyn/events{/privacy}","received_events_url":"https://api.github.com/users/ig-sinicyn/received_events","type":"User","site_admin":false},"created_at":"2016-05-19T08:56:39Z","updated_at":"2016-05-19T08:56:39Z","body":"@mattwarren \r\nWell, I can do it but I'm afraid that it will be not to soon.\r\n\r\nI'm quite busy at work for a week or two and also I still had to finish [the competition test runner](https://github.com/rsdn/CodeJam/tree/master/Main/src-perfomance). It's in almost-ready state, need to add some missing features (the main one - annotating perftests with actual timings obtained from ci-server). After that I'll do a post here."}}