{"action":"created","issue":{"url":"https://api.github.com/repos/DataDog/opencensus-go-exporter-datadog/issues/20","repository_url":"https://api.github.com/repos/DataDog/opencensus-go-exporter-datadog","labels_url":"https://api.github.com/repos/DataDog/opencensus-go-exporter-datadog/issues/20/labels{/name}","comments_url":"https://api.github.com/repos/DataDog/opencensus-go-exporter-datadog/issues/20/comments","events_url":"https://api.github.com/repos/DataDog/opencensus-go-exporter-datadog/issues/20/events","html_url":"https://github.com/DataDog/opencensus-go-exporter-datadog/issues/20","id":374479262,"node_id":"MDU6SXNzdWUzNzQ0NzkyNjI=","number":20,"title":"trace: incorrect APM metrics","user":{"login":"arguile-","id":756863,"node_id":"MDQ6VXNlcjc1Njg2Mw==","avatar_url":"https://avatars0.githubusercontent.com/u/756863?v=4","gravatar_id":"","url":"https://api.github.com/users/arguile-","html_url":"https://github.com/arguile-","followers_url":"https://api.github.com/users/arguile-/followers","following_url":"https://api.github.com/users/arguile-/following{/other_user}","gists_url":"https://api.github.com/users/arguile-/gists{/gist_id}","starred_url":"https://api.github.com/users/arguile-/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/arguile-/subscriptions","organizations_url":"https://api.github.com/users/arguile-/orgs","repos_url":"https://api.github.com/users/arguile-/repos","events_url":"https://api.github.com/users/arguile-/events{/privacy}","received_events_url":"https://api.github.com/users/arguile-/received_events","type":"User","site_admin":false},"labels":[{"id":962611973,"node_id":"MDU6TGFiZWw5NjI2MTE5NzM=","url":"https://api.github.com/repos/DataDog/opencensus-go-exporter-datadog/labels/stats","name":"stats","color":"6ad8cf","default":false}],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":4,"created_at":"2018-10-26T17:33:56Z","updated_at":"2019-01-04T15:44:35Z","closed_at":"2019-01-04T09:49:30Z","author_association":"CONTRIBUTOR","body":"- OpenCensus uses a sampler to determine if trace should be sampled or not.\r\n- Only sampled traces are sent to the exporters.\r\n- Datadog trace agent gets only those sampled traces.\r\n- That Datadog trace agent expects to get all traces for APM metrics and then do the sampling itself, so with OpenCensus these metrics will always be incomplete as sampling is done **before traces get to the Datadog trace agent**.\r\n\r\n\r\n_A scenario to illustrate this:_\r\n\r\n1. A probabilistic sampler with a 50% sample rate is given to the OpenCensus trace configuration.\r\n2. Ten requests are made to the application, of which four get `true` for sampling.\r\n3. Those four samples are sent through the exporter to the Datadog agent.\r\n4. Within the APM view, we see only those four requests.\r\n\r\nIn this case the APM would shows four requests, no errors, and a p99 of 647µs. All hitting a single route.\r\n\r\n_What could have happened:_\r\n\r\n- Ten requests were made, eight to the show route and two to another other.\r\n- Four of the requests resulted in errors.\r\n- The p99 of the request on the main route was actually in the seconds.\r\n\r\nThere is a mismatch between how the DD APM gets its metrics (from sampled traces) and how OpenCensus collects metrics.\r\n\r\nHow much of this within the purview of this exporter I'm not certain.\r\n\r\n(_This was also filed as a private support ticket with Datadog._)"},"comment":{"url":"https://api.github.com/repos/DataDog/opencensus-go-exporter-datadog/issues/comments/451480396","html_url":"https://github.com/DataDog/opencensus-go-exporter-datadog/issues/20#issuecomment-451480396","issue_url":"https://api.github.com/repos/DataDog/opencensus-go-exporter-datadog/issues/20","id":451480396,"node_id":"MDEyOklzc3VlQ29tbWVudDQ1MTQ4MDM5Ng==","user":{"login":"arguile-","id":756863,"node_id":"MDQ6VXNlcjc1Njg2Mw==","avatar_url":"https://avatars0.githubusercontent.com/u/756863?v=4","gravatar_id":"","url":"https://api.github.com/users/arguile-","html_url":"https://github.com/arguile-","followers_url":"https://api.github.com/users/arguile-/followers","following_url":"https://api.github.com/users/arguile-/following{/other_user}","gists_url":"https://api.github.com/users/arguile-/gists{/gist_id}","starred_url":"https://api.github.com/users/arguile-/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/arguile-/subscriptions","organizations_url":"https://api.github.com/users/arguile-/orgs","repos_url":"https://api.github.com/users/arguile-/repos","events_url":"https://api.github.com/users/arguile-/events{/privacy}","received_events_url":"https://api.github.com/users/arguile-/received_events","type":"User","site_admin":false},"created_at":"2019-01-04T15:44:35Z","updated_at":"2019-01-04T15:44:35Z","author_association":"CONTRIBUTOR","body":"@gbbr I think the request from @julianvmodesto is the same as my initial question, that is can we not use the stats from `ochttp` for example to populate the APM metrics instead of only taking them from the traces.\r\n\r\nI realize this could be a huge product change, but it's a reasonable approach from the point of a user."}}
{"actor":{"display_login":"arguile-"}}