{"action":"created","issue":{"url":"https://api.github.com/repos/getsentry/sentry/issues/2673","repository_url":"https://api.github.com/repos/getsentry/sentry","labels_url":"https://api.github.com/repos/getsentry/sentry/issues/2673/labels{/name}","comments_url":"https://api.github.com/repos/getsentry/sentry/issues/2673/comments","events_url":"https://api.github.com/repos/getsentry/sentry/issues/2673/events","html_url":"https://github.com/getsentry/sentry/issues/2673","id":133360555,"number":2673,"title":"Discussion: reducing email notification noise in Sentry","user":{"login":"benvinegar","id":2153,"avatar_url":"https://avatars.githubusercontent.com/u/2153?v=3","gravatar_id":"","url":"https://api.github.com/users/benvinegar","html_url":"https://github.com/benvinegar","followers_url":"https://api.github.com/users/benvinegar/followers","following_url":"https://api.github.com/users/benvinegar/following{/other_user}","gists_url":"https://api.github.com/users/benvinegar/gists{/gist_id}","starred_url":"https://api.github.com/users/benvinegar/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/benvinegar/subscriptions","organizations_url":"https://api.github.com/users/benvinegar/orgs","repos_url":"https://api.github.com/users/benvinegar/repos","events_url":"https://api.github.com/users/benvinegar/events{/privacy}","received_events_url":"https://api.github.com/users/benvinegar/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":6,"created_at":"2016-02-12T22:39:43Z","updated_at":"2016-02-12T23:54:22Z","closed_at":null,"body":"Today the Sentry team had a long conversation in our private Slack channel about reducing email notification noise in Sentry. We're going to open this discussion up to the community here on GitHub, in case anyone wants to chime in with their own experience and what potential solutions they feel might best help their team(s).\r\n\r\n### The Problem\r\n\r\nSentry can email you a lot, and if you get too much email, this can cause you to subconsciously \"tune out\" potential problem issues. This noise can be reduced by carefully tuning your project's notification rules, but limitations remain.\r\n\r\nEmail noise can also be particularly troublesome for client-side JavaScript projects for which stack trace grouping is less reliable, and for which 3rd-party code (console, browser extensions, bookmarklets, malware) can easily trigger one-off errors.\r\n\r\n### Some Proposed Solutions\r\n\r\n**New notification rule option based on absolute counts**\r\n\r\nToday you can configure notifications to occur when an event has been seen `more than {times} in {interval}`. However, you cannot simply specify `more than {times}` (without an interval). This could be helpful for client JavaScript projects where it is not uncommon to get errors triggered by a 3rd party that only occur once or twice by the same user.\r\n\r\n**A new daily/weekly summary email**\r\n\r\nSentry sends you a summary email every morning summarizing yesterday's (or last week's) issues. This email could hypothetically:\r\n\r\n- highlight issues that didn't get seen (viewed in Sentry) or actioned\r\n- highlight high-priority issues (as defined by you)\r\n- include event counts\r\n- include number of affected users\r\n\r\nMost importantly, the goal of the email is to highlight things **you may have missed**. The thinking here is that, if there is such a summary email, you may feel less obliged to configure aggressive _individual notifications_ for issues. Basically, the goal is to push individual notifications and have them rolled up (with more context) into this daily summary.\r\n\r\nSee: #647\r\n\r\n**Release summary emails**\r\n\r\nSimilar to daily/weekly summaries, but instead we summarize issues that have popped up after a release has occurred (30 minutes, 60 minutes, 2 hours ... as configured by you).\r\n\r\nThe idea here is that issues that may have been triggered by a release are likely to be of more interest and are more actionable than an issue that occurs randomly in the middle of the night. And that if this release summary exists, users will not feel it is as necessary to report as many individual notification emails.\r\n\r\n**Bring notifications into the UI**\r\n\r\nRight now email notifications are completely separate from the Sentry application. They appear only in email (or Slack, or Hipchat, etc) – and not in Sentry itself.\r\n\r\nIf we brought notifications into Sentry, for example, to a dashboard or notification view, potentially users might feel less need to email themselves for lower priority items. They can configure Sentry to report high priority notifications (e.g. issue seen 100 times in a day), while low priority notifications are still visible in this view for regular review.\r\n\r\n**Improve client JavaScript grouping/whitelisting**\r\n\r\nAs mentioned earlier, email notification noise is felt more in JavaScript projects. There is feeling that if we can improve our JavaScript grouping algorithm, this can reduce the amount of email notification noise as individual events are more likely to be grouped into an existing issue rather than a new one. Most projects are configured to receive emails when a new issue is detected.\r\n\r\nAdditionally, this could be improved by rejecting issues that are thought to be caused by third-parties: users playing in their browser's developer console, browser extensions, etc. But some of these can be particularly difficult to detect, e.g. if a browser extension changes the DOM which results in an error caused by your application's code, there's not much we can do.\r\n\r\n---\r\n\r\nAgain, we want to hear from you. Is email notification noise a problem for your organization/team? Do any of the proposed solutions feel like they'd be helpful? Do you have other ideas we should consider? Please chime in on this issue."},"comment":{"url":"https://api.github.com/repos/getsentry/sentry/issues/comments/183532660","html_url":"https://github.com/getsentry/sentry/issues/2673#issuecomment-183532660","issue_url":"https://api.github.com/repos/getsentry/sentry/issues/2673","id":183532660,"user":{"login":"dcramer","id":23610,"avatar_url":"https://avatars.githubusercontent.com/u/23610?v=3","gravatar_id":"","url":"https://api.github.com/users/dcramer","html_url":"https://github.com/dcramer","followers_url":"https://api.github.com/users/dcramer/followers","following_url":"https://api.github.com/users/dcramer/following{/other_user}","gists_url":"https://api.github.com/users/dcramer/gists{/gist_id}","starred_url":"https://api.github.com/users/dcramer/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dcramer/subscriptions","organizations_url":"https://api.github.com/users/dcramer/orgs","repos_url":"https://api.github.com/users/dcramer/repos","events_url":"https://api.github.com/users/dcramer/events{/privacy}","received_events_url":"https://api.github.com/users/dcramer/received_events","type":"User","site_admin":false},"created_at":"2016-02-12T23:54:22Z","updated_at":"2016-02-12T23:54:22Z","body":"I agree with @mallyvai and my biggest fear is always not seeing something critical (like a billing error, which may only happen once a week).\r\n\r\nHere's some additional proposal. We take level and shift it into an impact/priority concept. We make debug/info logs set the priority to low (by default), and others set to medium. We then support a way to dynamically prioritize items (maybe rules, part of the SDK, based on rates/etc). Finally, we could then make it so notifications (more specifically, actual alerts that you'd get in your inbox/slack) can be triggered based on priority. \"Email me when a new high priority item is created\". \"Email me when an item becomes high priority\". \"Email me when a high priority item is unresolved after 1 hour\"."}}