{"action":"created","issue":{"url":"https://api.github.com/repos/apache/incubator-superset/issues/5294","repository_url":"https://api.github.com/repos/apache/incubator-superset","labels_url":"https://api.github.com/repos/apache/incubator-superset/issues/5294/labels{/name}","comments_url":"https://api.github.com/repos/apache/incubator-superset/issues/5294/comments","events_url":"https://api.github.com/repos/apache/incubator-superset/issues/5294/events","html_url":"https://github.com/apache/incubator-superset/pull/5294","id":336031767,"node_id":"MDExOlB1bGxSZXF1ZXN0MTk3NTgzMDg0","number":5294,"title":"[SIP-3] Scheduled email reports for Slices / Dashboards","user":{"login":"mahendra","id":115988,"node_id":"MDQ6VXNlcjExNTk4OA==","avatar_url":"https://avatars1.githubusercontent.com/u/115988?v=4","gravatar_id":"","url":"https://api.github.com/users/mahendra","html_url":"https://github.com/mahendra","followers_url":"https://api.github.com/users/mahendra/followers","following_url":"https://api.github.com/users/mahendra/following{/other_user}","gists_url":"https://api.github.com/users/mahendra/gists{/gist_id}","starred_url":"https://api.github.com/users/mahendra/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mahendra/subscriptions","organizations_url":"https://api.github.com/users/mahendra/orgs","repos_url":"https://api.github.com/users/mahendra/repos","events_url":"https://api.github.com/users/mahendra/events{/privacy}","received_events_url":"https://api.github.com/users/mahendra/received_events","type":"User","site_admin":false},"labels":[{"id":1010577363,"node_id":"MDU6TGFiZWwxMDEwNTc3MzYz","url":"https://api.github.com/repos/apache/incubator-superset/labels/Superset%20Improvement%20Proposal","name":"Superset Improvement Proposal","color":"6a2ca0","default":false}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":95,"created_at":"2018-06-27T00:00:04Z","updated_at":"2018-11-16T10:33:30Z","closed_at":null,"author_association":"NONE","pull_request":{"url":"https://api.github.com/repos/apache/incubator-superset/pulls/5294","html_url":"https://github.com/apache/incubator-superset/pull/5294","diff_url":"https://github.com/apache/incubator-superset/pull/5294.diff","patch_url":"https://github.com/apache/incubator-superset/pull/5294.patch"},"body":"# Motivation \r\nScheduled email reports (of dashboards and slices) has been a long standing ask from the superset community. The feature also provides superset a competitive advantage over tools like Looker.\r\n\r\n# Proposed change\r\n* Scheduled email reports for slice and dashboard visualization (Attachment or inline)\r\n* Scheduled email reports for slice data (CSV attachment on inline table)\r\n* Each schedule has a list of recipients (all of them can receive a single mail, or separate mails)\r\n* All outgoing mails can have a mandatory `bcc` - for audit purposes.\r\n* Each dashboard/slice can have multiple schedules.\r\n\r\n# How does this work.\r\n* Users specify a crontab schedule for a slice / dashboard. \r\n* We use `celerybeat` to schedule tasks for delivering reports every hour.\r\n* Each `celery` task uses selenium (firefox/chrome) webdriver to crawl the actual dashboard (including visualizations) and mail it out to the recipients.\r\n\r\n# New dependencies\r\n* Python libraries\r\n  * selenium\r\n  * croniter\r\n* Firefox webdriver installable for your OS\r\n\r\n# New or Changed Public Interfaces\r\nThe main menu adds two new entries. This will be organized better in a future PR for categorizing dashboards and slices.\r\n\r\n# Migration plan\r\n* Requires a db migration - handled automatically via alembic\r\n\r\n# Rejected alternatives\r\n* Considered running PhantomJS, but selenium is deprecating support for it\r\n* Considered embedding image URLs within the email, but it can create issues where superset is hosted within corporate VPNs.\r\n\r\n# Future work\r\n* Better schedule management - currently the beat job will (re)-schedule all the jobs for an hour if it is re-run. We can get better at this.\r\n* Better visualization of configured schedules for each dashboard / slice (coming soon in another PR)\r\n* Better UI for scheduling reports (coming soon)\r\n* Better menu placement (coming soon as part of another PR)\r\n* Alternate report delivery mechanisms, if needed.\r\n* Try using an existing webdriver instance for all jobs. (to save on memory/cpu)\r\n\r\n# Screenshots\r\n## Admin\r\n### Menus\r\n![image](https://user-images.githubusercontent.com/115988/42000103-7fb513e4-7a14-11e8-8826-f99f6967b01e.png)\r\n\r\n### Dashboard schedules\r\n![image](https://user-images.githubusercontent.com/115988/42000140-a20e683c-7a14-11e8-948b-d51ccbfd15ae.png)\r\n\r\n### Slice schedules\r\n![image](https://user-images.githubusercontent.com/115988/42000168-bafa6634-7a14-11e8-9dd9-f8d4f5e8eba2.png)\r\n\r\n## Delivery\r\n### Dashboard via email\r\n![image](https://user-images.githubusercontent.com/115988/42001197-17da3d4a-7a18-11e8-930d-9094498f8510.png)\r\n![image](https://user-images.githubusercontent.com/115988/41999279-fd480f76-7a11-11e8-9c4b-2118a472dd1b.png)\r\n\r\n### Slice via email (inline)\r\n![image](https://user-images.githubusercontent.com/115988/41999352-2964507e-7a12-11e8-8472-a968eac19b31.png)\r\n\r\n### Slice data email (inline)\r\n![image](https://user-images.githubusercontent.com/115988/41999680-2db1ae78-7a13-11e8-9b80-9d67da4bd31e.png)\r\n\r\n### Slice data as CSV attachment\r\n![image](https://user-images.githubusercontent.com/115988/42000035-49027a94-7a14-11e8-8b66-4ef710519ef1.png)\r\n\r\n## FAQ's\r\n### Why did we not use phantomjs?\r\n(I can make the change if anyone needs/prefers `phantomjs`). \r\n\r\n`phantomjs` had the following issues\r\n1. The feature was originally developed using phantomjs. However, selenium emitted a deprecation notice for `phantomjs`, so I stopped working with it.\r\n1. Phantomjs does not support taking screenshots of elements (only the entire page). Firefox is the only browser which supports this.\r\n1. In my testing, I noticed that rendering quality was not as good in phantomjs (fonts). Chrome > Firefox > PhantomJS (did not investigate further)\r\n"},"comment":{"url":"https://api.github.com/repos/apache/incubator-superset/issues/comments/439352089","html_url":"https://github.com/apache/incubator-superset/pull/5294#issuecomment-439352089","issue_url":"https://api.github.com/repos/apache/incubator-superset/issues/5294","id":439352089,"node_id":"MDEyOklzc3VlQ29tbWVudDQzOTM1MjA4OQ==","user":{"login":"cloud26","id":5143017,"node_id":"MDQ6VXNlcjUxNDMwMTc=","avatar_url":"https://avatars1.githubusercontent.com/u/5143017?v=4","gravatar_id":"","url":"https://api.github.com/users/cloud26","html_url":"https://github.com/cloud26","followers_url":"https://api.github.com/users/cloud26/followers","following_url":"https://api.github.com/users/cloud26/following{/other_user}","gists_url":"https://api.github.com/users/cloud26/gists{/gist_id}","starred_url":"https://api.github.com/users/cloud26/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cloud26/subscriptions","organizations_url":"https://api.github.com/users/cloud26/orgs","repos_url":"https://api.github.com/users/cloud26/repos","events_url":"https://api.github.com/users/cloud26/events{/privacy}","received_events_url":"https://api.github.com/users/cloud26/received_events","type":"User","site_admin":false},"created_at":"2018-11-16T10:33:30Z","updated_at":"2018-11-16T10:33:30Z","author_association":"NONE","body":"> @mahendra - Can you help me with the following issue that I am facing when i schedule a dashboard email:\r\n> \r\n> [2018-09-18 17:02:04,499: ERROR/ForkPoolWorker-4] Task email_reports.send[e2fb2ff1-c80a-4cbe-9c97-5c7fa8f9dae1] raised unexpected: NoSuchElementException()\r\n> Traceback (most recent call last):\r\n> File \"/Library/Python/2.7/site-packages/celery/app/trace.py\", line 382, in trace_task\r\n> R = retval = fun(*args, **kwargs)\r\n> File \"/Library/Python/2.7/site-packages/celery/app/trace.py\", line 641, in **protected_call**\r\n> return self.run(*args, **kwargs)\r\n> File \"superset/tasks/schedules.py\", line 382, in schedule_email_report\r\n> deliver_dashboard(schedule)\r\n> File \"superset/tasks/schedules.py\", line 221, in deliver_dashboard\r\n> delay=PAGE_RENDER_WAIT,\r\n> File \"/Library/Python/2.7/site-packages/retry/api.py\", line 101, in retry_call\r\n> return __retry_internal(partial(f, *args, **kwargs), exceptions, tries, delay, max_delay, backoff, jitter, logger)\r\n> File \"/Library/Python/2.7/site-packages/retry/api.py\", line 33, in __retry_internal\r\n> return f()\r\n> File \"/Library/Python/2.7/site-packages/selenium/webdriver/remote/webdriver.py\", line 563, in find_element_by_class_name\r\n> return self.find_element(by=By.CLASS_NAME, value=name)\r\n> File \"/Library/Python/2.7/site-packages/selenium/webdriver/remote/webdriver.py\", line 966, in find_element\r\n> 'value': value})['value']\r\n> File \"/Library/Python/2.7/site-packages/selenium/webdriver/remote/webdriver.py\", line 320, in execute\r\n> self.error_handler.check_response(response)\r\n> File \"/Library/Python/2.7/site-packages/selenium/webdriver/remote/errorhandler.py\", line 242, in check_response\r\n> raise exception_class(message, screen, stacktrace)\r\n> NoSuchElementException: Message: no such element: Unable to locate element: {\"method\":\"class name\",\"selector\":\"grid-container\"}\r\n\r\n\r\n@akshayjamwal07 I came across this problem too. The default WEBDRIVER_BASEURL is http://0.0.0.0:8088/, which is the backend server address. Because the backend doesn't render the graph, so it is unable to locate the dashboard or chart.\r\n\r\nIn my case, change WEBDRIVER_BASEURL to http://127.0.0.1:9000 will solve this problem. "}}
{"actor":{"display_login":"cloud26"}}