{"action":"created","issue":{"url":"https://api.github.com/repos/jcrist/skein/issues/72","repository_url":"https://api.github.com/repos/jcrist/skein","labels_url":"https://api.github.com/repos/jcrist/skein/issues/72/labels{/name}","comments_url":"https://api.github.com/repos/jcrist/skein/issues/72/comments","events_url":"https://api.github.com/repos/jcrist/skein/issues/72/events","html_url":"https://github.com/jcrist/skein/pull/72","id":360410300,"node_id":"MDExOlB1bGxSZXF1ZXN0MjE1Njc2NTU5","number":72,"title":"[WIP] Add proxied pages through Web UI","user":{"login":"jcrist","id":2783717,"node_id":"MDQ6VXNlcjI3ODM3MTc=","avatar_url":"https://avatars3.githubusercontent.com/u/2783717?v=4","gravatar_id":"","url":"https://api.github.com/users/jcrist","html_url":"https://github.com/jcrist","followers_url":"https://api.github.com/users/jcrist/followers","following_url":"https://api.github.com/users/jcrist/following{/other_user}","gists_url":"https://api.github.com/users/jcrist/gists{/gist_id}","starred_url":"https://api.github.com/users/jcrist/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jcrist/subscriptions","organizations_url":"https://api.github.com/users/jcrist/orgs","repos_url":"https://api.github.com/users/jcrist/repos","events_url":"https://api.github.com/users/jcrist/events{/privacy}","received_events_url":"https://api.github.com/users/jcrist/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":3,"created_at":"2018-09-14T18:33:13Z","updated_at":"2018-09-14T18:59:43Z","closed_at":null,"author_association":"OWNER","pull_request":{"url":"https://api.github.com/repos/jcrist/skein/pulls/72","html_url":"https://github.com/jcrist/skein/pull/72","diff_url":"https://github.com/jcrist/skein/pull/72.diff","patch_url":"https://github.com/jcrist/skein/pull/72.patch"},"body":"*I'm not 100% settled on this being a good idea/worth it, but it was relatively quick to write up and try out.*\r\n\r\nThis adds the ability for applications to register additional web pages to proxy through our webui. This allows additional user-provided pages to be accessible through the webui (and thus outside the YARN cluster firewall). Proxied pages can be added and removed dynamically at runtime, and optionally presented as part of the WebUI\r\n\r\nNote that our WebUI already passes through the YARN proxy - this adds an additional proxy in our UI that will forward requests to additional pages.\r\n\r\n**Existing**\r\n`YARN Proxy -> WebUI`\r\n\r\n**With this PR**\r\n`YARN Proxy -> WebUI -> User pages`\r\n\r\n### Caveats:\r\n- Due to limitations in the YARN proxy, only `GET` methods are supported in the commonly deployed hadoop versions. In Hadoop 3, support for `PUT` was added, but CDH still deploys 2.6. The Spark web-ui is full of hacks abusing `GET` to get around this.\r\n\r\n- Modern web components like web sockets are *not* supported. In particular this means that the dask web ui (https://distributed.readthedocs.io/en/latest/web.html) won't work through the proxy.\r\n\r\n- YARN provides a security filter to secure the connection between their proxy and the webui, based on the ip of the requester (https://hadoop.apache.org/docs/stable/hadoop-yarn/hadoop-yarn-site/WebApplicationProxy.html). This won't work for the proxy from our UI to user pages. This means that a malicious actor *already inside the YARN cluster firewall* (meaning e.g. a coworker) can access the user provided pages provided they can find the addresses. For now I'm punting on this security issue, but it does mean that user provided pages are less secure."},"comment":{"url":"https://api.github.com/repos/jcrist/skein/issues/comments/421453822","html_url":"https://github.com/jcrist/skein/pull/72#issuecomment-421453822","issue_url":"https://api.github.com/repos/jcrist/skein/issues/72","id":421453822,"node_id":"MDEyOklzc3VlQ29tbWVudDQyMTQ1MzgyMg==","user":{"login":"mrocklin","id":306380,"node_id":"MDQ6VXNlcjMwNjM4MA==","avatar_url":"https://avatars3.githubusercontent.com/u/306380?v=4","gravatar_id":"","url":"https://api.github.com/users/mrocklin","html_url":"https://github.com/mrocklin","followers_url":"https://api.github.com/users/mrocklin/followers","following_url":"https://api.github.com/users/mrocklin/following{/other_user}","gists_url":"https://api.github.com/users/mrocklin/gists{/gist_id}","starred_url":"https://api.github.com/users/mrocklin/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mrocklin/subscriptions","organizations_url":"https://api.github.com/users/mrocklin/orgs","repos_url":"https://api.github.com/users/mrocklin/repos","events_url":"https://api.github.com/users/mrocklin/events{/privacy}","received_events_url":"https://api.github.com/users/mrocklin/received_events","type":"User","site_admin":false},"created_at":"2018-09-14T18:59:43Z","updated_at":"2018-09-14T18:59:43Z","author_association":"CONTRIBUTOR","body":"The /info pages of the current dask scheduler UI might be informative\n\nOn Fri, Sep 14, 2018 at 2:58 PM Jim Crist <notifications@github.com> wrote:\n\n> Also cc @mrocklin <https://github.com/mrocklin> for thoughts on providing\n> a simpler, GET only dask UI (which could be proxied out of the cluster\n> using this PR).\n>\n> —\n> You are receiving this because you were mentioned.\n> Reply to this email directly, view it on GitHub\n> <https://github.com/jcrist/skein/pull/72#issuecomment-421453619>, or mute\n> the thread\n> <https://github.com/notifications/unsubscribe-auth/AASszJHXfkTQxxBY4bf3B3sTzyMUfO4cks5ua_xrgaJpZM4Wpv0U>\n> .\n>\n"}}
{"actor":{"display_login":"mrocklin"}}