{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/9608","labels_url":"https://api.github.com/repos/apache/spark/issues/9608/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/9608/comments","events_url":"https://api.github.com/repos/apache/spark/issues/9608/events","html_url":"https://github.com/apache/spark/pull/9608","id":116218234,"number":9608,"title":"[SPARK-11638] [Mesos + Docker Bridge networking]: Run Spark on Mesos,…","user":{"login":"radekg","id":62946,"avatar_url":"https://avatars.githubusercontent.com/u/62946?v=3","gravatar_id":"","url":"https://api.github.com/users/radekg","html_url":"https://github.com/radekg","followers_url":"https://api.github.com/users/radekg/followers","following_url":"https://api.github.com/users/radekg/following{/other_user}","gists_url":"https://api.github.com/users/radekg/gists{/gist_id}","starred_url":"https://api.github.com/users/radekg/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/radekg/subscriptions","organizations_url":"https://api.github.com/users/radekg/orgs","repos_url":"https://api.github.com/users/radekg/repos","events_url":"https://api.github.com/users/radekg/events{/privacy}","received_events_url":"https://api.github.com/users/radekg/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":16,"created_at":"2015-11-10T22:36:37Z","updated_at":"2016-01-05T11:04:46Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/9608","html_url":"https://github.com/apache/spark/pull/9608","diff_url":"https://github.com/apache/spark/pull/9608.diff","patch_url":"https://github.com/apache/spark/pull/9608.patch"},"body":"… in Docker with Bridge networking\r\n\r\nProvides `spark.driver.advertisedPort`, `spark.fileserver.advertisedPort`, `spark.broadcast.advertisedPort` and `spark.replClassServer.advertisedPort` settings to enable running Spark in Mesos on Docker with Bridge networking. Provides patches for Akka Remote to enable Spark driver advertisement using alternative host and port (https://issues.apache.org/jira/browse/SPARK-11638).\r\n\r\nWith these settings, it is possible to run Spark Master in a Docker container and have the executors running on Mesos talk back correctly to such Master.\r\n\r\nThe problem is discussed on the Mesos mailing list here: https://mail-archives.apache.org/mod_mbox/mesos-user/201510.mbox/%3CCACTd3c9vjAMXk=bFOtj5LJZFRH5u7ix-ghppFqKnVg9mkKctjg@mail.gmail.com%3E\r\n\r\nTo take advantage of these, patched Akka Remote is required. Patches for Akka available here: https://issues.apache.org/jira/browse/SPARK-11638."},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/168972745","html_url":"https://github.com/apache/spark/pull/9608#issuecomment-168972745","issue_url":"https://api.github.com/repos/apache/spark/issues/9608","id":168972745,"user":{"login":"radekg","id":62946,"avatar_url":"https://avatars.githubusercontent.com/u/62946?v=3","gravatar_id":"","url":"https://api.github.com/users/radekg","html_url":"https://github.com/radekg","followers_url":"https://api.github.com/users/radekg/followers","following_url":"https://api.github.com/users/radekg/following{/other_user}","gists_url":"https://api.github.com/users/radekg/gists{/gist_id}","starred_url":"https://api.github.com/users/radekg/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/radekg/subscriptions","organizations_url":"https://api.github.com/users/radekg/orgs","repos_url":"https://api.github.com/users/radekg/repos","events_url":"https://api.github.com/users/radekg/events{/privacy}","received_events_url":"https://api.github.com/users/radekg/received_events","type":"User","site_admin":false},"created_at":"2016-01-05T11:04:46Z","updated_at":"2016-01-05T11:04:46Z","body":"@tnachen what would be the best place to publish such documentation? Regarding `worth nothing` comment. The whole concept behind this pr is to allow running spark master inside of a docker container on mesos and have the executors run on the same mesos infrastructure, outside of the container in which the master is running.\r\n\r\nAs such, this can only be achieved by knowing the ports on which the particular services will run inside of the mesos task. It's not possible otherwise as the ports have to be handed in to mesos at the time of the task submission. This also means that the ports can be dedicated to spark and the user is in full control of what ports will be available.\r\n\r\nDoes this clarify?"}}