{"action":"created","issue":{"url":"https://api.github.com/repos/big-data-europe/README/issues/2","repository_url":"https://api.github.com/repos/big-data-europe/README","labels_url":"https://api.github.com/repos/big-data-europe/README/issues/2/labels{/name}","comments_url":"https://api.github.com/repos/big-data-europe/README/issues/2/comments","events_url":"https://api.github.com/repos/big-data-europe/README/issues/2/events","html_url":"https://github.com/big-data-europe/README/issues/2","id":169592988,"number":2,"title":"Spark-cluster in NCSR-cluster","user":{"login":"gioargyr","id":17429563,"avatar_url":"https://avatars.githubusercontent.com/u/17429563?v=3","gravatar_id":"","url":"https://api.github.com/users/gioargyr","html_url":"https://github.com/gioargyr","followers_url":"https://api.github.com/users/gioargyr/followers","following_url":"https://api.github.com/users/gioargyr/following{/other_user}","gists_url":"https://api.github.com/users/gioargyr/gists{/gist_id}","starred_url":"https://api.github.com/users/gioargyr/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/gioargyr/subscriptions","organizations_url":"https://api.github.com/users/gioargyr/orgs","repos_url":"https://api.github.com/users/gioargyr/repos","events_url":"https://api.github.com/users/gioargyr/events{/privacy}","received_events_url":"https://api.github.com/users/gioargyr/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":1,"created_at":"2016-08-05T12:07:10Z","updated_at":"2016-08-05T12:27:36Z","closed_at":null,"body":"Hi! There is a docker-spark-cluster up&running the last 7 days in NCSR's cluster (from bde2020/spark-*:1.5.1-hadoop2.6). I submitted a spark-job and I got a spark-related error(https://github.com/big-data-europe/docker-spark/issues/4).\r\nI tried to run the next version of docker-spark (bde2020/spark-*:1.6.2-hadoop2.6), but the workers can't connect to the master, possibly because there is the already up&running spark-cluster.\r\n- Who else is using the up&running docker-spark-cluster?\r\n- Can I freely stop it whenever I want?\r\n- Can we stop it and run the newer version? "},"comment":{"url":"https://api.github.com/repos/big-data-europe/README/issues/comments/237837536","html_url":"https://github.com/big-data-europe/README/issues/2#issuecomment-237837536","issue_url":"https://api.github.com/repos/big-data-europe/README/issues/2","id":237837536,"user":{"login":"acharal","id":1780587,"avatar_url":"https://avatars.githubusercontent.com/u/1780587?v=3","gravatar_id":"","url":"https://api.github.com/users/acharal","html_url":"https://github.com/acharal","followers_url":"https://api.github.com/users/acharal/followers","following_url":"https://api.github.com/users/acharal/following{/other_user}","gists_url":"https://api.github.com/users/acharal/gists{/gist_id}","starred_url":"https://api.github.com/users/acharal/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/acharal/subscriptions","organizations_url":"https://api.github.com/users/acharal/orgs","repos_url":"https://api.github.com/users/acharal/repos","events_url":"https://api.github.com/users/acharal/events{/privacy}","received_events_url":"https://api.github.com/users/acharal/received_events","type":"User","site_admin":false},"created_at":"2016-08-05T12:27:36Z","updated_at":"2016-08-05T12:27:36Z","body":"Hi Giorgos,\r\nThe Docker instances you are referring to are leftovers from an experiment we (NCSR-D) were running. I shut them down. You can start you own spark cluster. \r\n\r\nAs a note, I think that it is desired to be able to have 2 different versions of spark up&running at the cluster without interfering with each other. I think the problems you have encountered are due to the fact that the spark workers are trying to connect to the predefined \"spark-master\". Do anyone know if it is possible to configure:\r\n- a docker spark master with a different container_name than \"spark-master\"\r\n- the docker spark workers to register to a different master other than \"spark-master\""}}
{"actor":{"display_login":"acharal"}}