{"action":"created","issue":{"url":"https://api.github.com/repos/mesos/chronos/issues/680","repository_url":"https://api.github.com/repos/mesos/chronos","labels_url":"https://api.github.com/repos/mesos/chronos/issues/680/labels{/name}","comments_url":"https://api.github.com/repos/mesos/chronos/issues/680/comments","events_url":"https://api.github.com/repos/mesos/chronos/issues/680/events","html_url":"https://github.com/mesos/chronos/issues/680","id":154073066,"number":680,"title":"Spark_python job not Working on Dcos chronos Env","user":{"login":"slankipalli","id":7545316,"avatar_url":"https://avatars.githubusercontent.com/u/7545316?v=3","gravatar_id":"","url":"https://api.github.com/users/slankipalli","html_url":"https://github.com/slankipalli","followers_url":"https://api.github.com/users/slankipalli/followers","following_url":"https://api.github.com/users/slankipalli/following{/other_user}","gists_url":"https://api.github.com/users/slankipalli/gists{/gist_id}","starred_url":"https://api.github.com/users/slankipalli/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/slankipalli/subscriptions","organizations_url":"https://api.github.com/users/slankipalli/orgs","repos_url":"https://api.github.com/users/slankipalli/repos","events_url":"https://api.github.com/users/slankipalli/events{/privacy}","received_events_url":"https://api.github.com/users/slankipalli/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2016-05-10T18:19:50Z","updated_at":"2016-05-12T17:27:46Z","closed_at":null,"body":"Spark python job not working on chronos, Chronos running on Dcos environment. I build Docker image with spark and python dependencies \r\n\r\nI0510 14:43:19.216224  6301 exec.cpp:143] Version: 0.28.1\r\nI0510 14:43:19.220377  6315 exec.cpp:217] Executor registered on slave 671bdf57-caeb-44c9-b382-a84043e7cc23-S5\r\nUsing Spark's default log4j profile: org/apache/spark/log4j-defaults.properties\r\n16/05/10 14:43:23 INFO SparkContext: Running Spark version 1.4.1\r\n16/05/10 14:43:23 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable\r\n16/05/10 14:43:23 INFO SecurityManager: Changing view acls to: root\r\n16/05/10 14:43:23 INFO SecurityManager: Changing modify acls to: root\r\n16/05/10 14:43:23 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(root); users with modify permissions: Set(root)\r\n16/05/10 14:43:24 INFO Slf4jLogger: Slf4jLogger started\r\n16/05/10 14:43:24 INFO Remoting: Starting remoting\r\n16/05/10 14:43:24 INFO Remoting: Remoting started; listening on addresses :[akka.tcp://sparkDriver@172.16.2.216:60004]\r\n16/05/10 14:43:24 INFO Utils: Successfully started service 'sparkDriver' on port 60004.\r\n16/05/10 14:43:24 INFO SparkEnv: Registering MapOutputTracker\r\n16/05/10 14:43:24 INFO SparkEnv: Registering BlockManagerMaster\r\n16/05/10 14:43:24 INFO DiskBlockManager: Created local directory at /tmp/spark-298324d8-a187-4ccd-a65f-ebae5d7cd362/blockmgr-10f614a8-d0e7-4804-ae4d-eeb0da2153c2\r\n16/05/10 14:43:24 INFO MemoryStore: MemoryStore started with capacity 265.4 MB\r\n16/05/10 14:43:24 INFO HttpFileServer: HTTP File server directory is /tmp/spark-298324d8-a187-4ccd-a65f-ebae5d7cd362/httpd-ce5ffb91-418c-463e-b676-02ace110d297\r\n16/05/10 14:43:24 INFO HttpServer: Starting HTTP Server\r\n16/05/10 14:43:24 INFO Utils: Successfully started service 'HTTP file server' on port 36004.\r\n16/05/10 14:43:24 INFO SparkEnv: Registering OutputCommitCoordinator\r\n16/05/10 14:43:25 INFO Utils: Successfully started service 'SparkUI' on port 4040.\r\n16/05/10 14:43:25 INFO SparkUI: Started SparkUI at http://172.16.2.216:4040\r\n16/05/10 14:43:25 INFO Utils: Copying /opt/spark/dist/cleantrips/bin/spark_test.py to /tmp/spark-298324d8-a187-4ccd-a65f-ebae5d7cd362/userFiles-c3b4b65f-065b-4c81-a087-0930e3606d32/process_with_spark.py\r\n16/05/10 14:43:25 INFO SparkContext: Added file file:/opt/spark/dist/clean/bin/spark_test.py at http://172.16.2.216:36004/files/spark_test.py with timestamp 1462891405289\r\nWARNING: Logging before InitGoogleLogging() is written to STDERR\r\nF0510 14:43:25.419803   127 process.cpp:868] Failed to initialize: Failed to bind on 10.0.1.202:0: Cannot assign requested address: Cannot assign requested address [99]\r\n*** Check failure stack trace: ***\r\nAborted (core dumped)\r\n\r\nspark-dcoker.json file\r\n\r\n{\r\n \"schedule\": \"R/2016-05-10T14:41:00Z/PT24H\",\r\n \"name\": \"sparkdockerjob1\",\r\n \"container\": {\r\n   \"type\": \"DOCKER\",\r\n   \"image\": \"dockers/tests:1.1\",\r\n   \"network\": \"BRIDGE\",\r\n   \"volumes\": [\r\n     {\r\n       \"containerPath\": \"/var/log/\",\r\n       \"hostPath\": \"/logs/\",\r\n       \"mode\": \"RW\"\r\n     }\r\n   ]\r\n },\r\n \"cpus\": \"0.5\",\r\n \"mem\": \"512\",\r\n \"uris\": [],\r\n  \"command\": \"/opt/spark/dist/bin/spark-submit  --master mesos://leader.mesos:5050/mesos --executor-memory 2g --conf spark.mesos.coarse=true --conf spark.mesos.executor.docker.image=dockers/tests:1.1 /opt/spark/dist/clean/bin/spark_test.py testbucket Dcos-d-in Dcos-d-op \"\r\n}"},"comment":{"url":"https://api.github.com/repos/mesos/chronos/issues/comments/218827655","html_url":"https://github.com/mesos/chronos/issues/680#issuecomment-218827655","issue_url":"https://api.github.com/repos/mesos/chronos/issues/680","id":218827655,"user":{"login":"jgarcia-mesosphere","id":13425618,"avatar_url":"https://avatars.githubusercontent.com/u/13425618?v=3","gravatar_id":"","url":"https://api.github.com/users/jgarcia-mesosphere","html_url":"https://github.com/jgarcia-mesosphere","followers_url":"https://api.github.com/users/jgarcia-mesosphere/followers","following_url":"https://api.github.com/users/jgarcia-mesosphere/following{/other_user}","gists_url":"https://api.github.com/users/jgarcia-mesosphere/gists{/gist_id}","starred_url":"https://api.github.com/users/jgarcia-mesosphere/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jgarcia-mesosphere/subscriptions","organizations_url":"https://api.github.com/users/jgarcia-mesosphere/orgs","repos_url":"https://api.github.com/users/jgarcia-mesosphere/repos","events_url":"https://api.github.com/users/jgarcia-mesosphere/events{/privacy}","received_events_url":"https://api.github.com/users/jgarcia-mesosphere/received_events","type":"User","site_admin":false},"created_at":"2016-05-12T17:27:46Z","updated_at":"2016-05-12T17:27:46Z","body":"Hello!\r\n\r\nLooking at this line \r\n```\r\nF0510 14:43:25.419803 127 process.cpp:868] Failed to initialize: Failed to bind on 10.0.1.202:0: Cannot assign requested address: Cannot assign requested address [99]\r\n```\r\n\r\nit looks like Spark's attempt to open a network port failed. Do you get better results trying a similar job in Marathon?"}}