title: With Spark,how to connect master or solve an error:"WARN TaskSchedulerImpl: Initial job has not accepted any resources"
body: <p>Please tell me to how to following problem. </p>
  
  <p>Firstly,I confirmed that following code run when master is "local". </p>
  
  <p>Then I started two EC2 instances(m1.large). 
  However,when master is "spark://MASTER_PUBLIC_DNS:7077",the error message,"TaskSchedulerImpl", appears and it fails. </p>
  
  <p>When I change to INVALID address as a master(spark://INVALID_DNS:7077) from VALID address,the same error message appears.</p>
  
  <p>Namely,"WARN TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient memory" </p>
  
  <p>It seems like <a href="http://apache-spark-user-list.1001560.n3.nabble.com/TaskSchedulerImpl-Initial-job-has-not-accepted-any-resources-check-your-cluster-UI-to-ensure-that-woy-td8247.html" rel="nofollow">this</a>.
  As this comment, I assigned 12G memory to this cluster but it fails.</p>
  
  <pre><code>#!/usr/bin/env python                                                                                     
  # -*- coding: utf-8 -*- 
  from pyspark import SparkContext, SparkConf 
  from pyspark.mllib.classification import LogisticRegressionWithSGD 
  from pyspark.mllib.regression import LabeledPoint 
  from numpy import array 
  
  # Load and parse the data 
  def parsePoint(line): 
    values = [float(x) for x in line.split(' ')] 
    return LabeledPoint(values[0], values[1:]) 
  appName = "testsparkapp" 
  master = "spark://MASTER_PUBLIC_DNS:7077" 
  #master = "local" 
  
  
  conf = SparkConf().setAppName(appName).setMaster(master) 
  sc = SparkContext(conf=conf) 
  
  data = sc.textFile("/root/spark/mllib/data/sample_svm_data.txt") 
  parsedData = data.map(parsePoint) 
  
  # Build the model 
  model = LogisticRegressionWithSGD.train(parsedData) 
  
  # Evaluating the model on training data 
  labelsAndPreds = parsedData.map(lambda p: (p.label, model.predict(p.features))) 
  trainErr = labelsAndPreds.filter(lambda (v, p): v != p).count() / float(parsedData.count()) 
  print("Training Error = " + str(trainErr))     
  </code></pre>
  
  <h2>Additional</h2>
  
  <p>I did three tasks that my friend adviced me to.</p>
  
  <p>1.I opened master port,7077.</p>
  
  <p>2.In master url, set host name not ip address. </p>
  
  <p>->There fore,I became to able to connect master server(I checked it by Cluster UI).</p>
  
  <p>3.I tried to set worker_max_heap,as following, but it fails probably.</p>
  
  <p>ScalaConf().set("spark.executor.memory", "4g").set("worker_max_heapsize","2g")</p>
  
  <p>the worker allow me to use 6.3GB(I checked it by UI).It is m1.large.</p>
  
  <p>->I recognized a warning in my executing log and an error in a worker stderr.</p>
  
  <h3>my executing log</h3>
  
  <pre><code>14/08/08 06:11:59 WARN TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient memory
  </code></pre>
  
  <h3>worker stderr</h3>
  
  <pre><code>14/08/08 06:14:04 INFO worker.WorkerWatcher: Successfully connected to akka.tcp://sparkWorker@PRIVATE_HOST_NAME1:52011/user/Worker
  14/08/08 06:15:07 ERROR executor.CoarseGrainedExecutorBackend: Driver Disassociated [akka.tcp://sparkExecutor@PRIVATE_HOST_NAME1:52201] -&gt; [akka.tcp://spark@PRIVATE_HOST_NAME2:38286] disassociated! Shutting down.
  </code></pre>