title: code execution inside Spark foreach
body: <p>I have two RDDs: <code>points</code> and <code>pointsWithinEps</code>. Each point in <code>points</code> represent <code>x, y</code> coordinate. <code>pointsWithinEps</code> represent two points and distance between them: <code>((x, y), distance)</code>. I want to loop all points and for every point filter only that elements which are in the <code>pointsWithinEps</code> as <code>x</code> (first) coordinate. So I do following:</p>
  
  <pre><code>    points.foreach(p =&gt;
        val distances = pointsWithinEps.filter{
          case((x, y), distance) =&gt; x == p
        }
        if (distances.count() &gt; 3) {
  //        do some other actions
        }
      )
  </code></pre>
  
  <p>But this syntax is not valid. As far as I understand it is not allowed to create variables inside Spark foreach. Should I do something like this?</p>
  
  <pre><code>for (i &lt;- 0 to points.count().toInt) {
    val p = points.take(i + 1).drop(i) // take the point
    val distances = pointsWithinEps.filter{
      case((x, y), distance) =&gt; x == p
    }
    if (distances.count() &gt; 3) {
      //        do some other actions
    }
  }
  </code></pre>
  
  <p>Or there is a better way to do this? Complete code is hosted here: <a href="https://github.com/timasjov/spark-learning/blob/master/src/DBSCAN.scala" rel="nofollow">https://github.com/timasjov/spark-learning/blob/master/src/DBSCAN.scala</a></p>
  
  <p><strong>EDIT:</strong></p>
  
  <pre><code>points.foreach({ p =&gt;
    val pointNeighbours = pointsWithinEps.filter {
      case ((x, y), distance) =&gt; x == p
    }
    println(pointNeighbours)
  })
  </code></pre>
  
  <p>Right now I have following code, but it throws a NullPointerException (pointsWithinEps). How it can be fixed any why <code>pointsWithinEps</code> is null (before foreach there are elements in it)?</p>