title: How can you view the result of RDD.join() in Scala?
body: <p>I am trying to compute the result of a PageRanking algorithm, where the scoring function is the number of outgoing links on a page.</p>
  <pre><code>val links = warcs.map{ wr =&gt; wr._2.getRecord()}.
                 map{ wb =&gt; {
                          val url = wb.getHeader().getUrl()
                          val d = Jsoup.parse(wb.getHttpStringBody())
                          val links = d.select(&quot;a&quot;).asScala
                          links.map(l =&gt; (url,l.attr(&quot;href&quot;))).toIterator
                      }
                  }.
                  flatMap(identity).map(t =&gt; (t._1,List(t._2))).reduceByKey(_:::_)
                  
  
  
  
  var ranks = warcs.map{ wr =&gt; wr._2.getRecord()}.
                    map{ wb =&gt; (wb.getHeader().getUrl(), Jsoup.parse(wb.getHttpStringBody()).select(&quot;a[href]&quot;).size())}.
                    filter{ l =&gt; l._2 &gt; 0}
  </code></pre>
  <p>The links RDD is of the form (URL, list of outgoing URLs) and ranks is of the form (URL, number of outgoing URLs).</p>
  <p>This is what the pageranking looks like:</p>
  <pre><code>for(i &lt;- 1 to 10){
      val contribs = links.join(ranks).flatMap { case (url, (links, rank)) =&gt; links.map(dest =&gt; (dest, rank/links.size)) }
  
      ranks = contribs.reduceByKey((x,y) =&gt; x+y).mapValues(sum =&gt; (0.15 + 0.85*sum).toInt)
  }
  </code></pre>
  <p>This being said, when I try to check the results of the ranking algorithm, I am met with an IndexOutOfBoundsException. I tried seeing if the resulting RDD is empty by printing <code>ranks.isEmpty()</code> and I get the same exception.</p>
  <p>I have tried out of curiosity to see the result of <code>links.join(ranks)</code>, but the same exception once again occurs.</p>
  <p>What is going wrong with the join() operation, and how can I progress?</p>