title: Compute similarity between sentences using Spark
body: <p>I have the following input for my problem statement:- </p>
  
  <pre><code> ID  -&gt; List of Words
  (101 -&gt; Array("a1","b2","c4","d2"))
  (102 -&gt; Array("a6","b1","c5","d3"))
  (103 -&gt; Array("a1","b4","c4","d2"))
  (104 -&gt; Array("a2","b2","c3","d2"))
  (105 -&gt; Array("a7","b6","c1","d3"))
  </code></pre>
  
  <p>Now, I want to find out the similarity between these input statements.</p>
  
  <p>Example:- </p>
  
  <pre><code>(101 -&gt; Array("a1","b2","c4","d2"))
  (103 -&gt; Array("a1","b4","c4","d2"))
  (104 -&gt; Array("a2","b2","c3",",d2"))
  </code></pre>
  
  <p>In Example output the statements are much similar to each other.</p>
  
  <p>How can I achieve this Using Spark? I can use any logical code or any machine learning Algorithm.</p>
  
  <p>Thanks</p>