title: Find diff for two datasets with Hadoop
body: <p>I have two datasets in files with the following data (~500 millinons lines in each file):</p>
  
  <pre><code>Dataset1:
  a1
  a2
  a3
  a4
  
  Dataset2:
  a1
  a2
  </code></pre>
  
  <p>I want to find elements in Dataset1 which isn't existing in Dataset2
  So, the result should be:</p>
  
  <pre><code>Dataset3
  a3
  a4
  </code></pre>
  
  <p>What is the best way to do this with Hadoop? Any Join-related machanisms, or Pig, or something else? Thanks.</p>