title: Join relatively small table with large table in Spark 2.1
body: <p>I am currently working on updating a table based on its existence on another table: </p>
  
  <p>Ex: </p>
  
  <p>Dataset A (relatively small, 300k of rows): <strong>DepartmentId, EmployeeId, Salary, Error</strong> </p>
  
  <p>Dataset B (relatively huge, millions of rows): <strong>DepartmentId, EmployeeId, Salary</strong></p>
  
  <p>The logic is:
  1. If A's (DepartmentId, EmployeeId) pair exists in B, then update A's salary with B's salary
  2. Otherwise, write a message to A's error field</p>
  
  <p>The solution I have now is doing a left outer join on A with B. Is there any other better practices for this type of problem? </p>
  
  <p>Thank you in advance!</p>