title: In Azure Data Factory, is it possible to filter the Source query of a Copy Activity, using a column from another Source?
body: <p>I have an invoicing table, 11M rows. I'd like to stop doing full refreshes each day and incrementally load. I essentially want to update changed rows and add new rows.</p>
  <p>My idea is to create a table key and use it to determine if an invoice line is present already in my Sink.</p>
  <p>Cases: Source Key is...</p>
  <ul>
  <li>In sink: Do nothing</li>
  <li>Not in Sink: Add to Sink</li>
  </ul>
  <p>Case: Sink key is...</p>
  <ul>
  <li>Not in Source: Delete</li>
  </ul>
  <p>To visualize what i'm going for...</p>
  <p><a href="https://i.stack.imgur.com/1lPkr.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/1lPkr.png" alt="enter image description here" /></a></p>
  <p>However, my problem is that my Source and Sink's are not of the same type. Source = ODBC (Hive), Sink = Azure SQL Database. So, in ADF, you cannot join them and the lookup's have a limit of 5000 rows.</p>
  <p>Is there any way to go about this? Anyway to use the column of one in the Where clause of another?</p>