title: Azure data factory copy activity mystery (one duped row, one missing row from source)
body: <p>Okay I am at a total loss here - I have tried tweaking every setting I can think of in the copy activity I am dealing with but for some reason on the sink it is inserting a duplicate row, and missing a row from the datasource it pulls.</p>
  <p>Data comes in via REST from our CRM system - this works fine, all records come through - then this mystery happens where the duplicated rows get inserted (this even occurred somehow when I changed it from insert to upsert and specified the GUID of the row as the key - how the heck ??)</p>
  <p>Overall flow is : Grab auth token from rest source -&gt; grab records via REST endpoint in a Copy data activity</p>
  <p><a href="https://i.stack.imgur.com/ufd3m.png" rel="nofollow noreferrer">Copy data settings - source settings</a></p>
  <p><a href="https://i.stack.imgur.com/CpYRh.png" rel="nofollow noreferrer">Sink settings copy data</a></p>
  <p>Settings of the copy data step are default - I have tried editing DIUs, parallelism, max concurrent connections, different batch sizes, etc to no impact.</p>
  <p>It appears to always be the 15,000 row that is duplicated, so 15,001 is 15,000 duped, so in my mind I am thinking it is something to do with the data being partitioned and it's not indexed properly? I just don't get how this is happening - it doesn't happen with any of our other 60+ pipelines we have run nightly.</p>
  <p>Any help or suggestions is greatly appreciated!</p>