 <p>Well, you defintitely need a ETL tool then and preferably SSIS. First it will drastically improve your transfer rates while also providing robust error handling. Additionally you will have to use lookup transforms to ensure duplicates do not enter the sytsem. I would suggest go for Cache Connection Manager in order to perform the look-ups. </p> <p>In terms of design, if your source system (OurDB) is having a primary key say recId, then have a column say source_rec_id in your InterfaceDB table. Say your first run has transferred 100 rows. Now in your second run, you would then need to pick 100+1th record and move on to the next rows. This way you will have a tracking mechanism and one-to-one correlation between source system and destination system to understand how many records have got transferred, how many are left etc.</p> <p>For best understanding of SSIS go to <a href="https://channel9.msdn.com/Search?term=ssis#ch9Search&amp;lang-en=en" rel="nofollow noreferrer">Channel 9 - msdn - SSIS</a>. Very helpful resource.</p>