title: Azure storage incremental copy by modified date
body: <p>I need to copy one storage account into another. I have created a <code>Runbook</code> and schedule it to run daily. This is an incremental copy.</p>
  
  <p>What I am doing is</p>
  
  <ol>
  <li>list the blobs in source storage container </li>
  <li>Check the blobs in destination storage container</li>
  <li>If it doesn't exist in destination container copy the blob <code>Start-AzureStorageBlobCopy</code></li>
  </ol>
  
  <p>While this works for containers with small size, this takes a very long time and is certainly cost ineffective for containers with say 10 million block blobs because every time I run the task I have to go through all those 10 million blobs.</p>
  
  <p>I don't see it in <a href="https://msdn.microsoft.com/en-us/library/dn806392.aspx" rel="nofollow">documentation</a> but is there any way i can use conditional headers like <code>DateModifedSince</code> some thing like <code>Get-AzureStorageBlob -DateModifiedSince date</code> in powershell.</p>
  
  <p>I have not tried but I can see it is possible to use <code>DateModifiedSince</code> in <a href="https://msdn.microsoft.com/en-us/library/dn806392.aspx" rel="nofollow">nodejs library</a></p>
  
  <p>Is there anyway I can do it with powershell so that I can be able to use <code>Runbooks</code>?</p>
  
  <p>EDIT:</p>
  
  <p>Using AzCopy made a copy of storage account that contains 7 million blobs, I uploaded few new blobs and started the azcopy again. It still takes significant amount of time to copy few new uploaded files.</p>
  
  <p><code>AzCopy /Source:$sourceUri /Dest:$destUri /SourceKey:$sourceStorageKey /DestKey:$destStorageAccountKey /S /XO /XN /Y</code></p>
  
  <p>It is possible to filter for a blob with blob name in no time</p>
  
  <p>For example <code>Get-AzureStorageBlob -Blob</code> will return the blob immediately from 7 million records</p>
  
  <p>It should have been possible to filter blob(s) with other properties too..</p>