title: Azure Data Factory : copying data from CosmosDB to SQL Database fails with "range-indexed" error
body: <p>I need to move data from Azure CosmosDB (previously known as DocumentDB) into Azure SQL database.</p>
  
  <p>I am using the "Copy Data" resource where I have setup the source and the destination as well as the mapping.</p>
  
  <p>The schedule has been setup for once every hour. The issues : </p>
  
  <ol>
  <li>I am noticing that the no. of instances or "runs" are more than once per hour. Is this because it is failing and it keeps retrying for a certain amount of times ? </li>
  <li>The main error why its failing seems to be this : </li>
  </ol>
  
  <blockquote>
    <p>Activity Copy_dbo_SubscriptionLocator failed: Failure happened on
    'Source' side.
    ErrorCode=UserErrorDocumentDBReadError,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=DocumentDb
    operation failed: Message: {"Errors":["An invalid query has been
    specified with filters against path(s) that are not range-indexed.
    Consider adding allow scan header in the request."]}</p>
  </blockquote>
  
  <p>What changes do I need to make on CosmosDB end to avoid the error ?</p>
  
  <p>Following is my current indexing policy : </p>
  
  <pre><code>{
      "indexingMode": "consistent",
      "automatic": true,
      "includedPaths": [
          {
              "path": "/*",
              "indexes": [
                  {
                      "kind": "Range",
                      "dataType": "Number",
                      "precision": -1
                  },
                  {
                      "kind": "Hash",
                      "dataType": "String",
                      "precision": 3
                  }
              ]
          }
      ],
      "excludedPaths": [
          {
              "path": "/OneOfTheColumns/*"
          }
      ]
  }
  </code></pre>
  
  <p>Looks like the issue is with the datetime field which needs to be range indexed as a string. I could possibly use the _ts field in the document. But how do I change the Copy job to convert the query's datetime to epoch time and use its value for _ts field.</p>