title: Load Google Datastore Backups from Data Storage to Google BigQuery
body: <p>Our requirement is to programmatically backup Google Datastore and load these backups to Google Big query for further analysis. We were successful in automating backups using the following approach</p>
  
  <pre><code>        Queue queue = QueueFactory.getQueue("datastoreBackupQueue");
  
          /*
           * Create a task which is equivalent to the backup URL mentioned in
           * above cron.xml, using new queue which has Datastore admin enabled
           */
          TaskOptions taskOptions = TaskOptions.Builder.withUrl("/_ah/datastore_admin/backup.create")
                  .method(TaskOptions.Method.GET).param("name", "").param("filesystem", "gs")
                  .param("gs_bucket_name",
                          "db-backup" + "/" + TimeUtils.parseDateToString(new Date(), "yyyy/MMM/dd"))
                  .param("queue", queue.getQueueName());
  
          /*
           * Get list of dynamic entity kind names from the datastore based on
           * the kinds present in the datastore at the start of backup
           */
          List&lt;String&gt; entityNames = getEntityNamesForBackup();
          for (String entityName : entityNames) {
              taskOptions.param("kind", entityName);
          }
  
          /* Add this task to above queue */
          queue.add(taskOptions);
  </code></pre>
  
  <p>I was able to then import this backups to Google Bigquery manually, But how do we automate this process?</p>
  
  <p>I have also looked at most of the docs and nothing helped
  <a href="https://cloud.google.com/bigquery/docs/loading-data-cloud-storage#loading_data_from_google_cloud_storage" rel="nofollow">https://cloud.google.com/bigquery/docs/loading-data-cloud-storage#loading_data_from_google_cloud_storage</a></p>