title: Google cloud functions python script creates bucket - doesn't upload files
body: <p><strong>What I'm looking to do:</strong></p>
  <ul>
  <li>Run some BigQuery queries</li>
  <li>Output results as JSON files</li>
  <li>Upload JSON files to GCS</li>
  </ul>
  <p><strong>How I'm trying to do it:</strong></p>
  <ol>
  <li>Install and initialise Google Cloud SDK: <code>gcloud auth activate-service-account --key-file=&quot;gcp-credentials.json&quot;</code></li>
  <li>Enable APIs:</li>
  </ol>
  <pre class="lang-sh prettyprint-override"><code>gcloud services enable \
      bigquery.googleapis.com \
      cloudbuild.googleapis.com \
      cloudfunctions.googleapis.com \
      cloudscheduler.googleapis.com \
      pubsub.googleapis.com \
      serviceusage.googleapis.com \
      storage-component.googleapis.com
  </code></pre>
  <ol start="3">
  <li>Write up code:</li>
  </ol>
  <pre><code>src
  |__data
  |__queries
        |__test_query_1.sql
        |__test_query_2.sql
        |__test_query_3.sql
  |__scripts
        |__config.py
        |__log.txt
        |__main.py
        |__requirements.txt
  </code></pre>
  <p>requirements.txt</p>
  <pre><code>google-cloud-bigquery
  google-cloud-storage
  </code></pre>
  <p>config.py:</p>
  <pre class="lang-py prettyprint-override"><code>from pathlib import Path
  
  src_dir = Path(__file__).absolute().parent
  
  config_vars = {
      &quot;data_dir&quot;: src_dir.parent / &quot;data&quot;,
      &quot;queries_dir&quot;: src_dir.parent / &quot;queries&quot;,
      &quot;bucket&quot;: &quot;...&quot;,
  }
  </code></pre>
  <p>main.py:</p>
  <pre class="lang-py prettyprint-override"><code>import ...
  data_dir = config.config_vars[&quot;data_dir&quot;]
  queries_dir = config.config_vars[&quot;queries_dir&quot;]
  
  def main(data, context):
      ...
  
  if __name__ == &quot;__main__&quot;:
      main(&quot;data&quot;, &quot;context&quot;)
  </code></pre>
  <p>So the <code>main.py</code> script, takes all queries in the queries folder, runs them, outputs them as JSON and then uploads them to a bucket called &quot;test-bucket-20201219&quot;. If the bucket doesn't exist then it creates it.</p>
  <p>The script runs fine locally but when it's deployed and scheduled in GCP via PubSub and Google Scheduler, it runs and creates the bucket but doesn't upload the files...I'm not sure what I'm doing wrong. Any help would be much appreciated. Tried everything - e.g. permitted the PROJECTID@appspot.gserviceaccount.com to add objects to bucket.</p>
  <p>Logging statements:</p>
  <pre><code>2020-12-20 18:43:50,656 | INFO | Uploading test_query_2.json to test-bucket-20201219.
  2020-12-20 18:43:50,962 | DEBUG | https://storage.googleapis.com:443 &quot;POST /upload/storage/v1/b/test-bucket-20201219/o?uploadType=multipart HTTP/1.1&quot; 200 776
  2020-12-20 18:43:50,963 | INFO | Uploading test_query_3.json to test-bucket-20201219.
  2020-12-20 18:43:51,238 | DEBUG | https://storage.googleapis.com:443 &quot;POST /upload/storage/v1/b/test-bucket-20201219/o?uploadType=multipart HTTP/1.1&quot; 200 776
  2020-12-20 18:43:51,239 | INFO | Uploading test_query_1.json to test-bucket-20201219.
  2020-12-20 18:43:51,466 | DEBUG | https://storage.googleapis.com:443 &quot;POST /upload/storage/v1/b/test-bucket-20201219/o?uploadType=multipart HTTP/1.1&quot; 200 775
  </code></pre>