<p>I have a simple application that Gets and Puts information from a Datastore.</p>

<p>It works everywhere, but when I run it from inside the Kubernetes Engine cluster, I get this output:</p>

<pre><code>Error from Get()
rpc error: code = PermissionDenied desc = Request had insufficient authentication scopes.
Error from Put()
rpc error: code = PermissionDenied desc = Request had insufficient authentication scopes.
</code></pre>

<p>I'm using the <code>cloud.google.com/go/datastore</code> package and the Go language.</p>

<p>I don't know why I'm getting this error since the application works everywhere else just fine.</p>

<p><strong>Update:</strong></p>

<p>Looking for an answer I found this comment on Google Groups:</p>

<blockquote>
  <p>In order to use Cloud Datastore from GCE, the instance needs to be
  configured with a couple of extra scopes. These can't be added to
  existing GCE instances, but you can create a new one with the
  following Cloud SDK command:</p>
  
  <p>gcloud compute instances create hello-datastore --project
   --zone  --scopes datastore userinfo-email</p>
</blockquote>

<p>Would that mean I can't use Datastore from GKE by default?</p>

<p><strong>Update 2:</strong></p>

<p>I can see that when creating my cluster I didn't enable any permissions (which are disabled for most services by default). I suppose that's what's causing the issue:</p>

<p><img src="https://i.stack.imgur.com/gTx4i.png" width="400"/></p>

<p>Strangely, I can use CloudSQL just fine even though it's disabled (using the <code>cloudsql_proxy</code> container).</p>

<p>So what I learnt in the process of debugging this issue was that:</p>

<ul>
<li><p>During the creation of a Kubernetes Cluster you can specify permissions for the GCE nodes that will be created.</p></li>
<li><p>If you for example enable Datastore access on the cluster nodes during creation, you will be able to access Datastore directly from the Pods without having to set up anything else.</p></li>
<li><p>If your cluster node permissions are disabled for most things (default settings) like mine were, you will need to create an appropriate Service Account for each application that wants to use a GCP resource like Datastore.</p></li>
<li><p>Another alternative is to create a new node pool with the <code>gcloud</code> command, set the desired permission scopes and then migrate all deployments to the new node pool (rather tedious).</p></li>
</ul>

<p>So at the end of the day I fixed the issue by creating a Service Account for my application, downloading the JSON authentication key, creating a Kubernetes secret which contains that key, and in the case of Datastore, I set the <code>GOOGLE_APPLICATION_CREDENTIALS</code> environment variable to the path of the mounted secret JSON key. </p>

<p>This way when my application starts, it checks if the <code>GOOGLE_APPLICATION_CREDENTIALS</code> variable is present, and authenticates Datastore API access based on the JSON key that the variable points to.</p>

<p>Deployment YAML snippet:</p>

<pre><code>  ...
  containers:
  - image: foo
    name: foo
    env:
    - name: GOOGLE_APPLICATION_CREDENTIALS
      value: /auth/credentials.json
    volumeMounts:
    - name: foo-service-account
      mountPath: "/auth"
      readOnly: true
  volumes:
  - name: foo-service-account
    secret:
      secretName: foo-service-account
</code></pre>
