title: Accessing GCS with Hadoop client from outside of Cloud
body: <p>I want to access Google Cloud Storage via Hadoop client. I want to use it on machine outside of Google Cloud.</p>
  
  <p>I followed instructions from <a href="https://cloud.google.com/dataproc/docs/concepts/connectors/install-storage-connector" rel="nofollow noreferrer">here</a>. 
  I created service account and generated key file. I also created core-site.xml file and downloaded the necessary library.</p>
  
  <p>However, when I am trying to run simple <code>hdfs dfs -ls gs://bucket-name</code> command, all I get is this:</p>
  
  <pre><code>Error getting access token from metadata server at: http://169.254.169.254/computeMetadata/v1/instance/service-accounts/default/token
  </code></pre>
  
  <p>When I am doing this <strong>inside</strong> the Google Cloud it works, but trying to connect to GCS from <strong>outside</strong>, it shows error above.</p>
  
  <p>How to connect to GCS with Hadoop Client in this way? Is it even possible? I have no route to 169.254.169.254 address.</p>
  
  <p>Here is my core-site.xml(I changed the key path and email in this example):</p>
  
  <pre><code>&lt;?xml version="1.0" encoding="UTF-8"?&gt;
  &lt;?xml-stylesheet type="text/xsl" href="configuration.xsl"?&gt;
  &lt;!--
    Licensed under the Apache License, Version 2.0 (the "License");
    you may not use this file except in compliance with the License.
    You may obtain a copy of the License at
  
      http://www.apache.org/licenses/LICENSE-2.0
  
    Unless required by applicable law or agreed to in writing, software
    distributed under the License is distributed on an "AS IS" BASIS,
    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    See the License for the specific language governing permissions and
    limitations under the License. See accompanying LICENSE file.
  --&gt;
  
  &lt;!-- Put site-specific property overrides in this file. --&gt;
  
  &lt;configuration&gt;
    &lt;property&gt;
      &lt;name&gt;spark.hadoop.google.cloud.auth.service.account.enable&lt;/name&gt;
      &lt;value&gt;true&lt;/value&gt;
    &lt;/property&gt;
    &lt;property&gt;
      &lt;name&gt;spark.hadoop.google.cloud.auth.service.account.json.keyfile&lt;/name&gt;
      &lt;value&gt;path/to/key.json&lt;/value&gt;
    &lt;/property&gt;
    &lt;property&gt;
      &lt;name&gt;fs.gs.project.id&lt;/name&gt;
      &lt;value&gt;ringgit-research&lt;/value&gt;
      &lt;description&gt;
        Optional. Google Cloud Project ID with access to GCS buckets.
        Required only for list buckets and create bucket operations.
      &lt;/description&gt;
    &lt;/property&gt;
    &lt;property&gt;
      &lt;name&gt;fs.AbstractFileSystem.gs.impl&lt;/name&gt;
      &lt;value&gt;com.google.cloud.hadoop.fs.gcs.GoogleHadoopFS&lt;/value&gt;
      &lt;description&gt;The AbstractFileSystem for gs: uris.&lt;/description&gt;
    &lt;/property&gt;
    &lt;property&gt;
      &lt;name&gt;fs.gs.auth.service.account.email&lt;/name&gt;
      &lt;value&gt;myserviceaccountaddress@google&lt;/value&gt;
      &lt;description&gt;
        The email address is associated with the service account used for GCS
        access when fs.gs.auth.service.account.enable is true. Required
        when authentication key specified in the Configuration file (Method 1)
        or a PKCS12 certificate (Method 3) is being used.
      &lt;/description&gt;
    &lt;/property&gt;
  &lt;/configuration&gt;
  </code></pre>