title: How to install Google Cloud SDK's app-engine-python on Compute Engine (Ubuntu 16.04)?
body: <p>I would like to use Google Cloud SDK (including the Python Extensions for App Engine) inside Compute Engine, so I can replicate my local development setup in a virtual machine. I.e. run local <code>dev_appserver.py</code> and unit-testing on the VM, or deploy new app versions to Google App Engine.</p>
  
  <p>After creating a new VM instance from the default Ubuntu 16.04 image and (machine type <code>n1-standard-1</code>), I have noticed that <code>gcloud</code> is already pre-installed.</p>
  
  <pre><code>anh@ani-dev-2:~$ gcloud components list
  
  Your current Cloud SDK version is: 163.0.0
  The latest available version is: 169.0.0
  
  ┌────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
  │                                                   Components                                                   │
  ├──────────────────┬──────────────────────────────────────────────────────┬──────────────────────────┬───────────┤
  │      Status      │                         Name                         │            ID            │    Size   │
  ├──────────────────┼──────────────────────────────────────────────────────┼──────────────────────────┼───────────┤
  │ Update Available │ BigQuery Command Line Tool                           │ bq                       │   &lt; 1 MiB │
  │ Update Available │ Cloud SDK Core Libraries                             │ core                     │   6.7 MiB │
  │ Update Available │ Cloud Storage Command Line Tool                      │ gsutil                   │   3.0 MiB │
  │ Not Installed    │ App Engine Go Extensions                             │ app-engine-go            │  98.0 MiB │
  │ Not Installed    │ Cloud Bigtable Command Line Tool                     │ cbt                      │   4.1 MiB │
  │ Not Installed    │ Cloud Bigtable Emulator                              │ bigtable                 │   3.5 MiB │
  │ Not Installed    │ Cloud Datalab Command Line Tool                      │ datalab                  │   &lt; 1 MiB │
  │ Not Installed    │ Cloud Datastore Emulator                             │ cloud-datastore-emulator │  15.4 MiB │
  │ Not Installed    │ Cloud Datastore Emulator (Legacy)                    │ gcd-emulator             │  38.1 MiB │
  │ Not Installed    │ Cloud Pub/Sub Emulator                               │ pubsub-emulator          │  33.2 MiB │
  │ Not Installed    │ Emulator Reverse Proxy                               │ emulator-reverse-proxy   │  14.5 MiB │
  │ Not Installed    │ Google Container Local Builder                       │ container-builder-local  │   3.7 MiB │
  │ Not Installed    │ Google Container Registry's Docker credential helper │ docker-credential-gcr    │   2.2 MiB │
  │ Not Installed    │ gcloud Alpha Commands                                │ alpha                    │   &lt; 1 MiB │
  │ Not Installed    │ gcloud Beta Commands                                 │ beta                     │   &lt; 1 MiB │
  │ Not Installed    │ gcloud app Java Extensions                           │ app-engine-java          │ 128.1 MiB │
  │ Not Installed    │ gcloud app Python Extensions                         │ app-engine-python        │   6.5 MiB │
  │ Not Installed    │ kubectl                                              │ kubectl                  │  16.0 MiB │
  │ Installed        │ Default set of gcloud commands                       │ gcloud                   │           │
  └──────────────────┴──────────────────────────────────────────────────────┴──────────────────────────┴───────────┘
  </code></pre>
  
  <p>Intuitively, I've tried to install the missing <code>app-engine-python</code> component.</p>
  
  <pre><code>anh@ani-dev-2:~$ gcloud components install app-engine-python
  You cannot perform this action because this Cloud SDK installation is 
  managed by an external package manager.  If you would like to get the 
  latest version, please see our main download page at:
  
  https://cloud.google.com/sdk/
  ERROR: (gcloud.components.install) The component manager is disabled for 
  this installation
  </code></pre>
  
  <p>As far as I understand, there are alternative installation methods available (yum, debian) and deb was used for the VM image, which is the reason the component manager is disabled:</p>
  
  <pre><code>anh@ani-dev-2:~$ dpkg -s google-cloud-sdk
  Package: google-cloud-sdk
  Status: install ok installed
  Priority: optional
  Section: partner/admin
  Installed-Size: 93228
  Maintainer: Canonical CPC Team &lt;cpc@canonical.com&gt;
  Architecture: all
  Version: 163.0.0-0ubuntu1~16.04.0
  Depends: findutils, python2.7, python:any (&gt;= 2.6.6-7~), python-crcmod, python-google-compute-engine
  Conflicts: gsutils
  Conffiles:
   /etc/bash_completion.d/completion.bash.inc 860b91bfb6a0e6b513321ea6ca429e4c
  Description: Google Cloud SDK for interacting with Google Cloud services.
   The command-line tools for Google App Engine, Compute Engine,
   Cloud Storage, BigQuery, Cloud SQL, and Cloud DNS are bundled as part of
   the Cloud SDK.
   .
   This is a packaged distribution of the Google Cloud SDK. For up-to-date
   versions please see https://developers.google.com/cloud/sdk.
  Homepage: https://developers.google.com/cloud/sdk/
  </code></pre>
  
  <p>So I've installed successfully the missing component <code>google-cloud-sdk-app-engine-python</code> via apt-get as explained in the docs <a href="https://cloud.google.com/sdk/downloads#apt-get" rel="nofollow noreferrer">Installing Cloud SDK</a>. It seems that the pre-installed Cloud SDK package 163.0.0 was replaced by the recent version 169.0.0, and the python component is now installed.</p>
  
  <pre><code>anh@ani-dev-2:~$ gcloud components list
  
  Your current Cloud SDK version is: 169.0.0
  The latest available version is: 169.0.0
  
  ┌─────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
  │                                                  Components                                                 │
  ├───────────────┬──────────────────────────────────────────────────────┬──────────────────────────┬───────────┤
  │     Status    │                         Name                         │            ID            │    Size   │
  ├───────────────┼──────────────────────────────────────────────────────┼──────────────────────────┼───────────┤
  │ Not Installed │ App Engine Go Extensions                             │ app-engine-go            │  98.0 MiB │
  │ Not Installed │ Cloud Bigtable Command Line Tool                     │ cbt                      │   4.1 MiB │
  │ Not Installed │ Cloud Bigtable Emulator                              │ bigtable                 │   3.5 MiB │
  │ Not Installed │ Cloud Datalab Command Line Tool                      │ datalab                  │   &lt; 1 MiB │
  │ Not Installed │ Cloud Datastore Emulator                             │ cloud-datastore-emulator │  15.4 MiB │
  │ Not Installed │ Cloud Datastore Emulator (Legacy)                    │ gcd-emulator             │  38.1 MiB │
  │ Not Installed │ Cloud Pub/Sub Emulator                               │ pubsub-emulator          │  33.2 MiB │
  │ Not Installed │ Emulator Reverse Proxy                               │ emulator-reverse-proxy   │  14.5 MiB │
  │ Not Installed │ Google Container Local Builder                       │ container-builder-local  │   3.7 MiB │
  │ Not Installed │ Google Container Registry's Docker credential helper │ docker-credential-gcr    │   2.2 MiB │
  │ Not Installed │ gcloud app Java Extensions                           │ app-engine-java          │ 128.1 MiB │
  │ Not Installed │ kubectl                                              │ kubectl                  │  16.0 MiB │
  │ Installed     │ BigQuery Command Line Tool                           │ bq                       │   &lt; 1 MiB │
  │ Installed     │ Cloud SDK Core Libraries                             │ core                     │   6.7 MiB │
  │ Installed     │ Cloud Storage Command Line Tool                      │ gsutil                   │   3.0 MiB │
  │ Installed     │ Default set of gcloud commands                       │ gcloud                   │           │
  │ Installed     │ gcloud Alpha Commands                                │ alpha                    │   &lt; 1 MiB │
  │ Installed     │ gcloud Beta Commands                                 │ beta                     │   &lt; 1 MiB │
  │ Installed     │ gcloud app Python Extensions                         │ app-engine-python        │   6.5 MiB │
  └───────────────┴──────────────────────────────────────────────────────┴──────────────────────────┴───────────┘
  </code></pre>
  
  <ol>
  <li><p>Is this approach the recommended way to add Cloud SDK components on Compute Engine?</p></li>
  <li><p>Is the same approach also supported with a micro or small machine-type? I assume, <code>f1-micro</code> (or <code>g1-small</code>) would be sufficient for my purpose. But maybe since in the smaller types the VM <em>shares</em> the CPU I wouldn't have permissions to change the installation. If this is the case, am I supposed to install an additional, custom Cloud SDK manually into my user folder, e.g. <code>$HOME/google-cloud-sdk</code> like I did on my Mac?</p></li>
  </ol>
  
  <h2>P.S. 1</h2>
  
  <p>This is the output when I perform the same steps in the <code>f1-micro</code> (same VM image), I've tried twice on two new instances:</p>
  
  <pre><code>anh@ani-dev-1:~$ sudo apt-get update &amp;&amp; sudo apt-get install google-cloud-sdk
  Hit:1 http://us-east1.gce.archive.ubuntu.com/ubuntu xenial InRelease
  Get:2 http://us-east1.gce.archive.ubuntu.com/ubuntu xenial-updates InRelease [102 kB]                             
  Get:3 http://us-east1.gce.archive.ubuntu.com/ubuntu xenial-backports InRelease [102 kB]                           
  Get:4 http://us-east1.gce.archive.ubuntu.com/ubuntu xenial/main Sources [868 kB]                                  
  Get:5 http://us-east1.gce.archive.ubuntu.com/ubuntu xenial/restricted Sources [4,808 B]                           
  Get:6 http://us-east1.gce.archive.ubuntu.com/ubuntu xenial/universe Sources [7,728 kB]                            
  Get:7 http://security.ubuntu.com/ubuntu xenial-security InRelease [102 kB]                                        
  Hit:8 http://archive.canonical.com/ubuntu xenial InRelease                                                        
  Get:9 http://packages.cloud.google.com/apt cloud-sdk-xenial InRelease [6,327 B]                                   
  Get:10 http://us-east1.gce.archive.ubuntu.com/ubuntu xenial/multiverse Sources [179 kB]                           
  Get:11 http://us-east1.gce.archive.ubuntu.com/ubuntu xenial-updates/main Sources [273 kB]                         
  Get:12 http://us-east1.gce.archive.ubuntu.com/ubuntu xenial-updates/restricted Sources [3,400 B]
  Get:13 http://us-east1.gce.archive.ubuntu.com/ubuntu xenial-updates/universe Sources [171 kB]
  Get:14 http://us-east1.gce.archive.ubuntu.com/ubuntu xenial-updates/multiverse Sources [7,232 B]
  Get:15 http://us-east1.gce.archive.ubuntu.com/ubuntu xenial-updates/main amd64 Packages [632 kB]
  Get:16 http://us-east1.gce.archive.ubuntu.com/ubuntu xenial-updates/main Translation-en [261 kB]
  Get:17 http://us-east1.gce.archive.ubuntu.com/ubuntu xenial-updates/restricted amd64 Packages [8,048 B]
  Get:18 http://us-east1.gce.archive.ubuntu.com/ubuntu xenial-updates/restricted Translation-en [2,688 B]
  Get:19 http://us-east1.gce.archive.ubuntu.com/ubuntu xenial-updates/universe amd64 Packages [529 kB]
  Get:20 http://us-east1.gce.archive.ubuntu.com/ubuntu xenial-updates/universe Translation-en [209 kB]
  Get:21 http://us-east1.gce.archive.ubuntu.com/ubuntu xenial-backports/main Sources [3,396 B]
  Get:22 http://us-east1.gce.archive.ubuntu.com/ubuntu xenial-backports/universe Sources [4,404 B]
  Get:23 http://us-east1.gce.archive.ubuntu.com/ubuntu xenial-backports/main amd64 Packages [4,884 B]
  Get:24 http://packages.cloud.google.com/apt cloud-sdk-xenial/main amd64 Packages [9,625 B]
  Get:25 http://security.ubuntu.com/ubuntu xenial-security/main Sources [90.1 kB]  
  Get:26 http://security.ubuntu.com/ubuntu xenial-security/restricted Sources [2,604 B]
  Get:27 http://security.ubuntu.com/ubuntu xenial-security/universe Sources [40.9 kB]
  Get:28 http://security.ubuntu.com/ubuntu xenial-security/multiverse Sources [1,144 B]
  Get:29 http://security.ubuntu.com/ubuntu xenial-security/main amd64 Packages [353 kB]
  Get:30 http://security.ubuntu.com/ubuntu xenial-security/main Translation-en [154 kB]
  Get:31 http://security.ubuntu.com/ubuntu xenial-security/universe amd64 Packages [166 kB]
  Get:32 http://security.ubuntu.com/ubuntu xenial-security/universe Translation-en [86.8 kB]
  Fetched 12.1 MB in 2s (4,922 kB/s)             
  Reading package lists... Done
  Reading package lists... Done
  Building dependency tree       
  Reading state information... Done
  Suggested packages:
    google-cloud-sdk-app-engine-java google-cloud-sdk-app-engine-python google-cloud-sdk-pubsub-emulator
    google-cloud-sdk-bigtable-emulator google-cloud-sdk-datastore-emulator kubectl
  The following packages will be upgraded:
    google-cloud-sdk
  1 upgraded, 0 newly installed, 0 to remove and 13 not upgraded.
  Need to get 13.8 MB of archives.
  After this operation, 10.7 MB of additional disk space will be used.
  Get:1 http://packages.cloud.google.com/apt cloud-sdk-xenial/main amd64 google-cloud-sdk all 169.0.0-0 [13.8 MB]
  Fetched 13.8 MB in 1s (11.4 MB/s)            
  (Reading database ... 64568 files and directories currently installed.)
  Preparing to unpack .../google-cloud-sdk_169.0.0-0_all.deb ...
  Unpacking google-cloud-sdk (169.0.0-0) over (163.0.0-0ubuntu1~16.04.0) ...
  Processing triggers for man-db (2.7.5-1) ...
  Setting up google-cloud-sdk (169.0.0-0) ...
  Killed
  dpkg: error processing package google-cloud-sdk (--configure):
   subprocess installed post-installation script returned error exit status 137
  E: Sub-process /usr/bin/dpkg returned an error code (1)
  </code></pre>
  
  <h2>P.S. 2</h2>
  
  <p>It seems the reason why the installation process is killed in <code>f1-micro</code> is some kind of out-of-memory error. See: 
  <a href="https://serverfault.com/questions/862019/how-to-resolve-dpkg-error-processing-package-google-cloud-sdk">https://serverfault.com/questions/862019/how-to-resolve-dpkg-error-processing-package-google-cloud-sdk</a>
  and <a href="https://groups.google.com/forum/#!topic/gce-discussion/BBHc2QebbBw" rel="nofollow noreferrer">https://groups.google.com/forum/#!topic/gce-discussion/BBHc2QebbBw</a></p>
  
  <p>However, I can achieve my goal with a bigger machine-type.</p>