title: Using custom dependencies in Google Cloud ML Engine
body: <p>I am trying to use a python <a href="https://github.com/lucasb-eyer/pydensecrf" rel="nofollow noreferrer">package</a> that is not listed in PyPI with the Google Cloud ML Engine. This package has itself dependencies that even though are listed in PyPI are not installed by default in the ML engine environment, namely the <a href="http://cython.org" rel="nofollow noreferrer">Cython</a> package.</p>
  
  <p>Looking at the <a href="https://cloud.google.com/ml-engine/docs/how-tos/packaging-trainer" rel="nofollow noreferrer">documentation</a> it is not really clear how to proceed in this case, I have tried packaging this package in a <code>.tar.gz</code> file and passing it under the <code>--packages</code> argument, but I got the following error:</p>
  
  <pre><code>File "&lt;string&gt;", line 1, in &lt;module&gt; IOError: [Errno 2] No such file or directory: '/tmp/pip-jnm3Ml-build/setup.py'
  </code></pre>
  
  <p>After I tried using a <code>setup.py</code> file and packaging my code but google cloud ml engine is not able to find the package in <code>dependency_links</code></p>
  
  <p>Here is my current <code>setup.py</code>:</p>
  
  <pre><code>from setuptools import find_packages, setup
  
  required_packages = ['cython', 'numpy', 'tensorflow', 'scipy', 'cython']
  dependency_links = ['git+https://github.com/lucasb-eyer/pydensecrf.git']
  
  setup(name='trainer',
        version='0.1',
        packages=['trainer'],
        install_requires=required_packages,
        dependency_links=dependency_links,
        include_package_data=True,
        description='description')
  </code></pre>
  
  <p>I would like to avoid doing this by trial and error since sending jobs to the cloud costs money even if they fail immediately.</p>
  
  <p>Thanks in advance.</p>