title: Scrapy setup ubuntu 16.04 or any other
body: <p>My today installed fresh Ubuntu 16.04 was preinstalled with python:</p>
  
  <pre><code>p@Scrapy:~$ python --version
  Python 2.7.11+
  p@Scrapy:~$ python3 --version
  Python 3.5.1+
  </code></pre>
  
  <p>As mentioned on manual page <a href="http://doc.scrapy.org/en/latest/intro/install.html" rel="nofollow noreferrer">http://doc.scrapy.org/en/latest/intro/install.html</a> I open this link <a href="http://doc.scrapy.org/en/latest/topics/ubuntu.html#topics-ubuntu" rel="nofollow noreferrer">http://doc.scrapy.org/en/latest/topics/ubuntu.html#topics-ubuntu</a> and tried to follow steps described to install Scrapy. </p>
  
  <p>But I get an error after step 3</p>
  
  <pre><code>sudo apt-get update &amp;&amp; sudo apt-get install scrapy
  </code></pre>
  
  <p>...</p>
  
  <pre><code>The following packages have unmet dependencies:
   scrapy : Depends: python-support (&gt;= 0.90.0) but it is not installable
            Recommends: python-setuptools but it is not going to be installed
  E: Unable to correct problems, you have held broken packages.
  </code></pre>
  
  <p>I post a problem yesterday about error with Scrapy and looks like it was setup issue too but on Windows. <a href="https://stackoverflow.com/questions/37818606/cannot-setup-scrapy-on-windows/37820028#37820028">Cannot setup Scrapy on windows</a> So I tried today with Ubuntu and once more without luck.</p>
  
  <p>So question how to setup Scrapy on Ubuntu 16.04 or may be other version? Looks like Scrapy manuals outdated. I would think Scrapy project died, but I know that people still use Scrapy. May be Scrapy only works with python 2.+? So I will stay with Windows. Cannot check all variants. It takes too much time. Can anyone mention stable config (operating system + python version) working with Scrapy?</p>
  
  <p>Thanks.</p>
  
  <p><strong>Update</strong></p>
  
  <p>Here I tried with Docker. I create Dockerfile, other steps from terminal:</p>
  
  <pre><code>p@ScrapyPython3:~$ cat Dockerfile
  $ cat Dockerfile
  FROM ubuntu:xenial
  
  ENV DEBIAN_FRONTEND noninteractive
  
  RUN apt-get update
  
  # Install Python3 and dev headers
  RUN apt-get install -y \
      python3 \
      python-dev \
      python3-dev
  
  # Install cryptography
  RUN apt-get install -y \
      build-essential \
      libssl-dev \
      libffi-dev
  
  # install lxml
  RUN apt-get install -y \
      libxml2-dev \
      libxslt-dev
  
  # install pip
  RUN apt-get install -y python-pip
  
  RUN useradd --create-home --shell /bin/bash scrapyuser
  
  USER scrapyuser
  WORKDIR /home/scrapyuser
  p@ScrapyPython3:~$ sudo docker build -t redapple/scrapy-ubuntu-xenial .
  Sending build context to Docker daemon 81.21 MB
  Step 1 : $ 
  Unknown instruction: $
  p@ScrapyPython3:~$ sudo docker run -t -i redapple/scrapy-ubuntu-xenial
  Unable to find image 'redapple/scrapy-ubuntu-xenial:latest' locally
  Pulling repository docker.io/redapple/scrapy-ubuntu-xenial
  docker: Error: image redapple/scrapy-ubuntu-xenial not found.
  See 'docker run --help'.
  p@ScrapyPython3:~$ pip install scrapy
  Requirement already satisfied (use --upgrade to upgrade): scrapy in ./.local/lib/python2.7/site-packages
  Requirement already satisfied (use --upgrade to upgrade): queuelib in ./.local/lib/python2.7/site-packages (from scrapy)
  Requirement already satisfied (use --upgrade to upgrade): pyOpenSSL in ./.local/lib/python2.7/site-packages (from scrapy)
  Requirement already satisfied (use --upgrade to upgrade): Twisted&gt;=10.0.0 in ./.local/lib/python2.7/site-packages (from scrapy)
  Requirement already satisfied (use --upgrade to upgrade): six&gt;=1.5.2 in ./.local/lib/python2.7/site-packages (from scrapy)
  Requirement already satisfied (use --upgrade to upgrade): w3lib&gt;=1.14.2 in ./.local/lib/python2.7/site-packages (from scrapy)
  Requirement already satisfied (use --upgrade to upgrade): service-identity in ./.local/lib/python2.7/site-packages (from scrapy)
  Requirement already satisfied (use --upgrade to upgrade): cssselect&gt;=0.9 in ./.local/lib/python2.7/site-packages (from scrapy)
  Requirement already satisfied (use --upgrade to upgrade): lxml in ./.local/lib/python2.7/site-packages (from scrapy)
  Requirement already satisfied (use --upgrade to upgrade): parsel&gt;=0.9.3 in ./.local/lib/python2.7/site-packages (from scrapy)
  Requirement already satisfied (use --upgrade to upgrade): PyDispatcher&gt;=2.0.5 in ./.local/lib/python2.7/site-packages (from scrapy)
  Requirement already satisfied (use --upgrade to upgrade): cryptography&gt;=1.3 in ./.local/lib/python2.7/site-packages (from pyOpenSSL-&gt;scrapy)
  Requirement already satisfied (use --upgrade to upgrade): zope.interface&gt;=3.6.0 in ./.local/lib/python2.7/site-packages (from Twisted&gt;=10.0.0-&gt;scrapy)
  Requirement already satisfied (use --upgrade to upgrade): pyasn1-modules in ./.local/lib/python2.7/site-packages (from service-identity-&gt;scrapy)
  Requirement already satisfied (use --upgrade to upgrade): pyasn1 in ./.local/lib/python2.7/site-packages (from service-identity-&gt;scrapy)
  Requirement already satisfied (use --upgrade to upgrade): attrs in ./.local/lib/python2.7/site-packages (from service-identity-&gt;scrapy)
  Requirement already satisfied (use --upgrade to upgrade): setuptools&gt;=11.3 in ./.local/lib/python2.7/site-packages (from cryptography&gt;=1.3-&gt;pyOpenSSL-&gt;scrapy)
  Requirement already satisfied (use --upgrade to upgrade): ipaddress in ./.local/lib/python2.7/site-packages (from cryptography&gt;=1.3-&gt;pyOpenSSL-&gt;scrapy)
  Requirement already satisfied (use --upgrade to upgrade): enum34 in ./.local/lib/python2.7/site-packages (from cryptography&gt;=1.3-&gt;pyOpenSSL-&gt;scrapy)
  Requirement already satisfied (use --upgrade to upgrade): idna&gt;=2.0 in ./.local/lib/python2.7/site-packages (from cryptography&gt;=1.3-&gt;pyOpenSSL-&gt;scrapy)
  Requirement already satisfied (use --upgrade to upgrade): cffi&gt;=1.4.1 in ./.local/lib/python2.7/site-packages (from cryptography&gt;=1.3-&gt;pyOpenSSL-&gt;scrapy)
  Requirement already satisfied (use --upgrade to upgrade): pycparser in ./.local/lib/python2.7/site-packages (from cffi&gt;=1.4.1-&gt;cryptography&gt;=1.3-&gt;pyOpenSSL-&gt;scrapy)
  p@ScrapyPython3:~$ scrapy version
  The program 'scrapy' is currently not installed. You can install it by typing:
  sudo apt install python-scrapy
  </code></pre>
  
  <p><strong>UPDATE 1</strong>
  Looks like first row in Dockerfile should not be there. If i remove it ($ cat Dockerfile)</p>
  
  <p>I can get docker image up and running, but once more no luck with pip install scrapy. More over I realize I have to do some overhead with docker to be able upload scrapy files there (I prefer GUI when coding). It there any tool installed in this Docker image? Here install log:</p>
  
  <pre><code>p@ScrapyPython3:~$ sudo docker run -t -i redapple/scrapy-ubuntu-xenial
      scrapyuser@41bef38de45d:~$ python --version   
      Python 2.7.11+
      scrapyuser@41bef38de45d:~$ python3 --version
      Python 3.5.1+
      scrapyuser@41bef38de45d:~$ pip install scrapy
      Collecting scrapy
        Downloading Scrapy-1.1.0-py2.py3-none-any.whl (294kB)
          100% |################################| 296kB 245kB/s 
      Collecting queuelib (from scrapy)
        Downloading queuelib-1.4.2-py2.py3-none-any.whl
      Collecting pyOpenSSL (from scrapy)
        Downloading pyOpenSSL-16.0.0-py2.py3-none-any.whl (45kB)
          100% |################################| 51kB 12.7MB/s 
      Collecting Twisted&gt;=10.0.0 (from scrapy)
        Downloading Twisted-16.2.0.tar.bz2 (2.9MB)
          100% |################################| 2.9MB 472kB/s 
      Collecting six&gt;=1.5.2 (from scrapy)
        Downloading six-1.10.0-py2.py3-none-any.whl
      Collecting w3lib&gt;=1.14.2 (from scrapy)
        Downloading w3lib-1.14.2-py2.py3-none-any.whl
      Collecting service-identity (from scrapy)
        Downloading service_identity-16.0.0-py2.py3-none-any.whl
      Collecting cssselect&gt;=0.9 (from scrapy)
        Downloading cssselect-0.9.2-py2.py3-none-any.whl
      Collecting lxml (from scrapy)
        Downloading lxml-3.6.0.tar.gz (3.7MB)
          100% |################################| 3.7MB 389kB/s 
      Collecting parsel&gt;=0.9.3 (from scrapy)
        Downloading parsel-1.0.2-py2.py3-none-any.whl
      Collecting PyDispatcher&gt;=2.0.5 (from scrapy)
        Downloading PyDispatcher-2.0.5.tar.gz
      Collecting cryptography&gt;=1.3 (from pyOpenSSL-&gt;scrapy)
        Downloading cryptography-1.4.tar.gz (399kB)
          100% |################################| 409kB 1.4MB/s 
      Collecting zope.interface&gt;=3.6.0 (from Twisted&gt;=10.0.0-&gt;scrapy)
        Downloading zope.interface-4.2.0.tar.gz (146kB)
          100% |################################| 153kB 1.2MB/s 
      Collecting pyasn1-modules (from service-identity-&gt;scrapy)
        Downloading pyasn1_modules-0.0.8-py2.py3-none-any.whl
      Collecting pyasn1 (from service-identity-&gt;scrapy)
        Downloading pyasn1-0.1.9-py2.py3-none-any.whl
      Collecting attrs (from service-identity-&gt;scrapy)
        Downloading attrs-16.0.0-py2.py3-none-any.whl
      Collecting idna&gt;=2.0 (from cryptography&gt;=1.3-&gt;pyOpenSSL-&gt;scrapy)
        Downloading idna-2.1-py2.py3-none-any.whl (54kB)
          100% |################################| 61kB 10.8MB/s 
      Collecting setuptools&gt;=11.3 (from cryptography&gt;=1.3-&gt;pyOpenSSL-&gt;scrapy)
        Downloading setuptools-23.0.0-py2.py3-none-any.whl (435kB)
          100% |################################| 440kB 1.2MB/s 
      Collecting enum34 (from cryptography&gt;=1.3-&gt;pyOpenSSL-&gt;scrapy)
        Downloading enum34-1.1.6-py2-none-any.whl
      Collecting ipaddress (from cryptography&gt;=1.3-&gt;pyOpenSSL-&gt;scrapy)
        Downloading ipaddress-1.0.16-py27-none-any.whl
      Collecting cffi&gt;=1.4.1 (from cryptography&gt;=1.3-&gt;pyOpenSSL-&gt;scrapy)
        Downloading cffi-1.6.0.tar.gz (397kB)
          100% |################################| 399kB 1.3MB/s 
      Collecting pycparser (from cffi&gt;=1.4.1-&gt;cryptography&gt;=1.3-&gt;pyOpenSSL-&gt;scrapy)
        Downloading pycparser-2.14.tar.gz (223kB)
          100% |################################| 225kB 1.1MB/s 
      Building wheels for collected packages: Twisted, lxml, PyDispatcher, cryptography, zope.interface, cffi, pycparser
        Running setup.py bdist_wheel for Twisted ... done
        Stored in directory: /home/scrapyuser/.cache/pip/wheels/fe/9d/3f/9f7b1c768889796c01929abb7cdfa2a9cdd32bae64eb7aa239
        Running setup.py bdist_wheel for lxml ... done
        Stored in directory: /home/scrapyuser/.cache/pip/wheels/6c/eb/a1/e4ff54c99630e3cc6ec659287c4fd88345cd78199923544412
        Running setup.py bdist_wheel for PyDispatcher ... done
        Stored in directory: /home/scrapyuser/.cache/pip/wheels/86/02/a1/5857c77600a28813aaf0f66d4e4568f50c9f133277a4122411
        Running setup.py bdist_wheel for cryptography ... done
        Stored in directory: /home/scrapyuser/.cache/pip/wheels/f6/6c/21/11ec069285a52d7fa8c735be5fc2edfb8b24012c0f78f93d20
        Running setup.py bdist_wheel for zope.interface ... done
        Stored in directory: /home/scrapyuser/.cache/pip/wheels/20/a2/bc/74fe87cee17134f5219ba01fe82dd8c10998377e0fb910bb22
        Running setup.py bdist_wheel for cffi ... done
        Stored in directory: /home/scrapyuser/.cache/pip/wheels/8f/00/29/553c1b1db38bbeec3fec428ae4e400cd8349ecd99fe86edea1
        Running setup.py bdist_wheel for pycparser ... done
        Stored in directory: /home/scrapyuser/.cache/pip/wheels/9b/f4/2e/d03e949a551719a1ffcb659f2c63d8444f4df12e994ce52112
      Successfully built Twisted lxml PyDispatcher cryptography zope.interface cffi pycparser
      Installing collected packages: queuelib, idna, pyasn1, six, setuptools, enum34, ipaddress, pycparser, cffi, cryptography, pyOpenSSL, zope.interface, Twisted, w3lib, pyasn1-modules, attrs, service-identity, cssselect, lxml, parsel, PyDispatcher, scrapy
      Successfully installed PyDispatcher Twisted attrs cffi cryptography cssselect enum34 idna ipaddress lxml parsel pyOpenSSL pyasn1 pyasn1-modules pycparser queuelib scrapy service-identity setuptools-20.7.0 six w3lib zope.interface
      You are using pip version 8.1.1, however version 8.1.2 is available.
      You should consider upgrading via the 'pip install --upgrade pip' command.
      scrapyuser@41bef38de45d:~$ scrapy version
      bash: scrapy: command not found
  </code></pre>