title: Airflow 1.10 Installation Failing
body: <p>I have a working Airflow environment using Airflow version 1.9 that is running on an Amazon EC2-Instance. I need to upgrade to the latest version of Airflow which is 1.10. I have the option of either upgrading from version 1.9 or installing 1.10 freshly on a new server. Airflow version 1.10 is not listed on Pip so I'm installing it from Git via this command,</p>
  
  <pre><code>pip-3.6 install git+git://github.com/apache/incubator-airflow.git@v1-10-stable
  </code></pre>
  
  <p>This command successfully installs Airflow version 1.10. You can see that by running the command <code>airflow version</code> and viewing the output,</p>
  
  <pre><code>  ____________       _____________
   ____    |__( )_________  __/__  /________      __
  ____  /| |_  /__  ___/_  /_ __  /_  __ \_ | /| / /
  ___  ___ |  / _  /   _  __/ _  / / /_/ /_ |/ |/ /
   _/_/  |_/_/  /_/    /_/    /_/  \____/____/|__/
     v1.10.0
  </code></pre>
  
  <p>When I tried starting up the Airflow scheduler with <code>airflow scheduler</code> I get the following exception,</p>
  
  <pre><code>ModuleNotFoundError: No module named 'MySQLdb'
  [2018-08-14 14:03:16,195] {celery_executor.py:112} ERROR - Error syncing the celery executor, ignoring it:
  [2018-08-14 14:03:16,195] {celery_executor.py:113} ERROR - No module named 'MySQLdb'
  Traceback (most recent call last):
    File "/usr/local/lib/python3.6/site-packages/airflow/executors/celery_executor.py", line 94, in sync
      state = task.state
    File "/usr/local/lib/python3.6/site-packages/celery/result.py", line 471, in state
      return self._get_task_meta()['status']
    File "/usr/local/lib/python3.6/site-packages/celery/result.py", line 410, in _get_task_meta
      return self._maybe_set_cache(self.backend.get_task_meta(self.id))
    File "/usr/local/lib/python3.6/site-packages/celery/backends/base.py", line 365, in get_task_meta
      meta = self._get_task_meta_for(task_id)
    File "/usr/local/lib/python3.6/site-packages/celery/backends/database/__init__.py", line 53, in _inner
      return fun(*args, **kwargs)
    File "/usr/local/lib/python3.6/site-packages/celery/backends/database/__init__.py", line 122, in _get_task_meta_for
      session = self.ResultSession()
    File "/usr/local/lib/python3.6/site-packages/celery/backends/database/__init__.py", line 99, in ResultSession
      **self.engine_options)
    File "/usr/local/lib/python3.6/site-packages/celery/backends/database/session.py", line 59, in session_factory
      engine, session = self.create_session(dburi, **kwargs)
    File "/usr/local/lib/python3.6/site-packages/celery/backends/database/session.py", line 45, in create_session
      engine = self.get_engine(dburi, **kwargs)
    File "/usr/local/lib/python3.6/site-packages/celery/backends/database/session.py", line 42, in get_engine
      return create_engine(dburi, poolclass=NullPool)
    File "/usr/local/lib64/python3.6/site-packages/sqlalchemy/engine/__init__.py", line 391, in create_engine
      return strategy.create(*args, **kwargs)
    File "/usr/local/lib64/python3.6/site-packages/sqlalchemy/engine/strategies.py", line 80, in create
      dbapi = dialect_cls.dbapi(**dbapi_args)
    File "/usr/local/lib64/python3.6/site-packages/sqlalchemy/dialects/mysql/mysqldb.py", line 110, in dbapi
      return __import__('MySQLdb')
  ModuleNotFoundError: No module named 'MySQLdb'
  [2018-08-14 14:03:16,196] {celery_executor.py:112} ERROR - Error syncing the celery executor, ignoring it:
  [2018-08-14 14:03:16,196] {celery_executor.py:113} ERROR - No module named 'MySQLdb'
  Traceback (most recent call last):
    File "/usr/local/lib/python3.6/site-packages/airflow/executors/celery_executor.py", line 94, in sync
      state = task.state
    File "/usr/local/lib/python3.6/site-packages/celery/result.py", line 471, in state
      return self._get_task_meta()['status']
    File "/usr/local/lib/python3.6/site-packages/celery/result.py", line 410, in _get_task_meta
      return self._maybe_set_cache(self.backend.get_task_meta(self.id))
    File "/usr/local/lib/python3.6/site-packages/celery/backends/base.py", line 365, in get_task_meta
      meta = self._get_task_meta_for(task_id)
    File "/usr/local/lib/python3.6/site-packages/celery/backends/database/__init__.py", line 53, in _inner
      return fun(*args, **kwargs)
    File "/usr/local/lib/python3.6/site-packages/celery/backends/database/__init__.py", line 122, in _get_task_meta_for
      session = self.ResultSession()
    File "/usr/local/lib/python3.6/site-packages/celery/backends/database/__init__.py", line 99, in ResultSession
      **self.engine_options)
    File "/usr/local/lib/python3.6/site-packages/celery/backends/database/session.py", line 59, in session_factory
      engine, session = self.create_session(dburi, **kwargs)
    File "/usr/local/lib/python3.6/site-packages/celery/backends/database/session.py", line 45, in create_session
      engine = self.get_engine(dburi, **kwargs)
    File "/usr/local/lib/python3.6/site-packages/celery/backends/database/session.py", line 42, in get_engine
      return create_engine(dburi, poolclass=NullPool)
    File "/usr/local/lib64/python3.6/site-packages/sqlalchemy/engine/__init__.py", line 391, in create_engine
      return strategy.create(*args, **kwargs)
    File "/usr/local/lib64/python3.6/site-packages/sqlalchemy/engine/strategies.py", line 80, in create
      dbapi = dialect_cls.dbapi(**dbapi_args)
    File "/usr/local/lib64/python3.6/site-packages/sqlalchemy/dialects/mysql/mysqldb.py", line 110, in dbapi
      return __import__('MySQLdb')
  ModuleNotFoundError: No module named 'MySQLdb'
  [2018-08-14 14:03:16,197] {celery_executor.py:112} ERROR - Error syncing the celery executor, ignoring it:
  [2018-08-14 14:03:16,197] {celery_executor.py:113} ERROR - No module named 'MySQLdb'
  Traceback (most recent call last):
    File "/usr/local/lib/python3.6/site-packages/airflow/executors/celery_executor.py", line 94, in sync
      state = task.state
    File "/usr/local/lib/python3.6/site-packages/celery/result.py", line 471, in state
      return self._get_task_meta()['status']
    File "/usr/local/lib/python3.6/site-packages/celery/result.py", line 410, in _get_task_meta
      return self._maybe_set_cache(self.backend.get_task_meta(self.id))
    File "/usr/local/lib/python3.6/site-packages/celery/backends/base.py", line 365, in get_task_meta
      meta = self._get_task_meta_for(task_id)
    File "/usr/local/lib/python3.6/site-packages/celery/backends/database/__init__.py", line 53, in _inner
      return fun(*args, **kwargs)
    File "/usr/local/lib/python3.6/site-packages/celery/backends/database/__init__.py", line 122, in _get_task_meta_for
      session = self.ResultSession()
    File "/usr/local/lib/python3.6/site-packages/celery/backends/database/__init__.py", line 99, in ResultSession
      **self.engine_options)
    File "/usr/local/lib/python3.6/site-packages/celery/backends/database/session.py", line 59, in session_factory
      engine, session = self.create_session(dburi, **kwargs)
    File "/usr/local/lib/python3.6/site-packages/celery/backends/database/session.py", line 45, in create_session
      engine = self.get_engine(dburi, **kwargs)
    File "/usr/local/lib/python3.6/site-packages/celery/backends/database/session.py", line 42, in get_engine
      return create_engine(dburi, poolclass=NullPool)
    File "/usr/local/lib64/python3.6/site-packages/sqlalchemy/engine/__init__.py", line 391, in create_engine
      return strategy.create(*args^C[2018-08-14 14:03:16,424] {jobs.py:1585} INFO - Exited execute loop
  [2018-08-14 14:03:16,433] {jobs.py:1599} INFO - Terminating child PID: 13615
  </code></pre>
  
  <p>Here's what my lib folder has,</p>
  
  <pre><code>[/usr/local/lib/python3.6/site-packages]# cd /usr/local/lib64/python3.6/site-packages/sqlalchemy/
  root@ip-1-2-3-4
  [/usr/local/lib64/python3.6/site-packages/sqlalchemy]# ll
  total 320
  drwxr-xr-x  3 root root  4096 Aug 13 17:17 connectors
  -rwxr-xr-x  1 root root 40456 Aug 13 17:17 cprocessors.cpython-36m-x86_64-linux-gnu.so
  -rwxr-xr-x  1 root root 51408 Aug 13 17:17 cresultproxy.cpython-36m-x86_64-linux-gnu.so
  -rwxr-xr-x  1 root root 21944 Aug 13 17:17 cutils.cpython-36m-x86_64-linux-gnu.so
  drwxr-xr-x  3 root root  4096 Aug 13 17:17 databases
  drwxr-xr-x 10 root root  4096 Aug 13 17:17 dialects
  drwxr-xr-x  3 root root  4096 Aug 13 17:17 engine
  drwxr-xr-x  3 root root  4096 Aug 13 17:17 event
  -rwxr-xr-x  1 root root 49746 Mar  6 14:01 events.py
  -rwxr-xr-x  1 root root 12030 Mar  6 14:01 exc.py
  drwxr-xr-x  4 root root  4096 Aug 13 17:17 ext
  -rwxr-xr-x  1 root root  2249 Mar  6 14:01 __init__.py
  -rwxr-xr-x  1 root root  3093 Mar  6 14:01 inspection.py
  -rwxr-xr-x  1 root root 10967 Mar  6 14:01 interfaces.py
  -rwxr-xr-x  1 root root  6712 Mar  6 14:01 log.py
  drwxr-xr-x  3 root root  4096 Aug 13 17:17 orm
  -rwxr-xr-x  1 root root 49883 Mar  6 14:01 pool.py
  -rwxr-xr-x  1 root root  5217 Mar  6 14:01 processors.py
  drwxr-xr-x  2 root root  4096 Aug 13 17:17 __pycache__
  -rwxr-xr-x  1 root root  1200 Mar  6 14:01 schema.py
  drwxr-xr-x  3 root root  4096 Aug 13 17:17 sql
  drwxr-xr-x  5 root root  4096 Aug 13 17:17 testing
  -rwxr-xr-x  1 root root  1713 Mar  6 14:01 types.py
  drwxr-xr-x  3 root root  4096 Aug 13 17:17 util
  root@ip-1-2-3-4
  [/usr/local/lib64/python3.6/site-packages/sqlalchemy]# pwd
  /usr/local/lib64/python3.6/site-packages/sqlalchemy
  root@ip-1-2-3-4
  [/usr/local/lib64/python3.6/site-packages/sqlalchemy]# cd /usr/local/lib/python3.6/site-packages/sqlalchemy/
  bash: cd: /usr/local/lib/python3.6/site-packages/sqlalchemy/: No such file or directory
  </code></pre>
  
  <p>I'm just confused why Airflow's installation didn't take care of all it's needed dependencies. Am I installing Airflow incorrectly? I really need to be on version 1.10 because version 1.9 has a major bug in it as discovered <a href="https://stackoverflow.com/questions/51365911/airflow-logs-brokenpipeexception/51790409#51790409">here</a> and <a href="https://stackoverflow.com/questions/51775370/airflowexception-celery-command-failed-the-recorded-hostname-does-not-match-t">here</a>.</p>