title: Access Django models with scrapy: defining path to Django project
body: <p>I'm very new to Python and Django. I'm currently exploring using Scrapy to scrape sites and save data to the Django database. My goal is to run a spider based on domain given by a user. </p>
  
  <p>I've written a spider that extracts the data I need and store it correctly in a json file when calling </p>
  
  <pre><code>scrapy crawl spider -o items.json -t json
  </code></pre>
  
  <p>As described in the <a href="http://doc.scrapy.org/en/latest/intro/tutorial.html" rel="noreferrer">scrapy tutorial</a>.</p>
  
  <p>My goal is now to get the spider to succesfully to save data to the Django database, and then work on getting the spider to run based on user input. </p>
  
  <p>I'm aware that various posts exists on this subject, such as these:
  <a href="https://stackoverflow.com/questions/4271975/access-django-models-inside-of-scrapy">link 1</a>
  <a href="https://stackoverflow.com/questions/7883196/saving-django-model-from-scrapy-project">link 2</a>
  <a href="https://stackoverflow.com/questions/4271975/access-django-models-inside-of-scrapy">link 3</a></p>
  
  <p>But having spend more than 8 hours on trying to get this to work, I'm assuming i'm not the only one still facing issues with this. I'll therefor try and gather all the knowledge i've gotten so far in this post, as well a hopefully posting a working solution at a later point. Because of this, this post is rather long. </p>
  
  <p>It appears to me that there is two different solutions to saving data to the Django database from Scrapy. One is to use <a href="https://scrapy.readthedocs.org/en/latest/topics/djangoitem.html" rel="noreferrer">DjangoItem</a>, another is to to import the models directly(as done <a href="https://stackoverflow.com/questions/7883196/saving-django-model-from-scrapy-project">here</a>).</p>
  
  <p>I'm not completely aware of the advantages and disadvantages of these two, but it seems like the difference is simply the using DjangoItem is just more convenient and shorter.</p>
  
  <p><strong>What i've done:</strong> </p>
  
  <p>I've added:</p>
  
  <pre><code>def setup_django_env(path):
      import imp, os
      from django.core.management import setup_environ
  
      f, filename, desc = imp.find_module('settings', [path])
      project = imp.load_module('settings', f, filename, desc)       
  
      setup_environ(project)
  
  setup_django_env('/Users/Anders/DjangoTraining/wsgi/')
  </code></pre>
  
  <p><strong>Error i'm getting is:</strong></p>
  
  <pre><code>ImportError: No module named settings
  </code></pre>
  
  <p>I'm thinking i'm defining the path to my Django project in a wrong way?</p>
  
  <p>I've also tried the following:</p>
  
  <pre><code>setup_django_env('../../') 
  </code></pre>
  
  <p>How do I define the path to my Django project correctly? (if that is the issue)</p>