title: Response 400 while using Requests package in Django
body: <p>I want to <strong>scrape</strong> Myntra using Django Framework but when I put my code in Django project I get 400 error. The same code runs without error when I run it on the same file even in Django. Means calling the MyntraScraperClass in MyntraScraper.py file.</p>
  <p>Here is my project directory</p>
  <pre><code>BackendController
      -MyntraScraper.py
  myDjangoApp
      -views.py
  </code></pre>
  <p>Inside views.py, there is a function where I am calling my MyntraScraperClass</p>
  <pre><code>def tst(request):
      ------------
  </code></pre>
  <p>The same code that is MyntraScraperClass runs error free when I call it on</p>
  <pre><code>BackendController
      -MyntraScraper.py
  </code></pre>
  <p>Here is my code:</p>
  <pre><code>import requests, json
  from bs4 import BeautifulSoup
  import os, ssl
  import certifi
  import urllib3
  
  class MyntraScraperClass:
  
      def __init__(self,url):
          self.url=url
  
  
      def myntra(self):
          mt={}
  
          http = urllib3.PoolManager(
                  cert_reqs='CERT_REQUIRED',
                  ca_certs=certifi.where())
  
          if (not os.environ.get('PYTHONHTTPSVERIFY', '') and getattr(ssl, '_create_unverified_context', None)):
              ssl._create_default_https_context = ssl._create_unverified_context
  
          proxy = {'http': '-------'}
  
          headers = {'User-Agent' : 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.89 Safari/537.36'}
          for x in range(0,5):
              print(x)
              try:
                  s = requests.Session()
                  res = s.get(self.url, headers=headers, verify=True, proxies=proxy)
                  print(res)
                  soup = BeautifulSoup(res.text, &quot;lxml&quot;)
                  print(soup)
                  if &quot;Access Denied&quot; in soup.title:
                      continue
                  break
              except requests.exceptions.ProxyError:
                  continue
              except Exception as e:
                  print(e)
                  mt['error']=e
  
  
          script = None
          for s in soup.find_all(&quot;script&quot;):
              if 'pdpData' in s.text:
                  script = s.get_text(strip=True)
                  break
              else:
                  continue
  
          mt['data'] =json.loads(script[script.index('{'):])
          return mt
  
  </code></pre>
  <p>I am calling MyntraScraperClass in my Django Views from BackendController directory.</p>
  <pre><code>def tst(request):
      url = request.GET.get('url')
      from BackendController.MyntraScraper import  MyntraScraperClass
      
      obj1 = MyntraScraperClass(url)
      kk = obj1.myntra()
      print(kk)
      return JsonResponse(kk)
  </code></pre>
  <p>The response is</p>
  <pre><code>&lt;Response [400]&gt;
  </code></pre>
  <p>I am getting this in soup</p>
  <pre><code>&lt;html&gt;&lt;head&gt;
  &lt;title&gt;Invalid URL&lt;/title&gt;
  &lt;/head&gt;&lt;body&gt;
  &lt;h1&gt;Invalid URL&lt;/h1&gt;
  The requested URL &quot;[no URL]&quot;, is invalid.&lt;p&gt;
  Reference #9.6b722c31.1596891375.7bfc4ae
  &lt;/p&gt;&lt;/body&gt;&lt;/html&gt;
  </code></pre>