title: Scraping websites which has api which is javascript enabled site but can't be scraped through api and give me an error
body: <pre><code>'# -*- coding: utf-8 -*-
  import scrapy
  import json
  
  class NtsschoolSpider(scrapy.Spider):
      name = 'ntsschool'
      start_urls = ['https://directory.ntschools.net/#/schools']
      headers = {        
              &quot;Accept&quot;: &quot;application/json&quot;,
              &quot;Accept-Encoding&quot;: &quot;gzip, deflate, br&quot;,
              &quot;Accept-Language&quot;: &quot;en-US,en;q=0.9,ur;q=0.8&quot;,
              &quot;Referer&quot;: &quot;https://directory.ntschools.net/&quot;,         
              &quot;Sec-Fetch-Mode&quot;: &quot;cors&quot;,
              &quot;Sec-Fetch-Site&quot;: &quot;same-origin&quot;,
              &quot;User-Agent&quot;: &quot;Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/85.0.4183.102 Safari/537.36&quot;,
              &quot;X-Requested-With&quot;: &quot;Fetch&quot;,
                 }
              
      def parse(self, response):
          url = 'https://directory.ntschools.net/api/System/GetAllSchools'
          yield  scrapy.Request(url, 
                                callback = self.parse_api,
                                headers = self.headers)
  
      def  parse_api(self, response):
           base_url = 'https://directory.ntschools.net/api/System/GetSchool?itSchoolCode'
           raw_data = response.body
           data = json.loads(raw_data)                  
           for school in data:
               school_code = school['itSchoolCode']
               school_url = base_url + school_code
               request = scrapy.Request(school_url,
                                        callback = self.parse_url, 
                                        headers = self.headers  )
  
               yield request
  
      def  parse_url(self, response):
           raw_data = response.body
           data = json.loads(raw_data) 
           yield {
                  'Name' : data['name'],
                  'Physical_address': data['physicalAddress']['displayAddress'],
                  'Postal_address': data['postalAddress']['displayAddress'],
                  'Email': data['mail'],
                  'Phone': data['telephoneNumber'] 
  
                   }
           '
  </code></pre>
  <p>The error is:</p>
  <pre class="lang-none prettyprint-override"><code>2020-11-26 12:18:42 [scrapy.spidermiddlewares.httperror] INFO: Ignoring response &lt;404 https://directory.ntschools.net/api/System/GetSchool?itSchoolCodelarapsch&gt;: HTTP status code is not handled or not allowed
  2020-11-26 12:18:42 [scrapy.spidermiddlewares.httperror] INFO: Ignoring response &lt;404 https://directory.ntschools.net/api/System/GetSchool?itSchoolCodelarrasch&gt;: HTTP status code is not handled or not allowed
  2020-11-26 12:18:42 [scrapy.spidermiddlewares.httperror] INFO: Ignoring response &lt;404 https://directory.ntschools.net/api/System/GetSchool?itSchoolCodekathesch&gt;: HTTP status code is not handled or not allowed
  </code></pre>