title: Scrapy: ValueError('Missing scheme in request url: %s' % self._url)
body: <p>I am trying to scrape data from a webpage. The webpage is simply a bullet list of 2500 URLs. Scrapy fetch and goes to each and every URL and fetch some data ... </p>
  
  <p>Here is my code </p>
  
  <pre><code>class MySpider(CrawlSpider):
      name = 'dknews'
      start_urls = ['http://www.example.org/uat-area/scrapy/all-news-listing']
      allowed_domains = ['example.org']
  
      def parse(self, response):
          hxs = Selector(response)
          soup = BeautifulSoup(response.body, 'lxml')
          nf = NewsFields()
          ptype = soup.find_all(attrs={"name":"dkpagetype"})
          ptitle = soup.find_all(attrs={"name":"dkpagetitle"})
          pturl = soup.find_all(attrs={"name":"dkpageurl"})
          ptdate = soup.find_all(attrs={"name":"dkpagedate"})
          ptdesc = soup.find_all(attrs={"name":"dkpagedescription"})
           for node in soup.find_all("div", class_="module_content-panel-sidebar-content"):
             ptbody = ''.join(node.find_all(text=True))  
             ptbody = ' '.join(ptbody.split())
             nf['pagetype'] = ptype[0]['content'].encode('ascii', 'ignore')
             nf['pagetitle'] = ptitle[0]['content'].encode('ascii', 'ignore')
             nf['pageurl'] = pturl[0]['content'].encode('ascii', 'ignore')
             nf['pagedate'] = ptdate[0]['content'].encode('ascii', 'ignore')
             nf['pagedescription'] = ptdesc[0]['content'].encode('ascii', 'ignore')
             nf['bodytext'] = ptbody.encode('ascii', 'ignore')
           yield nf
              for url in hxs.xpath('//ul[@class="scrapy"]/li/a/@href').extract():
               yield Request(url, callback=self.parse)
  </code></pre>
  
  <p>Now the problem is that the above code scrapes around 215 out of 2500 articles. It closes by giving this error ...</p>
  
  <blockquote>
    <p>ValueError('Missing scheme in request url: %s' % self._url)</p>
  </blockquote>
  
  <p>I have no idea what is causing this error ....</p>
  
  <p>Any help is very appreciated.</p>
  
  <p>Thanks</p>