title: Why are links with headers not filtered out in Scrapy?
body: <p>I was under the impression that scrapy automatically filters links with headers or anchors (i.e. example.com vs example.com#overview). However, using the code below I still see, for example:</p>
  <pre><code>https://www.warnermediaprivacy.com/opt-out/
  https://www.warnermediaprivacy.com/opt-out/#third-party-app
  False
  </code></pre>
  <pre><code>from scrapy.linkextractors import LinkExtractor
  
  def extract_links(self, response: Response) -&gt; List[Link]:
      return sorted(
          self.link_extractor.extract_links(response), key=lambda link: link.url
      )
  
  
  self.link_extractor = LinkExtractor()
  for link in self.extract_links(response):
      if response.url == link.url.split('#')[0]:
          print(response.url)
          print(link.url)
          print(link.nofollow)
  </code></pre>