title: href with 2 different links but with same content
body: <p>I wanted to create a small program using tutorials on youtube to scrape download links on a video website.</p>
  <p>but the problem is that the 'href' that hold the download links sometimes hold the same content.
  In fact it's a forum, there's the link to download a video and the same video in smaller size just below. If we click on the video we are redirected to the same page that the link above redirects us.</p>
  <p>Here is the small program:</p>
  <pre><code>import requests
  from bs4 import BeautifulSoup
  
  links =[]
  
  for p in range(21):
      url = 'website url' + str(p)
      response = requests.get(url)
      print(response)
  
      if response.ok:
          print('Page: ' + str(p))
          soup = BeautifulSoup(response.text, &quot;html.parser&quot;)
          iS = soup.findAll('i')
  
          for i in iS:
              a = i.find('a')
              link = a['href']
              links.append(link)
  
  print(len(links))
  
  with open('urls.txt', 'w') as file:
      for link in links:
          file.write(link + '\n')
  
  
  with open('urls.txt', 'r') as file:
      for row in file:
          print(row)
  </code></pre>
  <p><a href="https://i.stack.imgur.com/5GInq.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/5GInq.png" alt="enter image description here" /></a></p>