title: Python recursive Function returns extra 'None' once complete
body: <p>I am working to write a web-scraper for a school project which catalogs all valid URLs found on the page, and can follow a URL to the next webpage and perform the same action; up to a set number of layers.</p>
  <p>quick code intent:</p>
  <ol>
  <li>function takes a BeautifulSoup type, a url (to indicate where it started), the layer count, and the maximum layer depth</li>
  <li>check page for all href lines</li>
  <li>append to list of 'results' is populated each time an href tag is found containing a valid url (starts with http, https, HTTP, HTTPS; which I know may not be the perfect way to check but for now its what I am working with)</li>
  <li>the layer is incremented by 1 each time a valid URL is found the recursiveLinkSearch() function called again</li>
  <li>when layer count is reached, or no href's remain, return results list</li>
  </ol>
  <p>I am very out of practice with recursion, and am hitting an issue with python adding a 'None' to the list &quot;results&quot; at the end of the recursion.</p>
  <p>This link [https://stackoverflow.com/questions/61691657/python-recursive-function-returns-none] indicates that it may be where I am exiting my function from.  I am also not sure I have recursion operating properly because of the nested for loop.</p>
  <p>Any help or insight on recursion exit strategy is greatly appreciated.</p>
  <pre class="lang-py prettyprint-override"><code>def curlURL(url):
      # beautify with BS
      soup = BeautifulSoup(requests.get(url, timeout=3).text, &quot;html.parser&quot;)
      return soup
  
  
  def recursiveLinkSearch(soup, url, layer, depth):
      results = []
      # for each 'href' found on the page, check if it is a URL
      for a in soup.find_all(href=True):
          try:
              # for every href found, check if contains http or https
              if any(stringStartsWith in a.get('href')[0:4] for stringStartsWith in [&quot;http&quot;, &quot;https&quot;, &quot;HTTP&quot;, &quot;HTTPS&quot;]) \
                      and a.get('href') != url and layer &lt; depth:
  
                  print(f&quot;Found URL: {a.get('href')}&quot;)
                  print(f&quot;LOG: {colors.yellow}Current Layer: {layer}{colors.end}&quot;)
                  results.append(a.get('href'))
                  # BUG: adds an extra &quot;None&quot; type to the end of each list
                  results.append(recursiveLinkSearch(curlURL(a.get('href')), a.get('href'), layer+1, depth))
          # Exceptions Stack
          except requests.exceptions.InvalidSchema:
              print(f&quot;{a.get('href')}&quot;)
              print(f&quot;{colors.bad}Invalid Url Detected{colors.end}&quot;)
          except requests.exceptions.ConnectTimeout:
              print(f&quot;{a.get('href')}&quot;)
              print(f&quot;{colors.bad}Connection Timeout. Passing...&quot;)
          except requests.exceptions.SSLError:
              print(f&quot;{a.get('href')}&quot;)
              print(f&quot;{colors.bad}SSL Certificate Error.  Passing...&quot;)
          except requests.exceptions.ReadTimeout:
              print(f&quot;{a.get('href')}&quot;)
              print(f&quot;{colors.bad}Read Timeout.  Passing...&quot;)
      # exit recursion
      if results != []:
          print(f&quot;LOG: {results[-1]}&quot;)
          return results
  </code></pre>