title: Webscraping with beautifulsoup 'NoneType' object has no attribute 'get_text'
body: <p>I'm trying to learn beautifulsoup to <strong>scrape</strong> the text from NYT politics articles, currently with the code I have right now, it does manage to scrape through two paragraphs, but then after that, it spits out AttributeError: 'NoneType' object has no attribute 'get_text'. I've looked this error up and some threads claim that the error originates from using legacy functions from beautifulsoup3. But that doesn't seem to be the problem here, any ideas?</p>
  <p>Code:</p>
  <pre><code>import requests
  from urllib import request, response, error, parse
  from urllib.request import urlopen
  from bs4 import BeautifulSoup
  
  
  
  
  url = &quot;https://www.nytimes.com/2020/02/10/us/politics/trump-manchin-impeachment.html&quot;
  html = urlopen(url)
  soup = BeautifulSoup(html, &quot;html.parser&quot;)
  
  
  title = soup.title
  titleText = title.get_text()
  
  body = soup.find('article', class_='css-1vxca1d')
  
  section = soup.find('section', class_=&quot;css-1r7ky0e&quot;)
  for elem in section:
      div1 = elem.findAll('div')
      for x in div1:
          div2 = elem.findAll('div')
          for i in div2:
              text = i.find('p').get_text()
              print (text)
              print(&quot;----------&quot;)
  </code></pre>
  <p>output:</p>
  <pre><code>WASHINGTON — Senator Joe Manchin III votes with President Trump more than any other Democrat in the Senate. But his vote last week to convict Mr. Trump of impeachable offenses has eclipsed all of that, earning him the rage of a president who coveted a bipartisan acquittal.
  ----------
  “Munchkin means that you’re small, right?” he said. “I’m bigger than him — of course he has me by weight, now, he has more volume than I have by about 30 or 40 pounds. I’m far from being weak and pathetic, and I’m far from being a munchkin, and I still want him to succeed as president of the United States.”
  ----------
  Traceback (most recent call last):
    File &quot;/Users/user/PycharmProjects/project2/webscrapper.py&quot;, line 25, in &lt;module&gt;
      text = i.find('p').get_text()
  AttributeError: 'NoneType' object has no attribute 'get_text'
  
  Process finished with exit code 1
  
  </code></pre>