title: How to scrape data using pyqt5
body: <p>I would like to get the pop-up data from a <a href="https://www.autohome.com.cn" rel="nofollow noreferrer">website</a>.</p>
  
  <p>As shown in the first figure, I need to click a link.</p>
  
  <p>After that, a pop-up, as shown in the second figure will appear.</p>
  
  <p>The content of this pop-up is what I want.</p>
  
  <p>I tried to follow the example using <code>pyqyt5</code> to get the data.</p>
  
  <p>However, the program continue to run permanently.</p>
  
  <p>How to solve this problem?</p>
  
  <p>Thank you very much.</p>
  
  <pre><code>import sys
  from PyQt5.QtWidgets import QApplication
  from PyQt5.QtCore import QUrl
  from PyQt5.QtWebEngineWidgets import QWebEngineView
  from bs4 import BeautifulSoup
  
  
  class Render(QWebEngineView):
      def __init__(self, url):
          self.html = None
          self.first_pass = True
          self.app = QApplication(sys.argv)
          QWebEngineView.__init__(self)
          self.loadFinished.connect(self._load_finished)
          self.load(QUrl(url))
          self.app.exec_()
  
      def _load_finished(self, result):        
          if self.first_pass:
              self._first_finished()
              self.first_pass = False
          else:
              self._second_finished()
  
      def _first_finished(self):
          self.page().runJavaScript("document.getElementById('auto-header-citypop-citylist');")
  
  
      def _second_finished(self):
          self.page().toHtml(self.callable)
  
      def callable(self, data):
          self.html = data
          self.app.quit()
  
  url = r'https://www.autohome.com.cn'
  web = Render(url)
  
  with open('data2.html', 'w', encoding='utf-8-sig') as f:
      f.write(web.html)
  </code></pre>
  
  <p><a href="https://i.stack.imgur.com/WbPL4.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/WbPL4.png" alt="enter image description here"></a>
  <a href="https://i.stack.imgur.com/9Eu2n.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/9Eu2n.png" alt="enter image description here"></a></p>