title: Exchanging data between Python and React in real time with Electron JS
body: <p>I have to implement a web scraping tool and I choose to do with electron js with react and python.</p>
  <p>I could able to <a href="https://github.com/sahitpj/electron-react-python" rel="nofollow noreferrer">integrate</a> python and react using <a href="https://github.com/extrabacon/python-shell" rel="nofollow noreferrer">python shell</a> in electron as follow,</p>
  <blockquote>
  <p>React Code</p>
  </blockquote>
  <pre><code>import React from 'react';
  var path = require(&quot;path&quot;)
  
  const {PythonShell} = require(&quot;python-shell&quot;);
  const city = 'XYZ';
    var options = {
      scriptPath : path.join(__dirname, '../../python/'),
      args : [city]
    }
  
  class App extends React.Component {
  constructor(props) {
    super(props);
  }
  
  
  componentDidMount() {
    var shell = new PythonShell('main.py', options); //executes python script on python3
  
    shell.on('message', function(message) {
      console.log('message', message)
    })
  }
  
  
  render (){
    return (
     &lt;div className=&quot;header&quot;&gt;
          &lt;h1&gt;Hello, World, {this.state.test}&lt;/h1&gt;
     &lt;/div&gt;
    )
   }
  }
  
  export default App;
  </code></pre>
  <blockquote>
  <p>Python Code</p>
  </blockquote>
  <pre><code>import sys
  import requests
  from bs4 import BeautifulSoup
  from queue import Queue, Empty
  from concurrent.futures import ThreadPoolExecutor
  from urllib.parse import urljoin, urlparse
  
  city = sys.argv[1]
  
  class MultiThreadScraper:
  
  def __init__(self, base_url):
  
      self.base_url = base_url
      self.root_url = '{}://{}'.format(urlparse(self.base_url).scheme, urlparse(self.base_url).netloc)
      self.pool = ThreadPoolExecutor(max_workers=5)
      self.scraped_pages = set([])
      self.to_crawl = Queue()
      self.to_crawl.put(self.base_url)
  
  def parse_links(self, html):
      soup = BeautifulSoup(html, 'html.parser')
      links = soup.find_all('a', href=True)
      for link in links:
          url = link['href']
          if url.startswith('/') or url.startswith(self.root_url):
              url = urljoin(self.root_url, url)
              if url not in self.scraped_pages:
                  self.to_crawl.put(url)
  
  def scrape_info(self, html):
      return
  
  def post_scrape_callback(self, res):
      result = res.result()
      if result and result.status_code == 200:
          self.parse_links(result.text)
          self.scrape_info(result.text)
  
  def scrape_page(self, url):
      try:
          res = requests.get(url, timeout=(3, 30))
          return res
      except requests.RequestException:
          return
  
  def run_scraper(self):
      while True:
          try:
              target_url = self.to_crawl.get(timeout=60)
              if target_url not in self.scraped_pages:
                  print(&quot;Scraping URL: {}&quot;.format(target_url))
                  self.scraped_pages.add(target_url)
                  job = self.pool.submit(self.scrape_page, target_url)
                  job.add_done_callback(self.post_scrape_callback)
          except Empty:
              return
          except Exception as e:
              print(e)
              continue
  if __name__ == '__main__':
     s = MultiThreadScraper(&quot;http://websosite.com&quot;)
     s.run_scraper()
  </code></pre>
  <p>I could get all scraping URLs after executing the python shell in React but I want all URLs in real time in React front end.</p>
  <p>Following React code execute the python code and give the final result</p>
  <pre><code>var shell = new PythonShell('main.py', options); //executes python script on python3
  </code></pre>
  <p>This React code is used to receives a message from the python script with a simple 'print' statement.</p>
  <pre><code>pyshell.on('message', function (message) {
   console.log(message);
  </code></pre>
  <p>});</p>
  <p>Is there any way to get the result in real time while executing the python code?</p>