original_docstring: 
      Extracts the main content of a webpage. Similar to the
      Reader View in Firefox or Pocket.
      Main HTML content is extracted using python-readability (readability-lxml)
      HTML tags are removed using BeautifulSoup.
      
docstring: Extracts the main content of a webpage. Similar to the
  Reader View in Firefox or Pocket.
  Main HTML content is extracted using python-readability (readability-lxml)
  HTML tags are removed using BeautifulSoup.
docstring_tokens: Extracts the main content of a webpage . Similar to the Reader View in Firefox or Pocket . Main HTML content is extracted using python - readability ( readability - lxml ) HTML tags are removed using BeautifulSoup .
code: def extract_content_from_html(html):
      """
      Extracts the main content of a webpage. Similar to the
      Reader View in Firefox or Pocket.
      Main HTML content is extracted using python-readability (readability-lxml)
      HTML tags are removed using BeautifulSoup.
      """
      doc = Document(html)
      soup = BeautifulSoup(doc.content(), "lxml")
      plaintext = ''.join(soup.findAll(text=True))
      # the original soup comes with too many '\n', this messes up
      # word frequency calculation
      return plaintext.replace("\\n", "")
code_tokens: def extract_content_from_html ( html ) : doc = Document ( html ) soup = BeautifulSoup ( doc . content ( ) , "lxml" ) plaintext = '' . join ( soup . findAll ( text = True ) ) return plaintext . replace ( "\\n" , "" )
short_docstring: Extracts the main content of a webpage.
short_docstring_tokens: Extracts the main content of a webpage .
comment: """
      Extracts the main content of a webpage. Similar to the
      Reader View in Firefox or Pocket.
      Main HTML content is extracted using python-readability (readability-lxml)
      HTML tags are removed using BeautifulSoup.
      """$SEP$# the original soup comes with too many '\n', this messes up$SEP$# word frequency calculation