title: How can I clean data frame with html?
body: <p>I have a dataframe which looks like this :</p>
  <pre><code>    text
  0   &quot;&lt;p&gt;ROA1&lt;\/p&gt;&quot;
  1   &quot;&lt;p&gt;Supports de cours&lt;\/p&gt;&quot;
  2   &quot;&lt;p&gt;Textes&lt;\/p&gt;&quot;
  3   &quot;&lt;p&gt;Bonjour \u00e0 tous,&amp;nbsp;&lt;br&gt;&lt;br&gt;J'esp\u0...
  4   &quot;&lt;p style=\&quot;text-align:justify;\&quot;&gt;Chers appren...
  ...
  295 Rejoignez-moi &lt;a href=&quot;/video/liv...
  296 &quot;Pr\u00e9sente&quot;
  297 Rejoignez-moi dans la réunion &lt;a target='_blan...
  298 &quot;&lt;p style=\&quot;text-align:justify;\&quot;&gt;&lt;span style=...
  299 &quot;&lt;p style=\&quot;text-align:justify;\&quot;&gt;&lt;span style=...
  </code></pre>
  <p>I want to clean the text. I did this to remove everything bewtween tags :</p>
  <pre><code>df['text'] = df['text'].map(lambda x : re.sub('&lt;[^&gt;]+&gt;', '', str(x)))
  </code></pre>
  <p>Now I'm trying to use unescape to convert unicode characters to string. For example I want to convert '\u00e0' to 'à'. I don't know how to apply this to the entire data frame column.</p>
  <p><strong>UPDATE</strong></p>
  <p>I found out about html2text. It seems to work well :</p>
  <pre><code>def html_to_string(text):
    text = html2text.html2text(text)
  
    return text
  </code></pre>
  <p>The output looks like this :</p>
  <pre><code>Bonjour à tous,  
    
  J'espère que vous avez bien profité de la pause inter-semestres.  
  Nous voici ensemble pour traverser le 2e Semestre.  
  </code></pre>
  <p>This is exactly what I want so I tried to apply this to the dataframe.</p>
  <pre><code>df['text'] = df['text'].apply(html_to_string)
  </code></pre>
  <p>But it doesn't work on the dataframe, maybe I don't apply it correctly, here's the output :</p>
  <pre><code>    text
  0   &quot;\n\nROA1&lt;\/p&gt;&quot;\n\n
  1   &quot;\n\nSupports de cours&lt;\/p&gt;&quot;\n\n
  2   &quot;\n\nTextes&lt;\/p&gt;&quot;\n\n
  3   &quot;\n\nBonjour \u00e0 tous, \n \nJ'esp\u00e8re...
  4   &quot;\n\nChers apprenants,&lt;\/p&gt;\n\n \nVous trouve...
  ... ...
  295 Rejoignez-moi dans la Room [MISSIONS FANNY](/v...
  296 &quot;Pr\u00e9sente&quot;\n\n
  297 Rejoignez-moi dans la réunion [Marché des gemm...
  298 &quot;\n\nBonjour,&lt;\/span&gt; \n \n \nMerci d'\u00e...
  299 &quot;\n\nBonjour,&lt;\/span&gt; \n \n \nMerci d'\u00e..
  </code></pre>