title: Prepocessing textual data for Machine Learning
body: <pre><code>'''
      import pandas as pd 
      import re
      articles_data = pd.read_csv('C:/Users/amrit/Downloads/data.csv') 
      print(articles_data.apply(lambda x: sum(x.isnull()))) 
      articles_nonNull = articles_data.dropna(subset=['text']) 
      articles_nonNull.reset_index(inplace=True)
  
  
      def clean_text(text):
  
  
  
      #Make text lowercase, remove text in square brackets,remove \n,remove punctuation and 
      remove words containing numbers.
  
      
      text = str(text).lower()
      text = re.sub('&lt;.*?&gt;+', '', text)
      text = re.sub('[%s]' % re.escape(string.punctuation), '', text)
      text = re.sub('\n', '', text)
      text = re.sub('\w*\d\w*', '', text)
      return text
  
      articles_nonNull['text_clean’] = articles_nonNull['text']
                                       .apply(lambda x:clean_text(x))
  
  '''
  </code></pre>
  <p>I am trying to use this code to preprocess my data and keep hitting the &quot;invalid syntax error&quot; on the last two lines from articles_nonNull['text_clean']. Could someone help with this and why this is happening?</p>
  <p>P.S. I am new to NLP and this is the first time I am handling an exceptionally large unstructured dataset.</p>