title: Removing non-ascii and special character in pyspark dataframe column
body: <p>I am reading data from csv files which has about 50 columns, few of the columns(4 to 5) contain text data with non-ASCII characters and special characters.</p>
  
  <pre><code>df = spark.read.csv(path, header=True, schema=availSchema)
  </code></pre>
  
  <p>I am trying to remove all the non-Ascii and special characters and keep only English characters, and I tried to do it as below</p>
  
  <pre><code>df = df['textcolumn'].str.encode('ascii', 'ignore').str.decode('ascii')
  </code></pre>
  
  <p>There are no spaces in my column name. I receive an error </p>
  
  <pre><code>TypeError: 'Column' object is not callable
  ---------------------------------------------------------------------------
  TypeError                                 Traceback (most recent call last)
  &lt;command-1486957561378215&gt; in &lt;module&gt;
  ----&gt; 1 InvFilteredDF = InvFilteredDF['SearchResultDescription'].str.encode('ascii', 'ignore').str.decode('ascii')
  
  TypeError: 'Column' object is not callable
  </code></pre>
  
  <p>Is there an alternative to accomplish this, appreciate any help with this.</p>