title: Convert the M and K of an Currency to integers in Pyspark
body: <p>I have a dataset with column of type <code>StringType()</code>. The values in these money columns contains abbreviations like <code>K</code> and <code>M</code>.</p>
  <p>I would like to remove 'K' and 'M' and multiple them either by 1000 or 1000000 for K/M respectively. I tried creating a function and use it to add a new column in the dataframe. I keep getting the following error</p>
  <blockquote>
  <p>ValueError: Cannot convert column into bool: please use '&amp;' for 'and',
  '|' for 'or', '~' for 'not' when building DataFrame boolean
  expressions.</p>
  </blockquote>
  <p>The Column values are as follows:</p>
  <p><a href="https://i.stack.imgur.com/hKaQl.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/hKaQl.png" alt="enter image description here" /></a></p>
  <pre><code>def convertall(ReleaseClause):
    if ReleaseClause == None:
      return 0
    elif expr(&quot;substring(ReleaseClause,-1,length(ReleaseClause))&quot;)=='K':
      remove_euro=expr(&quot;substring(ReleaseClause,2,length(ReleaseClause))&quot;)
      remove_K=translate(remove_euro,'K','')
      remove_Kint=remove_K.cast(IntegerType())*lit(1000)
      return remove_Kint
    elif expr(&quot;substring(ReleaseClause,-1,length(ReleaseClause))&quot;)=='M':
      remove_euro=expr(&quot;substring(ReleaseClause,2,length(ReleaseClause))&quot;)
      remove_M=translate(remove_euro,'M','')
      remove_Mint=remove_M.cast(IntegerType())*lit(1000000)
      return remove_Mint
    else:
      return ReleaseClause
  </code></pre>