title: UnicodeEncodeError: 'utf-8' codec can't encode character '\ud83d' in position 388: surrogates not allowed
body: <p>When I try to use:</p>
  <pre><code>df[df.columns.difference(['pos', 'neu', 'neg', 'new_description'])].to_csv('sentiment_data.csv')
  </code></pre>
  <p>I get the error:</p>
  <pre><code>UnicodeEncodeError: 'utf-8' codec can't encode character '\ud83d' in position 388: surrogates not allowed
  </code></pre>
  <p>I don't understand what this error means and how I can fix this error and export my data to a csv/excel. I have referred to this <a href="https://stackoverflow.com/questions/27366479/python-3-os-walk-file-paths-unicodeencodeerror-utf-8-codec-cant-encode-s">question</a> but I don't understand much and it doesn't answer how to do this with pandas.</p>
  <p>What does position 388 mean? What is the character '\ud83d'?</p>
  <p>I get a different error position when I try to export to an excel:</p>
  <pre><code>df[df.columns.difference(['pos', 'neu', 'neg', 'new_description'])].to_excel('sentiment_data_new.xlsx')
  </code></pre>
  <p>Error while exporting to excel:</p>
  <pre><code>UnicodeEncodeError: 'utf-8' codec can't encode character '\ud83d' in position 261: surrogates not allowed
  </code></pre>
  <p>Why is the position different when it's the same encoding?</p>
  <blockquote>
  <p>The other duplicate questions don't answer how to escape this error with pandas DataFrame.</p>
  </blockquote>