title: How to read uploaded CSV UTF-8 for processing with CsvHelper?
body: <p>My WebAPI allows a user to upload a CSV file and then parses the file.  I use CsvHelper to do the heavy lifting of reading the CSV and mapping it to domain objects.</p>
  
  <p>However, I have one customer who's files are in CSV UTF-8 format.  The code that works for "vanilla" (ASCII) CSV files hurls when it tries to deal with CSV UTF-8.  </p>
  
  <p>Is there a way to import the CSV UTF-8 data and convert it to ASCII CSV so that my code will continue to work?</p>
  
  <p>My current code looks like this:</p>
  
  <pre><code>//In my WebAPI Controller
  //fileToProcess is IFormFile
  byte[] fileBytes = new byte[fileToProcess.Length];
  using(var stream = fileToProcess.OpenReadStream())
  {
      await stream.ReadAsync(fileBytes);
      stream.Close();
  }
  
  var result = await ProcessFileAsync(fileBytes);
  
  return OK(result);
  ...
  
  //In a Parsing Class
  public async Task&lt;List&lt;Client&gt;&gt; ProcessFileAsync(byte[] fileBytes)
  {
      List&lt;Client&gt; result = null;
      var fileText = Encoding.Default.GetString(fileBytes);
      using(var reader = new StringReader(fileText))
      {
         using(var csv = new CsvReader(reader))
         {
            csv.RegisterClassMap&lt;ClientMap&gt;();
            result = csv.GetRecords&lt;T&gt;().ToList();
            await PostProcess(result);
         }
      }
  
      return result;
   }
  </code></pre>
  
  <p>The problem is that CSV UTF-8 has the BOM so when CsvHelper tries to process a mapping that references the first column header</p>
  
  <pre><code>Map(c =&gt; c.ClientId).Name("CLIENT ID");
  </code></pre>
  
  <p>it fails because the column name includes the BOM.</p>
  
  <p>So, my questions are:</p>
  
  <ol>
  <li>How can I tell if the file coming in is UTF-8 or ASCII.</li>
  <li>How do I convert the UTF-8 to ASCII so it can be processed normally?</li>
  </ol>
  
  <p><strong>NOTE</strong></p>
  
  <p>I did try the following:</p>
  
  <pre><code>fileBytes = Encoding.Convert(Encoding.UTF8, Encoding.ASCII, fileBytes);
  </code></pre>
  
  <p>However, this replaced the BOM with a <code>?</code> which still causes CsvHelper to fail.</p>