title: Ultra-efficient UTF8 byte encoding in C#
body: <p>I'm trying to achieve maximum performance in a library that needs to convert C# strings to byte[] before sending them off somewhere.</p>
  
  <p>Since a char in UTF8 takes maximum 4 bytes, my current approach is to preallocate a large byte[]. When a string arrives for encoding, I can use <a href="http://msdn.microsoft.com/en-us/library/system.text.encoder(v=vs.110).aspx" rel="nofollow">System.Text.Encoder</a> to populate the byte array with it. If the string's char length * 4 is bigger than my buffer, I allocate a new one (optimizing via a buffer pool), but this is supposed to become relatively rare quite quickly.</p>
  
  <p>My only issue with this solution, is that System.Text.Encoder doesn't appear to accept a string - only a char[]. Retrieving a char[] from the string involves another seemingly needless copy. There's an unsafe version with char pointers, but I'm prohibited from using that in my library at the moment.</p>
  
  <p>As a side note, StringWriter, which also does UTF8 conversion, maintains an internal buffer. This is again unsuitable - I need my own buffer since I encode other data types as well (e.g. ints).</p>
  
  <p>So does anyone have any idea where Encoder doesn't provide a method that works directly on String?</p>