title: How to stop .net Xml Serialisation inserting illegal characters
body: <p>Anything below 0x20 (except for 0x09, 0x0a, 0x0d i.e. tab, carrige return and line feed) cannot be included in an XML document.</p>
  
  <p>I have some data coming out of a Database and being passed as a response to a Web Service request.</p>
  
  <p>The Soap formatter happily encodes 0x12 character (Ascii 18, Device Control 2) as <code>&amp;#12;</code> but the response fails on the client with <strong>hexadecimal value 0x12, is an invalid character</strong></p>
  
  <p><code>&lt;rant&gt;</code> What I find <em>quite</em> frustrating is these are two sides of the same coin, both client and service are .net apps. Why will the soap formatter write bad xml if nothing can read it?<code>&lt;/rant&gt;</code></p>
  
  <p>I'd like to either </p>
  
  <ol>
  <li>Get the Xml Serialiser to handle these odd characters correctly or</li>
  <li>Have the request fail in the Web Service</li>
  </ol>
  
  <p>I've googled and couldn't find much on this other than, a) "sanitise your Inputs" or b) "change your document structure". </p>
  
  <p>a) Isn't a runner as some of this data is +20 years old<br>
  b) isn't much of an option either, as other than our own front end, we have clients that code against the Web Service directly.</p>
  
  <p>Is there something obvious I'm missing? Or is it simply a case of code around AscII control codes?</p>
  
  <p>Thanks</p>
  
  <p><strong>Update</strong><br>
  This is actually a problem with the XmlSerialiser, the following code will serialise an invalid character to the stream, but will not de-serialise it</p>
  
  <pre><code>[Serializable]
  public class MyData 
  {
      public string Text { get; set; }
  
  }
  class Program
  {
      public static void Main(string[] args)
      {
          var myData = new MyData {Text = "hello " 
                  + ASCIIEncoding.ASCII.GetString(new byte[] { 0x12 }) 
                  + " world"};
  
          var serializer = new XmlSerializer(typeof(MyData));
  
          var xmlWriter = new StringWriter();
  
          serializer.Serialize(xmlWriter, myData);
  
          var xmlReader = new StringReader(xmlWriter.ToString());
  
          var newData = (MyData)serializer.Deserialize(xmlReader); // Exception 
          // hexadecimal value 0x12, is an invalid character.
  
      }
  }
  </code></pre>
  
  <p>I can get it to choke <strong>writing</strong> the xml by explicitly creating an XmlWriter and passing that to <code>Serialise</code> (I'll post that shortly as my own answer), but that still means I've to sanatize my data before sending it.<br>
  As these characters are significant I can't just strip them, I need to encode them before transmission and decode them when read, and I'm <em>really</em> quite surprised that there doesn't appear to be an existing framework method to do this.</p>