title: Another PHP XML parsing error: "Input is not proper UTF-8, indicate encoding!"
body: <p>Error:</p>
  
  <blockquote>
    <p>Warning: simplexml_load_string()
    [function.simplexml-load-string]:
    Entity: line 3: parser error : Input
    is not proper UTF-8, indicate encoding
    ! Bytes: 0xE7 0x61 0x69 0x73</p>
  </blockquote>
  
  <p>XML from database (output from view source in FF):</p>
  
  <pre><code>&lt;?xml version="1.0" encoding="UTF-8" ?&gt;&lt;audit&gt;&lt;audit_detail&gt;
      &lt;fieldname&gt;role_fra&lt;/fieldname&gt;
      &lt;old_value&gt;Role en fran&amp;#xe7;ais&lt;/old_value&gt;
      &lt;new_value&gt;Role &amp;#xe7; en fran&amp;#xe7;ais&lt;/new_value&gt;
  &lt;/audit_detail&gt;&lt;/audit&gt;&lt;/xml&gt;
  </code></pre>
  
  <p>If I understand correctly, the error is related to the first ç encoded in the old_value tag. To be precise, the error is related to this based on the bytes: "çais" ?</p>
  
  <p>Here's how I load the XML:</p>
  
  <pre><code>$xmlData = simplexml_load_string($ed['updates'][$i]['audit_data']);
  </code></pre>
  
  <p>The I loop through using this:</p>
  
  <pre><code>foreach ($xmlData-&gt;audit_detail as $a){
  //code here
  }
  </code></pre>
  
  <p>The field in the database is of data type text and is set utf8_general_ci.</p>
  
  <p>My function to create the audit_detail stubs:</p>
  
  <pre><code>function ed_audit_node($field, $new, $old){
  
  
      $old = htmlentities($old, ENT_QUOTES, "UTF-8");
      $new = htmlentities($new, ENT_QUOTES, "UTF-8");
  
      $out = &lt;&lt;&lt;EOF
          &lt;audit_detail&gt;
              &lt;fieldname&gt;{$field}&lt;/fieldname&gt;
              &lt;old_value&gt;{$old}&lt;/old_value&gt;
              &lt;new_value&gt;{$new}&lt;/new_value&gt;
          &lt;/audit_detail&gt;
  EOF;
      return $out;
  }
  </code></pre>
  
  <p>The insert in the database is done like this:</p>
  
  <pre><code>function ed_audit_insert($ed, $xml){
      global $visitor;
  
      $sql = &lt;&lt;&lt;EOF
      INSERT INTO ed.audit
      (employee_id, audit_date, audit_action, audit_data, user_id) 
      VALUES (
          {$ed[emp][employee_id]}, 
          now(), 
          '{$ed[audit_action]}', 
          '{$xml}', 
          {$visitor[user_id]}
      );      
  EOF;
      $req = mysql_query($sql,$ed['db']) or die(db_query_error($sql,mysql_error(),__FUNCTION__));
  
  }
  </code></pre>
  
  <p>The weirdest part is that the following works (without the xml declaration though) in a simple PHP file:</p>
  
  <pre><code>$testxml = &lt;&lt;&lt;EOF
  &lt;audit&gt;&lt;audit_detail&gt;
          &lt;fieldname&gt;role_fra&lt;/fieldname&gt;
          &lt;old_value&gt;Role en fran&amp;#xe7;ais&lt;/old_value&gt;
          &lt;new_value&gt;Role &amp;#xe7; en fran&amp;#xe7;ais&lt;/new_value&gt;
      &lt;/audit_detail&gt;&lt;/audit&gt;
  EOF;
  </code></pre>
  
  <p>$xmlData = simplexml_load_string($testxml);</p>
  
  <p>Can someone help shed some light on this?</p>
  
  <p><strong>Edit #1</strong> - I'm now using DOM to build the XML document and have gotten rid of the error. Function here:</p>
  
  <pre><code>$dom = new DomDocument();
  $root = $dom-&gt;appendChild($dom-&gt;createElement('audit'));
  $xmlCount = 0;
  
  if($role_fra != $curr['role']['role_fra']){
     $root-&gt;appendChild(ed_audit_node($dom, 'role_fra', $role_fra, $curr['role']['role_fra'])); 
     $xmlCount++;
  }
  
  ...
  
  function ed_audit_node($dom, $field, $new, $old){
  
      //create audit_detail node
      $ad = $dom-&gt;createElement('audit_detail');
  
      $fn = $dom-&gt;createElement('fieldname');
      $fn-&gt;appendChild($dom-&gt;createTextNode($field));
      $ad-&gt;appendChild($fn);
  
      $ov = $dom-&gt;createElement('old_value');
      $ov-&gt;appendChild($dom-&gt;createTextNode($old));
      $ad-&gt;appendChild($ov);
  
      $nv = $dom-&gt;createElement('new_value');
      $nv-&gt;appendChild($dom-&gt;createTextNode($new));
      $ad-&gt;appendChild($nv);
  
      //append to document
      return $ad;
  }
  
  if($xmlCount != 0){
      ed_audit_insert($ed,$dom-&gt;saveXML());   
  }
  </code></pre>
  
  <p>However, I think I now have a display problem as this text "Roééleç sé en franêais" (new_value) is being displayed as:</p>
  
  <p>display problem:
  <img src="https://i.stack.imgur.com/VSh1K.png" title="charset display issues?" /></p>
  
  <p>In my HTML document, I have the following declaration for content-type (unfortunately, I don't hold the keys to make changes here):</p>
  
  <pre><code>&lt;html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en"&gt;
  ...
  &lt;meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" /&gt;
  </code></pre>
  
  <p>I've tried iconv() to convert to ISO-8859-1, however, most of the special characters are being removed when doing the conversion. All that remains is "Ro" using this command:</p>
  
  <pre><code>iconv('UTF-8','ISO-8859-1',$node-&gt;new_value);
  </code></pre>
  
  <p>iconv output:
  <img src="https://i.stack.imgur.com/X3Z7Y.png" title="post iconv output" /></p>
  
  <p>The field in the db is: utf8_general_ci. However, the connection charset would be whatever is the default.</p>
  
  <p>Not quite sure where to go from here...</p>
  
  <p><strong>Edit #2</strong> - I tried utf8_decode to see if that wouldn't help, but it didn't.</p>
  
  <pre><code>utf8_decode($a-&gt;new_value);
  </code></pre>
  
  <p>Output:
  <img src="https://i.stack.imgur.com/T0bbK.png" title="utf8_decode output" /></p>
  
  <p>I also noticed that my field in the db did contain UTF-8. Which is good.</p>