title: special characters in Java
body: <p>I have a form (struts 1) that is being validated and during the validation I have been asked to remove MS Word's curly single and double quote marks. Seems like such a simple request and I am tearing my hair out over it.</p>
  
  <p>My test text is <code>’ “ ”</code>.</p>
  
  <p>First of all, I found that when I run my code in the debugger and watch what IntelliJ thinks the values are, it displays <code>â\u0080\u0099 â\u0080\u009C â\u0080\u009D</code> and it seems that <code>â\u0080</code> are nonprinting characters.</p>
  
  <p>I used a piece of code that iterates over a StringBuilder of the text in the field and tests each char in the text. It replaces or deletes some chars, as below:</p>
  
  <pre><code>switch (origCharAsInt) {
  
      case ((int)'\u00C2'): sbOriginal.deleteCharAt(isb); break; // weird Word A with the caret over it
      case ((int)'\u00C3'): sbOriginal.deleteCharAt(isb); break; // weird Word A with the tilde over it
      case ((int)'\u00E2'): sbOriginal.deleteCharAt(isb); break; // weird Word a with the caret over it
  
      case ((int)'\u0099'): sbOriginal.setCharAt(isb, '\''); break; // Word single quote
      case ((int)'\u009C'): sbOriginal.setCharAt(isb, '"'); break; // Word left double quote
      case ((int)'\u009D'): sbOriginal.setCharAt(isb, '"'); break; // Word right double quote
  
      case ((int)'\u2018'):  sbOriginal.setCharAt(isb, '\''); break;  // left single quote
      case ((int)'\u2019'):  sbOriginal.setCharAt(isb, '\''); break;  // right single quote
      case ((int)'\u201A'):  sbOriginal.setCharAt(isb, '\''); break;  // lower quotation mark
  
      case ((int)'\u201C'):  sbOriginal.setCharAt(isb, '"'); break;  // left double quote
      case ((int)'\u201D'):  sbOriginal.setCharAt(isb, '"'); break;  // right double quote
      case ((int)'\u201E'):  sbOriginal.setCharAt(isb, '"'); break;  // double low quotation mark
  
      case ((int)'\u2039'):  sbOriginal.setCharAt(isb, '\''); break;  // Single Left-Pointing Quotation Mark
      case ((int)'\u203A'):  sbOriginal.setCharAt(isb, '\''); break;  // Single right-Pointing Quotation Mark
  
      default: break;
  </code></pre>
  
  <p>}</p>
  
  <p>This seems to work, in that it replaces some of the more egregious cruft, and the form now appears to contain <code>' " "</code>. If I then save again, though, the IntelliJ thinks the field contains <code>Â\u0080 Â\u0080\" Â\u0080\"</code>.</p>
  
  <p>So I added a few more cases to remove those Â characters. But I'm flummoxed by the persistence of the <code>\u0080</code> characters. </p>
  
  <p>I tried adding in a few more cases to try to remove them, but they did not work.</p>
  
  <pre><code>case ((int)'\u0080'): sbOriginal.deleteCharAt(isb); break; // another weird Word non-printing char
  case ((int)'\u0082'): sbOriginal.deleteCharAt(isb); break; // another weird Word non-printing char
  case ((int)'\u0083'): sbOriginal.deleteCharAt(isb); break; // another weird Word non-printing char
  case ((int)'\u0000'): sbOriginal.deleteCharAt(isb); break; // why are these weird symbols showing up?
  </code></pre>
  
  <p>Any help/explanation would be greatly appreciated.</p>