title: Identifying and matching non ascii characters in files
body: <p>I'm trying to read a delimited file and parse its contents. Unlike a CSV, the delimiter, string qualifier etc are non ASCII ie. U0014 and U00FE respectively. However, I'm unable to detect the string qualifier character (FE) . Is this because the character's value is greater 128 or something else? </p>
  
  <p>Here's a simple program that illustrates the core issue. How can I make this work? Here's a link to a very small test file . <a href="https://www.dropbox.com/s/1cilircwc3pq78c/nonascii.dat?dl=0" rel="nofollow">https://www.dropbox.com/s/1cilircwc3pq78c/nonascii.dat?dl=0</a></p>
  
  <p>Thanks</p>
  
  <pre><code>import org.apache.commons.io.FileUtils;
  import org.apache.commons.io.LineIterator;
  import java.io.BufferedReader;
  import java.io.File;
  import java.io.FileInputStream;
  import java.io.InputStreamReader;
  import java.io.PrintStream;
  import java.io.Reader;
  
  public class CharMatch {
      public static void main(String[] args)
          throws Exception {
      final String pathname = "/home/vinayb/Downloads/nonascii.dat";
      final File file = new File(pathname);
      final String encoding = "UTF-8";
      final PrintStream out = new PrintStream(System.out, true, encoding);
      final Reader r = new BufferedReader(new InputStreamReader(
              new FileInputStream(file), encoding));
  
      final LineIterator it = FileUtils.lineIterator(file, encoding);
      try {
          //read a line
          final String line = it.nextLine();
          final char[] chars = line.toCharArray();
          for (char c : chars) {
              out.println(c + " , with decimal value: " + Character.getNumericValue(c) + " and hexa value: " + Integer.toHexString(Character.getNumericValue(c)));
          }
  
          out.println("------------------------------------");
          final String expectedDelimiter = fromUnicode("0014");
          final String expectedStringQualifier = fromUnicode("00FE");
          out.println("##### expected delimiter:" + expectedDelimiter);
  
          out.println("##### expected string qualifier:" + expectedStringQualifier);
          String[] items = line.split(expectedDelimiter);
          out.println("#### " + items.length + " " + items[0]);
  
          if (line.contains(expectedDelimiter)) {
              out.println("Found delimiter"); ////=======&gt; can match this
          }
  
          if (line.contains(expectedStringQualifier)) {
              out.println("Found string qualifier"); //=======&gt; can't match this
          }
      } finally {
          LineIterator.closeQuietly(it);
      }
  }
  
  private static String fromUnicode(String codePoint) {
      return "" + (char) Integer.parseInt(codePoint, 16);
  }
  </code></pre>
  
  <p>}</p>