title: Outputting special characters correctly (unicode) in PERL
body: <p>I am trying to get all the file names in the directory and determine which names contain special characters. I am using the regex</p>
  
  <pre><code>/[^-a-zA-Z0-9_.]/
  </code></pre>
  
  <p>SAMPLE FILES ( I created using touch ):</p>
  
  <pre><code>pdf-2014à014&amp;7_06_64-Os_O&amp;L,_Inc.pdf
  pdf-20_06_04-O_OnLine,_Inc.pdf
  pdf-20_0_0-Utà_d.wr.pdf
  pdf-20_12_28-20.Mga_Grf.Fwd_Notice_KDJFI789&amp;_JFK38.pdf
  pdf-2_0_0-C_—_DUKE.pdf
  pdf-2_1_3-f_s-M_F_D&amp;A.pdf
  pdf_-_2014à014&amp;1007_0617_06264-O_O&amp;L,_Inc.pdf
  </code></pre>
  
  <p>Perl can output the correct name once before I match the name for pattern in regex. Yes perl was somehow able to get match the special character but when outputing the character changes.</p>
  
  <pre><code>* pdf-2_0_0-C_—_DUKE.pdf          &gt;        pdf-2_0_0-C_???_DUKE.pdf
  </code></pre>
  
  <p>I can try uncomment this line </p>
  
  <pre><code>   #binmode(STDOUT, ":utf8");
  </code></pre>
  
  <p>and run the commmand script again. Sure the ? marks will be remove but the output is also different.</p>
  
  <pre><code>* pdf-2_0_0-C_—_DUKE.pdf          &gt;        pdf-2_0_0-C_â_DUKE.pdf
  </code></pre>
  
  <p>Here is my code:</p>
  
  <pre><code>use strict;
  use warnings;
  use File::Find;
  use Cwd;
  
  #binmode(STDOUT, ":utf8");
  
  my $starting_directory = cwd();
  
  use Term::ANSIColor;
  
  checkForSpecialChar(cwd());
  
  
  sub checkForSpecialChar{
      my ($source_dir) = @_;
  
      chdir $source_dir or die qq(Cannot change into "$source_dir");
  
      find ( sub {
          return unless -f;   #We want files only
          print "\n";
          while(m/([^-a-zA-Z0-9_.])/g){ 
              chomp($_);
              print "DETECTED: |" . $_ . "|\n";
              print $`;
              print color 'bold red';
              print "$1";
              print color 'reset';
              print  $' . "\n";
  
          }
  
      }, ".");
  
      chdir("$starting_directory");
  </code></pre>
  
  <p>Any Idea guys?</p>
  
  <p>UPDATE:
  hmm, you guys are right looks like its not a problem with regex. Hi AKHolland, I tried changing the code to look just like yours for testing.  but still produce the same problem with hypen and a small letter a-grave . 
  Instead of a small letter a-grave it gives me 
  a` when not using binmode(STDOUT, ":utf8"); 
  aÌ when using binmode(STDOUT, ":utf8"); </p>
  
  <pre><code>use strict;
  use warnings;
  use File::Find;
  use Cwd;
  use Encode;
  binmode(STDOUT, ":utf8");
  
  my $starting_directory = cwd();
  
  use Term::ANSIColor;
  
  checkForSpecialChar(cwd());
  
  
  sub checkForSpecialChar{
     my ($source_dir) = @_;
  
     chdir $source_dir
         or die qq(Cannot change into "$source_dir");
  
     find ( sub {
        return unless -f;   #We want files only
       print $_ . "\n";
        $_ = Encode::decode_utf8($_);
        for(my $counter =0; $counter &lt; length($_); $counter++) {
          print Encode::encode_utf8(substr($_,$counter,1)) .  "\n";
        } 
  
  }, ".");
  </code></pre>
  
  <p>chdir("$starting_directory");
  }</p>
  
  <pre>
  Output with
  
      binmode(STDOUT, ":utf8");
  
  pdf-2_0_0-C_â_DUKE.pdf
  p
  d
  f
  -
  2
  _
  0
  _
  0
  -
  C
  _
  â
  _
  D
  U
  K
  E
  .
  p
  d
  f
  pdf_-_2014aÌ014&1007_0617_06264-O_O&L,_Inc.pdf
  p
  d
  f
  _
  -
  _
  2
  0
  1
  4
  a
  Ì
  0
  1
  4
  &
  1
  0
  0
  7
  _
  0
  6
  1
  7
  _
  0
  6
  2
  6
  4
  -
  O
  _
  O
  &
  L
  ,
  _
  I
  n
  c
  .
  p
  d
  f
  </pre>
  
  <pre>
  OUTPUT without 
      binmode(STDOUT, ":utf8");
  
  pdf-2_0_0-C_—_DUKE.pdf
  p
  d
  f
  -
  2
  _
  0
  _
  0
  -
  C
  _
  —
  _
  D
  U
  K
  E
  .
  p
  d
  f
  pdf_-_2014à014&1007_0617_06264-O_O&L,_Inc.pdf
  p
  d
  f
  _
  -
  _
  2
  0
  1
  4
  a
  ̀
  0
  1
  4
  &
  1
  0
  0
  7
  _
  0
  6
  1
  7
  _
  0
  6
  2
  6
  4
  -
  O
  _
  O
  &
  L
  ,
  _
  I
  n
  c
  .
  p
  d
  f
  </pre>