title: Strawberry Perl -- where are encoding conversions done by default?
body: <p>Basically, I wrote a Perl script that creates an encoded command for Powershell and tries to run it. I had to explicitly convert the command string to utf-16 before base64-encoding it. I'm wondering why that's <strong>all</strong> I had to do get the script to work. What conversions is Perl on Windows*  performing by default in the run of an "ordinary" program that interacts with the console and perhaps the file system? For instance, is argv converted? Is stdin/stdout converted? Does file IO go through a conversion?</p>
  
  <p>✱ in particular, the Strawberry Perl distribution in case ActivePerl does something different</p>
  
  <hr>
  
  <p>I'm trying to write a Perl script that calls many PowerShell fragments and depends on Strawberry Perl distribution.</p>
  
  <p>PowerShell, rather conveniently, has an <code>-encodedCommand</code> flag that accepts a base64-encoded string and then processes it. This is helpful for avoiding quoting-related problems.</p>
  
  <p>I tried the Simplest Thing That Could Possibly Work.</p>
  
  <pre><code>// powersheller.pl
  
  #! /usr/bin/env perl
  
  use strict;
  use warnings;
  
  use MIME::Base64;
  use Encode qw/encode decode/;
  
  use vars ('$powershell_command');
  
  sub run_powershell_fragment {
      my ($contents) = @_;
      my $encoded = encode_base64($contents);
      printf "encoded: %s\n", $encoded;
      return `powershell.exe -noprofile -encodedCommand $encoded`;
  }
  
  printf "%s\n---\n", run_powershell_fragment($powershell_command);
  
  BEGIN {
  $powershell_command = &lt;&lt;EOF
  echo "hi"   
  EOF
  }
  </code></pre>
  
  <p>And ran it. Here's the output of the ... standard output channels (?) from running the perl script in the powershell window.</p>
  
  <pre><code>PS C\...&gt; perl .\powersheller.pl
  encoded: ZWNobyAiaGkiCQo=
  
  Redundant argument in printf at .\powersheller.pl line 18.
  ?????? : The term '??????' is not recognized as the name of a cmdlet, function, script file, or operable program.
  
  ---
  </code></pre>
  
  <p>This looked like an encoding issue. I guessed that Perl was using something resembling utf-8 by default and powershell was expecting utf16-le or similar.</p>
  
  <pre><code>sub run_powershell_fragment {
      my ($contents) = @_;
      my $utf16_le_contents = encode("utf-16le", $contents);
      my $encoded = encode_base64($utf16_le_contents);
      printf "encoded: %s\n", $encoded;
      return `powershell.exe -noprofile -encodedCommand $encoded`;
  }
  </code></pre>
  
  <p>Technically, using <code>"ucs-2le"</code> also works. I don't know which is appropriate.</p>
  
  <p>Anyway, all together, the program works as expected with the extra conversion inserted.</p>
  
  <pre><code>PS C:\...&gt; perl .\powersheller.pl
  encoded: ZQBjAGgAbwAgACIAaABpACIACQAKAA==
  
  hi
  
  ---
  </code></pre>
  
  <p>Why was this all that I needed to do? Is Perl handling conversions related to argv and stdout &amp;c?</p>