title: Unicode null symbol in text parsed from file leading to failing equality checks
body: <p>I'm running ruby 1.9.3 on Cygwin64. I'm downloading a zip-file and parsing the contained file based on the beginning of each line like so:</p>
  
  <pre><code>open(fpath, 'wb') do |file|
      file &lt;&lt; open(url).read
  end
  logpath = Pathname.new("./tmp.log")
  Zip::File.open(fpath) do |zipfile|
     zipfile.each do |file|
        file.extract(logpath.to_s){true}
     end
  end 
  
  logfile = open(logpath, 'r')
  logfile.each do |line|
     if line == "SOMETHING"
        #do something
     end
  end
  </code></pre>
  
  <p>The code inside the <code>if</code> statement never runs. That's because:</p>
  
  <pre><code>"SOMETHING".inspect = "SOMETHING"
  </code></pre>
  
  <p>and</p>
  
  <pre><code>line.inspect = "\u0000S\u0000O\u0000M\u0000E\u0000T\u0000H\u0000I\u0000N\u0000G
  </code></pre>
  
  <p>I'm stumped as to why the unicode null escape code is present before every character in <code>line</code>. The only solutions I've found are:</p>
  
  <pre><code>system("dos2unix #{logpath}")
  </code></pre>
  
  <p>and</p>
  
  <pre><code>line.encode!("UTF-16le", "UTF-8", :invalid =&gt; :replace, :undefined =&gt; :replace, replace: "")
  line.encode!("UTF-8", "UTF-16le")
  line.gsub!("\u0000", "")
  </code></pre>
  
  <p>Is there a better way to handle this? Am I doing something incorrectly extracting the logfile that's causing this problem?</p>
  
  <p>Any and all comments and answers much appreciated!</p>
  
  <p><strong>Solution</strong></p>
  
  <p>Open the file for reading as a UTF-16 little endian file:</p>
  
  <pre><code>logfile = open(logpath, 'rb:UTF-16le')
  </code></pre>
  
  <p>Then encode each line (or the full body of text if you want)</p>
  
  <pre><code>line.encode!("UTF-8", "UTF-16le", :invalid =&gt; :replace, :undefined =&gt; :replace, replace: "")
  </code></pre>
  
  <p>And voila! Thanks Denis!</p>