title: CSV Parser, buffer overflow issue with very specific data
body: <p>This may not fall into guidelines since it's a very specific issue, but I have exhausted every avenue of this to be a File issue.</p>
  <p>Ruby (2.6.0) native CSV parser crashes on a specific file. I have tried reproducing the exact set of bytes that cause the issue, but haven't been able to do so.</p>
  <p>What I did then was to replicate the file, but replaced all alphabets with 'a' and numbers with '0'. The <a href="https://www.dropbox.com/s/gsytqiqvo73df7o/illegal_quoting_case.csv?dl=0" rel="nofollow noreferrer">resulting file</a> also crashes on a very specific line (1165) claiming my quotes aren't balanced (which they are).</p>
  <p>Code that crashes:</p>
  <pre><code>CSV.foreach(File.expand_path(&quot;~/Downloads/illegal_quoting_case.csv&quot;), skip_lines: /^(?:,\s*)+$/) { |r| puts &quot;\n\n#{r.inspect}\n\n&quot; }
  </code></pre>
  <p>Interesting observations:</p>
  <ol>
  <li>if you change any byte (add a character, or remove) from ANY line above 1165, it works fine. Even a space will do, in ANY line above it. You can ADD or REMOVE one character and it works fine.</li>
  <li>It works fine if you take away <code>skip_lines</code></li>
  </ol>
  <p>Now I have attempted to debug main codebase, the issue seems to be when the scanner is near the end of buffer chunk size of 8192 then <a href="https://github.com/ruby/csv/blob/29cef9ea9dfb479f4231389a5ab7b754046d0edd/lib/csv/parser.rb#L905" rel="nofollow noreferrer">THIS line</a> somehow reads extra bytes, splitting the first column of the next line to cause the issue.</p>
  <p>This is a bizzare one to be able to reproduce, but the issue DOES lie somewhere in the <code>CSV::Parser::Scanner::StringScanner</code>'s method of reading bytes.</p>