title: Ruby unable to parse a CSV file: CSV::MalformedCSVError (Illegal quoting in line 1.)
body: <p><strong>Ubuntu 12.04 LTS</strong></p>
  
  <p><strong>Ruby ruby 1.9.3dev (2011-09-23 revision 33323) [i686-linux]</strong></p>
  
  <p><strong>Rails 3.2.9</strong></p>
  
  <p>Following is the content of my received CSV file:</p>
  
  <pre><code>"date/time","settlement id","type","order id","sku","description","quantity","marketplace","fulfillment","order city","order state","order postal","product sales","shipping credits","gift wrap credits","promotional rebates","sales tax collected","selling fees","fba fees","other transaction fees","other","total"
  "Mar 1, 2013 12:03:54 AM PST","5481545091","Order","108-0938567-7009852","ALS2GL36LED","Solar Two Directional 36 Bright White LED Security Flood Light with Motion Activated Sensor","1","amazon.com","Amazon","Pasadena","CA","91104-1056","43.00","3.25","0","-3.25","0","-6.45","-3.75","0","0","32.80"
  </code></pre>
  
  <p>However when I am trying to parse the CSV file I am getting error:</p>
  
  <pre><code>1.9.3dev :016 &gt; options = { col_sep: ",", quote_char:'"' }
  =&gt; {:col_sep=&gt;",", :quote_char=&gt;"\""} 
  
  1.9.3dev :022 &gt; CSV.foreach("/tmp/my_data.csv", options) { |row| puts row }
  CSV::MalformedCSVError: Illegal quoting in line 1.
      from /home/jigneshgohel/.rvm/rubies/ruby-1.9.3-rc1/lib/ruby/1.9.1/csv.rb:1925:in `block (2 levels) in shift'
      from /home/jigneshgohel/.rvm/rubies/ruby-1.9.3-rc1/lib/ruby/1.9.1/csv.rb:1887:in `each'
      from /home/jigneshgohel/.rvm/rubies/ruby-1.9.3-rc1/lib/ruby/1.9.1/csv.rb:1887:in `block in shift'
      from /home/jigneshgohel/.rvm/rubies/ruby-1.9.3-rc1/lib/ruby/1.9.1/csv.rb:1849:in `loop'
      from /home/jigneshgohel/.rvm/rubies/ruby-1.9.3-rc1/lib/ruby/1.9.1/csv.rb:1849:in `shift'
      from /home/jigneshgohel/.rvm/rubies/ruby-1.9.3-rc1/lib/ruby/1.9.1/csv.rb:1791:in `each'
      from /home/jigneshgohel/.rvm/rubies/ruby-1.9.3-rc1/lib/ruby/1.9.1/csv.rb:1208:in `block in foreach'
      from /home/jigneshgohel/.rvm/rubies/ruby-1.9.3-rc1/lib/ruby/1.9.1/csv.rb:1354:in `open'
      from /home/jigneshgohel/.rvm/rubies/ruby-1.9.3-rc1/lib/ruby/1.9.1/csv.rb:1207:in `foreach'
      from (irb):22
      from /home/jigneshgohel/.rvm/rubies/ruby-1.9.3-rc1/bin/irb:16:in `&lt;main&gt;'
  </code></pre>
  
  <p>Then I tried simplifying the data i.e.</p>
  
  <pre><code>"name","age","email"
  "jignesh","30","jignesh@example.com"
  </code></pre>
  
  <p>however still I am getting the same error:</p>
  
  <pre><code>      1.9.3dev :023 &gt; CSV.foreach("/tmp/my_data.csv", options) { |row| puts row }
    CSV::MalformedCSVError: Illegal quoting in line 1.
        from /home/jigneshgohel/.rvm/rubies/ruby-1.9.3-rc1/lib/ruby/1.9.1/csv.rb:1925:in `block (2 levels) in shift'
        from /home/jigneshgohel/.rvm/rubies/ruby-1.9.3-rc1/lib/ruby/1.9.1/csv.rb:1887:in `each'
        from /home/jigneshgohel/.rvm/rubies/ruby-1.9.3-rc1/lib/ruby/1.9.1/csv.rb:1887:in `block in shift'
        from /home/jigneshgohel/.rvm/rubies/ruby-1.9.3-rc1/lib/ruby/1.9.1/csv.rb:1849:in `loop'
        from /home/jigneshgohel/.rvm/rubies/ruby-1.9.3-rc1/lib/ruby/1.9.1/csv.rb:1849:in `shift'
        from /home/jigneshgohel/.rvm/rubies/ruby-1.9.3-rc1/lib/ruby/1.9.1/csv.rb:1791:in `each'
        from /home/jigneshgohel/.rvm/rubies/ruby-1.9.3-rc1/lib/ruby/1.9.1/csv.rb:1208:in `block in foreach'
        from /home/jigneshgohel/.rvm/rubies/ruby-1.9.3-rc1/lib/ruby/1.9.1/csv.rb:1354:in `open'
        from /home/jigneshgohel/.rvm/rubies/ruby-1.9.3-rc1/lib/ruby/1.9.1/csv.rb:1207:in `foreach'
        from (irb):23
        from /home/jigneshgohel/.rvm/rubies/ruby-1.9.3-rc1/bin/irb:16:in `&lt;main&gt;'
  </code></pre>
  
  <p>Again I tried simplifying the data like this:</p>
  
  <pre><code>name,age,email
  jignesh,30,jignesh@example.com
  </code></pre>
  
  <p>and it works.See the output below:</p>
  
  <pre><code>  1.9.3dev :024 &gt; CSV.foreach("/tmp/my_data.csv") { |row| puts row }
    name
    age
    email
    jignesh
    30
    jignesh@example.com
     =&gt; nil 
  </code></pre>
  
  <p>But I will be receiving the CSV files having quoted data so removing quotes solution is not actually I am looking for.I am unable to figure out what is causing the error: <strong>CSV::MalformedCSVError: Illegal quoting in line 1.</strong> in my earlier examples.</p>
  
  <p>I have verified that in the CSV there are no leading/trailing spaces by enabling "Show whitespace characters" and "Show Line Endings" in my text editor.Also I have verified the encoding using following.</p>
  
  <pre><code>  1.9.3dev :026 &gt; File.open("/tmp/my_data.csv").read.encoding
    =&gt; #&lt;Encoding:UTF-8&gt; 
  </code></pre>
  
  <p><em>Note: I tried using CSV.read too but same error with that method.</em></p>
  
  <p>Can anybody please help me getting out of the problem and make me understand where it is going wrong?</p>
  
  <p>=====================</p>
  
  <p>I just found following post at: <a href="http://www.ruby-forum.com/topic/448070" rel="noreferrer">http://www.ruby-forum.com/topic/448070</a> and tried following:</p>
  
  <pre><code>  file_data = file.read
    file_data.gsub!('"', "'")
    arr_of_arrs = CSV.parse(file_data)
  
    arr_of_arrs.each do |arr|
      Rails.logger.debug "=======#{arr}"
    end
  </code></pre>
  
  <p>and got the following output:</p>
  
  <pre><code>   =======["\xEF\xBB\xBF'date/time'", "'settlement id'", "'type'", "'order id'", "'sku'", "'description'", "'quantity'", "'marketplace'", "'fulfillment'", "'order city'", "'order state'", "'order postal'", "'product sales'", "'shipping credits'", "'gift wrap credits'", "'promotional rebates'", "'sales tax collected'", "'selling fees'", "'fba fees'", "'other transaction fees'", "'other'", "'total'"]
      =======["'Mar 1", " 2013 12:03:54 AM PST'", "'5481545091'", "'Order'", "'108-0938567-7009852'", "'ALS2GL36LED'", "'Solar Two Directional 36 Bright White LED Security Flood Light with Motion Activated Sensor'", "'1'", "'amazon.com'", "'Amazon'", "'Pasadena'", "'CA'", "'91104-1056'", "'43.00'", "'3.25'", "'0'", "'-3.25'", "'0'", "'-6.45'", "'-3.75'", "'0'", "'0'", "'32.80'"]
  </code></pre>
  
  <p>which messed up reading the data properly as the default <strong>col_sep</strong> used is a comma character.
  However I tried using <strong>quote_char</strong> option like this:</p>
  
  <pre><code>  arr_of_arrs = CSV.parse(file_data, :quote_char =&gt; "'")
  </code></pre>
  
  <p>but it ended up the following error:</p>
  
  <pre><code>   CSV::MalformedCSVError (Illegal quoting in line 1.):
  </code></pre>
  
  <p>Thanks,
  Jignesh</p>