title: large data in array
body: <p>I am developing a <strong>Rails</strong> app. I would like to use an array to hold <strong>2,000,000 data</strong>, then insert the data into database like following:</p>
  
  <pre><code>large_data = Get_data_Method() #get 2,000,000 raw data
  
  all_values = Array.new
  
  large_data.each{ |data|
  
      all_values &lt;&lt; data[1] #e.g. data[1] has the format "(2,'john','2002-09-12')"
  
  }
  
  sql="INSERT INTO cars (id,name,date) VALUES "+all_values.join(',')
  
  ActiveRecord::Base.connection.execute(sql)
  </code></pre>
  
  <p>When I run the code, it takes a long long time at the point of <code>large_data.each{...}</code> . Actually I am now still waiting for it to finish(it has been running for 1 hour already still not finish the <code>large_data.each{...}</code> part). </p>
  
  <p>Is it because of the number of elements is too large for the ruby array that the array can not hold <strong>2,000,000</strong> elements ?  or ruby array can hold that much elements and it is reasonable to wait this long?</p>
  
  <p>Since I would like to use <strong>bulk insertion</strong> in SQL to speed up the large data insertion time in mysql database, so I would like to use only one <code>INSERT INTO</code> statement, that's why I did the above thing. If this is a bad design, can you recommand me a better way?</p>