title: 600k rows csv file parsing / formating with python EXTREMELY slow
body: <p>My first question here since i started with python a couple days ago.
  I've great experience with VBA and Matlab, but I'm currently trying python as an exercise and because of the larger (is it?) quant community behind it.</p>
  
  <p>I searched quite a lot before asking here, i even used some code snippets which i found here from other users while doing mine (thanks all of you).</p>
  
  <p>The thing is, I'm reading a tick data csv that is ~630k rows long (15mb) so i can extract its third column (last ticks / deals) and them create a structure (matlab lingo) for them, a column vector of all the ticks in DataFrame format so i can compute their pct_change (pandas). </p>
  
  <p>I let it run overnight for about 6 hours and it was still @ ~150k/630k 
  I'm sure that I'm doing something that is extremely inefficient.</p>
  
  <p>I'm currently using spyder , running on windows 7, 4gb ram, i3 core, not really under heavy load.</p>
  
  <p>Here is the code:</p>
  
  <pre><code>"""""""""""""""""""""""""""""""""""""""
  created on Sun Jan 03 12:59:25 2016
  @author: eduardo
  """""""""""""""""""""""""""""""""""""""
  
  import pandas as pd
  import csv as csv
  from datetime import datetime
  startTime = datetime.now()
  
  path = "C:\Users\eduardo\.xy\startups"
  data = "C:\Users\eduardo\.xy\startups\INDV14.CSV"
  
  delimeters = [' ', ';'] # matrix [1,2]
  
  unique = '[]'  # empty struct ?
  close = [] # empty matrix for later use
  
  with open(data) as data: # data = csv
      for row in data: # counter to loop for inside csv
          for cols in data: # another counter for separating columns now
              for d in delimeters:cols = unique.join(cols.split(d))
              # last for loop does not need ":" ? 
              # from d to d+n, step 1
              # splits columns using "d" separators defined above
              # joins them after splitting, by a "[]" separator "space" ?
              row = cols.split(unique) # row = for each row splitted
              close.append(row[2]) # call third column of each (row) 
              # fill the empty matrix created above (close) row by row
              # with it up with a column vector of my 3rd col of the CSV
              ticks = map(int, close) # coverts strings to integers
              # format the column vector above to pandas DataFrame format
              deals = pd.DataFrame(ticks)
              # call pct_change function of pandas 
              daily_returns = deals.pct_change(periods=1)
              print(daily_returns)
      data.close() # closes csv file 
  
      # creates a new file ("W"rite), returns.csv
      dataCSV = open('returns.csv', 'w') 
      for line in daily_returns: # de for each line in the daily returns struct
          dataCSV.write(line) # writes them in the new csv file 
      dataCSV.close() # closes new file
  
      datetime.now() - startTime # time counter 
  </code></pre>
  
  <p>csv format:</p>
  
  <pre><code>20140801 105159;57085;5
  20140801 105206;57085;5
  </code></pre>