title: Converting a large ASCII to CSV file in python or java or something else on Linux or Win7
body: <p>Need a hint so I can convert a huge (300-400 mb) ASCII file to a CSV file.</p>
  
  <p>My ASCII file is a database with a lot of products (about 600,000 pcs = 55,200,000 lines in the file).</p>
  
  <p>Below is ONE product. It is like a tablerow in a database, with 88 columns.
  If you count the below lines, there is 92 lines.</p>
  
  <p>For every time we have the '00I+CR\LF' it indicates, that we have a new row/product.</p>
  
  <p>Each line is ended with a CR+LF.</p>
  
  <p>A whole product/row is ended with the following three lines:</p>
  
  <pre><code>A00
  A10
  A21
  </code></pre>
  
  <p>-as shown below.</p>
  
  <p>Between the starting line <code>'00I CR+LF'</code> and the three ending lines, we have lines, starting with 2 digits (column name), and what comes after those digits, is the data for the column.</p>
  
  <p>If we take the first line below the starting line <code>'00I CR+LF'</code> we will see:</p>
  
  <p><code>'0109321609'</code>. <code>01</code> indicates that it is the column named 01, and the rest is the data stored in that column: <code>'09321609'</code>.</p>
  
  <p>I want to strip out the two digits, indicating each column name/line-number, so the first line (after the starting indication <code>'00I'</code>): <code>0109321609</code> comes out as the following: <code>”09321609”</code>.</p>
  
  <p>Putting it together with the next line (<code>02</code>), it should give an output like:  </p>
  
  <pre><code>”09321609”,”15274”, etc.
  </code></pre>
  
  <p>When coming to the end, we want a new row.</p>
  
  <p>The first line <code>'00I'</code> and the three last lines <code>'A00'</code>, <code>'A10'</code> and <code>'A21'</code> we don't want to be included in the output file.</p>
  
  <p>Here is how a row looks like (every line is ended by a CR+LF):</p>
  
  <pre><code>00I
  0109321609
  0215274
  032
  0419685
  05
  062
  072
  081
  09
  111
  121
  15
  161
  17
  1814740
  1920120401
  2020120401
  2120120401
  22
  230
  240
  251
  26BLAHBLAH 1000MG
  27
  281
  29
  30
  31BLAHBLAH 1000 mg Filmtablets Hursutacinzki
  32
  3336
  341
  350
  361
  371
  401
  410
  420
  43
  445774
  45FTA
  46
  47AN03AX14
  48BLAHBLAH00000000000000000000010
  491
  501
  512
  522
  5317
  542
  552
  561
  572
  581
  591
  60
  61
  62
  631
  641
  65
  66
  67
  681
  69
  721
  74884
  761
  771
  780
  790
  801
  811
  831
  851474
  86
  871
  880
  891
  901
  911
  922
  930
  941
  951
  961
  97
  98
  990
  A00
  A10
  A21
  </code></pre>
  
  <p>Anyone got a hint on how it can be converted?</p>
  
  <p>The file is too big for a webserver with <code>php</code> and <code>mysql</code> to run. My thought was to put the file in a directory on my local server, and read the file, strip out the line numbers, and insert the data directly in a <code>mysql</code> database on the fly, but the file is too big, and the server stalls.</p>
  
  <p>I'm able to run under Linux (Ubuntu) and Windows 7.</p>
  
  <p>Maybe some python or java is recommended? I'm able to run both, but my experience with those is low, but I'm a quick learner, so if someone can give a hint?  :-)</p>
  
  <p>Best Regards
  Bjarke :-)</p>