title: How to read a big data in C?
body: <p>I practice with file handling in C. I need to read a file which recorded about 9000000 data.
  It works when I read the first 9000 data. However, it always crashes when I set NUM_DATA = 9000000.
  How can I solve this problem?</p>
  <pre><code>#include &lt;stdio.h&gt;
  #include &lt;string.h&gt;
  #include &lt;ctype.h&gt;
  #include &lt;stdlib.h&gt;
  
  #define MAX_LENGTH 10
  #define NUM_DATA 9000000
  
  int main(int argc, char **argv){
  
      char **signal = malloc( NUM_DATA * MAX_LENGTH * sizeof(char *) );
      char **svid = malloc( NUM_DATA * MAX_LENGTH * sizeof(char *) );
  
      char GetString[110];
      int num;
      double tw[NUM_DATA], ip[NUM_DATA], qp[NUM_DATA]
      char a[20], b[20], c[20], d[20], e[20], f[20], g[20], h[20], z[20], x[20], w[20];
  
      FILE *fIn;
  
      fIn = fopen(argv[1], &quot;r&quot;);
  
      // read header
      fgets(GetString, 109, fIn);
      fgets(GetString, 109, fIn);
  
      // read data
      num = 0;
      while (!feof(fIn)){
  
          fscanf(fIn, &quot;%[^','],%[^','],%[^','],%[^','],%[^','],%[^','],%[^','],%[^','],%[^','],%[^','],%f&quot;,
                 a, b, c, d, e, f, g, h, z, x, w);
  
          tw[num] = atof(a);
          ip[num] = atof(z);
          qp[num] = atof(x);
          signal[num] = malloc(MAX_LENGTH * sizeof(char *));
          strcpy(signal[num], f);
          strcpy(svid[num], h);
          num++;
      }
      fclose(fIn);
  }
  </code></pre>
  <p>The format of the file:</p>
  <p>353700.000,1352,0.020000,0.000000,1,ISR1,Main,E11,173,14,56.102000
  353700.000,1353,0.020000,0.000000,1,ISR1,Main,E11,156,7,14.367000
  353700.000,1354,0.020000,0.000000,1,ISR1,Main,E11,161,14,40.136000
  353700.000,1355,0.020000,0.000000,1,ISR1,Main,E11,316,23,31.326000
  353700.000,1356,0.020000,0.000000,1,ISR1,Main,E11,152,-1,3.806000
  353700.000,1357,0.020000,0.000000,2,ISR1,Main,G14,-101,10,51.833000
  353700.000,1358,0.020000,0.000000,3,ISR1,Main,G24,434,-3,56.414000
  353700.000,1359,0.020000,0.000000,3,ISR1,Main,G24,171,-4,17.890000
  353700.000,1360,0.020000,0.000000,3,ISR1,Main,G24,474,6,52.542000</p>
  <p>......</p>