title: How to speed up PHP to MySQL query with large query data
body: <p>Before code :
  for loop should run at least 143,792,640,000 times and create the table at least produce 563,760 rows without duplicated
  I want to know how to speed up or something parallel computing like Hadoop that could accelerate between php and MySQL.</p>
  
  <p>Code below:</p>
  
  <p>MySQL connection</p>
  
  <pre><code>$link=mysql_connect($servername,$username,$password);
  mysql_select_db($dbname);
  $sql= "INSERT INTO EM (source,target) VALUES ";
  </code></pre>
  
  <p>for loop read data into MySQL check function if duplicate not insert and update count=count+1</p>
  
  <pre><code>for($i=0;$i&lt;$combine_arr_size;$i++){
      for($j=0;$j&lt;$combine_arr_size;$j++){  
  </code></pre>
  
  <p>//below check if find the duplicated like a,b we recognize b,a is same thing</p>
  
  <pre><code>if(check($combine_words_array[$i],$combine_words_array[$j])) {
                  $update_query="UPDATE EM SET count = count+1 where (source='$combine_words_array[$i]' AND target='$combine_words_array[$j]') OR (source='$combine_words_array[$j]' AND target='$combine_words_array[$i]');";
                  mysql_query($update_query);
              } else {
                  if (!$link) {
                      die("Connection failed: " . mysql_error());
                  }
  </code></pre>
  
  <p>//else using insert into table () value to concatenate the string</p>
  
  <pre><code>    $sql.="('$combine_words_array[$i]','$combine_words_array[$j]'),";     
              mysql_query(substr($sql,0,-1));
              $sql= "INSERT INTO EM (source,target) VALUES ";        
          }
      }
  } 
  </code></pre>
  
  <p>read the all vector align from <code>comebine_word_array[]</code> to <code>combine_word_array[]</code></p>
  
  <p>below is check function , check if find the pair return value</p>
  
  <pre><code>function check($src, $trg) {
      $query = mysql_query("SELECT * FROM EM WHERE (source='$src' AND target='$trg') OR (source='$trg' AND target='$src');");
      if (mysql_num_rows($query) &gt; 0) {
          return 1;
      } else {
          return 0;
      }
  }
  </code></pre>
  
  <p>table </p>
  
  <pre><code>+--------+--------------+------+-----+---------+-------+
  | Field  | Type         | Null | Key | Default | Extra |
  +--------+--------------+------+-----+---------+-------+
  | source | varchar(255) | YES  |     | NULL    |       |
  | target | varchar(255) | YES  |     | NULL    |       |
  | count  | int(11)      | NO   |     | 0       |       |
  | prob   | double       | NO   |     | 0       |       |
  +--------+--------------+------+-----+---------+-------+
  </code></pre>
  
  <p>now the php code just influence the source ,target and count </p>