title: mysql - best way to avoid duplicates
body: <p>I am very new to the database game so forgive my ignorance.</p>
  
  <p>I am loading millions of rows into a simply structured MySQL database table</p>
  
  <pre><code>SQLStr = "LOAD DATA LOCAL INFILE 'f:/Smallscale/02 MMToTxt/flat.txt'
    INTO TABLE `GMLObjects` FIELDS TERMINATED BY ','
    LINES STARTING BY 'XXX';"
  </code></pre>
  
  <p>At the moment the table has been set to no duplicates against one field.</p>
  
  <p>However, I am wondering if it would be quicker to remove the no duplicates rule and deal with the problem of duplicates later, either by using <code>ALTER TABLE</code> or <code>SELECT DISTINCT</code>, or some such query.</p>
  
  <p>What are your thoughts?</p>
  
  <p>P.S the database engine is InnoDB</p>