 <p>1 million entries containing words shouldn't be slow if you have an index on the word column. This is because the words would be pretty short but with enough entropy (statistical dispersion) to leverage the key.</p> <p>If this were 1 million phrases, comparing the phrases might have taken a bit longer and in order to optimise you could've broken the phrases down into the first 3 words (in different columns) and a column for the rest of the phrase with 4 column index over them.</p> <p>Test the speed like this:</p> <pre><code>set_time_limit(60*60); $pdo = new PDO('mysql:host=localhost;dbname=db', 'user', 'pass'); $x = microtime(TRUE); for($i = 0; $i &lt; 1000000; $i++) { $word = ''; for($j = 0; $j &lt; mt_rand(0,40); $j++) { $word .= chr(97+mt_rand(0,25)); } if($_GET['select']) $pdo-&gt;query("SELECT FROM words WHERE word = '$word';"); else if($_GET['insert']) $pdo-&gt;exec("INSERT IGNORE INTO words (word) VALUES ('$word');"); } $x = microtime(TRUE)-$x; var_dump($x); CREATE TABLE IF NOT EXISTS `words` ( `word` varchar(40) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL, UNIQUE KEY `word` (`word`) ) ENGINE=MyISAM DEFAULT CHARSET=latin1; </code></pre> <p>The speed I clocked on my laptop was initially 80.765522003174 seconds, and I've done 10 tests and the average is around <strong>93.478111839294</strong> seconds for the 1million select statements, meaning 1 tenth of a millisecond for each select.</p> <p>Take into consideration the fact that I clocked it from PHP that means that the actual SQL execution speed is much higher, the 93.5 seconds include PHP communicating with MySQL over TPC.</p> <p>I have inserted an extra 9million values into the table and tested the same script running 1million select statements against 10million values. The overall duration is around 52 seconds.</p>