title: Speed up millions of regex replacements in Python 3
body: <p>I have two lists:</p>
  <ul>
  <li>a list of about 750K <em><strong>&quot;sentences&quot;</strong></em> (long strings)</li>
  <li>a list of about 20K <em><strong>&quot;words&quot;</strong></em> that I would like to delete from my 750K sentences</li>
  </ul>
  <p>So, I have to loop through 750K <em>sentences</em> and perform about 20K replacements, <strong>but ONLY if my words are actually <em>&quot;words&quot;</em> and are not part of a larger string of characters.</strong></p>
  <p>I am doing this by pre-compiling my <em>words</em> so that they are flanked by the <code>\b</code> word-boundary metacharacter:</p>
  <pre><code>compiled_words = [re.compile(r'\b' + word + r'\b') for word in my20000words]
  </code></pre>
  <p>Then I loop through my <em>&quot;sentences&quot;</em>:</p>
  <pre><code>import re
  
  for sentence in sentences:
    for word in compiled_words:
      sentence = re.sub(word, &quot;&quot;, sentence)
    # put sentence into a growing list
  </code></pre>
  <p>This nested loop is processing about <strong>50 sentences per second</strong>, which is nice, but it still takes several hours to process all of my sentences.</p>
  <ul>
  <li><p>Is there a way to using the <code>str.replace</code> method (which I believe is faster), but still requiring that replacements only happen at <strong>word boundaries</strong>?</p>
  </li>
  <li><p>Alternatively, is there a way to speed up the <code>re.sub</code> method? I have already improved the speed marginally by skipping over <code>re.sub</code> if the length of my word is &gt; than the length of my sentence, but it's not much of an improvement.</p>
  </li>
  </ul>
  <p>I'm using Python 3.5.2</p>