title: translate by replacing words inside existing text
body: <p>What are common approaches for translating certain words (or expressions) inside a given text, when the text must be reconstructed (with punctuations and everythin.) ?</p>
  
  <p>The translation comes from a lookup table, and covers words, collocations, and emoticons like L33t, CUL8R, :-), etc.</p>
  
  <p>Simple string search-and-replace is not enough since it can replace part of longer words (cat > dog  ≠>  caterpillar > dogerpillar).</p>
  
  <p>Assume the following input:</p>
  
  <pre><code>s = "dogbert, started a dilbert dilbertion proces cat-bert :-)"
  </code></pre>
  
  <p>after translation, i should receive something like:</p>
  
  <blockquote>
    <p>result = "<strong>anna</strong>, started a <strong>george</strong> dilbertion <strong>process</strong> cat-bert <strong>smiley</strong>"</p>
  </blockquote>
  
  <p>I can't simply tokenize, since i loose <strong>punctuations</strong> and <strong>word positions</strong>. </p>
  
  <p>Regular expressions, works for normal words, but don't catch special expressions like the smiley :-) but it does .</p>
  
  <pre><code>re.sub(r'\bword\b','translation',s) ==&gt; translation
  re.sub(r'\b:-\)\b','smiley',s) ==&gt; :-)
  </code></pre>
  
  <p>for now i'm using the above mentioned regex, and simple replace for the non-alphanumeric words, but it's far from being bulletproof.</p>
  
  <p>(p.s. i'm using python)</p>