title: Python pygments lexer state preservation
body: <p>Running <a href="http://pygments.org/" rel="noreferrer">pygments</a> default lexer on the following c++ text: <code>class foo{};</code>, results in this:</p>
  
  <pre><code>(Token.Keyword, 'class')
  (Token.Text, ' ')
  (Token.Name.Class, 'foo')
  (Token.Punctuation, '{')
  (Token.Punctuation, '}')
  (Token.Punctuation, ';')
  </code></pre>
  
  <p>Note that the toke <code>foo</code> has the type <code>Token.Name.Class</code>. </p>
  
  <p>If i change the class name to <code>foobar</code> i want to be able to run the <strong>default</strong> lexer only on the touched tokens, in this case original tokens <code>foo</code> and <code>{</code>.</p>
  
  <p>Q: How can i save the lexer state so that tokenizing <code>foobar{</code> will give tokens with type <code>Token.Name.Class</code>?</p>
  
  <p>Having this feature would optimize syntax highlighting for large source files that suffered a change (user is typing text) right in the middle of the file for example. There seems no documented way of doing this and no information on how to do this using the default <a href="http://pygments.org/docs/lexers/" rel="noreferrer">pygments lexers</a>.</p>
  
  <p>Are there any other syntax highlighting systems that have support for this behavior ?</p>
  
  <p>EDIT:</p>
  
  <p>Regarding performance here is an example: <a href="http://tpcg.io/ESYjiF" rel="noreferrer">http://tpcg.io/ESYjiF</a></p>