title: Poor man's "lexer" for C#
body: <p>I'm trying to write a very simple parser in C#.</p>
  
  <p>I need a lexer -- something that lets me associate regular expressions with tokens, so it reads in regexs and gives me back symbols.</p>
  
  <p>It seems like I ought to be able to use Regex to do the actual heavy lifting, but I can't see an easy way to do it. For one thing, Regex only seems to work on strings, not streams (why is that!?!?).</p>
  
  <p>Basically, I want an implementation of the following interface:</p>
  
  <pre><code>interface ILexer : IDisposable
  {
      /// &lt;summary&gt;
      /// Return true if there are more tokens to read
      /// &lt;/summary&gt;
      bool HasMoreTokens { get; }
      /// &lt;summary&gt;
      /// The actual contents that matched the token
      /// &lt;/summary&gt;
      string TokenContents { get; }
      /// &lt;summary&gt;
      /// The particular token in "tokenDefinitions" that was matched (e.g. "STRING", "NUMBER", "OPEN PARENS", "CLOSE PARENS"
      /// &lt;/summary&gt;
      object Token { get; }
      /// &lt;summary&gt;
      /// Move to the next token
      /// &lt;/summary&gt;
      void Next();
  }
  
  interface ILexerFactory
  {
      /// &lt;summary&gt;
      /// Create a Lexer for converting a stream of characters into tokens
      /// &lt;/summary&gt;
      /// &lt;param name="reader"&gt;TextReader that supplies the underlying stream&lt;/param&gt;
      /// &lt;param name="tokenDefinitions"&gt;A dictionary from regular expressions to their "token identifers"&lt;/param&gt;
      /// &lt;returns&gt;The lexer&lt;/returns&gt;
      ILexer CreateLexer(TextReader reader, IDictionary&lt;string, object&gt; tokenDefinitions);
  }
  </code></pre>
  
  <p>So, pluz send the codz...<br>
  No, seriously, I am about to start writing an implementation of the above interface yet I find it hard to believe that there isn't some simple way of doing this in .NET (2.0) already.</p>
  
  <p>So, any suggestions for a simple way to do the above? (Also, I don't want any "code generators".  Performance is not important for this thing and I don't want to introduce any complexity into the build process.)</p>