title: PLY - return multiple tokens
body: <p>AFAIK the technique for lexing Python source code is:</p>
  
  <ul>
  <li>When current line's indentation level is less than previous line's, produce DEDENT. Produce multiple DEDENTs if it is closing multiple INDENTs.</li>
  <li>When end of input is reached, produce DEDENT(s) if there's unclosed INDENT(s).</li>
  </ul>
  
  <p>Now, using PLY:</p>
  
  <ul>
  <li>How do I return multiple tokens from a t_definition?</li>
  <li>How do I make a t_definition that's called when EOF is reached? Simple <code>\Z</code> doesn't work -- PLY complains that it matches empty string.</li>
  </ul>