the present invention provides a method and apparatus for the lexical analysis of computer source code . the lexical analyzer is dynamically configured at runtime to recognize a one or more reserved words or operators . thus , the analyzer has the ability to interact with multiple languages . in one or more embodiments of the present invention , the analyzer is instantiated by a host application , for example , the parser of a compiler . the host application adds a list of tokens to the analyzer that must be recognized . these tokens comprise at least a subset of the reserved words and operators of the computer language . in one embodiment , the host application then queries the analyzer for the next token in the source code . in another embodiment , tokens are added during the query phase as needed . in a separate embodiment , tokens are dynamically removed from the analyzer as the needs of the host application change .