DYNAMIC LEXICON
In a system for content management, a dynamic lexicon allows dictionary and lexical data at NLP (natural language processing) engines at remote sites to stay current with table data at a central location without suffering the time loss involved in computing new tables at the remote sites, of computing new tables at the central site and distributing them. As new terms are added to the dictionary, each item is assigned a new token identifier. A first step involves downloading extensions to the table data in real time whenever a new word or expression is encountered. A second step involves periodically updating the table data in real time with recomputed data transmitted in compact data files from the central location. Content items in the local archive are re-indexed based on the updated table data. Maintaining tokens across generations of tables allows documents in different languages to be associated without requiring translation (Figure 1, 100, 112, 101, 105, 107, 111).