Create a new data.xml index file, by combining the xml description
  files for various packages and collections.  ``root`` should be the
  path to a directory containing the package xml and zip files; and
  the collection xml files.  The ``root`` directory is expected to
  have the following subdirectories::

    root/
    packages/ .................. subdirectory for packages
      corpora/ ................. zip & xml files for corpora
      grammars/ ................ zip & xml files for grammars
      taggers/ ................. zip & xml files for taggers
      tokenizers/ .............. zip & xml files for tokenizers
      etc.
    collections/ ............... xml files for collections

  For each package, there should be two files: ``package.zip``
  (where *package* is the package name)
  which contains the package itself as a compressed zip file; and
  ``package.xml``, which is an xml description of the package.  The
  zipfile ``package.zip`` should expand to a single subdirectory
  named ``package/``.  The base filename ``package`` must match
  the identifier given in the package's xml file.

  For each collection, there should be a single file ``collection.zip``
  describing the collection, where *collection* is the name of the collection.

  All identifiers (for both packages and collections) must be unique.

def build_index(root, base_url):
  """
  Create a new data.xml index file, by combining the xml description
  files for various packages and collections.  ``root`` should be the
  path to a directory containing the package xml and zip files; and
  the collection xml files.  The ``root`` directory is expected to
  have the following subdirectories::

    root/
    packages/ .................. subdirectory for packages
      corpora/ ................. zip & xml files for corpora
      grammars/ ................ zip & xml files for grammars
      taggers/ ................. zip & xml files for taggers
      tokenizers/ .............. zip & xml files for tokenizers
      etc.
    collections/ ............... xml files for collections

  For each package, there should be two files: ``package.zip``
  (where *package* is the package name)
  which contains the package itself as a compressed zip file; and
  ``package.xml``, which is an xml description of the package.  The
  zipfile ``package.zip`` should expand to a single subdirectory
  named ``package/``.  The base filename ``package`` must match
  the identifier given in the package's xml file.

  For each collection, there should be a single file ``collection.zip``
  describing the collection, where *collection* is the name of the collection.

  All identifiers (for both packages and collections) must be unique.
  """
  # Find all packages.
  packages = []
  for pkg_xml, zf, subdir in _find_packages(os.path.join(root, 'packages')):
    zipstat = os.stat(zf.filename)
    url = '%s/%s/%s' % (base_url, subdir, os.path.split(zf.filename)[1])
    unzipped_size = sum(zf_info.file_size for zf_info in zf.infolist())

    # Fill in several fields of the package xml with calculated values.
    pkg_xml.set('unzipped_size', '%s' % unzipped_size)
    pkg_xml.set('size', '%s' % zipstat.st_size)
    pkg_xml.set('subdir', subdir)
    pkg_xml.set('url', url)

    # Record the package.
    packages.append(pkg_xml)

  # Find all collections
  collections = list(_find_collections(os.path.join(root, 'collections')))

  # Check that all UIDs are unique
  uids = set()
  for item in packages+collections:
    if item.get('id') in uids:
      raise ValueError('Duplicate UID: %s' % item.get('id'))
    uids.add(item.get('id'))

  # Put it all together
  top_elt = ElementTree.Element('polyglot_data')
  top_elt.append(ElementTree.Element('packages'))
  for package in packages: top_elt[0].append(package)
  top_elt.append(ElementTree.Element('collections'))
  for collection in collections: top_elt[1].append(collection)

  _indent_xml(top_elt)
  return top_elt