Process Metadata from stored (or "packed") state to functional state.

    Metadata can come be read from a file "packed" in various ways, 
    e.g. with a string representation of a dialect or coloring dictionary.  
    This function "unpacks" the stored metadata into useable Python
    objects.  It consists of a list of quasi-modular parts, one for each 
    type of recognized metadata.

    **Parameters**

        **metadata** : dictionary

            This argument is a dictionary whose keys are strings denoting
            different kinds of metadata (e.g. "names" or "formats") and whose 
            values are the metadata of that type.  The metadata dictionary is 
            modified IN-PLACE by this function.

        **items** : string or list of strings, optional

            The items arguments specifies which metadata keys are to be 
            processed.  E.g. of items = 'names,formats', then the "names" 
            metadata and "formats" metadata will be processed, but no others.
            Note however, that sometimes, the processing of one type of metadata 
            requires that another be processed first, e.g. "dialect" must 
            processed into an actual CSV.dialect object before "names"  is 
            processed.  (The processed of "names" metadata involves splitting 
            the names metadata string into a list, using the delimiter.  This 
            delimiter is part of the dialect object.)   In these cases, if you 
            call processmetadata on one item before its requirements are 
            processed, nothing will happen.

        **comments** : single-character string, optional

            The comments character is used to process many pieces of metadata, 
            e.g. it is striped of the left side of names and formats strings
            before splitting on delimiter.

        **verbosity** : integer, optional

            Determines the level of verbosity in the printout of messages
            during the running of the procedure. 

   **Returns**
   
       Nothing.

def processmetadata(metadata, items=None, comments=None, delimiter_regex=None,
                    ncols=None, verbosity=DEFAULT_VERBOSITY):
    """
    Process Metadata from stored (or "packed") state to functional state.

    Metadata can come be read from a file "packed" in various ways, 
    e.g. with a string representation of a dialect or coloring dictionary.  
    This function "unpacks" the stored metadata into useable Python
    objects.  It consists of a list of quasi-modular parts, one for each 
    type of recognized metadata.

    **Parameters**

        **metadata** : dictionary

            This argument is a dictionary whose keys are strings denoting
            different kinds of metadata (e.g. "names" or "formats") and whose 
            values are the metadata of that type.  The metadata dictionary is 
            modified IN-PLACE by this function.

        **items** : string or list of strings, optional

            The items arguments specifies which metadata keys are to be 
            processed.  E.g. of items = 'names,formats', then the "names" 
            metadata and "formats" metadata will be processed, but no others.
            Note however, that sometimes, the processing of one type of metadata 
            requires that another be processed first, e.g. "dialect" must 
            processed into an actual CSV.dialect object before "names"  is 
            processed.  (The processed of "names" metadata involves splitting 
            the names metadata string into a list, using the delimiter.  This 
            delimiter is part of the dialect object.)   In these cases, if you 
            call processmetadata on one item before its requirements are 
            processed, nothing will happen.

        **comments** : single-character string, optional

            The comments character is used to process many pieces of metadata, 
            e.g. it is striped of the left side of names and formats strings
            before splitting on delimiter.

        **verbosity** : integer, optional

            Determines the level of verbosity in the printout of messages
            during the running of the procedure. 

   **Returns**
   
       Nothing.

    """
    items = items.split(',') if isinstance(items,str) else items

    if comments is None:
        if 'comments' in metadata.keys():
            comments = metadata['comments']
        else:
            comments = '#'
            if verbosity > 8:
                print 'processing metadata with comments char = #'
    else:
        if (('comments' in metadata.keys()) and 
            (comments != metadata['comments']) and (verbosity > 8)):
            print 'comments character specified to process metadata (', repr(comments) ,') is different from comments charcater set in metadata dictionary (', repr(metadata['comments']) , ').'
    
    if not items:
        for k in metadata.keys():
            if is_string_like(metadata[k]):
                metadata[k] = '\n'.join([x.lstrip(comments) 
                                         for x in metadata[k].split('\n') ])
    
    if not items or 'dialect' in items:
        if 'dialect' in metadata.keys():
            if isinstance(metadata['dialect'],str):
                D = dialectfromstring(metadata['dialect'].lstrip(comments))
                if D:
                    metadata['dialect'] = D
                    if (verbosity > 8):
                        print 'processed dialect from string'
                        
                else:
                    if (verbosity > 8):
                        print '''Dialect failed to be converted properly from 
                                 string representation in metadata.'''

            if 'delimiter' in dir(metadata['dialect']):
                for a in dir(metadata['dialect']):
                    if not a.startswith('_') and a in metadata.keys():
                        setattr(metadata['dialect'],a, metadata[a])
                        if ((verbosity > 2 and a == 'delimiter') or 
                            (verbosity >= 8)):
                            print 'Setting dialect attribute', a, 'to equal specified value:', repr(metadata[a])
                    elif not a.startswith('_') and a not in metadata.keys():
                        metadata[a] = getattr(metadata['dialect'], a)
                        if ((verbosity > 2 and a == 'delimiter') or (verbosity >= 8)):
                            print 'Setting metadata attribute from dialect', a , 'to equal specified value:', repr(metadata[a])

    if (not items or 'names' in items) and ('names' in metadata.keys()): 
        if is_string_like(metadata['names']):

            if delimiter_regex:
                metadata['names'] = delimiter_regex.split(metadata['names'])
            elif (('dialect' in metadata.keys()) and 
                  ('delimiter' in dir(metadata['dialect']))):
                d = metadata['dialect']
                n = metadata['names']
                metadata['names'] = list(csv.reader([n.lstrip(comments)],      
                                                     dialect=d))[0]
                if (verbosity > 8):
                    print '... splitting "names" metadata from string with delimiter', repr(d.delimiter), '. Resulting names:', metadata['names']

    if (not items or 'formats' in items) and 'formats' in metadata.keys(): 
        if is_string_like(metadata['formats']):   
            if delimiter_regex:
                metadata['formats'] = delimiter_regex.split(metadata['formats']) 
            elif (('dialect' in metadata.keys()) and ('delimiter' in dir(metadata['dialect']))):
                d = metadata['dialect']
                n = metadata['formats']
                metadata['formats'] = list(csv.reader([n.lstrip(comments)],  
                                           dialect=d))[0]

                if (verbosity > 8):
                    print '... splitting "formats" metadata from string with delimiter', repr(d.delimiter), '. Resulting names:', metadata['formats']       
 
        if ncols:
            metadata['formats'] = postprocessformats(metadata['formats'],ncols)


    if (not items or 'types' in items) and 'types' in metadata.keys(): 
        if is_string_like(metadata['types']):      
            if delimiter_regex:
                metadata['types'] = delimiter_regex.split(metadata['types']) 
            elif (('dialect' in metadata.keys()) and 
                  ('delimiter' in dir(metadata['dialect']))):
                d = metadata['dialect']
                n = metadata['types']
                metadata['types'] = list(csv.reader([n.lstrip(comments)],  
                                         dialect=d))[0]
                if (verbosity > 8):
                    print '... splitting "types" metadata from string with delimiter', repr(d.delimiter), '. Resulting names:', metadata['types']      
        
        if ncols:
            metadata['types'] = postprocessformats(metadata['types'],ncols)

    if (not items or 'coloring' in items) and ('coloring' in metadata.keys()):
        if is_string_like(metadata['coloring']):
            C = coloringfromstring(metadata['coloring'].lstrip(comments))
            if C:
                metadata['coloring'] = C
                if (verbosity > 8):  
                    print '... processed coloring from string'
            else:
                if verbosity > 1:
                    print 'Coloring failed to be converted properly from string representation in metadata ; removing coloring data from active metadata (putting it in "loaded_coloring").'
                    metadata['loaded_coloring'] = metadata['coloring']
                    metadata.pop('coloring')

    if (not items or 'headerlines' in items):
        if 'headerlines' in metadata.keys():
            if isinstance(metadata['headerlines'], str):
                try:
                    h = metadata['headerlines']
                    metadata['headerlines'] = int(h.lstrip(comments))
                except (ValueError,TypeError):
                    if verbosity > 6:
                        print 'headerlines metadata failed to convert to an integer.'
                else:
                    pass
                        

            if isinstance(metadata['headerlines'], int):
                if 'metametadata' in metadata.keys():
                    h= metadata['headerlines']
                    mmd = metadata['metametadata']
                    metadata['headerlines'] = max(h, 1 + max([v 
                                               if isinstance(v, int) else max(v) 
                                               for v in mmd.values()]))
                    if ((metadata['headerlines'] > h) and (verbosity > 8)):
                        print 'Resetting headerlines from', h, 'to', metadata['headerlines'], 'because of line number indications from metametadata.'