original_docstring: Process a transcript file, creating Colloquialism instances as needed,
         and returning a list of Tag instances for review.
         returns a tuple: (tags, errors)
  
         get_colloquialism should return Colloquialism instance
         get_tag should return a Tag instance
      
docstring: Process a transcript file, creating Colloquialism instances as needed,
  and returning a list of Tag instances for review.
  returns a tuple: (tags, errors)
  
  get_colloquialism should return Colloquialism instance
  get_tag should return a Tag instance
docstring_tokens: Process a transcript file creating Colloquialism instances as needed and returning a list of Tag instances for review . returns a tuple : ( tags errors ) get_colloquialism should return Colloquialism instance get_tag should return a Tag instance
code: def parse_transcript(transcript_file, language, valid_types, get_tag,
                       get_colloquialism):
      """Process a transcript file, creating Colloquialism instances as needed,
         and returning a list of Tag instances for review.
         returns a tuple: (tags, errors)
  
         get_colloquialism should return Colloquialism instance
         get_tag should return a Tag instance
      """
  
      webvttfile = get_webvttfile(transcript_file)
  
      errors = []
      tags = []
  
      unclosed = None
  
      def create_tag(tag_type, value, start, start_exact):
          # create the colloquialism entry if it doesn't exist.
          colloquialism = get_colloquialism(
              value=value, language=language, type=tag_type)
  
          start_exact = start + timedelta(milliseconds=int(length * pos))
          tag = get_tag(
              start=start, start_exact=start_exact,
              colloquialism=colloquialism)
          tags.append(tag)
  
      for entry in webvttfile:
          # convert WebVTTTime to timedelta
          start = timedelta(milliseconds=entry.start.ordinal)
          length = entry.end.ordinal - entry.start.ordinal
  
          # if there's a leftover unclosed tag, check if it's closed at the
          # start of the next entry
          if unclosed:
              # get text preceding any other tags
              pre_tag_text = TAG_RE.split(entry.text)[0]
  
              # look for a closing tag, and capture any text before it
              match = TAG_CLOSE_RE.match(pre_tag_text)
  
              if match:
                  # append the extra content to the unclosed tag
                  unclosed['value'] = '%s %s' % (
                      unclosed['value'], match.groups()[0])
  
                  create_tag(**unclosed)
                  unclosed = None
  
          for tag_type, value, pos, closed in parse_tags(entry.text):
              if tag_type not in valid_types:
                  # TODO - log error
                  errors.append('Invalid tag type at %s - %s' % (
                      entry.start, tag_type))
                  continue
  
              start_exact = start + timedelta(milliseconds=int(length * pos))
              tag_details = {
                  'tag_type': tag_type,
                  'value': value,
                  'start': start,
                  'start_exact': start_exact,
              }
  
              if closed:
                  create_tag(**tag_details)
                  unclosed = None
              else:
                  # save the details for the next iteration of the outer loop
                  unclosed = tag_details
  
      return tags, errors
code_tokens: def parse_transcript ( transcript_file , language , valid_types , get_tag , get_colloquialism ) : webvttfile = get_webvttfile ( transcript_file ) errors = [ ] tags = [ ] unclosed = None def create_tag ( tag_type , value , start , start_exact ) : colloquialism = get_colloquialism ( value = value , language = language , type = tag_type ) start_exact = start + timedelta ( milliseconds = int ( length * pos ) ) tag = get_tag ( start = start , start_exact = start_exact , colloquialism = colloquialism ) tags . append ( tag ) for entry in webvttfile : start = timedelta ( milliseconds = entry . start . ordinal ) length = entry . end . ordinal - entry . start . ordinal if unclosed : pre_tag_text = TAG_RE . split ( entry . text ) [ 0 ] match = TAG_CLOSE_RE . match ( pre_tag_text ) if match : unclosed [ 'value' ] = '%s %s' % ( unclosed [ 'value' ] , match . groups ( ) [ 0 ] ) create_tag ( ** unclosed ) unclosed = None for tag_type , value , pos , closed in parse_tags ( entry . text ) : if tag_type not in valid_types : errors . append ( 'Invalid tag type at %s - %s' % ( entry . start , tag_type ) ) continue start_exact = start + timedelta ( milliseconds = int ( length * pos ) ) tag_details = { 'tag_type' : tag_type , 'value' : value , 'start' : start , 'start_exact' : start_exact , } if closed : create_tag ( ** tag_details ) unclosed = None else : unclosed = tag_details return tags , errors
short_docstring: Process a transcript file, creating Colloquialism instances as needed,
  and returning a list of Tag instances for review.
short_docstring_tokens: Process a transcript file creating Colloquialism instances as needed and returning a list of Tag instances for review .
comment: """Process a transcript file, creating Colloquialism instances as needed,
         and returning a list of Tag instances for review.
         returns a tuple: (tags, errors)
  
         get_colloquialism should return Colloquialism instance
         get_tag should return a Tag instance
      """$SEP$# create the colloquialism entry if it doesn't exist.$SEP$# convert WebVTTTime to timedelta$SEP$# if there's a leftover unclosed tag, check if it's closed at the$SEP$# start of the next entry$SEP$# get text preceding any other tags$SEP$# look for a closing tag, and capture any text before it$SEP$# append the extra content to the unclosed tag$SEP$# TODO - log error$SEP$# save the details for the next iteration of the outer loop