original_docstring: Yield (type, value, pos, closed) tuples for a string of text containing
         tags, i.e.
  
         <c.tagtype>tag value</c>
  
         pos is a value between 0 and 1 giving the position of the tag in the
         text with tags stripped, rounded to 5 decimal places
      
docstring: Yield (type, value, pos, closed) tuples for a string of text containing
  tags, i.e.
  
  tag value
  
  pos is a value between 0 and 1 giving the position of the tag in the
  text with tags stripped, rounded to 5 decimal places
docstring_tokens: Yield ( type value pos closed ) tuples for a string of text containing tags i . e . tag value pos is a value between 0 and 1 giving the position of the tag in the text with tags stripped rounded to 5 decimal places
code: def parse_tags(text):
      """Yield (type, value, pos, closed) tuples for a string of text containing
         tags, i.e.
  
         <c.tagtype>tag value</c>
  
         pos is a value between 0 and 1 giving the position of the tag in the
         text with tags stripped, rounded to 5 decimal places
      """
  
      text = strip_voice_spans(text)
  
      stripped_len = len(strip_tags(text))
  
      for match in TAG_RE.finditer(text):
          # estimate position within the (stripped) string
          previous_text = strip_tags(text[:match.start()])
          pos = round(float(len(previous_text)) / stripped_len, 5)
  
          tag_type, value, closed = match.groups()
  
          yield tag_type, value, pos, bool(closed)
code_tokens: def parse_tags ( text ) : text = strip_voice_spans ( text ) stripped_len = len ( strip_tags ( text ) ) for match in TAG_RE . finditer ( text ) : previous_text = strip_tags ( text [ : match . start ( ) ] ) pos = round ( float ( len ( previous_text ) ) / stripped_len , 5 ) tag_type , value , closed = match . groups ( ) yield tag_type , value , pos , bool ( closed )
short_docstring: Yield (type, value, pos, closed) tuples for a string of text containing
  tags, i.e.
short_docstring_tokens: Yield ( type value pos closed ) tuples for a string of text containing tags i . e .
comment: """Yield (type, value, pos, closed) tuples for a string of text containing
         tags, i.e.
  
         <c.tagtype>tag value</c>
  
         pos is a value between 0 and 1 giving the position of the tag in the
         text with tags stripped, rounded to 5 decimal places
      """$SEP$# estimate position within the (stripped) string