original_docstring: Merges the 'new_schema_entry' into the 'old_schema_entry' and return
          a merged schema entry. Recursively merges in sub-fields as well.
  
          Returns the merged schema_entry. This method assumes that both
          'old_schema_entry' and 'new_schema_entry' can be modified in place and
          returned as the new schema_entry. Returns None if the field should
          be removed from the schema due to internal consistency errors.
  
          'base_path' is the string representing the current path within the
          nested record that leads to this specific entry. This is used during
          error logging.
  
          An Exception is thrown if an unexpected programming error is detected.
          The calling routine should stop processing the file.
          
docstring: Merges the 'new_schema_entry' into the 'old_schema_entry' and return
  a merged schema entry. Recursively merges in sub-fields as well.
  
  Returns the merged schema_entry. This method assumes that both
  'old_schema_entry' and 'new_schema_entry' can be modified in place and
  returned as the new schema_entry. Returns None if the field should
  be removed from the schema due to internal consistency errors.
  
  'base_path' is the string representing the current path within the
  nested record that leads to this specific entry. This is used during
  error logging.
  
  An Exception is thrown if an unexpected programming error is detected.
  The calling routine should stop processing the file.
docstring_tokens: Merges the ' new_schema_entry ' into the ' old_schema_entry ' and return a merged schema entry . Recursively merges in sub - fields as well . Returns the merged schema_entry . This method assumes that both ' old_schema_entry ' and ' new_schema_entry ' can be modified in place and returned as the new schema_entry . Returns None if the field should be removed from the schema due to internal consistency errors . ' base_path ' is the string representing the current path within the nested record that leads to this specific entry . This is used during error logging . An Exception is thrown if an unexpected programming error is detected . The calling routine should stop processing the file .
code: def merge_schema_entry(
          self,
          old_schema_entry,
          new_schema_entry,
          base_path=None,
      ):
          """Merges the 'new_schema_entry' into the 'old_schema_entry' and return
          a merged schema entry. Recursively merges in sub-fields as well.
  
          Returns the merged schema_entry. This method assumes that both
          'old_schema_entry' and 'new_schema_entry' can be modified in place and
          returned as the new schema_entry. Returns None if the field should
          be removed from the schema due to internal consistency errors.
  
          'base_path' is the string representing the current path within the
          nested record that leads to this specific entry. This is used during
          error logging.
  
          An Exception is thrown if an unexpected programming error is detected.
          The calling routine should stop processing the file.
          """
          if not old_schema_entry:
              return new_schema_entry
  
          # If the new schema is None, return immediately.
          if not new_schema_entry:
              return new_schema_entry
  
          # If a field value is missing, permanently set 'filled' to False.
          if not new_schema_entry['filled'] or not old_schema_entry['filled']:
              old_schema_entry['filled'] = False
              new_schema_entry['filled'] = False
  
          old_status = old_schema_entry['status']
          new_status = new_schema_entry['status']
  
          # new 'soft' does not clobber old 'hard'
          if old_status == 'hard' and new_status == 'soft':
              mode = self.merge_mode(old_schema_entry,
                                     new_schema_entry,
                                     base_path)
              if mode is None:
                  return None
              old_schema_entry['info']['mode'] = mode
              return old_schema_entry
  
          # new 'hard' clobbers old 'soft'
          if old_status == 'soft' and new_status == 'hard':
              mode = self.merge_mode(old_schema_entry,
                                     new_schema_entry,
                                     base_path)
              if mode is None:
                  return None
              new_schema_entry['info']['mode'] = mode
              return new_schema_entry
  
          # Verify that it's soft->soft or hard->hard
          if old_status != new_status:
              raise Exception(
                  f'Unexpected schema_entry type, this should never happen: '
                  f'old ({old_status}); new ({new_status})'
              )
  
          old_info = old_schema_entry['info']
          old_name = old_info['name']
          old_type = old_info['type']
          old_mode = old_info['mode']
          new_info = new_schema_entry['info']
          new_name = new_info['name']
          new_type = new_info['type']
          new_mode = new_info['mode']
  
          full_old_name = json_full_path(base_path, old_name)
          full_new_name = json_full_path(base_path, new_name)
  
          # Defensive check, names should always be the same.
          if old_name != new_name:
              if old_name.lower() != new_name.lower():
                  raise Exception(
                      'Unexpected difference in name, should never happen:'
                      f' old_name ({full_old_name}) != new_name ({full_new_name})'
                  )
              else:
                  # preserve old name if case is different
                  new_info['name'] = old_info['name']
  
          # Recursively merge in the subfields of a RECORD, allowing
          # NULLABLE to become REPEATED (because 'bq load' allows it).
          if old_type == 'RECORD' and new_type == 'RECORD':
              # Allow NULLABLE RECORD to be upgraded to REPEATED RECORD because
              # 'bq load' allows it.
              if old_mode == 'NULLABLE' and new_mode == 'REPEATED':
                  old_info['mode'] = 'REPEATED'
                  self.log_error(
                      f'Converting schema for "{full_old_name}" from '
                      'NULLABLE RECORD into REPEATED RECORD'
                  )
              elif old_mode == 'REPEATED' and new_mode == 'NULLABLE':
                  # TODO: Maybe remove this warning output. It was helpful during
                  # development, but maybe it's just natural.
                  self.log_error(
                      f'Leaving schema for "{full_old_name}" as REPEATED RECORD'
                  )
  
              # RECORD type needs a recursive merging of sub-fields. We merge into
              # the 'old_schema_entry' which assumes that the 'old_schema_entry'
              # can be modified in situ.
              old_fields = old_info['fields']
              new_fields = new_info['fields']
              for key, new_entry in new_fields.items():
                  old_entry = old_fields.get(key)
                  new_base_path = json_full_path(base_path, old_name)
                  old_fields[key] = self.merge_schema_entry(
                      old_schema_entry=old_entry,
                      new_schema_entry=new_entry,
                      base_path=new_base_path,
                  )
              return old_schema_entry
  
          new_mode = self.merge_mode(old_schema_entry,
                                     new_schema_entry,
                                     base_path)
          if new_mode is None:
              return None
          new_schema_entry['info']['mode'] = new_mode
  
          # For all other types...
          if old_type != new_type:
              # Check that the converted types are compatible.
              candidate_type = convert_type(old_type, new_type)
              if not candidate_type:
                  self.log_error(
                      f'Ignoring field with mismatched type: '
                      f'old=({old_status},{full_old_name},{old_mode},{old_type});'
                      f' new=({new_status},{full_new_name},{new_mode},{new_type})'
                  )
                  return None
  
              new_info['type'] = candidate_type
          return new_schema_entry
code_tokens: def merge_schema_entry ( self , old_schema_entry , new_schema_entry , base_path = None , ) : if not old_schema_entry : return new_schema_entry if not new_schema_entry : return new_schema_entry if not new_schema_entry [ 'filled' ] or not old_schema_entry [ 'filled' ] : old_schema_entry [ 'filled' ] = False new_schema_entry [ 'filled' ] = False old_status = old_schema_entry [ 'status' ] new_status = new_schema_entry [ 'status' ] if old_status == 'hard' and new_status == 'soft' : mode = self . merge_mode ( old_schema_entry , new_schema_entry , base_path ) if mode is None : return None old_schema_entry [ 'info' ] [ 'mode' ] = mode return old_schema_entry if old_status == 'soft' and new_status == 'hard' : mode = self . merge_mode ( old_schema_entry , new_schema_entry , base_path ) if mode is None : return None new_schema_entry [ 'info' ] [ 'mode' ] = mode return new_schema_entry if old_status != new_status : raise Exception ( f'Unexpected schema_entry type, this should never happen: ' f'old ({old_status}); new ({new_status})' ) old_info = old_schema_entry [ 'info' ] old_name = old_info [ 'name' ] old_type = old_info [ 'type' ] old_mode = old_info [ 'mode' ] new_info = new_schema_entry [ 'info' ] new_name = new_info [ 'name' ] new_type = new_info [ 'type' ] new_mode = new_info [ 'mode' ] full_old_name = json_full_path ( base_path , old_name ) full_new_name = json_full_path ( base_path , new_name ) if old_name != new_name : if old_name . lower ( ) != new_name . lower ( ) : raise Exception ( 'Unexpected difference in name, should never happen:' f' old_name ({full_old_name}) != new_name ({full_new_name})' ) else : new_info [ 'name' ] = old_info [ 'name' ] if old_type == 'RECORD' and new_type == 'RECORD' : if old_mode == 'NULLABLE' and new_mode == 'REPEATED' : old_info [ 'mode' ] = 'REPEATED' self . log_error ( f'Converting schema for "{full_old_name}" from ' 'NULLABLE RECORD into REPEATED RECORD' ) elif old_mode == 'REPEATED' and new_mode == 'NULLABLE' : self . log_error ( f'Leaving schema for "{full_old_name}" as REPEATED RECORD' ) old_fields = old_info [ 'fields' ] new_fields = new_info [ 'fields' ] for key , new_entry in new_fields . items ( ) : old_entry = old_fields . get ( key ) new_base_path = json_full_path ( base_path , old_name ) old_fields [ key ] = self . merge_schema_entry ( old_schema_entry = old_entry , new_schema_entry = new_entry , base_path = new_base_path , ) return old_schema_entry new_mode = self . merge_mode ( old_schema_entry , new_schema_entry , base_path ) if new_mode is None : return None new_schema_entry [ 'info' ] [ 'mode' ] = new_mode if old_type != new_type : candidate_type = convert_type ( old_type , new_type ) if not candidate_type : self . log_error ( f'Ignoring field with mismatched type: ' f'old=({old_status},{full_old_name},{old_mode},{old_type});' f' new=({new_status},{full_new_name},{new_mode},{new_type})' ) return None new_info [ 'type' ] = candidate_type return new_schema_entry
short_docstring: Merges the 'new_schema_entry' into the 'old_schema_entry' and return
  a merged schema entry.
short_docstring_tokens: Merges the ' new_schema_entry ' into the ' old_schema_entry ' and return a merged schema entry .
comment: """Merges the 'new_schema_entry' into the 'old_schema_entry' and return
          a merged schema entry. Recursively merges in sub-fields as well.
  
          Returns the merged schema_entry. This method assumes that both
          'old_schema_entry' and 'new_schema_entry' can be modified in place and
          returned as the new schema_entry. Returns None if the field should
          be removed from the schema due to internal consistency errors.
  
          'base_path' is the string representing the current path within the
          nested record that leads to this specific entry. This is used during
          error logging.
  
          An Exception is thrown if an unexpected programming error is detected.
          The calling routine should stop processing the file.
          """$SEP$# If the new schema is None, return immediately.$SEP$# If a field value is missing, permanently set 'filled' to False.$SEP$# new 'soft' does not clobber old 'hard'$SEP$# new 'hard' clobbers old 'soft'$SEP$# Verify that it's soft->soft or hard->hard$SEP$# Defensive check, names should always be the same.$SEP$# preserve old name if case is different$SEP$# Recursively merge in the subfields of a RECORD, allowing$SEP$# NULLABLE to become REPEATED (because 'bq load' allows it).$SEP$# Allow NULLABLE RECORD to be upgraded to REPEATED RECORD because$SEP$# 'bq load' allows it.$SEP$# TODO: Maybe remove this warning output. It was helpful during$SEP$# development, but maybe it's just natural.$SEP$# RECORD type needs a recursive merging of sub-fields. We merge into$SEP$# the 'old_schema_entry' which assumes that the 'old_schema_entry'$SEP$# can be modified in situ.$SEP$# For all other types...$SEP$# Check that the converted types are compatible.