original_docstring: Loop through each element of 'input_data' and deduce the
          BigQuery schema. The schema is returned as a recursive map that contains
          both the database schema and some additional metadata about each entry.
          It has the following form:
  
            schema_map := {
              key: schema_entry
            }
  
          The 'key' is the name of the table column.
  
            schema_entry := {
              'status': 'hard | soft',
              'filled': True | False,
              'info': {
                'name': key,
                'type': 'STRING | TIMESTAMP | DATE | TIME
                        | FLOAT | INTEGER | BOOLEAN | RECORD'
                'mode': 'NULLABLE | REPEATED',
                'fields': schema_map
              }
            }
  
          The status of 'hard' or 'soft' refers the reliability of the type
          inference that we made for a particular column element value. If the
          element value is a 'null', we assume that it's a 'soft' STRING. If the
          element value is an empty [] or {}, we assume that the element type is a
          'soft' REPEATED STRING or a 'soft' NULLABLE RECORD, respectively. When
          we come across a subsequent entry with non-null or non-empty values, we
          are able infer the type definitively, and we change the status to
          'hard'. The status can transition from 'soft' to 'hard' but not the
          reverse.
  
          The 'filled' entry indicates whether all input data records contained
          the given field. If the --infer_mode flag is given, the 'filled' entry
          is used to convert a NULLABLE schema entry to a REQUIRED schema entry or
          to relax an existing field in schema_map from REQUIRED to NULLABLE.
  
          The function returns a tuple of 2 things:
            * an OrderedDict which is sorted by the 'key' of the column name
            * a list of possible errors containing a map of 'line' and 'msg'
  
          An Exception is thrown in the lower-level calls only for programming
          errors, not for data validation errors. Therefore each line in the input
          file is processed without a try-except block and any exception will be
          allowed to escape to the calling routine.
          
docstring: Loop through each element of 'input_data' and deduce the
  BigQuery schema. The schema is returned as a recursive map that contains
  both the database schema and some additional metadata about each entry.
  It has the following form.
  
  
  
  The 'key' is the name of the table column.
  
  
  
  The status of 'hard' or 'soft' refers the reliability of the type
  inference that we made for a particular column element value.
  
  The 'filled' entry indicates whether all input data records contained
  the given field. If the --infer_mode flag is given, the 'filled' entry
  is used to convert a NULLABLE schema entry to a REQUIRED schema entry or
  to relax an existing field in schema_map from REQUIRED to NULLABLE.
  
  
  
  An Exception is thrown in the lower-level calls only for programming
  errors, not for data validation errors. Therefore each line in the input
  file is processed without a try-except block and any exception will be
  allowed to escape to the calling routine.
docstring_tokens: Loop through each element of ' input_data ' and deduce the BigQuery schema . The schema is returned as a recursive map that contains both the database schema and some additional metadata about each entry . It has the following form . The ' key ' is the name of the table column . The status of ' hard ' or ' soft ' refers the reliability of the type inference that we made for a particular column element value . The ' filled ' entry indicates whether all input data records contained the given field . If the -- infer_mode flag is given the ' filled ' entry is used to convert a NULLABLE schema entry to a REQUIRED schema entry or to relax an existing field in schema_map from REQUIRED to NULLABLE . An Exception is thrown in the lower - level calls only for programming errors not for data validation errors . Therefore each line in the input file is processed without a try - except block and any exception will be allowed to escape to the calling routine .
code: def deduce_schema(self, input_data, *, schema_map=None):
          """Loop through each element of 'input_data' and deduce the
          BigQuery schema. The schema is returned as a recursive map that contains
          both the database schema and some additional metadata about each entry.
          It has the following form:
  
            schema_map := {
              key: schema_entry
            }
  
          The 'key' is the name of the table column.
  
            schema_entry := {
              'status': 'hard | soft',
              'filled': True | False,
              'info': {
                'name': key,
                'type': 'STRING | TIMESTAMP | DATE | TIME
                        | FLOAT | INTEGER | BOOLEAN | RECORD'
                'mode': 'NULLABLE | REPEATED',
                'fields': schema_map
              }
            }
  
          The status of 'hard' or 'soft' refers the reliability of the type
          inference that we made for a particular column element value. If the
          element value is a 'null', we assume that it's a 'soft' STRING. If the
          element value is an empty [] or {}, we assume that the element type is a
          'soft' REPEATED STRING or a 'soft' NULLABLE RECORD, respectively. When
          we come across a subsequent entry with non-null or non-empty values, we
          are able infer the type definitively, and we change the status to
          'hard'. The status can transition from 'soft' to 'hard' but not the
          reverse.
  
          The 'filled' entry indicates whether all input data records contained
          the given field. If the --infer_mode flag is given, the 'filled' entry
          is used to convert a NULLABLE schema entry to a REQUIRED schema entry or
          to relax an existing field in schema_map from REQUIRED to NULLABLE.
  
          The function returns a tuple of 2 things:
            * an OrderedDict which is sorted by the 'key' of the column name
            * a list of possible errors containing a map of 'line' and 'msg'
  
          An Exception is thrown in the lower-level calls only for programming
          errors, not for data validation errors. Therefore each line in the input
          file is processed without a try-except block and any exception will be
          allowed to escape to the calling routine.
          """
  
          if self.input_format == 'csv':
              reader = csv.DictReader(input_data)
          elif self.input_format == 'json' or self.input_format is None:
              reader = json_reader(input_data)
          elif self.input_format == 'dict':
              reader = input_data
          else:
              raise Exception(f"Unknown input_format '{self.input_format}'")
  
          if schema_map is None:
              schema_map = OrderedDict()
  
          try:
              for json_object in reader:
  
                  # Print a progress message periodically.
                  self.line_number += 1
                  if self.line_number % self.debugging_interval == 0:
                      logging.info(f'Processing line {self.line_number}')
  
                  # Deduce the schema from this given data record.
                  if isinstance(json_object, dict):
                      self.deduce_schema_for_record(
                          json_object=json_object,
                          schema_map=schema_map,
                      )
                  elif isinstance(json_object, Exception):
                      self.log_error(
                          f'Record could not be parsed: Exception: {json_object}'
                      )
                      if not self.ignore_invalid_lines:
                          raise json_object
                  else:
                      self.log_error(
                          'Record should be a JSON Object '
                          f'but was a {type(json_object)}'
                      )
                      if not self.ignore_invalid_lines:
                          raise Exception(f'Record must be a JSON Object '
                                          f'but was a {type(json_object)}')
          finally:
              logging.info(f'Processed {self.line_number} lines')
  
          return schema_map, self.error_logs
code_tokens: def deduce_schema ( self , input_data , * , schema_map = None ) : if self . input_format == 'csv' : reader = csv . DictReader ( input_data ) elif self . input_format == 'json' or self . input_format is None : reader = json_reader ( input_data ) elif self . input_format == 'dict' : reader = input_data else : raise Exception ( f"Unknown input_format '{self.input_format}'" ) if schema_map is None : schema_map = OrderedDict ( ) try : for json_object in reader : self . line_number += 1 if self . line_number % self . debugging_interval == 0 : logging . info ( f'Processing line {self.line_number}' ) if isinstance ( json_object , dict ) : self . deduce_schema_for_record ( json_object = json_object , schema_map = schema_map , ) elif isinstance ( json_object , Exception ) : self . log_error ( f'Record could not be parsed: Exception: {json_object}' ) if not self . ignore_invalid_lines : raise json_object else : self . log_error ( 'Record should be a JSON Object ' f'but was a {type(json_object)}' ) if not self . ignore_invalid_lines : raise Exception ( f'Record must be a JSON Object ' f'but was a {type(json_object)}' ) finally : logging . info ( f'Processed {self.line_number} lines' ) return schema_map , self . error_logs
short_docstring: Loop through each element of 'input_data' and deduce the
  BigQuery schema.
short_docstring_tokens: Loop through each element of ' input_data ' and deduce the BigQuery schema .
comment: """Loop through each element of 'input_data' and deduce the
          BigQuery schema. The schema is returned as a recursive map that contains
          both the database schema and some additional metadata about each entry.
          It has the following form:
  
            schema_map := {
              key: schema_entry
            }
  
          The 'key' is the name of the table column.
  
            schema_entry := {
              'status': 'hard | soft',
              'filled': True | False,
              'info': {
                'name': key,
                'type': 'STRING | TIMESTAMP | DATE | TIME
                        | FLOAT | INTEGER | BOOLEAN | RECORD'
                'mode': 'NULLABLE | REPEATED',
                'fields': schema_map
              }
            }
  
          The status of 'hard' or 'soft' refers the reliability of the type
          inference that we made for a particular column element value. If the
          element value is a 'null', we assume that it's a 'soft' STRING. If the
          element value is an empty [] or {}, we assume that the element type is a
          'soft' REPEATED STRING or a 'soft' NULLABLE RECORD, respectively. When
          we come across a subsequent entry with non-null or non-empty values, we
          are able infer the type definitively, and we change the status to
          'hard'. The status can transition from 'soft' to 'hard' but not the
          reverse.
  
          The 'filled' entry indicates whether all input data records contained
          the given field. If the --infer_mode flag is given, the 'filled' entry
          is used to convert a NULLABLE schema entry to a REQUIRED schema entry or
          to relax an existing field in schema_map from REQUIRED to NULLABLE.
  
          The function returns a tuple of 2 things:
            * an OrderedDict which is sorted by the 'key' of the column name
            * a list of possible errors containing a map of 'line' and 'msg'
  
          An Exception is thrown in the lower-level calls only for programming
          errors, not for data validation errors. Therefore each line in the input
          file is processed without a try-except block and any exception will be
          allowed to escape to the calling routine.
          """$SEP$# Print a progress message periodically.$SEP$# Deduce the schema from this given data record.