original_docstring: Performing some alteration to a dataset node. A common alteration is to insert a node.
docstring: Performing some alteration to a dataset node. A common alteration is to insert a node.
docstring_tokens: Performing some alteration to a dataset node . A common alteration is to insert a node .
code: def _alter_node(node):
      """Performing some alteration to a dataset node. A common alteration is to insert a node."""
      if isinstance(node, (de.TFRecordDataset, de.TextFileDataset)) and node.shuffle_level == de.Shuffle.GLOBAL:
          # Remove the connection between the parent's node to the current node because we are inserting a node.
          if node.output:
              node.output.pop()
          # Perform a fast scan for average rows per file
          if isinstance(node, de.TFRecordDataset):
              avg_rows_per_file = node.get_dataset_size(True) // len(node.dataset_files)
          else:
              avg_rows_per_file = node.get_dataset_size() // len(node.dataset_files)
  
          # Shuffle between 4 files with a minimum size of 10000 rows
          new_shuffle = node.shuffle(max(avg_rows_per_file * 4, 10000))
          return new_shuffle
  
      if isinstance(node, de.MapDataset):
          if node.python_multiprocessing:
              # Bootstrap can only be performed on a copy of the original dataset node.
              # Bootstrap on original dataset node will make all iterators share the same process pool
              node.iterator_bootstrap()
          if node.columns_order is not None:
              # Remove the connection between the parent's node to the current node because we are inserting a node.
              if node.output:
                  node.output.pop()
  
              return node.project(node.columns_order)
      return node
code_tokens: def _alter_node ( node ) : if isinstance ( node , ( de . TFRecordDataset , de . TextFileDataset ) ) and node . shuffle_level == de . Shuffle . GLOBAL : if node . output : node . output . pop ( ) if isinstance ( node , de . TFRecordDataset ) : avg_rows_per_file = node . get_dataset_size ( True ) // len ( node . dataset_files ) else : avg_rows_per_file = node . get_dataset_size ( ) // len ( node . dataset_files ) new_shuffle = node . shuffle ( max ( avg_rows_per_file * 4 , 10000 ) ) return new_shuffle if isinstance ( node , de . MapDataset ) : if node . python_multiprocessing : node . iterator_bootstrap ( ) if node . columns_order is not None : if node . output : node . output . pop ( ) return node . project ( node . columns_order ) return node
short_docstring: Performing some alteration to a dataset node.
short_docstring_tokens: Performing some alteration to a dataset node .
comment: """Performing some alteration to a dataset node. A common alteration is to insert a node."""$SEP$# Remove the connection between the parent's node to the current node because we are inserting a node.$SEP$# Perform a fast scan for average rows per file$SEP$# Shuffle between 4 files with a minimum size of 10000 rows$SEP$# Bootstrap can only be performed on a copy of the original dataset node.$SEP$# Bootstrap on original dataset node will make all iterators share the same process pool$SEP$# Remove the connection between the parent's node to the current node because we are inserting a node.