original_docstring: 
      A data loader is created in a way similar to that of Detectron2.
      The main differences are:
       - it allows to combine datasets with different but compatible object category sets
  
      The data loader is created by the following steps:
      1. Use the dataset names in config to query :class:`DatasetCatalog`, and obtain a list of dicts.
      2. Start workers to work on the dicts. Each worker will:
          * Map each metadata dict into another format to be consumed by the model.
          * Batch them by simply putting dicts into a list.
      The batched ``list[mapped_dict]`` is what this dataloader will return.
  
      Args:
          cfg (CfgNode): the config
          mapper (callable): a callable which takes a sample (dict) from dataset and
              returns the format to be consumed by the model.
              By default it will be `DatasetMapper(cfg, True)`.
  
      Returns:
          an infinite iterator of training data
      
docstring: A data loader is created in a way similar to that of Detectron2.
  The main differences are:
  it allows to combine datasets with different but compatible object category sets
  
  The data loader is created by the following steps:
  1. Use the dataset names in config to query :class:`DatasetCatalog`, and obtain a list of dicts.
  2. Start workers to work on the dicts. Each worker will:
  Map each metadata dict into another format to be consumed by the model.
  Batch them by simply putting dicts into a list.
  The batched ``list[mapped_dict]`` is what this dataloader will return.
docstring_tokens: A data loader is created in a way similar to that of Detectron2 . The main differences are : it allows to combine datasets with different but compatible object category sets The data loader is created by the following steps : 1 . Use the dataset names in config to query : class : ` DatasetCatalog ` and obtain a list of dicts . 2 . Start workers to work on the dicts . Each worker will : Map each metadata dict into another format to be consumed by the model . Batch them by simply putting dicts into a list . The batched ` ` list [ mapped_dict ] ` ` is what this dataloader will return .
code: def build_detection_train_loader(cfg: CfgNode, mapper=None):
      """
      A data loader is created in a way similar to that of Detectron2.
      The main differences are:
       - it allows to combine datasets with different but compatible object category sets
  
      The data loader is created by the following steps:
      1. Use the dataset names in config to query :class:`DatasetCatalog`, and obtain a list of dicts.
      2. Start workers to work on the dicts. Each worker will:
          * Map each metadata dict into another format to be consumed by the model.
          * Batch them by simply putting dicts into a list.
      The batched ``list[mapped_dict]`` is what this dataloader will return.
  
      Args:
          cfg (CfgNode): the config
          mapper (callable): a callable which takes a sample (dict) from dataset and
              returns the format to be consumed by the model.
              By default it will be `DatasetMapper(cfg, True)`.
  
      Returns:
          an infinite iterator of training data
      """
      images_per_worker = _compute_num_images_per_worker(cfg)
  
      _add_category_whitelists_to_metadata(cfg)
      _add_category_maps_to_metadata(cfg)
      dataset_dicts = combine_detection_dataset_dicts(
          cfg.DATASETS.TRAIN,
          keep_instance_predicate=_get_keep_instance_predicate(cfg),
          proposal_files=cfg.DATASETS.PROPOSAL_FILES_TRAIN if cfg.MODEL.LOAD_PROPOSALS else None,
      )
      dataset = DatasetFromList(dataset_dicts, copy=False)
  
      if mapper is None:
          mapper = DatasetMapper(cfg, True)
      dataset = MapDataset(dataset, mapper)
  
      sampler_name = cfg.DATALOADER.SAMPLER_TRAIN
      logger = logging.getLogger(__name__)
      logger.info("Using training sampler {}".format(sampler_name))
      if sampler_name == "TrainingSampler":
          sampler = samplers.TrainingSampler(len(dataset))
      elif sampler_name == "RepeatFactorTrainingSampler":
          sampler = samplers.RepeatFactorTrainingSampler(
              dataset_dicts, cfg.DATALOADER.REPEAT_THRESHOLD
          )
      else:
          raise ValueError("Unknown training sampler: {}".format(sampler_name))
  
      if cfg.DATALOADER.ASPECT_RATIO_GROUPING:
          data_loader = torch.utils.data.DataLoader(
              dataset,
              sampler=sampler,
              num_workers=cfg.DATALOADER.NUM_WORKERS,
              batch_sampler=None,
              collate_fn=operator.itemgetter(0),  # don't batch, but yield individual elements
              worker_init_fn=worker_init_reset_seed,
          )  # yield individual mapped dict
          data_loader = AspectRatioGroupedDataset(data_loader, images_per_worker)
      else:
          batch_sampler = torch.utils.data.sampler.BatchSampler(
              sampler, images_per_worker, drop_last=True
          )
          # drop_last so the batch always have the same size
          data_loader = torch.utils.data.DataLoader(
              dataset,
              num_workers=cfg.DATALOADER.NUM_WORKERS,
              batch_sampler=batch_sampler,
              collate_fn=trivial_batch_collator,
              worker_init_fn=worker_init_reset_seed,
          )
  
      return data_loader
code_tokens: def build_detection_train_loader ( cfg : CfgNode , mapper = None ) : images_per_worker = _compute_num_images_per_worker ( cfg ) _add_category_whitelists_to_metadata ( cfg ) _add_category_maps_to_metadata ( cfg ) dataset_dicts = combine_detection_dataset_dicts ( cfg . DATASETS . TRAIN , keep_instance_predicate = _get_keep_instance_predicate ( cfg ) , proposal_files = cfg . DATASETS . PROPOSAL_FILES_TRAIN if cfg . MODEL . LOAD_PROPOSALS else None , ) dataset = DatasetFromList ( dataset_dicts , copy = False ) if mapper is None : mapper = DatasetMapper ( cfg , True ) dataset = MapDataset ( dataset , mapper ) sampler_name = cfg . DATALOADER . SAMPLER_TRAIN logger = logging . getLogger ( __name__ ) logger . info ( "Using training sampler {}" . format ( sampler_name ) ) if sampler_name == "TrainingSampler" : sampler = samplers . TrainingSampler ( len ( dataset ) ) elif sampler_name == "RepeatFactorTrainingSampler" : sampler = samplers . RepeatFactorTrainingSampler ( dataset_dicts , cfg . DATALOADER . REPEAT_THRESHOLD ) else : raise ValueError ( "Unknown training sampler: {}" . format ( sampler_name ) ) if cfg . DATALOADER . ASPECT_RATIO_GROUPING : data_loader = torch . utils . data . DataLoader ( dataset , sampler = sampler , num_workers = cfg . DATALOADER . NUM_WORKERS , batch_sampler = None , collate_fn = operator . itemgetter ( 0 ) , worker_init_fn = worker_init_reset_seed , ) data_loader = AspectRatioGroupedDataset ( data_loader , images_per_worker ) else : batch_sampler = torch . utils . data . sampler . BatchSampler ( sampler , images_per_worker , drop_last = True ) data_loader = torch . utils . data . DataLoader ( dataset , num_workers = cfg . DATALOADER . NUM_WORKERS , batch_sampler = batch_sampler , collate_fn = trivial_batch_collator , worker_init_fn = worker_init_reset_seed , ) return data_loader
short_docstring: A data loader is created in a way similar to that of Detectron2.
short_docstring_tokens: A data loader is created in a way similar to that of Detectron2 .
comment: """
      A data loader is created in a way similar to that of Detectron2.
      The main differences are:
       - it allows to combine datasets with different but compatible object category sets
  
      The data loader is created by the following steps:
      1. Use the dataset names in config to query :class:`DatasetCatalog`, and obtain a list of dicts.
      2. Start workers to work on the dicts. Each worker will:
          * Map each metadata dict into another format to be consumed by the model.
          * Batch them by simply putting dicts into a list.
      The batched ``list[mapped_dict]`` is what this dataloader will return.
  
      Args:
          cfg (CfgNode): the config
          mapper (callable): a callable which takes a sample (dict) from dataset and
              returns the format to be consumed by the model.
              By default it will be `DatasetMapper(cfg, True)`.
  
      Returns:
          an infinite iterator of training data
      """$SEP$# don't batch, but yield individual elements$SEP$# yield individual mapped dict$SEP$# drop_last so the batch always have the same size