original_docstring: 
      Loads training or test data using image_processing.load_images func
      which applies the supplied processing function as required.
  
      This function is rapidly becoming a hack and will probably be replace with
      Scott's HighLevelFeatures idea.
  
      If the classes kwarg is not none assumed to be loading labelled train
      data and returns two np objs:
          * data - image matrix
          * labels - vector of labels
  
      if classes kwarg is none, data will be loaded as test data and just return
          * data - list of image vectors
      
docstring: Loads training or test data using image_processing.load_images func
  which applies the supplied processing function as required.
  
  This function is rapidly becoming a hack and will probably be replace with
  Scott's HighLevelFeatures idea.
  
  If the classes kwarg is not none assumed to be loading labelled train
  data and returns two np objs:
  data - image matrix
  labels - vector of labels
  
  if classes kwarg is none, data will be loaded as test data and just return
  data - list of image vectors
docstring_tokens: Loads training or test data using image_processing . load_images func which applies the supplied processing function as required . This function is rapidly becoming a hack and will probably be replace with Scott ' s HighLevelFeatures idea . If the classes kwarg is not none assumed to be loading labelled train data and returns two np objs : data - image matrix labels - vector of labels if classes kwarg is none data will be loaded as test data and just return data - list of image vectors
code: def load_data(image_fname_dict, classes=None,
                processing=None, verbose=False):
      """
      Loads training or test data using image_processing.load_images func
      which applies the supplied processing function as required.
  
      This function is rapidly becoming a hack and will probably be replace with
      Scott's HighLevelFeatures idea.
  
      If the classes kwarg is not none assumed to be loading labelled train
      data and returns two np objs:
          * data - image matrix
          * labels - vector of labels
  
      if classes kwarg is none, data will be loaded as test data and just return
          * data - list of image vectors
      """
  
      if not processing:
          if verbose:
              print("Warning: no processing applied, it will \
              not be possible to stack these images due to \
              varying sizes.")
          # use the raw loading without processing
          return load_rawdata(image_fname_dict, classes=classes, verbose=verbose)
  
      # check augmentation factor
      dummy_images = processing(np.zeros((100,100)))
      if type(dummy_images) is not list:
          dummy_images = [dummy_images]
      if dummy_images[0].shape == ():
          dummy_images[0] = np.array(dummy_images[0])[np.newaxis]
      augmentation_factor = len(dummy_images)
      expected_shape = dummy_images[0].shape
      # second dummy images
      dummy_images = processing(np.zeros((42,42)))
      if type(dummy_images) is not list:
          dummy_images = [dummy_images]
      if dummy_images[0].shape == ():
          dummy_images[0] = np.array(dummy_images[0])[np.newaxis]
      # use this opportunity to check if we're going to be able to process these
      if expected_shape != dummy_images[0].shape:
          # then we'll just load them as a list and process them after
          X,labels = load_rawdata(image_fname_dict, classes=classes, verbose=verbose)
          # reprocess X
          X_new = []
          labels_new = []
          for image,label in zip(X,labels):
              p_images = processing(image)
              if type(p_images) is not list:
                  p_images = [p_images]
              X_new += p_images
              labels_new += len(p_images)*[label]
          return X_new,labels_new
  
      # e.g. labelled training data
      if classes:
          labels = []
  
          image_fpaths = []
  
          for class_index, class_name in enumerate(classes):
              if verbose:
                  print("class: {0} of 120: {1}".format(class_index, class_name))
  
              class_fpaths = image_fname_dict['train'][class_name]
              array_labels = augmentation_factor * len(class_fpaths) * [class_name]
              labels = labels + array_labels
              image_fpaths += class_fpaths
              #data_subset = image_processing.load_images(image_fpaths,
              #                                              processing,
              #                                              verbose)
              #data.append(data_subset)
              #num_images = len(data_subset)
              # generate the class labels and add them to the list
          data = image_processing.load_images(image_fpaths, processing, verbose)
          return data, np.array(labels)
  
      # e.g. test data
      else:
          data = image_processing.load_images(image_fname_dict['test'],
                                              processing,
                                              verbose)
          names = [os.path.basename(fpath) for fpath in image_fname_dict['test']]
          return np.vstack(data), names
code_tokens: def load_data ( image_fname_dict , classes = None , processing = None , verbose = False ) : if not processing : if verbose : print ( "Warning: no processing applied, it will \
              not be possible to stack these images due to \
              varying sizes." ) return load_rawdata ( image_fname_dict , classes = classes , verbose = verbose ) dummy_images = processing ( np . zeros ( ( 100 , 100 ) ) ) if type ( dummy_images ) is not list : dummy_images = [ dummy_images ] if dummy_images [ 0 ] . shape == ( ) : dummy_images [ 0 ] = np . array ( dummy_images [ 0 ] ) [ np . newaxis ] augmentation_factor = len ( dummy_images ) expected_shape = dummy_images [ 0 ] . shape dummy_images = processing ( np . zeros ( ( 42 , 42 ) ) ) if type ( dummy_images ) is not list : dummy_images = [ dummy_images ] if dummy_images [ 0 ] . shape == ( ) : dummy_images [ 0 ] = np . array ( dummy_images [ 0 ] ) [ np . newaxis ] if expected_shape != dummy_images [ 0 ] . shape : X , labels = load_rawdata ( image_fname_dict , classes = classes , verbose = verbose ) X_new = [ ] labels_new = [ ] for image , label in zip ( X , labels ) : p_images = processing ( image ) if type ( p_images ) is not list : p_images = [ p_images ] X_new += p_images labels_new += len ( p_images ) * [ label ] return X_new , labels_new if classes : labels = [ ] image_fpaths = [ ] for class_index , class_name in enumerate ( classes ) : if verbose : print ( "class: {0} of 120: {1}" . format ( class_index , class_name ) ) class_fpaths = image_fname_dict [ 'train' ] [ class_name ] array_labels = augmentation_factor * len ( class_fpaths ) * [ class_name ] labels = labels + array_labels image_fpaths += class_fpaths data = image_processing . load_images ( image_fpaths , processing , verbose ) return data , np . array ( labels ) else : data = image_processing . load_images ( image_fname_dict [ 'test' ] , processing , verbose ) names = [ os . path . basename ( fpath ) for fpath in image_fname_dict [ 'test' ] ] return np . vstack ( data ) , names
short_docstring: Loads training or test data using image_processing.load_images func
  which applies the supplied processing function as required.
short_docstring_tokens: Loads training or test data using image_processing . load_images func which applies the supplied processing function as required .
comment: """
      Loads training or test data using image_processing.load_images func
      which applies the supplied processing function as required.
  
      This function is rapidly becoming a hack and will probably be replace with
      Scott's HighLevelFeatures idea.
  
      If the classes kwarg is not none assumed to be loading labelled train
      data and returns two np objs:
          * data - image matrix
          * labels - vector of labels
  
      if classes kwarg is none, data will be loaded as test data and just return
          * data - list of image vectors
      """$SEP$# use the raw loading without processing$SEP$# check augmentation factor$SEP$# second dummy images$SEP$# use this opportunity to check if we're going to be able to process these$SEP$# then we'll just load them as a list and process them after$SEP$# reprocess X$SEP$# e.g. labelled training data$SEP$#data_subset = image_processing.load_images(image_fpaths,$SEP$#                                              processing,$SEP$#                                              verbose)$SEP$#data.append(data_subset)$SEP$#num_images = len(data_subset)$SEP$# generate the class labels and add them to the list$SEP$# e.g. test data