original_docstring:  Discard data from training set according to some criteria.
  
              :param data:  A new sample for the training set.
              :type  data:  list of float
              :param class_label:    The label of the new sample.
              :type  class_label:    str
              :param new_data_in_training_set: flag if new data is in training set
              :type  new_data_in_training_set: bool
              :param retraining_required: flag if retraining is
                      requiered (the new point is a potentiell sv or a removed
                      one was a sv)
              :type  retraining_required: bool
              :rtype: [flag if new data is in training set, flag if retraining is
                      requiered (the new point is a potentiell sv or a removed
                      one was a sv)]
  
          
docstring: Discard data from training set according to some criteria.
docstring_tokens: Discard data from training set according to some criteria .
code: def discard_data(self, data, class_label,\
                       new_data_in_training_set, retraining_required,
                       label=None):
          """ Discard data from training set according to some criteria.
  
              :param data:  A new sample for the training set.
              :type  data:  list of float
              :param class_label:    The label of the new sample.
              :type  class_label:    str
              :param new_data_in_training_set: flag if new data is in training set
              :type  new_data_in_training_set: bool
              :param retraining_required: flag if retraining is
                      requiered (the new point is a potentiell sv or a removed
                      one was a sv)
              :type  retraining_required: bool
              :rtype: [flag if new data is in training set, flag if retraining is
                      requiered (the new point is a potentiell sv or a removed
                      one was a sv)]
  
          """
          # Reset retraining_required flag if a new chunk is not full
          if self.discard_type == "INC_BATCH"\
              and len(self.future_samples) < self.basket_size:
              retraining_required = False
  
          while self.num_samples > self.basket_size\
              and (self.discard_type=="REMOVE_OLDEST"\
                   or self.discard_type=="REMOVE_FARTHEST"):
              if self.discard_type == "REMOVE_OLDEST":
                  # remove the oldest sample
                  idx = 0# in case "DONT_HANDLE_RATIO"
                  if self.training_set_ratio == "KEEP_RATIO_AS_IT_IS":
                      # choose from the training set the oldest sample with the
                      # same label as the added sample
                      idx = next((i for i in numpy.arange(len(self.samples))\
                                  if self.labels[i] == label), 0)
                  elif self.training_set_ratio == "BALANCED_RATIO":
                      # try to keep the number of samples for each class equal
                      num_target = sum(l == 1 for l in self.labels)
                      num_standard = sum(l == 0 for l in self.labels)
                      if num_target != num_standard:
                          label = (num_target > num_standard)
                      idx = next((i for i in numpy.arange(len(self.samples))\
                                  if self.labels[i] == label), 0)
                  retraining_required = self.remove_samples([idx])\
                      or retraining_required
              elif self.discard_type == "REMOVE_FARTHEST":
                  # remove the sample which distance is maximal to the
                  # hyperplane
                  samples = self.samples  # in case "DONT_HANDLE_RATIO"
                  if self.training_set_ratio == "KEEP_RATIO_AS_IT_IS":
                      # choose only from that samples with the same label as the
                      # added sample
                      samples = []
                      idxs_label = []
                      for i in numpy.arange(len(self.samples)):
                          if self.labels[i] == label:
                              samples.append(self.samples[i])
                              idxs_label.append(i)
                  elif self.training_set_ratio == "BALANCED_RATIO":
                      # try to keep the number of samples for each class equal
                      num_target = sum(l == 1 for l in self.labels)
                      num_standard = sum(l == 0 for l in self.labels)
                      if num_target != num_standard:
                          label = (num_target > num_standard)
  
                      samples = []
                      idxs_label = []
                      for i in numpy.arange(len(self.samples)):
                          if self.labels[i] == label:
                              samples.append(self.samples[i])
                              idxs_label.append(i)
  
                  idx = numpy.argmax(map(\
                          lambda x: abs((self._execute(\
                              numpy.atleast_2d(x))).prediction),\
                          samples))
  
                  if self.training_set_ratio == "KEEP_RATIO_AS_IT_IS" or\
                          self.training_set_ratio == "BALANCED_RATIO":
                      idx = idxs_label[idx]
  
                  retraining_required = self.remove_samples([idx])\
                      or retraining_required
  
          # TODO: add parameter to specify possible overlap
          #       like x times basket size?
          if self.discard_type == "INC_BATCH"\
                  and len(self.future_samples) == self.basket_size:
              # remove all old samples
              self.remove_samples(list(numpy.arange(self.num_samples)))
              # and add all samples from the future knowledge base
              for (d, c_l) in zip(self.future_samples, self.future_labels):
                  self.add_new_sample(d, c_l, True)
              # The whole training set changes so retraining is required
              retraining_required = True
  
          if self.discard_type == "REMOVE_NO_BORDER_POINTS":
              if len(self.samples) < self.basket_size:
                  # Don't retrain if basket size is not reached
                  retraining_required = False
              elif len(self.samples) == self.basket_size:
                  # Retrain if basket size is reached
                  retraining_required = True
              if len(self.samples) > self.basket_size:
                  # Discard useless data for next iterations
                  self.remove_no_border_points(retraining_required)
                  retraining_required = False
  
          if self.discard_type == "CDT":
              # test if a change occurred
              changeDetected = self.change_detection_test(data, class_label)
              # if a change is detected remove old samples
              if changeDetected or (numpy.floor_divide(
                      len(self.future_samples),
                      self.num_samples) > self.cdt_threshold):
                  self.remove_samples(numpy.arange(len(self.samples)))
                  # if a change is detected or many new samples arrived add
                  # current samples to training set
                  for (s, l) in zip(self.future_samples, self.future_labels):
                      self.add_new_sample(s, l, True)
                  retraining_required = True
              else:
                  retraining_required = False
  
          return [new_data_in_training_set, retraining_required]
code_tokens: def discard_data ( self , data , class_label , new_data_in_training_set , retraining_required , label = None ) : if self . discard_type == "INC_BATCH" and len ( self . future_samples ) < self . basket_size : retraining_required = False while self . num_samples > self . basket_size and ( self . discard_type == "REMOVE_OLDEST" or self . discard_type == "REMOVE_FARTHEST" ) : if self . discard_type == "REMOVE_OLDEST" : idx = 0 if self . training_set_ratio == "KEEP_RATIO_AS_IT_IS" : idx = next ( ( i for i in numpy . arange ( len ( self . samples ) ) if self . labels [ i ] == label ) , 0 ) elif self . training_set_ratio == "BALANCED_RATIO" : num_target = sum ( l == 1 for l in self . labels ) num_standard = sum ( l == 0 for l in self . labels ) if num_target != num_standard : label = ( num_target > num_standard ) idx = next ( ( i for i in numpy . arange ( len ( self . samples ) ) if self . labels [ i ] == label ) , 0 ) retraining_required = self . remove_samples ( [ idx ] ) or retraining_required elif self . discard_type == "REMOVE_FARTHEST" : samples = self . samples if self . training_set_ratio == "KEEP_RATIO_AS_IT_IS" : samples = [ ] idxs_label = [ ] for i in numpy . arange ( len ( self . samples ) ) : if self . labels [ i ] == label : samples . append ( self . samples [ i ] ) idxs_label . append ( i ) elif self . training_set_ratio == "BALANCED_RATIO" : num_target = sum ( l == 1 for l in self . labels ) num_standard = sum ( l == 0 for l in self . labels ) if num_target != num_standard : label = ( num_target > num_standard ) samples = [ ] idxs_label = [ ] for i in numpy . arange ( len ( self . samples ) ) : if self . labels [ i ] == label : samples . append ( self . samples [ i ] ) idxs_label . append ( i ) idx = numpy . argmax ( map ( lambda x : abs ( ( self . _execute ( numpy . atleast_2d ( x ) ) ) . prediction ) , samples ) ) if self . training_set_ratio == "KEEP_RATIO_AS_IT_IS" or self . training_set_ratio == "BALANCED_RATIO" : idx = idxs_label [ idx ] retraining_required = self . remove_samples ( [ idx ] ) or retraining_required if self . discard_type == "INC_BATCH" and len ( self . future_samples ) == self . basket_size : self . remove_samples ( list ( numpy . arange ( self . num_samples ) ) ) for ( d , c_l ) in zip ( self . future_samples , self . future_labels ) : self . add_new_sample ( d , c_l , True ) retraining_required = True if self . discard_type == "REMOVE_NO_BORDER_POINTS" : if len ( self . samples ) < self . basket_size : retraining_required = False elif len ( self . samples ) == self . basket_size : retraining_required = True if len ( self . samples ) > self . basket_size : self . remove_no_border_points ( retraining_required ) retraining_required = False if self . discard_type == "CDT" : changeDetected = self . change_detection_test ( data , class_label ) if changeDetected or ( numpy . floor_divide ( len ( self . future_samples ) , self . num_samples ) > self . cdt_threshold ) : self . remove_samples ( numpy . arange ( len ( self . samples ) ) ) for ( s , l ) in zip ( self . future_samples , self . future_labels ) : self . add_new_sample ( s , l , True ) retraining_required = True else : retraining_required = False return [ new_data_in_training_set , retraining_required ]
short_docstring: Discard data from training set according to some criteria.
short_docstring_tokens: Discard data from training set according to some criteria .
comment: """ Discard data from training set according to some criteria.
  
              :param data:  A new sample for the training set.
              :type  data:  list of float
              :param class_label:    The label of the new sample.
              :type  class_label:    str
              :param new_data_in_training_set: flag if new data is in training set
              :type  new_data_in_training_set: bool
              :param retraining_required: flag if retraining is
                      requiered (the new point is a potentiell sv or a removed
                      one was a sv)
              :type  retraining_required: bool
              :rtype: [flag if new data is in training set, flag if retraining is
                      requiered (the new point is a potentiell sv or a removed
                      one was a sv)]
  
          """$SEP$# Reset retraining_required flag if a new chunk is not full$SEP$# remove the oldest sample$SEP$# in case "DONT_HANDLE_RATIO"$SEP$# choose from the training set the oldest sample with the$SEP$# same label as the added sample$SEP$# try to keep the number of samples for each class equal$SEP$# remove the sample which distance is maximal to the$SEP$# hyperplane$SEP$# in case "DONT_HANDLE_RATIO"$SEP$# choose only from that samples with the same label as the$SEP$# added sample$SEP$# try to keep the number of samples for each class equal$SEP$# TODO: add parameter to specify possible overlap$SEP$#       like x times basket size?$SEP$# remove all old samples$SEP$# and add all samples from the future knowledge base$SEP$# The whole training set changes so retraining is required$SEP$# Don't retrain if basket size is not reached$SEP$# Retrain if basket size is reached$SEP$# Discard useless data for next iterations$SEP$# test if a change occurred$SEP$# if a change is detected remove old samples$SEP$# if a change is detected or many new samples arrived add$SEP$# current samples to training set