lib/rocket_job/sliced/slices.rb
# Insert a new slice into the collection
#
# Returns [Integer] the number of records uploaded
#
# Parameters
#   slice [RocketJob::Sliced::Slice | Array]
#     The slice to write to the slices collection
#     If slice is an Array, it will be converted to a Slice before inserting
#     into the slices collection
#
#   input_slice [RocketJob::Sliced::Slice]
#     The input slice to which this slice corresponds
#     The id of the input slice is copied across
#     If the insert results in a duplicate record it is ignored, to support
#     restarting of jobs that failed in the middle of processing.
#     A warning is logged that the slice has already been processed.
#
# Note:
#   `slice_size` is not enforced.
#   However many records are present in the slice will be written as a
#   single slice to the slices collection
#
Insert a new slice into the collection
Returns [Integer] the number of records uploaded

Parameters
slice [RocketJob::Sliced::Slice | Array]
The slice to write to the slices collection
If slice is an Array, it will be converted to a Slice before inserting
into the slices collection

input_slice [RocketJob::Sliced::Slice]
The input slice to which this slice corresponds
The id of the input slice is copied across
If the insert results in a duplicate record it is ignored, to support
restarting of jobs that failed in the middle of processing.
A warning is logged that the slice has already been processed.

`slice_size` is not enforced.
However many records are present in the slice will be written as a
single slice to the slices collection
Insert a new slice into the collection Returns [ Integer ] the number of records uploaded Parameters slice [ RocketJob :: Sliced :: Slice | Array ] The slice to write to the slices collection If slice is an Array it will be converted to a Slice before inserting into the slices collection input_slice [ RocketJob :: Sliced :: Slice ] The input slice to which this slice corresponds The id of the input slice is copied across If the insert results in a duplicate record it is ignored to support restarting of jobs that failed in the middle of processing . A warning is logged that the slice has already been processed . ` slice_size ` is not enforced . However many records are present in the slice will be written as a single slice to the slices collection
def insert(slice, input_slice = nil)
        slice = new(records: slice) unless slice.is_a?(Slice)
        if input_slice
          slice.id                  = input_slice.id
          slice.first_record_number = input_slice.first_record_number
        end
        begin
          slice.save!
        rescue Mongo::Error::OperationFailure => exc
          raise(exc) unless exc.message.include?('E11000')
          logger.warn "Skipped already processed slice# #{slice.id}"
        end
        slice
      end
def insert ( slice , input_slice = nil ) slice = new ( records : slice ) unless slice . is_a? ( Slice ) if input_slice slice . id = input_slice . id slice . first_record_number = input_slice . first_record_number end begin slice . save! rescue Mongo :: Error :: OperationFailure => exc raise ( exc ) unless exc . message . include? ( 'E11000' ) logger . warn "Skipped already processed slice# #{slice.id}" end slice end
Insert a new slice into the collection
Returns [Integer] the number of records uploaded
Insert a new slice into the collection Returns [ Integer ] the number of records uploaded
# Retain input_slice id in the new output slice # Ignore duplicates since it means the job was restarted
slice
input_slice