tensorflow/lite/delegates/gpu/common/memory_management/greedy_by_size_assignment.cc
// Assigns given tensors to shared objects, using the following greedy
// algorithm:
// - We have tensor usage records of all intermideate tensors as an input. Each
// record consists of tensor size, first and last tasks, that use it. Let's call
// [first_task..last_task] a tensor usage interval;
// - Distance between two usage intervals is the absolute difference between
// closest tasks in their intervals. If two usage intervals don't intersect,
// than the distance between them is positive;
// - Calculate positional maximums vector, e.g. the vector of lower bounds on
// size of each shared object;
// - For each tensor find the rightmost positional maximum, that is greater or
// equal, than current tensor's size (call it position);
// - Iterate through all tensors in non-decreasing order of their
// SizeDistPriority (described above);
// - For every such tensor, assign it to the object, that already has tensor,
// which usage interval has the smallest existing positive distance to the
// current tensor's usage interval (this distance and object id are already
// precalculated in its SizeDistPriority record). Size of the chosen object can
// possible increase;
// - If there are several such objects, use the largest one;
// - If there are no suitable shared objects, assign current tensor to the new
// object with size equal to current tensor's size;
// - Modify SizeDistPriority records of tensors, that haven't been assigned yet,
// to reflect distance changes after that assignment.
Assigns given tensors to shared objects, using the following greedy
algorithm:
We have tensor usage records of all intermideate tensors as an input. Each
record consists of tensor size, first and last tasks, that use it. Let's call
[first_task..last_task] a tensor usage interval;
Distance between two usage intervals is the absolute difference between
closest tasks in their intervals. If two usage intervals don't intersect,
than the distance between them is positive;
Calculate positional maximums vector, e.g. the vector of lower bounds on
size of each shared object;
For each tensor find the rightmost positional maximum, that is greater or
equal, than current tensor's size (call it position);
Iterate through all tensors in non-decreasing order of their
SizeDistPriority (described above);
For every such tensor, assign it to the object, that already has tensor,
which usage interval has the smallest existing positive distance to the
current tensor's usage interval (this distance and object id are already
precalculated in its SizeDistPriority record). Size of the chosen object can
possible increase;
If there are several such objects, use the largest one;
If there are no suitable shared objects, assign current tensor to the new
object with size equal to current tensor's size;
Modify SizeDistPriority records of tensors, that haven't been assigned yet,
to reflect distance changes after that assignment.
Assigns given tensors to shared objects using the following greedy algorithm : We have tensor usage records of all intermideate tensors as an input . Each record consists of tensor size first and last tasks that use it . Let ' s call [ first_task .. last_task ] a tensor usage interval ; Distance between two usage intervals is the absolute difference between closest tasks in their intervals . If two usage intervals don ' t intersect than the distance between them is positive ; Calculate positional maximums vector e . g . the vector of lower bounds on size of each shared object ; For each tensor find the rightmost positional maximum that is greater or equal than current tensor ' s size ( call it position ) ; Iterate through all tensors in non - decreasing order of their SizeDistPriority ( described above ) ; For every such tensor assign it to the object that already has tensor which usage interval has the smallest existing positive distance to the current tensor ' s usage interval ( this distance and object id are already precalculated in its SizeDistPriority record ) . Size of the chosen object can possible increase ; If there are several such objects use the largest one ; If there are no suitable shared objects assign current tensor to the new object with size equal to current tensor ' s size ; Modify SizeDistPriority records of tensors that haven ' t been assigned yet to reflect distance changes after that assignment .
absl::Status GreedyBySizeDistPriorityAssignment(
    const std::vector<TensorUsageRecord<size_t>>& usage_records,
    ObjectsAssignment<size_t>* assignment) {
  std::vector<size_t> positional_max =
      CalculatePositionalMaximums(usage_records);
  size_t num_records = usage_records.size();
  std::vector<SizeDistPriorityInfo> priority_info(num_records);
  for (size_t rec_id = 0; rec_id < usage_records.size(); ++rec_id) {
    priority_info[rec_id].tensor_usage_id = rec_id;
    priority_info[rec_id].tensor_size = usage_records[rec_id].tensor_size;
    priority_info[rec_id].best_dist = kNotAssigned;
    priority_info[rec_id].best_object = kNotAssigned;
    size_t pos = 0;
    while (pos < positional_max.size() &&
           positional_max[pos] >= priority_info[rec_id].tensor_size) {
      ++pos;
    }
    if (pos == 0) {
      return absl::InternalError("Variable pos must be positive.");
    }
    priority_info[rec_id].position = pos - 1;
  }
  assignment->object_sizes.clear();
  assignment->object_ids.assign(num_records, kNotAssigned);
  for (size_t it = 0; it < num_records; ++it) {
    size_t best_info_id = kNotAssigned;
    for (size_t info_id = 0; info_id < num_records; ++info_id) {
      if (assignment->object_ids[priority_info[info_id].tensor_usage_id] !=
          kNotAssigned) {
        continue;
      }
      if (best_info_id == kNotAssigned ||
          priority_info[info_id] > priority_info[best_info_id]) {
        best_info_id = info_id;
      }
    }
    if (best_info_id == kNotAssigned) {
      return absl::InternalError("Invalid value for variable best_info_id.");
    }
    size_t best_rec_id = priority_info[best_info_id].tensor_usage_id;
    size_t best_obj_id = priority_info[best_info_id].best_object;
    bool new_object = false;
    if (priority_info[best_info_id].best_dist == kNotAssigned) {
      new_object = true;
      best_obj_id = assignment->object_sizes.size();
      assignment->object_ids[best_rec_id] = best_obj_id;
      assignment->object_sizes.push_back(
          usage_records[best_rec_id].tensor_size);
    } else {
      assignment->object_ids[best_rec_id] = best_obj_id;
      assignment->object_sizes[best_obj_id] =
          std::max(assignment->object_sizes[best_obj_id],
                   usage_records[best_rec_id].tensor_size);
    }
    for (size_t info_id = 0; info_id < num_records; ++info_id) {
      size_t rec_id = priority_info[info_id].tensor_usage_id;
      if (assignment->object_ids[rec_id] != kNotAssigned) {
        continue;
      }
      if (!new_object &&
          priority_info[info_id].dist[best_obj_id] == kNotAssigned) {
        continue;
      }
      size_t dist = kNotAssigned;
      if (usage_records[rec_id].last_task <
          usage_records[best_rec_id].first_task) {
        dist = usage_records[best_rec_id].first_task -
               usage_records[rec_id].last_task;
      } else if (usage_records[best_rec_id].last_task <
                 usage_records[rec_id].first_task) {
        dist = usage_records[rec_id].first_task -
               usage_records[best_rec_id].last_task;
      }
      if (new_object) {
        priority_info[info_id].dist.push_back(dist);
      } else if (dist == kNotAssigned) {
        priority_info[info_id].dist[best_obj_id] = kNotAssigned;
        if (priority_info[info_id].best_object == best_obj_id) {
          priority_info[info_id].RecalcBestDist();
        }
      } else {
        priority_info[info_id].dist[best_obj_id] =
            std::min(priority_info[info_id].dist[best_obj_id], dist);
      }
      if (dist < priority_info[info_id].best_dist) {
        priority_info[info_id].best_dist = dist;
        priority_info[info_id].best_object = best_obj_id;
      }
    }
  }
  return absl::OkStatus();
}
absl :: Status GreedyBySizeDistPriorityAssignment ( const std :: vector < TensorUsageRecord < size_t > > & usage_records , ObjectsAssignment < size_t > * assignment ) { std :: vector < size_t > positional_max = CalculatePositionalMaximums ( usage_records ) ; size_t num_records = usage_records . size ( ) ; std :: vector < SizeDistPriorityInfo > priority_info ( num_records ) ; for ( size_t rec_id = 0 ; rec_id < usage_records . size ( ) ; ++ rec_id ) { priority_info [ rec_id ] . tensor_usage_id = rec_id ; priority_info [ rec_id ] . tensor_size = usage_records [ rec_id ] . tensor_size ; priority_info [ rec_id ] . best_dist = kNotAssigned ; priority_info [ rec_id ] . best_object = kNotAssigned ; size_t pos = 0 ; while ( pos < positional_max . size ( ) && positional_max [ pos ] >= priority_info [ rec_id ] . tensor_size ) { ++ pos ; } if ( pos == 0 ) { return absl :: InternalError ( " " ) ; } priority_info [ rec_id ] . position = pos - 1 ; } assignment -> object_sizes . clear ( ) ; assignment -> object_ids . assign ( num_records , kNotAssigned ) ; for ( size_t it = 0 ; it < num_records ; ++ it ) { size_t best_info_id = kNotAssigned ; for ( size_t info_id = 0 ; info_id < num_records ; ++ info_id ) { if ( assignment -> object_ids [ priority_info [ info_id ] . tensor_usage_id ] != kNotAssigned ) { continue ; } if ( best_info_id == kNotAssigned || priority_info [ info_id ] > priority_info [ best_info_id ] ) { best_info_id = info_id ; } } if ( best_info_id == kNotAssigned ) { return absl :: InternalError ( " " ) ; } size_t best_rec_id = priority_info [ best_info_id ] . tensor_usage_id ; size_t best_obj_id = priority_info [ best_info_id ] . best_object ; bool new_object = false ; if ( priority_info [ best_info_id ] . best_dist == kNotAssigned ) { new_object = true ; best_obj_id = assignment -> object_sizes . size ( ) ; assignment -> object_ids [ best_rec_id ] = best_obj_id ; assignment -> object_sizes . push_back ( usage_records [ best_rec_id ] . tensor_size ) ; } else { assignment -> object_ids [ best_rec_id ] = best_obj_id ; assignment -> object_sizes [ best_obj_id ] = std :: max ( assignment -> object_sizes [ best_obj_id ] , usage_records [ best_rec_id ] . tensor_size ) ; } for ( size_t info_id = 0 ; info_id < num_records ; ++ info_id ) { size_t rec_id = priority_info [ info_id ] . tensor_usage_id ; if ( assignment -> object_ids [ rec_id ] != kNotAssigned ) { continue ; } if ( ! new_object && priority_info [ info_id ] . dist [ best_obj_id ] == kNotAssigned ) { continue ; } size_t dist = kNotAssigned ; if ( usage_records [ rec_id ] . last_task < usage_records [ best_rec_id ] . first_task ) { dist = usage_records [ best_rec_id ] . first_task - usage_records [ rec_id ] . last_task ; } else if ( usage_records [ best_rec_id ] . last_task < usage_records [ rec_id ] . first_task ) { dist = usage_records [ rec_id ] . first_task - usage_records [ best_rec_id ] . last_task ; } if ( new_object ) { priority_info [ info_id ] . dist . push_back ( dist ) ; } else if ( dist == kNotAssigned ) { priority_info [ info_id ] . dist [ best_obj_id ] = kNotAssigned ; if ( priority_info [ info_id ] . best_object == best_obj_id ) { priority_info [ info_id ] . RecalcBestDist ( ) ; } } else { priority_info [ info_id ] . dist [ best_obj_id ] = std :: min ( priority_info [ info_id ] . dist [ best_obj_id ] , dist ) ; } if ( dist < priority_info [ info_id ] . best_dist ) { priority_info [ info_id ] . best_dist = dist ; priority_info [ info_id ] . best_object = best_obj_id ; } } } return absl :: OkStatus ( ) ; }
Assigns given tensors to shared objects, using the following greedy
algorithm:
 We have tensor usage records of all intermideate tensors as an input.
Assigns given tensors to shared objects using the following greedy algorithm : We have tensor usage records of all intermideate tensors as an input .
// No objects have been created yet. // Find the rightmost positional maximum, that is greater or // Tensor already assigned. // During each iteration we assign exactly one of the tensors, so some not // yet assigned tensors must exist. // No suitable shared object, so we create a new one. // Assign tensor best_rec_id to the already existing object best_obj_id. // Modify SizeDistPriority records of tensors, that haven't been assigned // yet, to reflect distance changes after that assignment. // SizeDistPriority record info_id contains priority of tensor rec_id. // Tensor rec_id is already assigned. // Tensor rec_id intersects with some of the tensors, that are assigned // to object best_obj_id. // best_rec_id is the only tensor, assigned to the new object. // Usage intervals of tensors rec_id and best_rec_id intersect. So // rec_id can't be assigned to best_obj_id anymore. // best_obj_id was the best shared object for tensor rec_id, but now // it's not suitable anymore, so we need some recalculation. // Update distance, because it has probably been changed. // Update best distance and best object for tensor rec_id.