Chunks given table into multiple smaller ones.

  Tables that osquery yields can be arbitrarily large. Because GRR's messages
  cannot be arbitrarily large, it might happen that the table has to be split
  into multiple smaller ones.

  Note that that serialized response protos are going to be slightly bigger than
  the specified limit. For regular values the additional payload should be
  negligible.

  Args:
    table: A table to split into multiple smaller ones.
    max_chunk_size: A maximum size of the returned table in bytes.

  Yields:
    Tables with the same query and headers as the input table and a subset of
    rows.

def ChunkTable(table,
               max_chunk_size):
  """Chunks given table into multiple smaller ones.

  Tables that osquery yields can be arbitrarily large. Because GRR's messages
  cannot be arbitrarily large, it might happen that the table has to be split
  into multiple smaller ones.

  Note that that serialized response protos are going to be slightly bigger than
  the specified limit. For regular values the additional payload should be
  negligible.

  Args:
    table: A table to split into multiple smaller ones.
    max_chunk_size: A maximum size of the returned table in bytes.

  Yields:
    Tables with the same query and headers as the input table and a subset of
    rows.
  """

  def ByteLength(string):
    return len(string.encode("utf-8"))

  def Chunk():
    result = rdf_osquery.OsqueryTable()
    result.query = table.query
    result.header = table.header
    return result

  chunk = Chunk()
  chunk_size = 0

  for row in table.rows:
    row_size = sum(map(ByteLength, row.values))

    if chunk_size + row_size > max_chunk_size:
      yield chunk

      chunk = Chunk()
      chunk_size = 0

    chunk.rows.append(row)
    chunk_size += row_size

  # We want to yield extra chunk in two cases:
  # * there are some rows that did not cause the chunk to overflow so it has not
  #   been yielded as part of the loop.
  # * the initial table has no rows but we still need to yield some table even
  #   if it is empty.
  if chunk.rows or not table.rows:
    yield chunk