def loss_denominator(targets, num_microbatches):
    """Denominator applied to losses.

    This is usually the size of the targets tensor (omitting ensemble
    dimensions).  Alternatively, it is an override value passed to the
    class constructor.

    Args:
      targets: a mtf.Tensor
      num_microbatches: an integer - greater than one if the step has been
        serialized into multiple microbatches to save memory.
    Returns:
      a float
    """
    ret = float(targets.shape.size) * num_microbatches
    return float(ret)
Denominator applied to losses.

This is usually the size of the targets tensor (omitting ensemble
dimensions).  Alternatively, it is an override value passed to the
class constructor.

Args:
targets: a mtf.Tensor
num_microbatches: an integer - greater than one if the step has been
serialized into multiple microbatches to save memory.
Returns:
a float
utils.py