def dist_reduce_sum(value: _V) -> _V:
    """
    Sums the given `value` across distributed worker nodes.
    This is equivalent to calling `dist_reduce(v, dist.ReduceOp.SUM)`.
    """
    # NOTE: Why have this check here even though the same check is in `dist_reduce()`?
    # Because we want to be able to call this function even when torch's distributed framework
    # is not available...
    # If torch's distributed framework is not available on the system, then `torch.distributed`
    # (imported here as `dist`) will just be an empty module. So calling `dist.ReduceOp.SUM` would
    # result in an `AttributeError`.
    if not is_distributed():
        return value
    return dist_reduce(value, dist.ReduceOp.SUM)
Sums the given `value` across distributed worker nodes.
This is equivalent to calling `dist_reduce(v, dist.ReduceOp.SUM)`.
allennlp/nn/util.py