Have we tested it with fp16 inputs?  Looks a little scary here as we are returning stats with pre-promoted data type. -> Later in our all_gather, we'll have inconsistent data type across processes. IDK if it's safe to do so.    We can move the promotion up and use the updated `input_options` to be on the safe side.