In different pytorch version, dropout performs differently(I set the same random seed)

recently, I am migrating code from PyTorch 1.4 to PyTorch 1.7. I set the random seeds of these two pieces of code to 42. When I test my code, I find that torch.nn.Dropout in PyTorch 1.7 dropped different elements compared to torch.nn.Dropout in 1.4.

My function to set random seed:

def seed_all(num: int = 42):
    if torch.cuda.device_count() > 0:
        torch.backends.cudnn.deterministic = True

The internal implementations might have changed (e.g. vectorization could have been added) and I don’t think there is a guarantee for determinism between different PyTorch releases.

1 Like

Oh, I get it. Thx a lot