title: Output of nn.Linear is different for the same input
body: <p>In <code>torch==1.7.1+cu101</code>, I have two tensors</p>
  <pre><code>import torch
  a = torch.rand(1,5,10)
  b = torch.rand(100,1,10)
  </code></pre>
  <p>and a feed-forward network</p>
  <pre><code>import torch.nn as nn
  l = nn.Linear(10,10)
  </code></pre>
  <p>I force one row of them to be equal:</p>
  <pre><code>a[0,0] = b[80][0].clone()
  </code></pre>
  <p>Then I feed both tensors to <code>l</code>:</p>
  <pre><code>r1 = l(a)
  r2 = l(b)
  </code></pre>
  <p>Apparently, since <code>a[0,0]</code> is equal to <code>b[80,0]</code>, <code>r1[0,0]</code> <strong>must be equal to</strong> <code>r2[80,0]</code>.
  But it turns out to be like:</p>
  <pre><code>(r1[0,0] == r2[80,0]).all()
  &gt;&gt;&gt; False
  </code></pre>
  <p>I've fixed the randomness by:</p>
  <pre><code>seed = 42
  
  random.seed(seed)
  os.environ['PYTHONHASHSEED'] = str(seed)
  np.random.seed(seed)
  torch.manual_seed(seed)
  torch.cuda.manual_seed(seed)
  torch.cuda.manual_seed_all(seed)
  torch.backends.cudnn.deterministic = True
  torch.backends.cudnn.benchmark = True
  </code></pre>
  <p>Does anyone know why <code>(r1[0,0] == r2[80,0]).all()</code> is False?</p>