<h2 dir="auto">Issue description</h2>
<p dir="auto">Using the +=, -=, *=, /= operators gives the wrong answer in some cases without any warning. The matrix multiply assignment operator @= seems to work fine. I have tested in both PyTorch 0.3.0.post4 and PyTorch 0.4.0 (the latest version at the time of this issue creation) on the CPU. I have not tested whether this bug happens with CUDA.</p>
<h2 dir="auto">Code example</h2>
<p dir="auto">I was able to produce a minimal case of this bug:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="import torch

## Addition
x = torch.Tensor([[1],[2],[3]])
t1 = x.expand(3, 3)
t2 = x.t().expand(3, 3)

t1 += t2
print(&quot;Incorrect answer&quot;)
print(t1)

x = torch.Tensor([[1],[2],[3]])
t1 = x.expand(3, 3)
t2 = x.t().expand(3, 3)

print(&quot;Correct answer&quot;)
print(t1 + t2)

## Subtraction
x = torch.Tensor([[1],[2],[3]])
t1 = x.expand(3, 3)
t2 = x.t().expand(3, 3)

t1 -= t2
print(&quot;Incorrect answer&quot;)
print(t1)

x = torch.Tensor([[1],[2],[3]])
t1 = x.expand(3, 3)
t2 = x.t().expand(3, 3)

print(&quot;Correct answer&quot;)
print(t1 - t2)

## Multiplication
x = torch.Tensor([[1],[2],[3]])
t1 = x.expand(3, 3)
t2 = x.t().expand(3, 3)

t1 *= t2
print(&quot;Incorrect answer&quot;)
print(t1)

x = torch.Tensor([[1],[2],[3]])
t1 = x.expand(3, 3)
t2 = x.t().expand(3, 3)

print(&quot;Correct answer&quot;)
print(t1 * t2)

## Division
x = torch.Tensor([[1],[2],[3]])
t1 = x.expand(3, 3)
t2 = x.t().expand(3, 3)

t1 /= t2
print(&quot;Incorrect answer&quot;)
print(t1)

x = torch.Tensor([[1],[2],[3]])
t1 = x.expand(3, 3)
t2 = x.t().expand(3, 3)

print(&quot;Correct answer&quot;)
print(t1 / t2)"><pre class="notranslate"><code class="notranslate">import torch

## Addition
x = torch.Tensor([[1],[2],[3]])
t1 = x.expand(3, 3)
t2 = x.t().expand(3, 3)

t1 += t2
print("Incorrect answer")
print(t1)

x = torch.Tensor([[1],[2],[3]])
t1 = x.expand(3, 3)
t2 = x.t().expand(3, 3)

print("Correct answer")
print(t1 + t2)

## Subtraction
x = torch.Tensor([[1],[2],[3]])
t1 = x.expand(3, 3)
t2 = x.t().expand(3, 3)

t1 -= t2
print("Incorrect answer")
print(t1)

x = torch.Tensor([[1],[2],[3]])
t1 = x.expand(3, 3)
t2 = x.t().expand(3, 3)

print("Correct answer")
print(t1 - t2)

## Multiplication
x = torch.Tensor([[1],[2],[3]])
t1 = x.expand(3, 3)
t2 = x.t().expand(3, 3)

t1 *= t2
print("Incorrect answer")
print(t1)

x = torch.Tensor([[1],[2],[3]])
t1 = x.expand(3, 3)
t2 = x.t().expand(3, 3)

print("Correct answer")
print(t1 * t2)

## Division
x = torch.Tensor([[1],[2],[3]])
t1 = x.expand(3, 3)
t2 = x.t().expand(3, 3)

t1 /= t2
print("Incorrect answer")
print(t1)

x = torch.Tensor([[1],[2],[3]])
t1 = x.expand(3, 3)
t2 = x.t().expand(3, 3)

print("Correct answer")
print(t1 / t2)
</code></pre></div>
<h2 dir="auto">Output</h2>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="Incorrect answer
tensor([[  7.,   7.,   7.],
        [ 21.,  21.,  21.],
        [ 62.,  62.,  62.]])
Correct answer
tensor([[ 2.,  3.,  4.],
        [ 3.,  4.,  5.],
        [ 4.,  5.,  6.]])
Incorrect answer
tensor([[-5., -5., -5.],
        [-3., -3., -3.],
        [ 0.,  0.,  0.]])
Correct answer
tensor([[ 0., -1., -2.],
        [ 1.,  0., -1.],
        [ 2.,  1.,  0.]])
Incorrect answer
tensor([[ 6.0000e+00,  6.0000e+00,  6.0000e+00],
        [ 4.3200e+02,  4.3200e+02,  4.3200e+02],
        [ 6.0466e+07,  6.0466e+07,  6.0466e+07]])
Correct answer
tensor([[ 1.,  2.,  3.],
        [ 2.,  4.,  6.],
        [ 3.,  6.,  9.]])
Incorrect answer
tensor([[ 0.1667,  0.1667,  0.1667],
        [ 0.3333,  0.3333,  0.3333],
        [ 1.0000,  1.0000,  1.0000]])
Correct answer
tensor([[ 1.0000,  0.5000,  0.3333],
        [ 2.0000,  1.0000,  0.6667],
        [ 3.0000,  1.5000,  1.0000]])"><pre class="notranslate"><code class="notranslate">Incorrect answer
tensor([[  7.,   7.,   7.],
        [ 21.,  21.,  21.],
        [ 62.,  62.,  62.]])
Correct answer
tensor([[ 2.,  3.,  4.],
        [ 3.,  4.,  5.],
        [ 4.,  5.,  6.]])
Incorrect answer
tensor([[-5., -5., -5.],
        [-3., -3., -3.],
        [ 0.,  0.,  0.]])
Correct answer
tensor([[ 0., -1., -2.],
        [ 1.,  0., -1.],
        [ 2.,  1.,  0.]])
Incorrect answer
tensor([[ 6.0000e+00,  6.0000e+00,  6.0000e+00],
        [ 4.3200e+02,  4.3200e+02,  4.3200e+02],
        [ 6.0466e+07,  6.0466e+07,  6.0466e+07]])
Correct answer
tensor([[ 1.,  2.,  3.],
        [ 2.,  4.,  6.],
        [ 3.,  6.,  9.]])
Incorrect answer
tensor([[ 0.1667,  0.1667,  0.1667],
        [ 0.3333,  0.3333,  0.3333],
        [ 1.0000,  1.0000,  1.0000]])
Correct answer
tensor([[ 1.0000,  0.5000,  0.3333],
        [ 2.0000,  1.0000,  0.6667],
        [ 3.0000,  1.5000,  1.0000]])
</code></pre></div>
<h2 dir="auto">System Info</h2>
<p dir="auto">PyTorch version: 0.4.0<br>
Is debug build: No<br>
CUDA used to build PyTorch: 8.0.61</p>
<p dir="auto">OS: Ubuntu 17.10<br>
GCC version: (Ubuntu 7.2.0-8ubuntu3.2) 7.2.0<br>
CMake version: version 3.9.1</p>
<p dir="auto">Python version: 3.6<br>
Is CUDA available: No<br>
CUDA runtime version: No CUDA<br>
GPU models and configuration: No CUDA<br>
Nvidia driver version: No CUDA<br>
cuDNN version: No CUDA</p>
<p dir="auto">Versions of relevant libraries:<br>
[pip3] numpy (1.14.3)<br>
[pip3] torch (0.4.0)<br>
[conda] Could not collect</p>
<p dir="auto">Sometimes an in-place division where the denominator comes from the same tensor will yield unpredictable results.  For example, if you want an array that counts from 0 to 100 in each of 50 rows, but actually you want to normalize to the range [0..1], it seems natural to write:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="import torch
c = torch.arange(101)[None,:].repeat(50, 1)
c /= c[:,100:]
print(c.max())"><pre class="notranslate"><code class="notranslate">import torch
c = torch.arange(101)[None,:].repeat(50, 1)
c /= c[:,100:]
print(c.max())
</code></pre></div>
<p dir="auto">However, running this on CPU repeatedly will produce an unpredictable max, e.g., 82, 95, 87, etc, rather than the expected 1.0.</p>
<p dir="auto">This operation appears to be reliable on numpy, and for many array shapes it also works fine in pytorch.  But it fails on certain sizes, leading to oddly rare and difficult-to-find bugs.</p>
<p dir="auto">Should <code class="notranslate">c /= c[:,-1:]</code> be reliable?  If not, would it be reasonable to warn on such operations?</p>