<h2 dir="auto"><g-emoji class="g-emoji" alias="bug" fallback-src="https://github.githubassets.com/images/icons/emoji/unicode/1f41b.png">🐛</g-emoji> Bug</h2>
<p dir="auto">In the example below, I want to subtract the value of the first column from all columns of a 2D array/tensor.<br>
i.e. <code class="notranslate">X -= X[:,0][:,np.newaxis]</code><br>
(or with pytorch can also do <code class="notranslate">unsqueeze(1)</code> instead of <code class="notranslate">[:,np.newaxis]</code>)</p>
<p dir="auto">However, the above expression gives different results when X is a numpy array vs when it is a pytorch Tensor.  Specifically, with pytorch it seems as though the broadcasting is "lazy", and only broadcasts the first column after the first column's self-subtraction.  While with numpy gives the "expected" results.<br>
Is it possible that the above expression, with X appearing on both LHS and RHS and relying on broadcasting, does not have well defined semantics?  Or is this a bug?</p>
<h2 dir="auto">To Reproduce</h2>
<p dir="auto">Steps to reproduce the behavior:</p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="import torch
import numpy as np

A = torch.Tensor([[1,2,3], [4,5,6]])
B = A.numpy().copy()

print(A) # [[1,2,3],[4,5,6]]
print(B) # [[1,2,3],[4,5,6]]

# Subtract first column from all columns
A -= A[:,0][:,np.newaxis] # or unsqueeze(1)
B -= B[:,0][:,np.newaxis]

print(A) # [[0,2,3],[0,5,6]] (unexpected result / bug)
print(B) # [[0,1,2],[0,1,2]] (this is expected result in my opinion)"><pre class="notranslate"><span class="pl-k">import</span> <span class="pl-s1">torch</span>
<span class="pl-k">import</span> <span class="pl-s1">numpy</span> <span class="pl-k">as</span> <span class="pl-s1">np</span>

<span class="pl-v">A</span> <span class="pl-c1">=</span> <span class="pl-s1">torch</span>.<span class="pl-v">Tensor</span>([[<span class="pl-c1">1</span>,<span class="pl-c1">2</span>,<span class="pl-c1">3</span>], [<span class="pl-c1">4</span>,<span class="pl-c1">5</span>,<span class="pl-c1">6</span>]])
<span class="pl-v">B</span> <span class="pl-c1">=</span> <span class="pl-v">A</span>.<span class="pl-en">numpy</span>().<span class="pl-en">copy</span>()

<span class="pl-en">print</span>(<span class="pl-v">A</span>) <span class="pl-c"># [[1,2,3],[4,5,6]]</span>
<span class="pl-en">print</span>(<span class="pl-v">B</span>) <span class="pl-c"># [[1,2,3],[4,5,6]]</span>

<span class="pl-c"># Subtract first column from all columns</span>
<span class="pl-v">A</span> <span class="pl-c1">-=</span> <span class="pl-v">A</span>[:,<span class="pl-c1">0</span>][:,<span class="pl-s1">np</span>.<span class="pl-s1">newaxis</span>] <span class="pl-c"># or unsqueeze(1)</span>
<span class="pl-v">B</span> <span class="pl-c1">-=</span> <span class="pl-v">B</span>[:,<span class="pl-c1">0</span>][:,<span class="pl-s1">np</span>.<span class="pl-s1">newaxis</span>]

<span class="pl-en">print</span>(<span class="pl-v">A</span>) <span class="pl-c"># [[0,2,3],[0,5,6]] (unexpected result / bug)</span>
<span class="pl-en">print</span>(<span class="pl-v">B</span>) <span class="pl-c"># [[0,1,2],[0,1,2]] (this is expected result in my opinion)</span></pre></div>
<h2 dir="auto">Expected behavior</h2>
<p dir="auto">Expected pytorch result (A) to match numpy result (B).</p>
<h2 dir="auto">Environment</h2>
<p dir="auto">PyTorch version: 1.1.0<br>
Is debug build: No<br>
CUDA used to build PyTorch: 9.0.176</p>
<p dir="auto">OS: Ubuntu 14.04.5 LTS<br>
GCC version: (Ubuntu 4.9.4-2ubuntu1~14.04.1) 4.9.4<br>
CMake version: version 3.2.2</p>
<p dir="auto">Python version: 3.6<br>
Is CUDA available: Yes<br>
CUDA runtime version: 8.0.61<br>
GPU models and configuration:<br>
GPU 0: NVS 310<br>
GPU 1: TITAN X (Pascal)</p>
<p dir="auto">Nvidia driver version: 384.130<br>
cuDNN version: Probably one of the following:<br>
/usr/local/cuda-8.0/targets/x86_64-linux/lib/libcudnn.so.6.0.21<br>
/usr/local/cuda-8.0/targets/x86_64-linux/lib/libcudnn.so.7.0.2</p>
<p dir="auto">Versions of relevant libraries:<br>
[pip3] numpy==1.16.2<br>
[pip3] torch==1.1.0<br>
[pip3] torchfile==0.1.0<br>
[pip3] torchvision==0.2.2.post3<br>
[conda] Could not collect</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="t = torch.range(1,10)
t[1:] = t[:-1] # wrong result
print(t)
t = torch.range(1,10)
t[1:] = t[:-1].clone() # right result
print(t)
t = torch.range(1, 10).numpy()
t[1:] = t[:-1] # right result
print(t)"><pre class="notranslate"><code class="notranslate">t = torch.range(1,10)
t[1:] = t[:-1] # wrong result
print(t)
t = torch.range(1,10)
t[1:] = t[:-1].clone() # right result
print(t)
t = torch.range(1, 10).numpy()
t[1:] = t[:-1] # right result
print(t)
</code></pre></div>
<p dir="auto">Note that the same (wrong?) behaviour is present in lua torch.</p>