<h2 dir="auto">🐛 Bug</h2>

<p dir="auto">torch.max seems to be acting differently on CPU/GPU for tensors with equal elements, for example, with all 0 elements.</p>
<p dir="auto">torch.max on CPU will return the last index while on GPU the first index.</p>
<h2 dir="auto">To Reproduce</h2>
<p dir="auto">Steps to reproduce the behavior:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="import torch
print (torch.max(torch.zeros([1, 2001]).to(torch.device('cpu')), 1)[1])
print (torch.max(torch.zeros([1, 2001]).to(torch.device('cuda:0')), 1)[1])"><pre class="notranslate"><code class="notranslate">import torch
print (torch.max(torch.zeros([1, 2001]).to(torch.device('cpu')), 1)[1])
print (torch.max(torch.zeros([1, 2001]).to(torch.device('cuda:0')), 1)[1])
</code></pre></div>
<p dir="auto">The result will be:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="tensor([2000])
tensor([0], device='cuda:0')"><pre class="notranslate"><code class="notranslate">tensor([2000])
tensor([0], device='cuda:0')
</code></pre></div>
<p dir="auto">On CPU it returns the index 2000, while on GPU 0.</p>

<h2 dir="auto">Expected behavior</h2>
<p dir="auto">Expected to return the same index on CPU/GPU, all last index or all first index.</p>

<h2 dir="auto">Environment</h2>
<p dir="auto">The output of <a href="https://raw.githubusercontent.com/pytorch/pytorch/master/torch/utils/collect_env.py" rel="nofollow">environment collection script</a>:</p>
<p dir="auto">PyTorch version: 1.4.0<br>
Is debug build: No<br>
CUDA used to build PyTorch: 10.1</p>
<p dir="auto">OS: Ubuntu 18.04.4 LTS<br>
GCC version: (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0<br>
CMake version: version 3.13.3</p>
<p dir="auto">Python version: 3.6<br>
Is CUDA available: Yes<br>
CUDA runtime version: 10.0.130<br>
GPU models and configuration: GPU 0: Tesla V100-SXM2-16GB<br>
Nvidia driver version: 440.33.01<br>
cuDNN version: Probably one of the following:<br>
/usr/local/cuda-10.1/targets/x86_64-linux/lib/libcudnn.so.7.6.5<br>
/usr/local/cuda-10.2/targets/x86_64-linux/lib/libcudnn.so.7.6.5</p>
<p dir="auto">Versions of relevant libraries:<br>
[pip3] numpy==1.15.4<br>
[conda] blas                      1.0                         mkl<br>
[conda] cudatoolkit               10.1.243             h6bb024c_0<br>
[conda] mkl                       2020.0                      166<br>
[conda] mkl-service               2.3.0            py36he904b0f_0<br>
[conda] mkl_fft                   1.0.15           py36ha843d7b_0<br>
[conda] mkl_random                1.1.0            py36hd6b4f25_0<br>
[conda] numpy                     1.18.1           py36h4f9e942_0<br>
[conda] numpy-base                1.18.1           py36hde5b4d6_1<br>
[conda] numpydoc                  0.9.2                      py_0<br>
[conda] pytorch                   1.4.0           py3.6_cuda10.1.243_cudnn7.6.3_0    pytorch<br>
[conda] torchvision               0.5.0                py36_cu101    pytorch</p>
<h2 dir="auto">🐛 Bug</h2>
<h2 dir="auto">To Reproduce</h2>
<p dir="auto">This code:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="
import time
import torch
import numpy as np

data = [np.random.rand(8, 800, 1333) &gt; 0.5 for _ in range(2)]
dtype_np = np.bool
dtype_pt = torch.bool

def f1():
    return np.asarray(data, dtype=dtype_np)

def f2():
    return np.stack(data)

def f3():
    return torch.as_tensor(data, dtype=dtype_pt)

def f4():
    return torch.stack([torch.from_numpy(x) for x in data])

def benchmark(f, iter, warmup):
    for k in range(warmup): f()
    start = time.perf_counter()
    for k in range(iter): f()
    torch.cuda.synchronize()
    return time.perf_counter() - start

print(benchmark(f1, 10, 1))
print(benchmark(f2, 10, 1))
print(benchmark(f3, 10, 1))
print(benchmark(f4, 10, 1))"><pre class="notranslate"><code class="notranslate">
import time
import torch
import numpy as np

data = [np.random.rand(8, 800, 1333) &gt; 0.5 for _ in range(2)]
dtype_np = np.bool
dtype_pt = torch.bool

def f1():
    return np.asarray(data, dtype=dtype_np)

def f2():
    return np.stack(data)

def f3():
    return torch.as_tensor(data, dtype=dtype_pt)

def f4():
    return torch.stack([torch.from_numpy(x) for x in data])

def benchmark(f, iter, warmup):
    for k in range(warmup): f()
    start = time.perf_counter()
    for k in range(iter): f()
    torch.cuda.synchronize()
    return time.perf_counter() - start

print(benchmark(f1, 10, 1))
print(benchmark(f2, 10, 1))
print(benchmark(f3, 10, 1))
print(benchmark(f4, 10, 1))
</code></pre></div>
<p dir="auto">prints</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="0.2839105408638716
0.013459203764796257
20.43221192806959
0.012655530124902725"><pre class="notranslate"><code class="notranslate">0.2839105408638716
0.013459203764796257
20.43221192806959
0.012655530124902725
</code></pre></div>
<p dir="auto">I don't expect <code class="notranslate">torch.as_tensor</code> to be 100x slower than <code class="notranslate">np.asarray()</code>.</p>
<h2 dir="auto">Environment</h2>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="PyTorch version: 1.2.0.dev20190805
Is debug build: No
CUDA used to build PyTorch: 10.0.130

OS: Ubuntu 18.04.2 LTS
GCC version: (Ubuntu 7.4.0-1ubuntu1~18.04) 7.4.0
CMake version: version 3.12.2

Python version: 3.6
Is CUDA available: Yes
CUDA runtime version: 10.0.130
GPU models and configuration: 
GPU 0: Quadro GP100
GPU 1: Quadro GP100

Nvidia driver version: 410.79
cuDNN version: Could not collect

Versions of relevant libraries:
[pip3] msgpack-numpy==0.4.4.3
[pip3] numpy==1.16.4
[pip3] numpydoc==0.7.0
[pip3] torch-nightly==1.2.0.dev20190805
[pip3] torchvision==0.4.0a0+6c7189f
[conda] blas                      1.0                         mkl  
[conda] mkl                       2019.1                      144  
[conda] mkl-include               2019.1                      144  
[conda] mkl-service               1.1.2            py36he904b0f_5  
[conda] mkl_fft                   1.0.6            py36hd81dba3_0  
[conda] mkl_random                1.0.2            py36hd81dba3_0"><pre class="notranslate"><code class="notranslate">PyTorch version: 1.2.0.dev20190805
Is debug build: No
CUDA used to build PyTorch: 10.0.130

OS: Ubuntu 18.04.2 LTS
GCC version: (Ubuntu 7.4.0-1ubuntu1~18.04) 7.4.0
CMake version: version 3.12.2

Python version: 3.6
Is CUDA available: Yes
CUDA runtime version: 10.0.130
GPU models and configuration: 
GPU 0: Quadro GP100
GPU 1: Quadro GP100

Nvidia driver version: 410.79
cuDNN version: Could not collect

Versions of relevant libraries:
[pip3] msgpack-numpy==0.4.4.3
[pip3] numpy==1.16.4
[pip3] numpydoc==0.7.0
[pip3] torch-nightly==1.2.0.dev20190805
[pip3] torchvision==0.4.0a0+6c7189f
[conda] blas                      1.0                         mkl  
[conda] mkl                       2019.1                      144  
[conda] mkl-include               2019.1                      144  
[conda] mkl-service               1.1.2            py36he904b0f_5  
[conda] mkl_fft                   1.0.6            py36hd81dba3_0  
[conda] mkl_random                1.0.2            py36hd81dba3_0
</code></pre></div>