<h2 dir="auto">Issue description</h2>
<p dir="auto">Hello! I am trying to install pytorch from source in conda, and I got these error message:</p>
<blockquote>
<p dir="auto">/home/lin/pytorch/torch/lib/THD/base/data_channels/DataChannelNccl.cpp:31:17: error: ‘ncclInt8’ was not declared in this scope<br>
{at::kChar, ncclInt8},<br>
^<br>
/home/lin/pytorch/torch/lib/THD/base/data_channels/DataChannelNccl.cpp:32:17: error: ‘ncclUint8’ was not declared in this scope<br>
{at::kByte, ncclUint8},<br>
^<br>
/home/lin/pytorch/torch/lib/THD/base/data_channels/DataChannelNccl.cpp:35:16: error: ‘ncclInt32’ was not declared in this scope<br>
{at::kInt, ncclInt32},<br>
^<br>
/home/lin/pytorch/torch/lib/THD/base/data_channels/DataChannelNccl.cpp:38:1: error: could not convert ‘{{at::kChar, }, {at::kByte, }, {at::kFloat, ncclFloat}, {at::kDouble, ncclDouble}, {at::kInt, }, {at::kLong, ncclInt64}, {at::kHalf, ncclHalf}}’ from ‘’ to ‘std::unordered_map&lt;at::ScalarType, ncclDataType_t&gt;’<br>
};<br>
^<br>
In file included from /home/lin/pytorch/torch/lib/THD/base/data_channels/DataChannelNccl.cpp:1:0:<br>
/home/lin/pytorch/torch/lib/THD/base/data_channels/DataChannelNccl.cpp: In member function ‘void thd::DataChannelNccl::_destroyNcclResources(THDGroup)’:<br>
/home/lin/pytorch/torch/lib/THD/base/data_channels/DataChannelNccl.cpp:187:40: error: void value not ignored as it ought to be<br>
NCCL_CHECK(ncclCommDestroy(comm));<br>
^<br>
/home/lin/pytorch/torch/lib/THD/base/data_channels/DataChannelNccl.hpp:16:26: note: in definition of macro ‘NCCL_CHECK’<br>
ncclResult_t error = cmd;                                             <br>
                          ^<br>
/home/lin/pytorch/torch/lib/THD/base/data_channels/DataChannelNccl.cpp: In member function ‘thd::NcclResourcePair thd::DataChannelNccl::_getNcclResourcePair(std::vectorat::Tensor&amp;, THDGroup)’:<br>
/home/lin/pytorch/torch/lib/THD/base/data_channels/DataChannelNccl.cpp:295:29: error: ‘ncclGroupStart’ was not declared in this scope<br>
NCCL_CHECK(ncclGroupStart());<br>
^<br>
/home/lin/pytorch/torch/lib/THD/base/data_channels/DataChannelNccl.hpp:16:26: note: in definition of macro ‘NCCL_CHECK’<br>
ncclResult_t error = cmd;                                             <br>
                          ^<br>
/home/lin/pytorch/torch/lib/THD/base/data_channels/DataChannelNccl.cpp:302:27: error: ‘ncclGroupEnd’ was not declared in this scope<br>
NCCL_CHECK(ncclGroupEnd());<br>
^<br>
/home/lin/pytorch/torch/lib/THD/base/data_channels/DataChannelNccl.hpp:16:26: note: in definition of macro ‘NCCL_CHECK’<br>
ncclResult_t error = cmd;                                             <br>
                          ^<br>
/home/lin/pytorch/torch/lib/THD/base/data_channels/DataChannelNccl.cpp: In member function ‘virtual void thd::DataChannelNccl::allReduce(std::vectorat::Tensor&amp;, THDReduceOp, THDGroup)’:<br>
/home/lin/pytorch/torch/lib/THD/base/data_channels/DataChannelNccl.cpp:417:29: error: ‘ncclGroupStart’ was not declared in this scope<br>
NCCL_CHECK(ncclGroupStart());<br>
^<br>
/home/lin/pytorch/torch/lib/THD/base/data_channels/DataChannelNccl.hpp:16:26: note: in definition of macro ‘NCCL_CHECK’<br>
ncclResult_t error = cmd;                                             <br>
                          ^<br>
/home/lin/pytorch/torch/lib/THD/base/data_channels/DataChannelNccl.cpp:432:27: error: ‘ncclGroupEnd’ was not declared in this scope<br>
NCCL_CHECK(ncclGroupEnd());<br>
^<br>
/home/lin/pytorch/torch/lib/THD/base/data_channels/DataChannelNccl.hpp:16:26: note: in definition of macro ‘NCCL_CHECK’<br>
ncclResult_t error = cmd;                                             <br>
                          ^<br>
/home/lin/pytorch/torch/lib/THD/base/data_channels/DataChannelNccl.cpp: In member function ‘virtual void thd::DataChannelNccl::allGather(std::vectorat::Tensor&amp;, std::vectorat::Tensor&amp;, THDGroup)’:<br>
/home/lin/pytorch/torch/lib/THD/base/data_channels/DataChannelNccl.cpp:466:29: error: ‘ncclGroupStart’ was not declared in this scope<br>
NCCL_CHECK(ncclGroupStart());<br>
^<br>
/home/lin/pytorch/torch/lib/THD/base/data_channels/DataChannelNccl.hpp:16:26: note: in definition of macro ‘NCCL_CHECK’<br>
ncclResult_t error = cmd;                                             <br>
                          ^<br>
/home/lin/pytorch/torch/lib/THD/base/data_channels/DataChannelNccl.cpp:473:27: error: invalid conversion from ‘void*’ to ‘int’ [-fpermissive]<br>
output[i].data_ptr(),<br>
^<br>
/home/lin/pytorch/torch/lib/THD/base/data_channels/DataChannelNccl.hpp:16:26: note: in definition of macro ‘NCCL_CHECK’<br>
ncclResult_t error = cmd;                                             <br>
                          ^<br>
/home/lin/pytorch/torch/lib/THD/base/data_channels/DataChannelNccl.cpp:474:23: error: invalid conversion from ‘int64_t {aka long int}’ to ‘ncclDataType_t’ [-fpermissive]<br>
input[i].numel(),<br>
^<br>
/home/lin/pytorch/torch/lib/THD/base/data_channels/DataChannelNccl.hpp:16:26: note: in definition of macro ‘NCCL_CHECK’<br>
ncclResult_t error = cmd;                                             <br>
                          ^<br>
/home/lin/pytorch/torch/lib/THD/base/data_channels/DataChannelNccl.cpp:477:15: error: cannot convert ‘ncclDataType_t’ to ‘void*’ for argument ‘4’ to ‘ncclResult_t ncclAllGather(const void*, int, ncclDataType_t, void*, ncclComm_t, cudaStream_t)’<br>
stream));<br>
^<br>
/home/lin/pytorch/torch/lib/THD/base/data_channels/DataChannelNccl.hpp:16:26: note: in definition of macro ‘NCCL_CHECK’<br>
ncclResult_t error = cmd;                                             <br>
                          ^<br>
/home/lin/pytorch/torch/lib/THD/base/data_channels/DataChannelNccl.cpp:480:27: error: ‘ncclGroupEnd’ was not declared in this scope<br>
NCCL_CHECK(ncclGroupEnd());<br>
^<br>
/home/lin/pytorch/torch/lib/THD/base/data_channels/DataChannelNccl.hpp:16:26: note: in definition of macro ‘NCCL_CHECK’<br>
ncclResult_t error = cmd;                                             <br>
                          ^<br>
/home/lin/pytorch/torch/lib/THD/base/data_channels/DataChannelNccl.cpp: In member function ‘virtual void thd::DataChannelNccl::reduce(std::vectorat::Tensor&amp;, THDReduceOp, thd::rank_type, THDGroup)’:<br>
/home/lin/pytorch/torch/lib/THD/base/data_channels/DataChannelNccl.cpp:516:29: error: ‘ncclGroupStart’ was not declared in this scope<br>
NCCL_CHECK(ncclGroupStart());<br>
^<br>
/home/lin/pytorch/torch/lib/THD/base/data_channels/DataChannelNccl.hpp:16:26: note: in definition of macro ‘NCCL_CHECK’<br>
ncclResult_t error = cmd;                                             <br>
                          ^<br>
/home/lin/pytorch/torch/lib/THD/base/data_channels/DataChannelNccl.cpp:532:27: error: ‘ncclGroupEnd’ was not declared in this scope<br>
NCCL_CHECK(ncclGroupEnd());<br>
^<br>
/home/lin/pytorch/torch/lib/THD/base/data_channels/DataChannelNccl.hpp:16:26: note: in definition of macro ‘NCCL_CHECK’<br>
ncclResult_t error = cmd;                                             <br>
                          ^<br>
/home/lin/pytorch/torch/lib/THD/base/data_channels/DataChannelNccl.cpp: In member function ‘virtual void thd::DataChannelNccl::broadcast(std::vectorat::Tensor&amp;, thd::rank_type, THDGroup)’:<br>
/home/lin/pytorch/torch/lib/THD/base/data_channels/DataChannelNccl.cpp:568:29: error: ‘ncclGroupStart’ was not declared in this scope<br>
NCCL_CHECK(ncclGroupStart());<br>
^<br>
/home/lin/pytorch/torch/lib/THD/base/data_channels/DataChannelNccl.hpp:16:26: note: in definition of macro ‘NCCL_CHECK’<br>
ncclResult_t error = cmd;                                             <br>
                          ^<br>
/home/lin/pytorch/torch/lib/THD/base/data_channels/DataChannelNccl.cpp:582:27: error: ‘ncclGroupEnd’ was not declared in this scope<br>
NCCL_CHECK(ncclGroupEnd());<br>
^<br>
/home/lin/pytorch/torch/lib/THD/base/data_channels/DataChannelNccl.hpp:16:26: note: in definition of macro ‘NCCL_CHECK’<br>
ncclResult_t error = cmd;                                             \</p>
</blockquote>
<p dir="auto">I have searched on google. And I find it may be the problem of nccl. I run</p>
<blockquote>
<p dir="auto">export NO_SYSTEM_NCCL=1</p>
</blockquote>
<p dir="auto">but it didn't work.</p>
<h2 dir="auto">System Info</h2>
<ul dir="auto">
<li>PyTorch</li>
<li>How you installed PyTorch (conda, pip, source):conda</li>
<li>Build command you used (if compiling from source):python setup install</li>
<li>OS: Ubuntu 16.04</li>
<li>PyTorch version: 1.0.0</li>
<li>Python version: 3.7.1</li>
<li>CUDA/cuDNN version: Cuda8 cudnn 7.0.5</li>
<li>GPU models and configuration: 1080Ti</li>
<li>GCC version (if compiling from source): gcc 5.3.1</li>
<li>CMake version: 3.12.2</li>
</ul>
<p dir="auto">We currently only support int varargs when they are the only argument in the Python torch wrapping. This limits, for example, the use of varargs in einsum.</p>
<p dir="auto">The root cause is in <a href="https://github.com/pytorch/pytorch/blob/c58893eb9eb9fca2c61aaee1c7013983569dad61/torch/csrc/utils/python_arg_parser.h#L14">the pattern for using <code class="notranslate">PythonArgParser</code> with the templated <code class="notranslate">ParsedArgs</code> as documented in the comment at the top of <code class="notranslate">torch/csrc/utils/python_arg_parser.h</code></a>.<br>
The fact that <code class="notranslate">ParsedArgs</code> are accessible without <code class="notranslate">PythonArgParser</code> keeps us from doing Python object allocations (for a slice of arguments to assign to the vararg) that could be cleaned up in the destructor of <code class="notranslate">PythonArgParser</code> without feeling dirty.<br>
I see two obivous ways out:</p>
<ol dir="auto">
<li>move the <code class="notranslate">ParsedArgs</code> structure into PythonArgParser and make that templated (but I would guess  that there was a reason to not do that all along - possibly duplicated functions)</li>
<li>add a <code class="notranslate">std::vector&lt;PyObject*&gt; varargs</code> to <code class="notranslate">ParsedArgs</code> that gets the varargs.</li>
</ol>
<p dir="auto">Would either of these be acceptable?</p>
<p dir="auto">Best regards</p>
<p dir="auto">Thomas</p>