ReduceOp is a C++ enum, and is exposed to the python interface using pybind (https://github.com/pytorch/pytorch/blob/master/torch/csrc/distributed/c10d/init.cpp#L145 5). That enum is defined here: https://github.com/pytorch/pytorch/blob/master/torch/lib/c10d/Types.hpp#L8 4