ptrblck:

t code cannot be build using a recent source build and fails with:


Hi, @ptrblck Thank so much for your attention on the question.
It seems the api was changed since pytorch 1.8.
In my eary trival, I didn’t build the code again the source code of the pytorch. Instead I build it with a pre-installed version.
I found the function definition in path something like
 ~/.pyenv/versions/3.6.5/lib/python3.6/site-packages/torch/include/ATen/NativeFunctions.h
The path prefix might be different on different machines.
If I ran
grep layer_norm ~/.pyenv/versions/3.6.5/lib/python3.6/site-packages/torch/include/ATen/NativeFunctions.h on pytorch1.5/1.6/1.7  it gave
CAFFE2_API Tensor layer_norm(const Tensor & input, IntArrayRef normalized_shape, const Tensor & weight={}, const Tensor & bias={}, double eps=1e-05, bool cudnn_enable=true);
CAFFE2_API std::tuple<Tensor,Tensor,Tensor> layer_norm_cpu(const Tensor & input, const Tensor & weight, const Tensor & bias, int64_t M, int64_t N, double eps);
CAFFE2_API std::tuple<Tensor,Tensor,Tensor> layer_norm_cuda(const Tensor & input, const Tensor & weight, const Tensor & bias, int64_t M, int64_t N, double eps);
CAFFE2_API std::tuple<Tensor,Tensor,Tensor> layer_norm_backward_cpu(const Tensor & grad_out, const Tensor & input, const Tensor & mean, const Tensor & rstd, const Tensor & weight, int64_t M, int64_t N, std::array<bool,3> output_mask);
CAFFE2_API std::tuple<Tensor,Tensor,Tensor> layer_norm_backward_cuda(const Tensor & grad_out, const Tensor & input, const Tensor & mean, const Tensor & rstd, const Tensor & weight, int64_t M, int64_t N, std::array<bool,3> output_mask);

I just tried the grep on pytorch 1.8, it gave
TORCH_API Tensor layer_norm(const Tensor & input, IntArrayRef normalized_shape, const Tensor & weight={}, const Tensor & bias={}, double eps=1e-05, bool cudnn_enable=true);
TORCH_API std::tuple<Tensor,Tensor,Tensor> layer_norm_cpu(const Tensor & input, IntArrayRef normalized_shape, const Tensor & weight, const Tensor & bias, double eps);
TORCH_API std::tuple<Tensor,Tensor,Tensor> layer_norm_cuda(const Tensor & input, IntArrayRef normalized_shape, const Tensor & weight, const Tensor & bias, double eps);
TORCH_API std::tuple<Tensor,Tensor,Tensor> math_native_layer_norm(const Tensor & input, IntArrayRef normalized_shape, const Tensor & weight, const Tensor & bias, double eps);
TORCH_API std::tuple<Tensor,Tensor,Tensor> layer_norm_backward_cpu(const Tensor & grad_out, const Tensor & input, IntArrayRef normalized_shape, const Tensor & mean, const Tensor & rstd, const Tensor & weight, const Tensor & bias, std::array<bool,3> output_mask);
TORCH_API std::tuple<Tensor,Tensor,Tensor> layer_norm_backward_cuda(const Tensor & grad_out, const Tensor & input, IntArrayRef normalized_shape, const Tensor & mean, const Tensor & rstd, const Tensor & weight, const Tensor & bias, std::array<bool,3> output_mask);

I have updated the code on github repo (run git pull, please ) to make it support pytorch 1.8. However the error was still there.