Thanks for the answer! In the mean time I came to the same conclusion and used the following snippet as a workaround. That way I can import the nn.Module defined in deform_conv.py from anywhere I want without having to worry about getting the path right. I’m just a bit surprised, that the torch.utils.cpp_extension.load() function doesn’t take care of that.
# deform_conv.py

import os
from torch.utils.cpp_extension import load
parent_dir = os.path.dirname(os.path.abspath(__file__))
sources = ['src/deform_conv_cuda.cpp', 'src/deform_conv_cuda_kernel.cu']  # Paths of sources relative to this file
abs_sources = [os.path.join(parent_dir, source) for source in sources]  # Absolute paths of sources
deform_conv_cuda = load(name='deform_conv_cuda', sources=abs_sources)  # JIT compilation of extensions