@TimZaman Could you share a little bit more about how the hacky solution is done exactly?
For example, in the description:
Per extension, create a bazel genrule, that just invokes a python setup.py build, and set the resulting .so file as the output artifact. This can then be loaded in the code. This leverages all the nice abstractions and build arguments that are set through the torch utilities (torch.utils.cpp_extension.$)
How do these files looks like?

BUILD
The .cpp file of custom C++ extension
The .py file calling the custom C++ extension

I am also interesting in writing a post about this for future reference. Thank you.