No problem. Actually, while I think the basic idea of selective compilation is correct, you can do the setup side more easily by just checking for the CUDA toolkit and if detected install a CUDAExtension otherwise install a CppExtension. See torchvision for an example of how to do this nicely. Especially around here 4 where, removing all the actual config, you have this sort of thing:
def get_extensions():
    extension=CppExtension
    if (torch.cuda.is_available() and CUDA_HOME is not None) or os.getenv('FORCE_CUDA', '0') == '1':
        extension = CUDAExtension
        define_macros += [('WITH_CUDA', None)]
    return extension(define_macros)

setup(
    ext_modules=get_extensions()
)

Where that define is used to add in CUDA stuff in the C++. Much cleaner than my suggestion of a command.