Thank you for your comments! With your help, I hacked my way through. I did using sligtly different argument of ffibuilder.set_source that described previously:
I first compile all cuda kernels:
Cmake:
CUDA_ADD_LIBRARY(~ STATIC c~CUDAKernels.cu)
TARGET_LINK_LIBRARIES(~)

and then link them as an extra link argument:
...
extra_link_args=['~.a']
ffi = create_extension(
    '~',
    headers=headers,
    sources=sources,
    define_macros=defines,
    relative_to=__file__,
    with_cuda=with_cuda,
    include_dirs = include_dirs,
    extra_compile_args=["-fopenmp"],
    extra_link_args = extra_link_args
)
...

This workaround does work with a static library. I had problems trying to link a shared library this way (haven’t tried enough though).