According to the video, they have an a runtime library (StreamExecutor) which abstracts over CUDA and OpenCL. That seems like the hard part to me. Switching out the LLVM backend for kernels ought to be relatively easy.