Your custom layer isn’t using the initialized self.BConv_layer and is using undefined arguments, such as kernel and y, instead, so this won’t work.
Besides that, I’m unsure if you need the no_grad block. If my_function isn’t using any parameters, you could probably remove it.
Also, if you can replace all numpy operations with their PyTorch equivalent, you should be able to use the GPU for all operations. If that’s not possible, note that each data transfer to the CPU would be blocking and might yield a slowdown.