Since the image size is small, and the batch size is small, and you are using a suitable GPU, this is probably due to the number of filters that you are sending to this Module.
(assuming you are on a *nix system) can you try starting watch nvidia-smi in the terminal before you run this?
Also try various values for your kernels param. Start with 1, work up till it crashes, etc.