I’d guess that it’s because you’re using 5x300 convolutional kernels, that are incredibly expensive to compute (7x7 kernels are considered large and expensive).
Apart from this I can’t see any mistakes at a glance. There might be some discrepancies with the original code, but I don’t know it so it’s hard to tell.