L2 regularization in PyTorch is available in all the optimizers, through the weight_decay parameter.
I modified the code accordingly.
optimizer = torch.optim.Adam(self.parameters(), lr=lrate, weight_decay=0.001)
I tried different values of weight_decay (0.0001, 0.001, 0.01). There is little improvement, but test accuracy is still far worse than TF.
The default weight_decay in tflearn is 0.001. http://tflearn.org/layers/conv/