Did you make sure the Dataset is working properly?
Just try to get some random samples and see, if the values make sense:
print(trainset[0])
print(trainset[10])
print(trainset[-1])

If that looks fine, try to lower your learning rate as it might be a bit high so that your training gets stuck from the beginning.