First, the normalization std looks like big,
Second, add the Batch normalization is recommended.
Third, the convolution channel is quite shallow why don’t you make more deeper.
If you read the code of any famous architecture they start such as 16 or 32 channel.
nn.conv2d(3,16,kernel_size=3)
like this.
I hope this answer is helpful.
Thank you.