To add on, using activation functions from torch.nn seems to cause the same problem before any training occurs. When torch or torch.nn.functional activation functions are used, the results displayed are before training are random.