Hello all, I am beginner in neural net, Just want to understand Activation Function.
Is this applicable as activation function? By testing the two codes, the loss of the first one is not decreasing too much than the second one.
self.sigmoid(self.relu(x))
instead to
self.sigmoid(x)