Hi Daniel!



 Danielr13:

class Network(nn.Module):
    def __init__(self):
...
        # hidden to output layer, 3 classes - one for each cultivar
        self.output = nn.Linear(25, 3)
...
        z2 = self.output(out1)
        out2 = self.sigmoid(z2)
        return out2



Don’t apply a Sigmoid to the output of your last `Linear’ layer.
That is, use, for example:
        return z2

CrossEntropyLoss wants the output you pass to it to be raw-score
logits that run from -inf to inf.  The Sigmoid compresses these
values to run from 0.0 to 1.0 (so they look like probabilities), and,
in effect, you would have to train your network to undo the damage
caused by the Sigmoid – something that will be hard (impossible?)
to do with a single-hidden-layer network.


I am using a Cross-Entropy Loss and have my targets one-hot encoded.
#one-hot encoding targets
y_train_encoded = to_categorical(y_train)[:,1:4]



I assume that this line is converting your targets to one-hot format.


…
criterion = nn.CrossEntropyLoss() #cross-entropy loss
…
output = model.forward(X_train)
…
loss = criterion(output.squeeze(), torch.max(y_train, 1)[1])



But the torch.max(y_train, 1)[1] that you pass to criterion
converts your one-hot encoded targets back to integer class labels.
This is correct for CrossEntropyLoss (as CrossEntropyLoss does
not work with one-hot targets), but it seems unnecessary to convert
your targets to one-hot, and then back again.
(As an aside, torch.max()[1] is an idiom for torch.argmax().
I believe your version of pytorch is new enough that it supports
torch.argmax(), so it would likely be a bit more readable to use
that.)
Best.
K. Frank