Thanks for the reply.
In keras model I had 2 Dense layers, hence 2 hidden layers in the pytorch example. If I remove F.relu(self.hidden2(x)), then it seems that self.hidden2() is not needed anymore? If so, then I will only have one hidden layer in the pytorch and both examples will not be equivalent?
Reminder, in keras I had:
network.summary()

Layer (type)                 Output Shape              Param #   
=================================================================
dense_1 (Dense)              (None, 392)               307720    
_________________________________________________________________
dense_2 (Dense)              (None, 10)                3930      
=================================================================
Total params: 311,650
Trainable params: 311,650
Non-trainable params: 0

Following your suggestions I refactored my pytorch code to the following:
class Sequential(nn.Module):
    def __init__(self):
        super().__init__()
        self.hidden1 = nn.Linear(784, 10)

    def forward(self, x):
        x = F.relu(self.hidden1(x))
        x = F.softmax(x, dim=1)
        return x

# Build Network
network = Sequential().cuda()
network

Sequential(
  (hidden1): Linear(in_features=784, out_features=10, bias=True)
)

optimizer_fn = optim.RMSprop(network.parameters(), lr=0.001)
loss_fn = nn.MSELoss()

# Train
epochs = 10
for epoch in range(1, epochs+1):
    # forward pass
    trained = network(train_img[:-10000])
    loss = loss_fn(trained, train_lbl[:-10000])
    # validation
    val_trained = network(train_img[-10000:])
    val_loss = loss_fn(val_trained, train_lbl[-10000:])
    
    print('Epoch {epoch}/{epochs}. loss: {loss}, val_loss: {val_loss}'.format(**vars()))
    
    # backward pass
    optimizer_fn.zero_grad()
    loss.backward()
    optimizer_fn.step()

Epoch 1/10. loss: 0.08996006101369858, val_loss: 0.08998527377843857
Epoch 2/10. loss: 0.0835660770535469, val_loss: 0.08396155387163162
Epoch 3/10. loss: 0.07465378940105438, val_loss: 0.07502195239067078
Epoch 4/10. loss: 0.07493914663791656, val_loss: 0.07497056573629379
Epoch 5/10. loss: 0.09106428176164627, val_loss: 0.0923982784152031
Epoch 6/10. loss: 0.07770228385925293, val_loss: 0.07810225337743759
Epoch 7/10. loss: 0.07583754509687424, val_loss: 0.07622469961643219
Epoch 8/10. loss: 0.06659538298845291, val_loss: 0.0668170228600502
Epoch 9/10. loss: 0.06550104916095734, val_loss: 0.06539501249790192
Epoch 10/10. loss: 0.0665767565369606, val_loss: 0.06721500307321548

While in keras the loss value seems to have a clear downwards direction, in my pytorch example it rather fluctuates back and forth.