Thanks for the input.  Using the same arch for Keras/TF and pyTorch was very important.  I did get the accuracy very close last night with the following addition.
for epoch in range(200):
    _ = model.train()
    for X_mb,y_mb in train_loader:
        ...

    _ = model.eval()
    yhat_train = model(...)
    yhat_valid = model(...)
    ...

Adding those those line brings the accuracy up to where it is about the same as TF.  Without those (and I just verified) even with the changed arch, the accuracy is lower.  I read somewhere that those are important if one is using Dropout or BatchNorm…