Neda:

torch.save(model.state_dict(), ‘train_valid_exp4.pth’)


You just need to move the above line inside the for-loop (for epoch in ...) , and give it a filename that has epoch number as well:
      # inside the for-loop:
      torch.save(model.state_dict(), 'train_valid_exp4-epoch{}.pth'.format(epoch)) 

You may also consider every 10 epochs, instead of every epoch if the model takes too much space:
      # inside the for-loop:
      if epoch % 10 == 9:
          torch.save(model.state_dict(), 'train_valid_exp4-epoch{}.pth'.format(epoch+1))