Yes, this link (Saving and loading a model in Pytorch? 346) has an example of optimizer as well. So, basically, you create a dictionary and save the checkpoint as follows:
        save_checkpoint({
            'epoch': epoch + 1,
            'arch': args.arch,
            'state_dict': model.state_dict(),
            'best_prec1': best_prec1,
            'optimizer' : optimizer.state_dict(),
        }, is_best)

Note that I just copied this from that link.