I forget to mention how I saved the weight ,I’ve edit that in the train function,by adding
torch.save(model.state_dict(),join(weight_dir,"Epoch_{
}_{:.3f}.pth".format(epoch,smooth_loss)))

So it looks like:
global best_loss
    model.eval()    
    if smooth_loss<best_loss:
        weight_dir = join("weights",args.files.split(".")[0])
        if not exists(weight_dir):
            mkdir(weight_dir) 
        torch.save(model.state_dict(),join(weight_dir,"Epoch_{
}_{:.3f}.pth".format(epoch,smooth_loss)))
        best_loss = smooth_loss

Basically I save every epochs weight cause I would predict the data in the training set to check if model learns well.I turn the model in to eval model.
And then I do prediction ,I didn’t process the weight What I do is simply do :
model = Retina_Net()
losses=[]
#sort and use the one with lowest loss 
for weight in weights:
    losses.append((weight,float(weight.split("/")[-1].split("_")[-1].split(".pth")[0])))
losses.sort(key= lambda x:x[1])

model.load_state_dict(torch.load(losses[0][0]))

#turn the model into test mode 
model.eval()

#then predict

If I miss any information ,please let me know,Thanks!