Thanks for your help. This is my simplified source code.
My network contains conv2d, batchnorm2d, batchnorm1d, relu, sigmoid and linear layers.

I load the best chkpt since I only save the best ckpt.
I test it firstly before backprob in main loop.
I recreate my optimizer and network and load_state_dict
I do network.eval() before testing.
please see the following code to find my problems.


	net = Naive5(n_way, k_shot, imgsz).cuda() 

	if os.path.exists(mdl_file):
		print('load checkpoint ...', mdl_file)
		net.load_state_dict(torch.load(mdl_file), strict=False)
	else:
		print('training from scratch.')
 
	optimizer = optim.Adam(net.parameters(), lr=lr) 
 
 

		for step, batch in enumerate(db):

			accuracy = test_accuracy()
			print('<<<<>>>>accuracy:', accuracy, 'best accuracy:', best_accuracy)
 

			# 2. train
			support_x = Variable(batch[0]).cuda()
			support_y = Variable(batch[1]).cuda()
			query_x = Variable(batch[2]).cuda()
			query_y = Variable(batch[3]).cuda()

			net.train()
			loss = net(support_x, support_y, query_x, query_y)
			total_train_loss += loss.data[0]

			optimizer.zero_grad()
			loss.backward()
			optimizer.step()