I train these models in a single script. Actually they are a Generator(G) and a Classifier(C ), and this error occurs when the Generator are generating data for the Classifier. Because the batchsize of the Generator is smaller than the Classifier, I run the Generator for a couple of times in order to generate enough data for the Classifier. And I observed that if I run the Generator for only one time, this error did not occur.
Here is an example snippet.
num = C_batch_size // G_batch_size # if I set num=1, this error won't occur
for i in range(num):
    input_i = input[i*num : (i+1)*num]
    data.append(G(input_i)) # G generates data for C, this is where the error occurs
C(data)