Solved by yassersouri in post #3 
              
              
                What I found is that, Although is it mentioned in the docs that the batch size should be larger than the number of GPUs, at least during evaluation, there is no error if the batch size is smaller than the number of GPUs. 
So I guess the answer is to do nothing. Do not drop the last batch when evalua…