Solved by mhubii in post #4 
              
              
                you could just create a tensor of size BxCxHxW, instead of pushing multiple tensors of size 1xCxHxW to your  std::vector<torch::jit::IValue>. For example 
auto module = torch::jit::load("../model.pt");

// Thats how to forward a whole batch.
std::vector<torch::jit::IValue> inputs;
inputs.push_back(t…