Found the solution and posting it here, in case someone else needs it. i had to call insert a clone to make everything work:
inputs.push_back(torch::from_blob(data.data(), {375,375}).clone().to(device));