Hi guys,
I am loading weights and performing only the feedforward step using libtorch in C++. Since I am on CPU it comes out that this prediction step is slower than the next iteration of the outer while loop. The scenario is the following:
while (...){
  output = torch_predict(input);
  display_results(output);
}

Very often output is empty, or not complete. If I add a sleep on my executable this appear less frequently. I guess is a problem related to the different thread(s) in which libtorch is operating. Is there any way I can use a mutex? In case I can know anyhow when libtorch finished is computation for the current iteration. A solution could be migrating to GPU?
Thanks in advance