@wanchaol, ok it was a stupid error on my side. Thanks for the help, now the inference works fine, I’d say we can close.
PS: I really would like to check the weights and the general state of the model, is there something similar to load_state_dict() in C++? That would make eveything easier.