Heiko @marshmellow77 upon closer inspection using Inference Toolkit suits me better.
However the documentation is very scarce and there are no examples I found implementing all these functions like model_fn and etc for custom model. Do you have link to examples?
And is there a way to locally debug them to avoid repeatedly creating endpoints for debugging?
Thanks again