@markl
model.forward(test_data) works and outputs some tensors (boxes).
But the with torch.jit.trace I still have the issue
RuntimeError: Only tensors or tuples of tensors can be output from traced functions (getNestedOutputTrace at /pytorch/torch/csrc/jit/tracer.cpp:200)

Not sure if jit trace fully supports maskrcnn model for now.
@tancl
I updated torch and torchvision versions to 1.3 and 0.4.1 respectively. still the same error.