In fact, it turns out that when saving the model with weights (export_params = True in the call to torch.onnx.export()) the saved model contains a field initializer in the model graph which allows to distinguish network inputs (no corresponding entry in initializer) from model weights (which have an initializer)