I didn’t know that I need to provide engine in the predict net definition.
After running:
for op in predict_net.op:
if op.type == ‘Conv’:
op.engine = ‘NNPACK’
more cores are used (although usage doesn’t cross 400% for 8 cores device)