One can use simpler approach with deepC compiler 70 and convert exported onnx model to c++.
Check out simple example at deepC compiler sample test 51
Compile onnx model for your target machine
Checkout mnist.ir 18
Step 1:
Generate intermediate code
% onnx2cpp mnist.onnx
Step 2:
Optimize and compile
% g++ -O3 mnist.cpp -I ../../../include/ -isystem ../../../packages/eigen-eigen-323c052e1731/ -o mnist.exe
Step 3:
Test run
% ./mnist.exe