Scratch my previous message below. I had forgotten, indeed we use TensorFlow librayr for IO, as you mentioned.
Maybe trying to remove the Tensorflow_io dependency ? It should be optional, if you don’t need it – and I assume you won’t need it in Raspberry PI.

This is odd. For Raspberry PI I assume you only want the C++ inference, by building Yggdrasil directly (GitHub - google/yggdrasil-decision-forests: A collection of state-of-the-art algorithms for the traini), there should be no dependencies to TensorFlow.
Notice the model you train with TensorFlow can be used by the C++ library in Yggdrasil – no conversions needed, the yggdrasil model is saved under assets/ subdirectory.