Bonjour/Hello Laurent :),
To make it more visual, here is the model you describe as I understand it:

graphviz491×641 20.5 KB

(Assuming this is the model you describe), this configuration is possible, but there are some caveats:
The TF-DF models don’t backpropagate gradients (yet 4). It means that the parameters of the “Common learnable pre-processing” part cannot be trained through the GBT. Instead, the pre-processing needs be trained with the gradients back-propagated through the “model #1” and “model #2”.
This implies a 2 steps training:

Training of “Common learnable pre-processing” + “model #1” + “model #2”.
Training of “GBT model”.

If there are no “Common learnable pre-processing” stages, all the models (including the GBT) can be trained in parallel (i.e. in the same session call). However, this might not be optimal as GBT trains in exactly on epoch, while NNs generally train over multiple ones. Instead, it will be more interesting to train the GBT and the NN independently (but possibly at the same time).
This colab 18 demonstrates how to train the ensemble shown in the figure. I hope this helps. The API is a bit verbosy, but hopefully, this will improve :).
Cheers,