Hi Bhack,
Thank you for responding.
It looks like a great resource, and it might be quite helpful for me to try it out.
I am trying to implement Pipeline(data + model) parallelism. I am using multithreading to do so but it seems like GradientTape cannot follow all the operations, possibly due to multithreading. I reckon that wrapping the function that has been parallelised, within a tf.function and using tf.gradients, could solve this issue.
I will try out the implementation 2 suggested in that issue and see if it can help me out.
Thank you once again.