Hi Sam,
I don’t have an answer to your question, sorry but I have one suggestions:

there’s this tutorial: Image captioning with visual attention  |  TensorFlow Core 5 that might give you some ideas.

good luck