Hi Callum,
I think you’re in the right direction but I’d suggest you try this tutorial: Text classification with an RNN  |  TensorFlow 3
it will give you some insights and maybe you can even adapt your model a little bit.