I think I have now found the issue. By the looks of it the audio input seems to the cause of the problem. I have not worked out how to fix that yet. Once I have a solution for that, if it is useful, I will update this thread with such.
To get to this outcome I tried to the train the model using an implementation with the CIFAR-10 dataset. This worked but on moving back to my dataset the issue returned. I also tried with the video modality (as a series of frames) which worked.
Thanks ptrblock for your help. I would have tried your latest suggestion but I was already part way through testing with CIFAR-10 which ended up with leading me to where I am now.