Yes singleton dimensions are dimensions of size 1.
This is relevant here as we do automatic broadcasting of singleton dimensions: an addition (500 x 1) + (500 x 500) = (500 x 500) where the first tensor dimensions is expanded to 500.
In your case, you have a dimension where one is of size 5000 and the other 60 so it does not know how to perform that operation.