Solved by albanD in post #2 
              
              
                Yes singleton dimensions are dimensions of size 1. 
This is relevant here as we do automatic broadcasting of singleton dimensions: an addition (500 x 1) + (500 x 500) = (500 x 500) where the first tensor dimensions is expanded to 500. 
In your case, you have a dimension where one is of size 5000 and…