This is because usually, the concept of row/column correspond to the last two dimensions of a Tensor.
And when you add a new dimension, what used to be rows become columns. And if you stack stuff on that new dimension, they are columns.