It means: give me the mean of the elements iterated of the axis dim.
In your example what it does is:
(t[0] + t[1]) / num_elements = ([1, 2]+ [3, 4]) / 2 = [2., 3.]

Similarly if you specified dim1, it would do:
(t[:,0] + t[:,1]) / 2 = tensor([ 7., 13.])

same result as
>>> t.mean(dim=1)
tensor([ 7., 13.])