A tensor of size (2, 2, 2, 2) is four-dimensional. Yes, it can hold 2 * 2 * 2 * 2 = 16 elements.
A tensor is a generic n-dimensional array, very similar to numpy arrays. The nice thing about using pytorch tensors in general is that you can easily put them on the gpu (as opposed to numpy arrays, which only exist on the cpu).