Each layer specifies the input and target dimensions.
E.g. in the docs 25 of nn.Conv2d you see the input defined as [N, Cin​, Hin​, Win​].
Generally, all layers usually found in a CNN will accept this shape (pooling, normalization layers etc.).
Your example of the 5-dimensional tensor would be a possible input to nn.Conv3d.
However, you have to be careful when it comes to e.g. RNNs, as the default shape is expected as [seq_len, batch_size, features]. You could use batch_first=True while creating the RNN to use inputs as [batch_size, seq_len, features].
The design choices were most likely made due to performance reasons.