So, the parameters are the following:

in_channels = 4
out_channels = 32
kernel_size = 8
stride = 4

The input is of size (batch_size, n_channels, height, width). Batch dim is 1 in both, so that can be discarded. The second dim of the input is 4, which is the in_channels, and the second dim of the output is 32, which is the out_channels.
Now, how (84, 84) turns into (20, 20) follows the formula described in the conv2d documentation:

h_out = floor(((84 + 2 * 0 - 1 * (8 - 1) - 1))/4 +1) = floor(19 + 1) = 20
w_out is the same.