I don’t think “downscaling” is referring to the spatial size, but to a multiplicative factor.
Here normalization is mentioned:



 collinarnett:

Importantly, we did not normalize
input maps but scaled them down by a constant factor.


which seems to point towards using a constant factor for the downscaling via:
act = act * 1/factor