@szymonk92  I faced the exact same problem… and I have the explanation. 
In the paper from He et al. (2015), the authors explain in 4.2 that they use a narrower ResNet for CIFAR-10 compared to the ImageNet reference model:

The network inputs are 32×32 images, with the per-pixel mean subtracted. The first layer is 3×3 convolutions.  Then we use a stack of 6n layers with 3×3 convolutions on the feature maps of sizes{32,16,8} respectively,with 2n layers for each feature map size.  The numbers of filters are{16,32,64}respectively. The subsampling is performed by convolutions with a stride of 2. The network ends with a global average pooling, a 10-way  fully-connected layer, and softmax.

This means that the Resnets for CIFAR-10 use 3 residual blocks with 16, 32 and 64 filters. Yet, the torchvision models are all designed for ImageNet. If you look at the code (in resnet.py) 14 you’ll see that the Resnets there use 4 blocks with an exponentially growing number of filters from 64 to 512.
Alas this behaviour cannot be modified directly from PyTorch. But this unofficial implementation 123 will allow you to reproduce the CIFAR-10 baselines using Resnets.