If I understand your use case correctly, you would just want to extract the features from a resnet without the last classification layer.
If that’s correct, you could wrap the inference code in a with torch.no_grad() block to avoid storing the intermediate tensors, which would be needed for the gradient calculation.
Also calling model.eval() might be necessary to use the running stats in batchnorm layers and disable dropout.
If you have multiple GPUs, you could also use nn.DataParallel and increase the batch size.
This will chunk the data in dim0 and send each chunk to a model replica on the different devices.