I checked your suggestions and turns out I have the same result. After isolating the problem, It seems that the issue is caused by the profiler to measure the performance of the model over all the test set, as shown in the code below:
import torch
import torchvision.transforms as transforms
import os
from PIL import Image
import psutil
from torch.profiler import profile, record_function, ProfilerActivity

mean = (0.485, 0.456, 0.406)
std = (0.229, 0.224, 0.225)

img_folder = "/path/to/img"
img_size = 224

def test():
  preprocess = transforms.Compose([
      transforms.Resize((img_size,img_size)),
      transforms.ToTensor(),
      transforms.Normalize(mean=mean, std=std)
  ])

  for dir1 in os.listdir(img_folder):
    for file in os.listdir(os.path.join(img_folder, dir1)):
      image_path = os.path.join(img_folder, dir1,  file)
      with Image.open(image_path) as img_pil:
        img_pil = preprocess(img_pil)
      memory = psutil.virtual_memory()
      totmemory = memory.total >> 20
      usedmemory = memory.used >> 20
      print(usedmemory)

with profile(activities=[ProfilerActivity.CPU], profile_memory=True, record_shapes=True) as prof:
  test()
print(prof.key_averages().table(sort_by="self_cpu_memory_usage", row_limit=10))

Is there a better-performing way to profile the model without building up the RAM?
I have created a colab here:
  
      

      colab.research.google.com
  

  
    

Google Colaboratory 1