You could write a custom Dataset to lazily load each numpy file:
class MyDataset(Dataset):
    def __init__(self, np_file_paths):
        self.files = np_file_paths
    
    def __getitem__(self, index):
        x = np.load(self.files[index])
        x = torch.from_numpy(x).float()
        return x
    
    def __len__(self):
        return len(self.files)

After creating the Dataset instance, you could wrap it in a DataLoader, which will create a batches of [batch_size, 25, 512, 512].