Hey @IlllIIII you can specify the split you want to load by passing split="test" to the load_dataset() function (docs). This will still download all the splits in the dataset, so if space is an issue you can stream the elements of the test set one by one:
from datasets import load_dataset

dset = load_dataset("librispeech_asr", 'clean', split="test", streaming=True)
next(iter(dset))