You can pass a RGB color code as a tuple to this function:
trans = transforms.RandomAffine(45, fillcolor=(0, 0, 255))
img = TF.to_pil_image(torch.randn(3, 100, 100))
trans(img)

This will use blue to fill the pixels.
Could you create an issue so that the docs might be updated, as this seems indeed to be confusing.