I found this answer in Albumentations website:
Many non-spatial transformations like, [RandomBrightness], [RandomContrast] [RandomGamma] can be also added. They will be applied only to the image and not the mask.