Thank you very much! The mistake was indeed that I had still RGB images as masks.
Converting them with the following solved it.

mask = Image.open(mask_path).convert(‘L’)

I can also now save my masks with:
img2 = Image.fromarray(prediction[0]['masks'][0, 0].mul(255).byte().cpu().numpy())
img2.save('1.png')

img1 = Image.fromarray(prediction[0]['masks'][1, 0].mul(255).byte().cpu().numpy())
img1.save('2.png')

Did you find any way to draw the results such as the tutorial https://pytorch.org/tutorials/intermediate/torchvision_tutorial.html#defining-the-dataset 15
With the bounding boxes and the outline?