Hello,
Indeed setting the weight in CrossEntropyLoss set weights with respects to the classes which seems not to be what you need.
Not sure my solution could help you but in my opinion you could create 3 datasets, one with labelled, one with unlabelelled and one that return the 2. You can create them via module Dataset of Pytorch, you can make a call only on the labelled data to compute the crossentropyloss, something like:
mixed_dataset = MixedDataset() #the __getitem__() function return ( labeleddata, unlabeleddata)
labeled_dataset = LabeledDataset()
unlabelled_dataset = UnlabeledDataset()

for data in mixed_dataset:
     whatever you need
     for labeled in labeled_dataset:
             compute loss
    whatever you need

just be sure to not mix the index and always return the corresponding data