nn.CrossEntropyLoss is used for a multi-class classification/segmentation use case and thus uses class indices in the target tensor.
Based on your description your target contains values in the range [0, 1], so you could try to use e.g. nn.BCEWithLogitsLoss or nn.MSELoss. I’m unsure what your exact use case is based on “I want to learn this vector for each of my input images”. If it’s a regression task, use nn.MSELoss, if it’s a “classification” task, you could probably try to use nn.BCEWithLogitsLoss.