Packed Sequences are used to not influence the states of a recurrent layer with the padded values. Then, if you want to compute a loss, you have to unpack the Packed Sequences (pad_packed_sequence) and you will get again the padded sequences with the actual lengths. Therefore, you should apply a masked loss to the padded sequence if you think that the padded values are going to influence the loss.