Because deep learning is based on computing gradients.
If you just manually change the value of a tensor you cannot compute gradient because it simply does not exist.
The closest thing you can do is
Criteria(labels[p>t],scores[p>t])
You can compute the loss over those values, but you cannot modify the tensor unless you apply a continuous function over those values