You would have to pick a value, which is not in your input space. Something like this:
ARBITRARY_VAL = -20000

tensor = torch.ones(10,10) * ARBITRARY_VAL 
tensor[0:8, 0:8] = 2
new_tensor = tensor[tensor!=ARBITRARY_VAL]

Instead of != you could also use other logical operators like >= (depending on your arbitrary value).