The problem with this snippet is that there is no gradient for the variable prob. Indeed, the first problem was related to the fact that your loss didn’t require gradient, and it was fixed by enabling gradients for t and f. Now the problem is that it doesn’t work right.
import torch
prob = torch.rand(5)
prob.requires_grad_(True)
t = torch.ones_like(prob)
f = torch.zeros_like(prob)
t.requires_grad_(True)
f.requires_grad_(True)
p = torch.where(prob < 0.3, t, f)
print(p.requires_grad) # prints True
loss = (p ** 2).sum() # Meaningless loss
loss.backward()
print(prob.grad) # No gradients
print(t.grad) # Gradients
print(f.grad) # Gradients

Maybe the following hack should work better for you:
import torch
prob = torch.rand(5)
prob.requires_grad_(True)
max_prob = torch.max(prob)
t = torch.ones_like(prob) * max_prob
f = torch.zeros_like(prob)
p = torch.where(prob < 0.3, t, f) / max_prob
print(p.requires_grad) # prints True
loss = (p ** 2).sum() # Meaningless loss
loss.backward()
print(prob.grad) # Gradients
print(t.grad) # No gradients
print(f.grad) # No gradients