Yes, these tensors should be freed after the backward().
To double check it, you could use this example and add some print statements to check the memory:
for t in range(5):
    # To apply our Function, we use Function.apply method. We alias this as 'relu'.
    relu = MyReLU.apply

    # Forward pass: compute predicted y using operations; we compute
    # ReLU using our custom autograd operation.
    print('='*10, t, '='*10)
    print(torch.cuda.memory_allocated()/1024)
    y_pred = relu(x.mm(w1)).mm(w2)
    print(torch.cuda.memory_allocated()/1024)

    # Compute and print loss
    loss = (y_pred - y).pow(2).sum()
    print(torch.cuda.memory_allocated()/1024)
    if t % 100 == 99:
        print(t, loss.item())

    # Use autograd to compute the backward pass.
    loss.backward()
    print(torch.cuda.memory_allocated()/1024)

    # Update weights using gradient descent
    with torch.no_grad():
        w1 -= learning_rate * w1.grad
        w2 -= learning_rate * w2.grad

        # Manually zero the gradients after updating weights
        w1.grad = None
        w2.grad = None
    
    print(torch.cuda.memory_allocated()/1024)

which shows that the memory falls down to the initial usage:
========== 0 ==========
647.5
700.0
703.0
1045.5
650.5
========== 1 ==========
650.5
700.5
703.0
1045.5
650.5
========== 2 ==========
650.5
700.5
703.0
1045.5
650.5
========== 3 ==========
650.5
700.5
703.0
1045.5
650.5
========== 4 ==========
650.5
700.5
703.0
1045.5
650.5

Note that I’ve replaced the .grad attributes with None to free these tensors as well instead of zeroing them out.