No, sorry, I edited the answer to clarify. So the gradient would be w.r.t. the new value of my_param and you could compute the new_value in the no_grad block.
The remaining computation - the bit that needs to be backproped - would not be in no_grad, of course.
Best regards
Thomas