oh my god, that was dumb 
using
cost = -(target.log_prob(detached_samples) - dist.log_prob(detached_samples)).sum()

doesn’t solve the problem though  I think it’s impossible to do this because there is a direct dependence between log_prob and the parameters of Normal, detaching samples doesn’t actually forbid the gradients to flow up to log_std and mean.
Thanks for taking a look!