The reason you get this error is because you are using the self.embed = self.embed * mask version in forward (I will call this self.embed2 in the following, as if you did self.embed2 = self.embed * mask). This tensor object is however not a leaf in the computation graph; the leaf is the original self.embed = torch.randn((1,10,10),requires_grad=True) object (I will continue to call this self.embed in the following). So your computation graph goes like this: self.embed --(a)--> self.embed2 --(b)--> .... In forward you are effectively using the self.embed2 tensor, so the edge --(a)--> never gets renewed, and hence it will remain in the graph also for the second forward pass and hence the error that you attempt to backprop another time through (that part of) the graph. Also note that if this worked (you could make it work by using backward(retain_graph=True)), your updates to the original self.embed tensor would never be effective, since you’re stuck with the self.embed2 version (which contains the original values of self.embed).
So the solution is to recompute self.embed2 on every forward pass. Basically you need to shift the relevant lines to forward:
def __init__(self):
    ...
    self.embed = torch.randn((1,10,10),requires_grad=True)
    # Move the `mask`ing part to `forward`.

def forward(self):
    mask = torch.zeros((1,10,10),requires_grad=False)
    mask[0,:,5] = 1
    embed = self.embed * mask  # don't overwrite `self.embed`
    x = torch.cat([embed,x])  # use the newly created `embed` tensor
    ...