The minimal example that I came up fails in a different way.
import torch as torch
import torch.nn as nn
import torch.nn.functional as F
from torch.autograd import Variable
    
(n, h, m) = (3, 4, 5)
fc1 = nn.Linear(n, h)
fc2 = nn.Linear(h, m)
emb = nn.Embedding(3, n)

f = lambda z, i: fc2( F.relu( fc1( emb( z[i:i + 2] ).sum(0).view(1, -1) ) ) )

x = torch.LongTensor([0, 1, 2])
y1 = torch.LongTensor([0])
y2 = torch.LongTensor([1])

y1_p = f( Variable( x ), 0 )
l1 = nn.CrossEntropyLoss()(y1_p, Variable( y1 ) )
l1.backward()

y2_p = y1_p + f(Variable( x ), 1)
l2 = nn.CrossEntropyLoss()(y2_p, Variable( y2 ) )
l2.backward()

This yields RuntimeError: Trying to backward through the graph second time, but the buffers have already been freed. Please specify retain_variables=True when calling backward for the first time., which is solved by doing l1.backward(retain_variables=True) rather than l1.backward().