Hello,
I have a problem that might have a very simple solution but it bugs me for a few hours now.
I have several models defined as:
class Model(nn.Module):
	def __init__(self, base):
		super(Model, self).__init__()
		self.base = base
		self.coeff = nn.Parameter(torch.randn(len(base)), requires_grad=True)

	def forward(self, x):
		res = []
		for x_i in x:
			res.append(torch.tensor([c*f(x_i) for c, f in zip(self.coeff, self.base)], requires_grad=True).sum())
		return torch.stack(res)


I use a custom loss function, defined as:
class RegressionLoss(nn.Module):
	def __init__(self):
		super(RegressionLoss, self).__init__()

	def forward(self, a, p, y_, y, q, w):
		return (w * a * p * torch.abs(y_ - y) ** q).mean()


All the models and the loss modules sit in seperate lists, i.e. I have 2 lists names models[] and loss_fn[]. I also have a list of LBFGS optimizers, named optimizers[].
I call the following function in a loop:
	def lbfs_KHM(self, k, a_values, p_values, x, y, w, q=2):
		def closure():
			self.optimizers[k].zero_grad()
			output = self.models[k](x)
			loss = self.loss_fn[k](a_values, p_values, output, y, q, w)
			loss.backward()
			return loss

		self.optimizers[k].step(closure)

Altough I assumed this will create different graphs for different models and losses, I get the following error when calling loss.backward:
RuntimeError: Trying to backward through the graph a second time, but the buffers have already been freed. Specify retain_graph=True when calling backward the first time.