This should work:
lin = nn.Linear(5, 5, bias=False)
alpha = nn.Parameter(torch.randn(5, 2))
eign_vect = torch.rand(2, 5)
#weight = nn.Parameter(torch.matmul(alpha, eign_vect))
optimizer = torch.optim.SGD([alpha], lr=1.)

# Freeze lin.weight
lin.weight.requires_grad = False

# Forward pass
x = torch.randn(1, 5)
weight = torch.matmul(alpha, eign_vect)
out = F.linear(x, lin.weight + weight)

# Backward pass
out.mean().backward()

# Check grads
print(lin.weight.grad)
print(alpha.grad)
print(weight.grad)

This approach would calculate the gradients for alpha and would recompute weight using the new and updated alpha value.