Thank you @ptrblck,
your implemetation is working, but when I put F.linear() inside the for loop like below;
def my_model(x):
    for i, old_param in enumerate(model.parameters()):
        x = F.linear(x, old_param + new_weight[i])

    return x

I am getting an error;

RuntimeError: Trying to backward through the graph a second time, but the saved intermediate results have already been freed. Specify retain_graph=True when calling backward the first time.

When I do loss.backward(retain_graph=True) the following error is commig in:
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.FloatTensor [10, 399]] is at version 1; expected version 0 instead. Hint: the backtrace further above shows the operation that failed to compute its gradient. The variable in question was changed in there or anywhere later. Good luck!

Here is my full implementation:

alphas = []
new_weights = []
for i, param in enumerate(model.parameters()):
    eign_shape = params_dicts[f"layer_{i}"]["eign_vector"].shape
    layer_shape = param.shape
    alpha = nn.Parameter(torch.randn(layer_shape[0], eign_shape[0]))
    alphas.append(alpha)
    new_weights.append(torch.matmul(alpha, params_dicts[f"layer_{i}"]["eign_vector"]))

def new_model(x): # x represents batch images.
    
    for i, old_param in enumerate(model.parameters()):
        x = func.linear(x, torch.add(old_param, new_weights[i]))
    
    return x


for epoch in range(num_epochs):
    for batch, (images, labels) in enumerate(train_loader):
        # print(batch)
        images, labels = images.to(device), labels.to(device)

        optimizer.zero_grad()
        with torch.set_grad_enabled(True):
            outputs = new_model(images)
            loss = criterion(outputs, labels)
        
            if epoch == (num_epochs-1) and batch == (train_loader):
                loss.backward()
            else:
                loss.backward(retain_graph=True)
            optimizer.step()