I think you shouldn’t use retain_graph=True in each backward() which leads to large amount of memory even OOM problem. In addition, there are some problems in you code snippet.
optimizer = Adam(NN.parameters()) # the frist optimizer of NN parameters
for l in range(total_number_of_layers):
    # add new layers
    ...
    optimizer.add_param_group({"params": new_module.parameters()})
    optimizer_new_layer = Adam(new_module.parameters())

    # only train new layers
    for t in range(T1):
        optimizer_new_layer.zero_grad()
        loss = get_loss(...)
        loss.backward()      # don't need retrain_graph
        optimizer_new_layer.step()

    # Fully train
    for t in range(T2):
        optimizer.zero_grad()
        loss = get_loss(...)
        loss.backward()
        optimizer.step()

Does it still have a large memory consumption after removing retrain_graph=True? Let me know.