Hello Guys,
I need your help.
I only have one GPU installed in my computer. I am currently working on architecture selection hence I test different architectures (e.g. 10 architectures) with one run.  I do it one by one.
for i in Architectures:
    model = network(architecture = i)
    ....

    for mini_epoch in Total_epochs:
        ##trains the architecture
        loss = model(input)
        loss.backward()
        optimizer.step()

My problem is that something is accumulating in the GPU memory that I cannot finish training all the architectures even though I treat them as a single model at a time. Testing the architectures one at a time works but it is too tedious. Is there a way to release GPU memory when a new architecture is passed as new model?
Thanks in advance!