Hello,
Not sure what I’m doing wrong or if it’s an expected behavior.
My experiment is to take a trained ResNet-101 and then prune it using the pytorch functionality in torch.nn.utils.prune, I’m currently using the code provided in the official turorial so nothing fancy yet.
The issue is that, as far as I monitored, the inference speed and the memory footprint remain exactly the same as the original unpruned model, which is weird for me.
# model has been loaded properly before
for name, module in model.named_modules():
    # prune 20% of connections in all 2D-conv layers
    if isinstance(module, torch.nn.Conv2d):
        prune.l1_unstructured(module, name='weight', amount=0.2)
        prune.remove(module, 'weight') # make it permanent
    # prune 40% of connections in all linear layers
    elif isinstance(module, torch.nn.Linear):
        prune.l1_unstructured(module, name='weight', amount=0.4)
        prune.remove(module, 'weight')

For timing I use :
    start = time.time()
    # get predictions
    output = model(inputs)
    end = time.time()  
    print(end - start)

For memory I use :
def get_memory(model):
    memory = 0
    for p in model.parameters():
        memory += 32 * p.nelement()
    return memory

Am I doing something wrong ? How can I achieve speed-up using torch.nn.utils.prune ?
Thank you very much for your help,
Cyril