Thanks a lot really appreciate it.
But when the whole channel is removed, we are basically left with a dense model at the end, so we still need to be able to benefit from the dense operation efficiency! In my case I’m using Torch-Pruning 5 which removes entire channels and not setting neurons to zeros only  as far as I understand.  So to me it should be able to run as fast at the very least.
Also note that the final pruned model is then retrained therefore even if we have sparse neurons (set to zero) at first, we should be dealing with a dense model at the end that shouldn’t be performing like this.
Am I missing something here?