Are you attempting to prune the weights of each layer or each layer as a whole?
IIRC “The Lottery Ticket Hypothesis” paper provides a good overview of common methods for weight pruning;
1803.03635v1.pdf (arxiv.org) 1
Though I’m not sure how well these would generalize to pruning an entire layer all at once (something iterative might still be needed).