SteveMama:

is it possible to prune or reduce model size post-training and train again?


i think the answer strongly depends on the type of pruning method (structured vs unstructured), but it is possible in principle to iteratively prune a neural network - this was essentially the strategy in a now-famous paper by han et al:

Screen Shot 2021-05-31 at 2.05.42 pm817×299 48.9 KB

now, when it comes to transformers, my current understanding is that you can either prune the attention heads post-training (see e.g. here 1), but if you want to keep training you’ll probably need to use something like movement pruning that i linked to above.
my personal experience is that pruning is still quite tricky to get right and requires some care to preserve the performance of the original model. i would suggest trying quantization or optimizations with frameworks like ONNX Runtime before devoting a lot of time into pruning