Based on sparsity being applied in an unstructured manner, I don’t think there is an out-of-the-box solution that speeds up inference based on the fact that the matrices are more sparse. I suggest that you try pruning in a structured way and see how that changes inference speed.