I did a tiny bit of digging, and the slowdown is 99% due to a single line in Inception3’s initialization:
values = torch.as_tensor(X.rvs(m.weight.numel()), dtype=m.weight.dtype)

Specifically, getting the truncated normal distribution samples via X.rvs(m.weight.numel()). Could it be that a change to scipy.stats slowed this down? Also, is it possible to perform the same truncated norm with torch’s built-in tensor operations?