I think it should work.
It looks like you’ve posted the shapes of the weight matrices of both models.
However, if that’s the case, the shapes of the weight matrices of the downsized model don’t fit together. Is this a typo or do I misunderstand something?