I think loading the entire trained state_dict and re-initializing the first layers would be the easiest and cleanest approach. Alternatively, you could also try to load the parameters using a per-layer approach, but this sounds unnecessary complicated.