I don’t think there is a clean approach besides performing a “warmup” iteration using your real inputs to create the parameters. Afterwards, you could push the parameters to the desired device and pass them also to the optimizer. There is a newly added meta device, but I don’t think this would help here given that the shapes are unknown. If I understand its use case correctly, it’s used to abstract the device, but not necessarily the data/shape.