I’m not really convinced about this code. There are issues:

Layer is not allocated properly. It will raise an Exception.
Layer is not tracked by the nn.Module. Thus, optimizer won’t track it and this will create a silent bug.

Despite I don’t know the details of the new lazy modules, I would recommend you to check them to code this.
https://pytorch.org/docs/stable/_modules/torch/nn/modules/conv.html#LazyConv2d