I mean, conceptually, you can add any network module to a module list.  The only example I have that is opensource is unfortunately in C++, and not for pytorch, https://github.com/hughperkins/DeepCL/blob/master/src/netdef/NetdefToNet.cpp#L104 6
you’re going to make _create_layer have maybe an if statement like:
if layer_def['type'] == 'relu':
    return nn.ReLU()
elif layer_def['type'] == 'conv':
    return [stuff here to create a conv layer]
elif ... etc ...