torch::nn::AnyModule should be the class you are looking for as a container for any type of layers. Please see https://github.com/pytorch/pytorch/blob/e312801453d1121f929b956c2862dfa58d6b3ac3/torch/csrc/api/include/torch/nn/modules/any.h#L116-L128 34 for its supported constructors.