It might depend on your use case, but nn.ModuleDict is just a container (dict), which stores modules and is used to register these modules properly inside a parent nn.Module.
Based on your code snippet, I would derive from nn.Module (your X class).