I figured it out. The answer is in the __setattr__ of the class Module (from which DDP inherits). So the Module class __setattr__ removes everything from __dict__ which is of type Parameter or Module and stores them in the self._parameters and self._modules OrderedDict.
Here’s the corresponding snippet from the source code:
    def __setattr__(self, name: str, value: Union[Tensor, 'Module']) -> None:
        def remove_from(*dicts_or_sets):
            for d in dicts_or_sets:
                if name in d:
                    if isinstance(d, dict):
                        del d[name]
                    else:
                        d.discard(name)

and it is called for attributes of type Module later in the same function like so:
else:
        modules = self.__dict__.get('_modules')
        if isinstance(value, Module):
            if modules is None:
                raise AttributeError(
                    "cannot assign module before Module.__init__() call")
            remove_from(self.__dict__, self._parameters, self._buffers, self._non_persistent_buffers_set)
            modules[name] = value