Hi,
You have two kinds of Tensors that a Module want to hold to:

Some that are learnable. Represented as nn.Parameter and that should be registered with mod.register_parameter("name", value) where value can be either None or an nn.Parameter
Some that are not learnable. Represented as regular torch.Tensors and that should be registered with mod.register_buffer(“name”, value) where value can be either None or a torch.Tensor.

Note that for simplicity, when you do mod.name = something. If something is an nn.Parameter, register_parameter() will be called automatically. EDITED: not true for Tensors.
As you can see, the only way to set a parameter or a buffer to None is to call the method directly. Otherwise, you can use natural assignment.