Right. So these are class weights. I am not sure if I correctly understand the meaning of “internal state”. How would you define it and does internal state varies from module to module? In source code for nn.Softmax(), I can see there are some lines on ‘state’, i.e.,
def __setstate__(self, state):
        self.__dict__.update(state)
        if not hasattr(self, 'dim'):
            self.dim = None

Is it the internal state you are refering towards? I also noticed the modules and functions inheriting nn.Module are mostly moved to CUDA, any thoughts?