@kl_divergence @ptrblck Thanks all for replying my question.
@kl_divergence Regarding to your answer, it’s true to say that “sub module that you aren’t declaring as attribute won’t be added as module of the model and so it’s parameters won’t be trained.”. But when we implement those lines in __init__ method, two submodules augmented1(2048) and augmented2(2048) are declared as the attributes of the module already.
Is it correct? If so, both of the approaches are correct. 
def __init__(self, ...):
    self.fc0 = augmented1(2048)
    self.fc1 = augmented2(2048)