Hi,
I have this snippet code, I have divided Alexnet to 3 sub-modules because I want to take the output of each sub-module to feed it to another network later.
class alex (nn.Module):
def init(self):
super(alex,self).init()
self.net= models.alexnet(pretrained= True)
for param in self.net.parameters():
param.requires_grad=False
self.feat_list=list(self.net.features.children())
self.feat_model= nn.Sequential(*self.feat_list)
        self.c1= list(self.net.classifier.children())[:-2]
        self.c1.pop(2)
        self.c1.insert(2, nn.ReLU(0.2))            
        self.sub_cl1=nn.Sequential(*self.c1)            
        self.c2= list(self.net.classifier.children())[5:7]
        self.sub_cl2=nn.Sequential(*self.c2)  
def forward(self,x):
          x_feat= self.feat_model(x)
          x_feat=self.net.avgpool(x_feat)
          x_feat=x_feat.view(-1,x_feat.size(1)*x_feat.size(2)*x_feat.size(3))

===>    x_sub_classifier1= self.sub_cl1(x_feat)
===>    model=self.sub_cl2(x_sub_classifier1)
return model, x_feat, x_sub_classifier1
The problem is during the runtime I debug the code and I can see the output of these variables:
x_feat
==>x_sub_classifier1
model
everything seems fine but step ahead from x_sub_classifier1 to model all negative values in x_sub_classifier1 become zero (note that I take the output after the Linear layer so there is no Relu function after the linear function to make the negative values Zeros) and the sub_cl1 has this sub-architecture
(sub_cl1): Sequential(
(0): Dropout(p=0.5, inplace=False)
(1): Linear(in_features=9216, out_features=4096, bias=True)
(2): ReLU(inplace=True)
(3): Dropout(p=0.5, inplace=False)
(4): Linear(in_features=4096, out_features=4096, bias=True)
)
Am I missing something or there is something I don’t understand?