Hi albanD,
Thanks for replying. Yes, I see what you said. Since my variable only affects the backward operation, it is not used in the forward pass operation. Say the following is my forward operation
def forward(self,x,alpha):
        self.c1 = self.pool(F.relu(self.conv1(x)))
        self.c2 = self.pool(F.relu(self.conv2(self.c1)))
        self.c2 = self.c2.view(-1, 16 * 5 * 5)
        self.f1 = F.relu(self.fc1(self.c2))
        self.f2 = F.relu(self.fc2(self.f1))
        self.c_out = self.fc3(self.f2[:self.f2.size(0)/2,:])
        self.c_out.retain_grad()
        h=self.c_out.register_hook(lambda grad: grad * alpha)
        self.f_out = self.fc4(self.f2[self.f2.size(0)/2:,:])
        self.f_out.retain_grad()        
        h=self.f_out.register_hook(lambda grad: grad * (1.0-alpha))
        return self.c_out,self.f_out

How can I formulate an objective (classification loss) as a function of alpha ? Please let me know if you need more info. Thanks!