I am currently experimenting with Freezeout Code of Andy Brocks.

  
      
      GitHub
  
  
    

ajbrock/FreezeOut

FreezeOut - Accelerate Neural Net Training by Progressively Freezing Layers


  
  
    
    
  
  


I am trying to precompute the output of the layer. To completely remove the layer,  I have set volatility of out as False and detach it. I am able to train the model .

if self.active and not test:
out = self.conv1(F.relu(self.bn1(x)))
out = self.conv2(F.relu(self.bn2(out)))
out = torch.cat((x, out), 1)
if(self.layer_index < DenseNet.freezeLayerIndex):
detach = out.detach()
detach.volatile = False
Bottleneck.out_saved[self.counter] = detach
self.counter+=1
if(self.counter >= self.maxCounter):
self.counter = 0
self.active = False

But when i used the trained model to validate on a test data , it throws argument 0 is not a variable error. My test function is below:

def test_fn(x, y):
output = net(V(x, volatile=True))
test_loss = F.nll_loss(output, V(y, volatile=True)).data[0]
    # Get the index of the max log-probability as the prediction.
    pred = output.data.max(1)[1].cpu()
    test_error = pred.ne(y).sum()

    return test_loss, test_error


I am not sure what is error since the input X I have sent is a variable.