How to extract features of an image from a trained model
Hi all,
big_tree
<SCODE>torchvision.models.resnet18(pretrained=True)
<ECODE>
apaszke
You can either reconstruct the classifier once the model was instantiated, as in the following example: <SCODE>import torch
import torch.nn as nn
from torchvision import models

model = models.alexnet(pretrained=True)

# remove last fully-connected layer
new_classifier = nn.Sequential(*list(model.classifier.children())[:-1])
model.classifier = new_classifier
<ECODE> Or, if instead you want to extract other parts of the model, you might need to recreate the model structure, and reusing the parts of the pre-trained model in the new model. <SCODE>import torch
import torch.nn as nn
from torchvision import models

original_model = models.alexnet(pretrained=True)

class AlexNetConv4(nn.Module):
            def __init__(self):
                super(AlexNetConv4, self).__init__()
                self.features = nn.Sequential(
                    # stop at conv4
                    *list(original_model.features.children())[:-3]
                )
            def forward(self, x):
                x = self.features(x)
                return x

model = AlexNetConv4()
<ECODE>
fmassa
big_tree
Is there a way to get values from multiple layers with one forward pass (for neural style transfer etc.)? In Tensorflow it would be along the lines of <SCODE>features = sess.run([net.conv1, net.conv2, net.conv3])
<ECODE>
kpar
<SCODE>class Net(nn.Module):
    def __init__(self):
        self.conv1 = nn.Conv2d(1, 1, 3)
        self.conv2 = nn.Conv2d(1, 1, 3)
        self.conv3 = nn.Conv2d(1, 1, 3)

    def forward(self, x):
        out1 = F.relu(self.conv1(x))
        out2 = F.relu(self.conv2(out1))
        out3 = F.relu(self.conv3(out2))
        return out1, out2, out3
<ECODE>
fmassa
That looks like it’ll do the trick. Is there a convenient way to fetch the intermediate values when the forward behavior is defined by nn.Sequential()? It seems like right now the only way to compose multiple responses would be to split off all the individual layers and forward the values manually in forward(). Essentially what I want to do is take an existing network (e.g. VGG) and just pick some responses of some layers (conv1_1, pool1, pool2, etc.) and concatenate them into a feature vector.
kpar
You could write your own sequential version that keeps track of all intermediate results in a list. Something like <SCODE>class SelectiveSequential(nn.Module):
    def __init__(self, to_select, modules_dict):
        super(SelectiveSequential, self).__init__()
        for key, module in modules_dict.items():
            self.add_module(key, module)
        self._to_select = to_select
    
    def forward(x):
        list = []
        for name, module in self._modules.iteritems():
            x = module(x)
            if name in self._to_select:
                list.append(x)
        return list
<ECODE> And then you could use it like <SCODE>class Net(nn.Module):
    def __init__(self):
        super(Net, self).__init__()
        self.features = SelectiveSequential(
            ['conv1', 'conv3'],
            {'conv1': nn.Conv2d(1, 1, 3),
             'conv2': nn.Conv2d(1, 1, 3),
             'conv3': nn.Conv2d(1, 1, 3)}
        )

    def forward(self, x):
        return self.features(x)
<ECODE>
fmassa
Thank you for your help
lcelona
<SCODE>net = models.alexnet(pretrained=True).features`

class Feature_extractor(nn.module):
    def forward(self, input):
        self.feature = input.clone()
        return input

new_net = nn.Sequential().cuda() # the new network

target_layers = [conv_1, conv_2, conv_4] # layers you want to extract`

i = 1
for layer in list(cnn):
    if isinstance(layer,nn.Conv2d):
        name = "conv_"+str(i)
        art_net.add_module(name,layer)

        if name in target_layers:
            new_net.add_module("extractor_"+str(i),Feature_extractor())

        i+=1

    if isinstance(layer,nn.ReLU):
        name = "relu_"+str(i)
        new_net.add_module(name,layer)

    if isinstance(layer,nn.MaxPool2d):
        name = "pool_"+str(i)
        new_net.add_module(name,layer)

new_net.forward(your_image)
print new_net.extractor_3.feature<ECODE>
alexis-jacq
If you really want to do something like that, I’d recommend this: <SCODE>class FeatureExtractor(nn.Module):
    def __init__(self, submodule, extracted_layers):
        self.submodule = submodule

    def forward(self, x):
        outputs = []
        for name, module in self.submodule._modules.items():
            x = module(x)
            if name in self.extracted_layers:
                outputs += [x]
        return outputs + [x]
<ECODE>
apaszke
Wow! good to know that… Thanks! That would be really useful!
alexis-jacq
<SCODE>x = Variable(torch.rand(1, 3, 224, 224))
h_x = resnet_18.forward(x)
last_view = h_x.creator.previous_functions[0][0]
last_pool = last_view.previous_functions[0][0]
embedding = last_pool.saved_tensors[0]
<ECODE>
Atcold
Why do you want to inspect the graph? A new instance is going to be created at every forward.
apaszke
<SCODE>for name, module in self.submodule._modules.items():
    x = module(x)
<ECODE>
Atcold
apaszke
alexis-jacq
Atcold
We managed to quickly write what we currently have because getting TensorBoard to take 2d values / histograms was relatively straightforward, however adding integration for more complex data types is not a one-day job. After the ICML deadline we’ll give it a look.
edran