Thank you!
It works but for some reason the network doesn’t learn.
My new model looks like that:
class TLModel(torch.nn.Module):
    def __init__(self, pretrained_model, num_inputs):
        super(TLModel, self).__init__()
        self.new_layer = nn.Conv2d(num_inputs, num_inputs, 1)
        self.pretrained_model = pretrained_model

    def forward(self, inputs):
        inputs, (hx, cx) = inputs
        x = F.elu(self.new_layer(inputs))
        return self.pretrained_model.forward((x, (hx, cx)))

Here is how I initialized the network:
ac_model = ActorCritic(env.observation_space.shape[0], env.action_space)
checkpoint = torch.load(fname)
ac_model.load_state_dict(checkpoint['state_dict'])
for param in ac_model.parameters():
    param.requires_grad = False
tlmodel = TLModel(shared_model, env.observation_space.shape[0], env.action_space)

and the optimizer is:
optimizer = my_optim.SharedAdam(tlmodel.new_layer.parameters(), lr=args.lr)