The identity layer shouldn’t have any weights that could be updated.
import torch.nn
>>> a = torch.nn.Linear(100, 100)
>>> a.weight
tensor([[-0.0092, -0.0083, -0.0101,  ..., -0.0416,  0.0169, -0.0232],
        [ 0.0707,  0.0684, -0.0826,  ..., -0.0583, -0.0801, -0.0349],
        [ 0.0531,  0.0917, -0.0934,  ...,  0.0632, -0.0696, -0.0597],
        ...,
        [-0.0661,  0.0780,  0.0926,  ...,  0.0099, -0.0024, -0.0690],
        [ 0.0313,  0.0154, -0.0628,  ...,  0.0512,  0.0821, -0.0196],
        [ 0.0760,  0.0127, -0.0037,  ..., -0.0742, -0.0545, -0.0989]],
       requires_grad=True)
>>> b = torch.nn.Identity()
>>> b.weight
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/pytorch/torch/nn/modules/module.py", line 1130, in __getattr__
    raise AttributeError("'{}' object has no attribute '{}'".format(
AttributeError: 'Identity' object has no attribute 'weight'