Hi, thanks for responding! I added your code and got results like this:
FE_layer.0.weight tensor([[[[  43103.2578]],

         [[-176214.8594]],

         [[ 193186.0156]]],


        [[[ -65900.7109]],

         [[   9959.4365]],

         [[   7424.0977]]],


        [[[  -2614.8594]],

         [[ -31229.7559]],

         [[ -17630.7383]]],


        [[[  46508.3984]],

         [[  66506.5938]],

         [[ -16597.5312]]],


        [[[ -29938.4004]],

         [[  20681.8320]],

         [[  16445.1836]]],



I’m not sure if I’m interpreting it correctly, but it seems like the gradients are flowing, it is just the model doesn’t learn.