Solved by MSS in post #2 
              
              
                Update: The layers receive gradients. My problem has to do something with My GNN. 
Maybe the integration in pytorch geometric is breaking