This is expected, as you are assigning net.x only in forward. If you are wrapping your model in DataParallel with more than 1 devices (your failing line, device_ids=[0,1], DataParallel operates on model replicas, and changes made to replicas (during forward) are not visible outside forward/backward calls.