I’ve executed your code on a machine with 8 GPUs and got the following output:
 model(torch.randn(8, 100, device='cuda'))

>  gpu 0  calculate_running: True
 gpu 1  calculate_running: True
 gpu 2  calculate_running: True
 gpu 4  calculate_running: True
 gpu 3  calculate_running: True
 gpu 7  calculate_running: True
 gpu 6  calculate_running: True
 gpu 5  calculate_running: True

Which seems to indicates the attribute is properly set.