I was not sure if reducing the bucket size helped (I tried with a low value in the first place). Perhaps Adam still updates all tensors (even if they were not involved in gradient computation). Can you suggest me if and how I could change this behavior?