Hello!
I’m using the default distanceType (‘cosine’) and the regular CX loss (‘calculate_CX_Loss()’), so the meshgrid computation is not being used. _random_sampling(), _random_pooling() and _crop_quarters() are also not being used, only _create_using_dotP (cosine distance) and calculate_CX_Loss, in principle.
So far only by taking the whole loss calculation out of the autocasting allows for the scale to remain stable. If I only autocast parts of the loss, wouldn’t calculations become inacurate?
In the case of SSIM, the scale also drops heavily, but stabilizes at a scale of 2. The only similarity I find between the two loss functions is that both are in principle distance metrics, but no idea if that is related.