import torch
x = torch.randn(10,2048).cuda()
y = torch.randn(10,2048).cuda()
print(torch.nn.CosineSimilarity()(x,y).mean())
x= x.half()
y =y.half()
print(torch.nn.CosineSimilarity()(x,y).mean())
print(torch.diagonal(x @ y.T).mean())
x = torch.nn.functional.normalize(x)
y = torch.nn.functional.normalize(y)
print(torch.nn.CosineSimilarity()(x,y).mean())
print(torch.diagonal(x @ y.T).mean())

gives

tensor(-0.0104, device=‘cuda:0’)
tensor(0., device=‘cuda:0’, dtype=torch.float16)
tensor(-20.9531, device=‘cuda:0’, dtype=torch.float16)
tensor(-0.0104, device=‘cuda:0’, dtype=torch.float16)
tensor(-0.0104, device=‘cuda:0’, dtype=torch.float16)

Normalizing the vectors give same result but that is not the expected behavior. There should not be any difference whether the vector is normalized or not normalized.
Is this a bug?