I have a few questions about bfloat16

how can I tell via pytorch if the gpu it’s running on supports bf16 natively? I tried:

$ python -c "import torch; print(torch.tensor(1).cuda().bfloat16().type())"
torch.cuda.BFloat16Tensor

and it works on any card, whether it’s supported natively or not.
non-pytorch way will do too. I wasn’t able to find any.

What’s the cost/overheard - how does pytorch handle bf16 on gpus that don’t have native support for it?

e.g. I’m trying to check whether rtx-3090 supports bf16 natively. The information is inconsistent - the Ampere arch supports bf16 but some comments I found suggest that the non-high end cards may have it disabled.
Thank you!