Hello, I have a question about FP16, I found if I apply FP 16 in forward process, it will very slow.
e.g. FP32 forward time 3.27s
FP16 forward time 13s
there are nn.Linear and torch.matmul in my nework. I want to ask does FP16 optimized for special ops not for all ops?