Curious about best practices.

When should I use cuda for matrix operations and when should I not use it?
Are cuda operations only suggested for large tensor multiplications?
What is a reasonable size after which it is advantageous to convert to cuda tensors?
Are there situations when one should not use cuda?
What’s the best way to convert between cuda and standard tensors?
Does sparsity affect the performance of cuda tensors?