Hi,
If you’re working only in python. The main idea is to perform as few ops as possible (making them as big as possible). The GPU is amazing to speeding up a given op but is very bad at switching between op and very very bad to execute small ops.
So make sure everything is stored in Tensors as large as possible and always work with the full Tensor.
Do you have particular code in mind?