I am trying to optimize the performance using Spyder profiler. This line, which is part of the code, z = torch.tensor(x).float().cuda(), seems to take too much time, approximately 20 ms per image.
I’ve tried some other initialization ways;  z =torch.cuda.FloatTensor(x), and
 to.device(...); yet they both take slightly more time.
Image Info:
Image size on average, (x as a numpy array):
210,000 bytes (~70,000 bytes per channel).
Note: No batching is used
The profiler refers to this as the source of time depletion:
<method 'cuda' of 'torch._C._TensorBase' objects>
and probably this:
..\Anaconda3\envs\PyTorchEight\lib\site-packages\torch\cuda\__init__.py : 144
Lazy init

System info:
PyTorch 1.8.1; Python 3.8; CUDA Version: 11.2; Windows 10 pro. Everything works well and the GUP does what it is supposed to do. Installed via (Cu 11.2 vs 11.1):
conda install pytorch torchvision torchaudio cudatoolkit=11.1 -c pytorch -c conda-forge