If you’ve moved the model as well as the inputs to the GPU, it would be used (I don’t know what exactly TensorBoard is showing).
You could verify it by checking the device argument of some parameters of the model and check the memory usage as well as the utilization of the GPU via nvidia-smi.