NoamAS:

How can I check in real time that my GPU is working ?


Push your model to the device and execute a training step:
device = 'cuda'
model.to(device)
data = torch.randn(1, 5, device=device)
out = model(data)
out.mean().backward()

If this code runs fine then your GPU is used.
For more testing, you could execute some examples 2.