Update - by moving the job to the CPU the Training loss is getting values.
steps so far

make cuda unavailable
import torch

torch.cuda.is_available = lambda : False

disabling Mixed precision
training_args = TrainingArguments(
…
# fp16=True,
…)

*Mixed precision training with AMP or APEX (--fp16) and FP16 evaluation can only be used on CUDA devices.  *