actually, i looked at the cloudwatch log, and it has print out the training loss, eval metrics, etc. and no error information. in the sagemaker training job dashboard, just show below, it’s quite confusing to me since i normally only encounter oom problem during training, not after training.

Screenshot 2021-09-07 at 2.41.23 PM1626×734 76.4 KB