Thanks @philschmid
Just to be sure, here is my understanding.

Notebook
In the notebook notebooks/sagemaker-notebook.ipynb at master · huggingface/notebooks · GitHub 1 that you mentioned, the libraries versions are the following ones:

sagemaker>=2.69.0
transformers==4.12.3
datasets==1.13

The installation is done by the following code:
!pip install "sagemaker>=2.69.0" "transformers==4.12.3" --upgrade
# using older dataset due to incompatibility of sagemaker notebook & aws-cli with > s3fs and fsspec to >= 2021.10
!pip install  "datasets==1.13" --upgrade

I noticed that unlike the first notebook (used in your workshop), you no longer install datasets[s3] but directly datasets. I’m just curious as to why and what are the differences between these 2 libraries? Thanks.

Reference & PR
In the Reference page, the transformers version goes up to but 4.6.1 (and compatible datasets 1.6.2) but your recent PR shows that the current situation of Training DLC is
transformers 4.12.3, datasets 1.15.1, PyTorch 1.9.1, python 3.8
Does it mean I could run the following code in a notebook instance of AWS SageMaker?
!pip install "sagemaker>=2.69.0" "transformers==4.12.3" --upgrade
!pip install  "datasets==1.15.1" --upgrade

If yes, I would need to update as well Pytorch and python as I do not see in the available kernel, this configuration. How can I do this? (or do you recommend to create a new conda kernel? through the jupyter terminal? could you give use the commands?)