The Trainer code will run on distributed or one GPU without any change. Regarding your other questions:

you need to define your model in all processes, they will see different part of the data each and all copies will be kept the same.
the tokenizer and dataset preprocessing can either be done on all processes if it doesn’t slow you down, or you can use the with training_args.main_process_first(desc="dataset map pre-processing"): context manager to make sure the preprocessing is done only on process 0.