do not support deepcopy when using deepspeed?
## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.16.2
- Platform: Ubuntu
- Python version: 3.8
- PyTorch version (GPU?): 1.8
- Tensorflow version (GPU?): none
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: yes

### Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- ALBERT, BERT, XLM, DeBERTa, DeBERTa-v2, ELECTRA, MobileBert, SqueezeBert: @LysandreJik
- T5, BART, Marian, Pegasus, EncoderDecoder: @patrickvonplaten
- Blenderbot, MBART: @patil-suraj
- Longformer, Reformer, TransfoXL, XLNet, FNet, BigBird: @patrickvonplaten
- FSMT: @stas00
- Funnel: @sgugger
- GPT-2, GPT: @patrickvonplaten, @LysandreJik
- RAG, DPR: @patrickvonplaten, @lhoestq
- TensorFlow: @Rocketknight1
- JAX/Flax: @patil-suraj
- TAPAS, LayoutLM, LayoutLMv2, LUKE, ViT, BEiT, DEiT, DETR, CANINE: @NielsRogge
- GPT-Neo, GPT-J, CLIP: @patil-suraj
- Wav2Vec2, HuBERT, SpeechEncoderDecoder, UniSpeech, UniSpeechSAT, SEW, SEW-D, Speech2Text: @patrickvonplaten, @anton-l

If the model isn't in the list, ping @LysandreJik who will redirect you to the correct contributor.

Library:

- Benchmarks: @patrickvonplaten
- Deepspeed: @stas00
- Ray/raytune: @richardliaw, @amogkam
- Text generation: @patrickvonplaten @narsil
- Tokenizers: @SaulLu
- Trainer: @sgugger
- Pipelines: @Narsil
- Speech: @patrickvonplaten, @anton-l
- Vision: @NielsRogge, @sgugger

Documentation: @sgugger

Model hub:

- for issues with a model, report at https://discuss.huggingface.co/ and tag the model's creator.

HF projects:

- datasets: [different repo](https://github.com/huggingface/datasets)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj

For research projetcs, please ping the contributor directly. For example, on the following projects:

- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->
@stas00 

## Information

Model I am using (Bert, XLNet ...): m2m100

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [ ] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. copy the follwing code as ```copy_test.py```
```
import copy

from transformers import M2M100Config, M2M100ForConditionalGeneration, M2M100Tokenizer

from transformers.deepspeed import HfDeepSpeedConfig
import deepspeed

config = M2M100Config.from_pretrained('facebook/m2m100_418M')
model_hidden_size = config.d_model

ds_config = {
    "fp16": {
        "enabled": False,
    },
    "bf16": {
        "enabled": False,
    },
    "zero_optimization": {
        "stage": 3,
        "offload_param": {
            "device": "cpu",
            "pin_memory": True
        },
        "overlap_comm": True,
        "contiguous_gradients": True,
        "reduce_bucket_size": model_hidden_size * model_hidden_size,
        "stage3_prefetch_bucket_size": 0.9 * model_hidden_size * model_hidden_size,
        "stage3_param_persistence_threshold": 10 * model_hidden_size
    },
    "steps_per_print": 2000,
    "train_batch_size": 1,
    "gradient_accumulation_steps": 1,
    "train_micro_batch_size_per_gpu": 1,
    "wall_clock_breakdown": False
}

dschf = HfDeepSpeedConfig(ds_config)

model = M2M100ForConditionalGeneration.from_pretrained("facebook/m2m100_418M")
tokenizer = M2M100Tokenizer.from_pretrained("facebook/m2m100_418M", src_lang="en", tgt_lang="fr")

src_text = "Life is like a box of chocolates."
tgt_text = "La vie est comme une boîte de chocolat."

model_inputs = tokenizer(src_text, return_tensors="pt")
with tokenizer.as_target_tokenizer():
    labels = tokenizer(tgt_text, return_tensors="pt").input_ids

engine = deepspeed.initialize(model=model, config_params=ds_config)

copy_model = copy.deepcopy(model)

loss = copy_model(**model_inputs, labels=labels)  # forward pass
```
2. Run ```deepspeed --num_gpus=1 copy_test.py```


<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->
we can using ```copy.deepcopy(model)``` successful when do not using ```deepspeed```, but failed when using ```deepspeed```.
+ The error message can be seen as follwing:
```
Using /mounts/Users/student/lavine/.cache/torch_extensions as PyTorch extensions root...
No modifications detected for re-loaded extension module utils, skipping build step...
Loading extension module utils...
Time to load utils op: 0.0015609264373779297 seconds
Traceback (most recent call last):
  File "test_copy.py", line 66, in <module>
    copy_model = copy.deepcopy(model)
  File "/mounts/work/lavine/envs/anaconda/envs/lavine-notebook/lib/python3.8/copy.py", line 172, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "/mounts/work/lavine/envs/anaconda/envs/lavine-notebook/lib/python3.8/copy.py", line 270, in _reconstruct
    state = deepcopy(state, memo)
  File "/mounts/work/lavine/envs/anaconda/envs/lavine-notebook/lib/python3.8/copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "/mounts/work/lavine/envs/anaconda/envs/lavine-notebook/lib/python3.8/copy.py", line 230, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/mounts/work/lavine/envs/anaconda/envs/lavine-notebook/lib/python3.8/copy.py", line 172, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "/mounts/work/lavine/envs/anaconda/envs/lavine-notebook/lib/python3.8/copy.py", line 264, in _reconstruct
    y = func(*args)
TypeError: __init__() missing 1 required positional argument: 'parent_module'
[2022-02-22 11:11:14,919] [INFO] [launch.py:160:sigkill_handler] Killing subprocess 28425
[2022-02-22 11:11:14,920] [ERROR] [launch.py:166:sigkill_handler] ['/mounts/work/lavine/envs/anaconda/envs/lavine-notebook/bin/python', '-u', 'test_copy.py', '--local_rank=0'] exits with return code = 1
```

@lavine-lmu,

deepspeed installs all kinds of hooks into the model, which perhaps makes it impossible to `deepcopy` 

Have you tried copying the model before deepspeed init? and then initing on the copy?

Overall this question belongs to https://github.com/microsoft/DeepSpeed as it's not related to the transformers integration.

But if you look at:

https://github.com/microsoft/DeepSpeed/blob/2151c787a27166f795eb4516f1e191e6730e823d/deepspeed/runtime/zero/stage3.py#L131-L132

you can see that deepspeed uses a custom dict class which it appears that `deepcopy` can't handle, as it's probably not designed to handle objects with custom dict classes. I don't know if it can be instrumented to support custom dict classes.

<|||||>Hi @stas00, I am doing meta-learning now, so I need to do the ```copy``` process after deepspeed init. Thanks for your reply, I will try other methods to solve this problem.