Expected object of device type cuda but got device type cpu for argument #1 'self' in call to _th_index_selec
I wrapped the code base into a flask container and tried to run it on a GPU and I am running into a GPU device copy issue. Looking for pointers.

I have verified that the model actually gets moved to the GPU:
`        self.model.to(torch.device("cuda" if torch.cuda.is_available() and not "store_true" else "cpu"))
        self.model.eval()  # TO HERE`

```
backend_1        |   File "/app/run_generation.py", line 285, in hook
backend_1        |     num_return_sequences=args.num_return_sequences,
backend_1        |   File "/usr/local/lib/python3.6/dist-packages/torch/autograd/grad_mode.py", line 49, in decorate_no_grad
backend_1        |     return func(*args, **kwargs)
backend_1        |   File "/app/transformers/modeling_utils.py", line 979, in generate
backend_1        |     attention_mask=attention_mask,
backend_1        |   File "/app/transformers/modeling_utils.py", line 1016, in _generate_no_beam_search
backend_1        |     outputs = self(**model_inputs)
backend_1        |   File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 541, in __call__
backend_1        |     result = self.forward(*input, **kwargs)
backend_1        |   File "/app/transformers/modeling_gpt2.py", line 599, in forward
backend_1        |     inputs_embeds=inputs_embeds,
backend_1        |   File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 541, in __call__
backend_1        |     result = self.forward(*input, **kwargs)
backend_1        |   File "/app/transformers/modeling_gpt2.py", line 465, in forward
backend_1        |     inputs_embeds = self.wte(input_ids)
backend_1        |   File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 541, in __call__
backend_1        |     result = self.forward(*input, **kwargs)
backend_1        |   File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/sparse.py", line 114, in forward
backend_1        |     self.norm_type, self.scale_grad_by_freq, self.sparse)
backend_1        |   File "/usr/local/lib/python3.6/dist-packages/torch/nn/functional.py", line 1484, in embedding
backend_1        |     return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)
backend_1        | RuntimeError: Expected object of device type cuda but got device type cpu for argument #1 'self' in call to _th_index_selec
```
Are the inputs also being cast to GPU?<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
