trainer/lightning_base: Arbitrary config updates through command line
this issue [https://github.com/huggingface/transformers/issues/6367], and a recent one to add dropout to the command line, as well as the usage of task_specific_params during finetuning, are all one off solutions to address a larger problem. During finetuning/training, it is very difficult to arbitrarily set config attributes. For `examples/lightning_base.py`, you need to save a whole new config to json and put it in a directory, which is fairly annoying method for changing hyperparameters, so we add lots of them, like `--dropout --attention_dropout --encoder_layerdrop --decoder_layerdrop` through `argparse.add_argument`. 
It would be a better user experience if I could just pass any kwarg without editing the code.

This seems possible with the `fire` package. But I would prefer an `argparse` solution as there is another issue open to delete the `fire` dependency, I also asked a similar question on
[stackoverflow](https://stackoverflow.com/questions/63329044/python-argparse-allow-unregistered-arguments)
see `argparse.remainder` https://stackoverflow.com/questions/22850332/getting-the-remaining-arguments-in-argparse/46250042#46250042

h/t @stas00 

<|||||>You could probably extract all the known args via `argparse`'s:
```
args, unknown = parser.parse_known_args()
```
and then use another tool to parse `unknown` (which is just `argv` minus known args) e.g. could even use `fire` or write a core function to do that.

surely a cheat, but it would make your query work, while having `arparse` as the main solution still.<|||||>You could also take a look at https://github.com/huggingface/transformers/blob/155288f04ba9a5d0a0e4d5be4f6d4e808ad8cfff/src/transformers/hf_argparser.py#L128-L146<|||||>If I read the code correctly, it doesn't do anything with `remaining_args`. It either just returns them as is (`argv` list) or throws an error.<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
<|||||>Fixed by @stas00 for both trainers, thanks!