Extending XLM Roberta for Question Answering
# ❓ Questions & Help

The AutoModelForQuestionAnswering is supported by many models, but not yet by XLM Roberta. In the current implementation I could see that most task-specific classes for XLM-R, e.g. XLMRobertaForSequenceClassification are just inheriting from Roberta. However, when I try to extend the class analogously, the process fails. This is my extension:

`from transformers.modeling_roberta import (
    RobertaForQuestionAnswering,
)`

and

`@add_start_docstrings(
    XLM_ROBERTA_START_DOCSTRING,
)`

`class XLMRobertaForQuestionAnswering(RobertaForQuestionAnswering):
    config_class = XLMRobertaConfig
    pretrained_model_archive_map = XLM_ROBERTA_PRETRAINED_MODEL_ARCHIVE_MAP
`

The error message I get is 

> Traceback (most recent call last):
  File "..miniconda3/lib/python3.6/site-packages/torch/serialization.py", line 289, in _check_seekable
    f.seek(f.tell())
AttributeError: 'NoneType' object has no attribute 'seek'

> During handling of the above exception, another exception occurred:

> Traceback (most recent call last):
  File "../miniconda3/lib/python3.6/site-packages/transformers/modeling_utils.py", line 516, in from_pretrained
    state_dict = torch.load(resolved_archive_file, map_location="cpu")
  File "../miniconda3/lib/python3.6/site-packages/torch/serialization.py", line 525, in load
    with _open_file_like(f, 'rb') as opened_file:
  File "../miniconda3/lib/python3.6/site-packages/torch/serialization.py", line 217, in _open_file_like
    return _open_buffer_reader(name_or_buffer)
  File "../miniconda3/lib/python3.6/site-packages/torch/serialization.py", line 202, in __init__
    _check_seekable(buffer)
  File "../miniconda3/lib/python3.6/site-packages/torch/serialization.py", line 292, in _check_seekable
    raise_err_msg(["seek", "tell"], e)
  File "/home/anlausch/miniconda3/lib/python3.6/site-packages/torch/serialization.py", line 285, in raise_err_msg
    raise type(e)(msg)
AttributeError: 'NoneType' object has no attribute 'seek'. You can only torch.load from a file that is seekable. Please pre-load the data into a buffer like io.BytesIO and try to load from it instead.

>  File "../modeling_auto.py", line 968, in from_pretrained
    return model_class.from_pretrained(pretrained_model_name_or_path, *model_args, config=config, **kwargs)
  File "..miniconda3/lib/python3.6/site-packages/transformers/modeling_utils.py", line 519, in from_pretrained
    "Unable to load weights from pytorch checkpoint file. "

Any idea what's going on here? 
I've worked around this problem now by not extending the huggingface framework directly but implementing XLMRobertaForQuestionAnswering externally (using huggingface classes etc.). <|||||>Would make sense to have it as a new feature though but that's a different issue type. ;)<|||||>Could you share the code or make PR? <|||||>@djstrong The code corresponds to my solution presented above. The issue was related to me changing the library instead of just adding the file externally.