tapas-base model is not predicting answers well.
Hi,
I was trying to get the answers for my own table. however results are not even reaching expectations. Please find the below code and output. actually, I am extracting tables from PDF,so here instead of providing pdf's i am providing excel sheet for you to test.

```
import pandas as pd
from transformers import  TapasForQuestionAnswering,TapasTokenizer
import camelot
import os
import numpy as np
path='/home/jupyter/Projects/ExtractiveQnA/fastapi/knowledgebase/'

appended_data = []

for file in os.listdir(path):
    print(file)

    tables = camelot.read_pdf(path+file, pages="1-6")
    if len(tables) != 0:
        for i in range(len(tables)):
            table = tables[0].df
            table_clean = table.replace("", np.nan).dropna()
            table_clean.rename(columns=table_clean.iloc[0], inplace=True)
            table_clean.drop(table_clean.index[0], inplace=True)
            table_clean.reset_index(drop=True, inplace=True)

            appended_data.append(table_clean.astype("str"))
# appended_data=[df.set_index("Model") for df in appended_data]
final_tables=pd.concat(appended_data,axis=1)
final_tables = final_tables.loc[:,~final_tables.columns.duplicated()]
final_tables=final_tables.fillna("NA")
model_name = 'google/tapas-base'
model = TapasForQuestionAnswering.from_pretrained(model_name)
tokenizer = TapasTokenizer.from_pretrained(model_name)
queries='What is the service flow rate  '
inputs = tokenizer(table=final_tables,
                   queries=queries,
                   padding='max_length',
                   return_tensors="pt")
outputs = model(**inputs)
predicted_answer_coordinates, = tokenizer.convert_logits_to_predictions(
       inputs,
      outputs.logits.detach(),
        
)

answers = []

for coordinates in predicted_answer_coordinates:
  if len(coordinates) == 1:
    # only a single cell:
    answers.append(table.iat[coordinates[0]])
  else:
    # multiple cells
    cell_values = []
    for coordinate in coordinates:
       cell_values.append(table.iat[coordinate])
    answers.append(", ".join(cell_values))

print("")
for query, answer, in zip(queries, answers,):
  print(query)
  print("Predicted answer: " + answer)
  




```
and the answer I am getting is :

```
GXSHC40N.pdf
GXSF30V.pdf
GXMH31H.pdf
GXSH40V_GXSH45V.pdf
Some weights of TapasForQuestionAnswering were not initialized from the model checkpoint at google/tapas-base and are newly initialized: ['output_bias', 'column_output_bias', 'column_output_weights', 'output_weights']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
Token indices sequence length is longer than the specified maximum sequence length for this model (567 > 512). Running this sequence through the model will result in indexing errors.

W
Predicted answer: Rated Capacity* (Grains@ Salt Dose), Total Water Used per Regeneration @ Maximum Salt Dose, Pressure Drop at Rated Service Flow (psig), Water Supply Maximum Hardness (gpg), Water Supply Maximum Clear Water Iron (ppm)***, Water Pressure Limits (minimum-maximum psi)****
```

can you please helpus with this.
[final_tables (2).zip](https://github.com/huggingface/transformers/files/6917872/final_tables.2.zip)

You are initializing `TapasForQuestionAnswering` with randomly initialized classification heads, hence the predictions will be random. The warning also prints this:

```
Some weights of TapasForQuestionAnswering were not initialized from the model checkpoint at google/tapas-base and are newly initialized: ['output_bias', 'column_output_bias', 'column_output_weights', 'output_weights']
```
Instead of initializing from `google/tapas-base`, you can initialize from any of the [checkpoints on the hub](https://huggingface.co/models?search=google/tapas) which have "finetuned" in their name, like `google/tapas-base-finetuned-wtq` for example. <|||||>Hey @NielsRogge ,
Thanks for your answer. unfortunately the same error is showing even for google/tapas-base-finetuned-tabfact. but for google/tapas-large-finetuned-wtq. answer is showing wrongly and its not for every run. 

```
queries = 'What is the service flow rate value'

GXSHC40N.pdf
GXSF30V.pdf
GXMH31H.pdf
GXSH40V_GXSH45V.pdf
Token indices sequence length is longer than the specified maximum sequence length for this model (568 > 512). Running this sequence through the model will result in indexing errors.
W
Predicted answer: 57.56/1.11, 20-125
```
but ideally it should show 7.5<|||||>Can you provide a colab to reproduce?<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.