layoutlmv2 input tensor shape
![question_hugging_face](https://user-images.githubusercontent.com/20882775/140917182-f88fc896-a3fd-40c5-988d-d588110aac45.JPG)
Hi, 
AFAIK encoder in layoutlmv2 only takes 512 tokes but this snippet from hugging face documentation attached above contradicts this.
could anyone help me understand this?

Hi,

LayoutLMv2 (and LayoutXLM) take both an `image` input and the regular `input_ids`. The `input_ids` indeed have a length of max 512 tokens (if you pad them up to the max length), however the image input is turned into a feature map, which is concatenated with the text tokens. This feature map is then flattened into a list of "image tokens", which are then concatenated with the text tokens.
<|||||>@NielsRogge As you mentioned, the feature map (7*7) is flattened into a list of image tokens, which in this case is 49 (documentation), which is then concatenated with 512 to make the entire length of input 561, which is then sent into the encoder, which only accepts 512 tokens as input. What is going on with the "image tokens"? Aint they're going into the encoder? 
but as written "these are then concatenated with the text tokens, and send through the Transformer encoder"
What exactly am I missing here?<|||||>I'm not sure what you mean. You just need to give a resized `image` (224x224) as well as `input_ids` (and `token_type_ids`, `bbox`) to the model. The model will internally create the image tokens using its visual backbone.<|||||>![layoutlmV2](https://user-images.githubusercontent.com/20882775/140938612-035f3e36-f64b-482a-b880-af2a17054d7c.jpg)
are these 49 tokens fed into the transformer encoder after concatenation?
<|||||>Yes, the text and image tokens are first concatenated, before being fed to the Transformer encoder.<|||||> layoutlm paper says "We initialize the weight of LayoutLM model with the pre-trained BERT base model."  with is trained with 512 tokens only.
how come are we able to pass 561 tokens into the encoder in layoutlmv2 which is initialized with the pre-trained BERT model?<|||||>As stated in the [LayoutLMv2 paper](https://arxiv.org/abs/2012.14740):

> The model is initialized from the existing pre-trained model checkpoints. For the encoder along
with the text embedding layer, LayoutLMv2 uses the same architecture as UniLMv2 (Bao et al.,
2020), thus it is initialized from UniLMv2. For the ResNeXt-FPN part in the visual embedding
layer, the backbone of a Mask-RCNN (He et al., 2017) model trained on PubLayNet is leveraged. 
The rest of the parameters in the model are randomly initialized.<|||||>thanks for the clarification @NielsRogge 
