layoutlmv2 visual backbone pretrained on publaynet 
@NielsRogge Amazing work thanks for sharing,
Does layoutlmv2's visual backbone utilize ResNeXt-FPN trained on Publaynet as described in the paper? If so, could you please point out where the weights are used to initialize the model in the repo?
Hi,

The LayoutLMv2 authors indeed leveraged a ResNeXT-FPN for the visual backbone. It was taken from this repo: https://github.com/hpanwar08/detectron2.

However, we've just added the [Document Image Transformer (DiT)](https://huggingface.co/docs/transformers/master/en/model_doc/dit), which actually outperforms the ResNeXT model on PubLayNet with the same object detection framework (Mask R-CNN), reaching an AP of 0.945/0.949 for DiT-base/large respectively (vs. 0.906 for ResNeXT).

Take a look at this Space for a demo: https://huggingface.co/spaces/nielsr/dit-document-layout-analysis<|||||>@NielsRogge Thanks for responding; yes, I got my hands on DIT, which is a fantastic model, but I'm trying to figure out where the ResNeXT-FPN weights are initialized in the layoutlmv2 model for the current hugging face implementation. 
https://github.com/microsoft/unilm/blob/master/layoutlmft/layoutlmft/models/layoutlmv2/detectron2_config.py i don't see any weights path given here <|||||>I don't think Microsoft open-sourced this weights initialization. This was probably part of their pre-training code (which is not open-sourced).

Weights can be found here: https://github.com/hpanwar08/detectron2#benchmarking<|||||>Then how does the visual backbone extracts features from the image? I can see the model config for detectron2 but I can't figure out what weights the model is getting initialized with? 
for example in detectron2, we pass config fille and YAML for weights 
```
cfg.merge_from_file(model_zoo.get_config_file("COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x.yaml"))
cfg.MODEL.WEIGHTS = model_zoo.get_checkpoint_url("COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x.yaml")
```<|||||>Yes I know, they probably set the MODEL.WEIGHTS attribute in their pre-training code, which they didn't open-source.

But as linked above, you can easily get the backbone weights.<|||||>I want to override the weights for the model but when I pass the weight's path in the visual backbone class the model is not picking up the weights but the MODEL.WEIGHTS key in config is showing the model path string,
 Any reasons why? @NielsRogge <|||||>got the weights from the backbone and loaded the custom weights <|||||>Hi @gauravlochab,
Could you please share how you managed load the custom weights?