Hi @philschmid and @marshmellow77
Thanks a lot for information comments!
I gave a first try on Serverless LayoutLMv2. On the first try I managed to create an endpoint successfully, however after invoking it I saw an error that detectron2 which LayoutLMv2 depends on was not found.
So I tried re-creating the checkpoint by adding git+https://github.com/facebookresearch/detectron2.git in the requirements. However this time the endpoint creation failed due to timeout.
Failure reason
Unable to successfully stand up your model within the allotted 180 second timeout. 
Please ensure that downloading your model artifacts, starting your model container and passing the ping health checks can be completed within 180 seconds.

I looked at the logs and the latest messages I saw were regarding the detectron2 installation. It is very likely that the endpoint creation is timing out due to additional time taking to install it, but I could be wrong.
I digged a bit into documentation to see if I can increase the delay. I added the WaiterConfig flags to the waiter wait
waiter.wait(EndpointName=huggingface_endpoint_name,
           WaiterConfig={
               'Delay': 480,
               'MaxAttempts': 120
           })

However still getting the same error.
I am not sure if I am setting up these delay flags correctly, so I would appreciate your help in pointing out how to potentially resolve this timeout issue. Thanks!