Use LMDeploy as default backend
#2
by
syarm-gd
- opened
Hi! First of all, thanks for the efforts you put in the project.
I have tried the freshly updated setup.sh and it works like a charm! Application starts without any issues after installation is over.
But when I try to process any file via frontend webpage, it fails with an error
"Error processing document: LMDeploy is not installed. Please install it following: https://github.com/Yuliang-Liu/MonkeyOCR/blob/main/docs/install_cuda.md to use MonkeyChat_LMDeploy."
AFAIK, LMDeploy is CUDA profile which is not runnable on Apple Silicon processors. It seems like there should be some MonkeyChat_MPS implementation for the chat which is not there yet. Please let me know if I am missing something!
The logs are provided below:
2025-06-25 12:01:43.923 | INFO | __main__:initialize_model:49 - Initializing MonkeyOCR model with MPS acceleration...
2025-06-25 12:01:43.924 | INFO | __main__:initialize_model:62 - MPS is available and will be used for acceleration
2025-06-25 12:01:43.927 | INFO | magic_pdf.model.custom_model:__init__:26 - using configs: {'device': 'mps', 'weights': {'doclayout_yolo': 'Structure/doclayout_yolo_docstructbench_imgsz1280_2501.pt', 'layoutreader': 'Relation'}, 'models_dir': 'MonkeyOCR/model_weight', 'layout_config': {'model': 'doclayout_yolo', 'reader': {'name': 'layoutreader'}}, 'chat_config': {'weight_path': 'MonkeyOCR/model_weight/Recognition', 'backend': 'auto', 'batch_size': 1, 'dtype': 'float16', 'max_new_tokens': 256, 'temperature': 0.0, 'do_sample': False}}
2025-06-25 12:01:43.927 | INFO | magic_pdf.model.custom_model:__init__:29 - using device: mps
2025-06-25 12:01:43.927 | INFO | magic_pdf.model.custom_model:__init__:41 - using models_dir: MonkeyOCR/model_weight
2025-06-25 12:01:44.246 | INFO | magic_pdf.model.custom_model:__init__:69 - layout model loaded: doclayout_yolo
2025-06-25 12:01:46.936 | INFO | magic_pdf.model.custom_model:__init__:95 - layoutreader model loaded: layoutreader
2025-06-25 12:01:46.937 | WARNING | magic_pdf.model.custom_model:__init__:121 - Use LMDeploy as default backend
2025-06-25 12:01:46.937 | ERROR | __main__:initialize_model:71 - Failed to initialize model: LMDeploy is not installed. Please install it following: https://github.com/Yuliang-Liu/MonkeyOCR/blob/main/docs/install_cuda.md to use MonkeyChat_LMDeploy.
Hi
LMdeploy is fall back for NvdA
Setup.sh enforces mlx