Frontiers in Intelligent Colonoscopy
Colonoscopy is currently one of the most sensitive screening methods for
colorectal cancer. This study investigates the frontiers of intelligent
colonoscopy techniques and their prospective implications for multimodal
medical applications. With this goal, we begin by assessing the current
data-centric and model-centric landscapes through four tasks for colonoscopic
scene perception, including classification, detection, segmentation, and
vision-language understanding. This assessment enables us to identify
domain-specific challenges and reveals that multimodal research in colonoscopy
remains open for further exploration. To embrace the coming multimodal era, we
establish three foundational initiatives: a large-scale multimodal instruction
tuning dataset ColonINST, a colonoscopy-designed multimodal language model
ColonGPT, and a multimodal benchmark. To facilitate ongoing monitoring of this
rapidly evolving field, we provide a public website for the latest updates:
https://github.com/ai4colonoscopy/IntelliScope.
67188d1ce4c594992b657d03
Ge-Peng Ji
67188d1ce4c594992b657d04
Jingyi Liu
67188d1ce4c594992b657d05
Peng Xu
admin_assigned
2024-10-24T08:37:11.566Z
66a1d926da490fbd69091715
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/Z5AFfJxktltm_45EAN6U3.jpeg
pengxu
user
pengxu2024
67188d1ce4c594992b657d06
Nick Barnes
67188d1ce4c594992b657d07
Fahad Shahbaz Khan
67188d1ce4c594992b657d08
Salman Khan
67188d1ce4c594992b657d09
Deng-Ping Fan
Frontiers in Intelligent Colonoscopy
Colonoscopy is currently one of the most sensitive screening methods for
colorectal cancer. This study investigates the frontiers of intelligent
colonoscopy techniques and their prospective implications for multimodal
medical applications. With this goal, we begin by assessing the current
data-centric and model-centric landscapes through four tasks for colonoscopic
scene perception, including classification, detection, segmentation, and
vision-language understanding. This assessment enables us to identify
domain-specific challenges and reveals that multimodal research in colonoscopy
remains open for further exploration. To embrace the coming multimodal era, we
establish three foundational initiatives: a large-scale multimodal instruction
tuning dataset ColonINST, a colonoscopy-designed multimodal language model
ColonGPT, and a multimodal benchmark. To facilitate ongoing monitoring of this
rapidly evolving field, we provide a public website for the latest updates:
https://github.com/ai4colonoscopy/IntelliScope.
Intelligent colonoscopy techniques are explored through classification, detection, segmentation, and vision-language understanding tasks, addressing domain-specific challenges with the development of ColonINST, ColonGPT, and a multimodal benchmark.
data-centric model-centric colonoscopic scene perception classification detection segmentation vision-language understanding multimodal instruction tuning multimodal language model multimodal benchmark