GPT Sonograpy: Hand Gesture Decoding from Forearm Ultrasound Images via VLM
<|reference_start|>GPT Sonograpy: Hand Gesture Decoding from Forearm Ultrasound Images via VLM: Large vision-language models (LVLMs), such as the Generative Pre-trained Transformer 4-omni (GPT-4o), are emerging multi-modal foundation models which have great potential as powerful artificial-intelligence (AI) assistance tools for a myriad of applications, including healthcare, industrial, and academic sectors. Although such foundation models perform well in a wide range of general tasks, their capability without fine-tuning is often limited in specialized tasks. However, full fine-tuning of large foundation models is challenging due to enormous computation/memory/dataset requirements. We show that GPT-4o can decode hand gestures from forearm ultrasound data even with no fine-tuning, and improves with few-shot, in-context learning.<|reference_end|>
@article{bimbraw2024gpt,
    title={GPT Sonograpy: Hand Gesture Decoding from Forearm Ultrasound Images via
  VLM},
    author={Keshav Bimbraw, Ye Wang, Jing Liu, Toshiaki Koike-Akino},
    journal={arXiv preprint arXiv:2407.10870},
    year={2024},
    archivePrefix={arXiv},
    eprint={2407.10870},
    primaryClass={cs.CV cs.AI cs.HC cs.LG}
}