title: VPA: Fully Test-Time Visual Prompt Adaptation
summary: Textual prompt tuning has demonstrated significant performance improvements
  in adapting natural language processing models to a variety of downstream tasks
  by treating hand-engineered prompts as trainable parameters. Inspired by the
  success of textual prompting, several studies have investigated the efficacy of
  visual prompt tuning. In this work, we present Visual Prompt Adaptation (VPA),
  the first framework that generalizes visual prompting with test-time
  adaptation. VPA introduces a small number of learnable tokens, enabling fully
  test-time and storage-efficient adaptation without necessitating source-domain
  information. We examine our VPA design under diverse adaptation settings,
  encompassing single-image, batched-image, and pseudo-label adaptation. We
  evaluate VPA on multiple tasks, including out-of-distribution (OOD)
  generalization, corruption robustness, and domain adaptation. Experimental
  results reveal that VPA effectively enhances OOD generalization by 3.3% across
  various models, surpassing previous test-time approaches. Furthermore, we show
  that VPA improves corruption robustness by 6.5% compared to strong baselines.
  Finally, we demonstrate that VPA also boosts domain adaptation performance by
  relatively 5.2%. Our VPA also exhibits marked effectiveness in improving the
  robustness of zero-shot recognition for vision-language models.
paper_authors._id: 6514f5f30885410769abebe2
paper_authors.name: Jiachen Sun
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2023-09-28T03:41:40.652Z
paper_authors._id: 643ed00b0d1194da249b7850
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/643ed00b0d1194da249b7850/AyrfPjxLM7BrHlAZzhsA0.jpeg
paper_authors.fullname: Jason
paper_authors.type: user
paper_authors.user: jcjcjcjcsun
paper_authors._id: 6514f5f30885410769abebe3
paper_authors.name: Mark Ibrahim
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2024-11-18T15:18:15.898Z
paper_authors._id: 64c17345e82e55936cf971bc
paper_authors.avatarUrl: /avatars/b39c5637e00291410620afb7c770d587.svg
paper_authors.fullname: Mark Ibrahim
paper_authors.type: user
paper_authors.user: marksibrahim
paper_authors._id: 6514f5f30885410769abebe4
paper_authors.name: Melissa Hall
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2024-07-17T13:00:53.313Z
paper_authors._id: 6697bfb4acbced03ebd133f8
paper_authors.avatarUrl: /avatars/a7d41367c6f7974303feb5af2a387ce4.svg
paper_authors.fullname: Melissa Hall
paper_authors.type: user
paper_authors.user: melissah14
paper_authors._id: 6514f5f30885410769abebe5
paper_authors.name: Ivan Evtimov
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2024-06-27T23:59:15.505Z
paper_authors._id: 667dbd8fcef5b7fa35acd25d
paper_authors.avatarUrl: /avatars/d484c9034d79b465744c831b30f04623.svg
paper_authors.fullname: Ivan Evtimov
paper_authors.type: user
paper_authors.user: ivanevtimov
paper_authors._id: 6514f5f30885410769abebe6
paper_authors.name: Z. Morley Mao
paper_authors._id: 6514f5f30885410769abebe7
paper_authors.name: Cristian Canton Ferrer
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-09-28T09:20:47.831Z
paper_authors._id: 63127ad9fa3e7ba13f17b11a
paper_authors.avatarUrl: /avatars/3b91b48702488526511ac86fde4f5e79.svg
paper_authors.fullname: cristian ferrer
paper_authors.type: user
paper_authors.user: chiky222
paper_authors._id: 6514f5f30885410769abebe8
paper_authors.name: Caner Hazirbas
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-09-28T19:03:40.682Z
paper_authors._id: 6515b9414eb20107d01eba58
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6515b9414eb20107d01eba58/8LSqNIgItKUVkWlEQERMD.jpeg
paper_authors.fullname: Caner Hazirbas
paper_authors.type: user
paper_authors.user: hazirbas
paper_title: VPA: Fully Test-Time Visual Prompt Adaptation
paper_summary: Textual prompt tuning has demonstrated significant performance improvements
  in adapting natural language processing models to a variety of downstream tasks
  by treating hand-engineered prompts as trainable parameters. Inspired by the
  success of textual prompting, several studies have investigated the efficacy of
  visual prompt tuning. In this work, we present Visual Prompt Adaptation (VPA),
  the first framework that generalizes visual prompting with test-time
  adaptation. VPA introduces a small number of learnable tokens, enabling fully
  test-time and storage-efficient adaptation without necessitating source-domain
  information. We examine our VPA design under diverse adaptation settings,
  encompassing single-image, batched-image, and pseudo-label adaptation. We
  evaluate VPA on multiple tasks, including out-of-distribution (OOD)
  generalization, corruption robustness, and domain adaptation. Experimental
  results reveal that VPA effectively enhances OOD generalization by 3.3% across
  various models, surpassing previous test-time approaches. Furthermore, we show
  that VPA improves corruption robustness by 6.5% compared to strong baselines.
  Finally, we demonstrate that VPA also boosts domain adaptation performance by
  relatively 5.2%. Our VPA also exhibits marked effectiveness in improving the
  robustness of zero-shot recognition for vision-language models.
paper_ai_summary: Visual Prompt Adaptation (VPA) enhances the performance of natural language processing models on various tasks through test-time adaptation using learnable tokens, improving OOD generalization, corruption robustness, and domain adaptation.
paper_ai_keywords: Visual Prompt Adaptation VPA learnable tokens test-time adaptation storage-efficient adaptation single-image batched-image pseudo-label adaptation out-of-distribution OOD generalization corruption robustness domain adaptation zero-shot recognition vision-language models