title: MVLLaVA: An Intelligent Agent for Unified and Flexible Novel View
    Synthesis
summary: This paper introduces MVLLaVA, an intelligent agent designed for novel view
  synthesis tasks. MVLLaVA integrates multiple multi-view diffusion models with a
  large multimodal model, LLaVA, enabling it to handle a wide range of tasks
  efficiently. MVLLaVA represents a versatile and unified platform that adapts to
  diverse input types, including a single image, a descriptive caption, or a
  specific change in viewing azimuth, guided by language instructions for
  viewpoint generation. We carefully craft task-specific instruction templates,
  which are subsequently used to fine-tune LLaVA. As a result, MVLLaVA acquires
  the capability to generate novel view images based on user instructions,
  demonstrating its flexibility across diverse tasks. Experiments are conducted
  to validate the effectiveness of MVLLaVA, demonstrating its robust performance
  and versatility in tackling diverse novel view synthesis challenges.
paper_authors._id: 66e256dc239f8302156d9417
paper_authors.name: Hanyu Jiang
paper_authors._id: 66e256dc239f8302156d9418
paper_authors.name: Jian Xue
paper_authors._id: 66e256dc239f8302156d9419
paper_authors.name: Xing Lan
paper_authors._id: 66e256dc239f8302156d941a
paper_authors.name: Guohong Hu
paper_authors._id: 66e256dc239f8302156d941b
paper_authors.name: Ke Lu
paper_title: MVLLaVA: An Intelligent Agent for Unified and Flexible Novel View
    Synthesis
paper_summary: This paper introduces MVLLaVA, an intelligent agent designed for novel view
  synthesis tasks. MVLLaVA integrates multiple multi-view diffusion models with a
  large multimodal model, LLaVA, enabling it to handle a wide range of tasks
  efficiently. MVLLaVA represents a versatile and unified platform that adapts to
  diverse input types, including a single image, a descriptive caption, or a
  specific change in viewing azimuth, guided by language instructions for
  viewpoint generation. We carefully craft task-specific instruction templates,
  which are subsequently used to fine-tune LLaVA. As a result, MVLLaVA acquires
  the capability to generate novel view images based on user instructions,
  demonstrating its flexibility across diverse tasks. Experiments are conducted
  to validate the effectiveness of MVLLaVA, demonstrating its robust performance
  and versatility in tackling diverse novel view synthesis challenges.
paper_ai_summary: MVLLaVA combines multi-view diffusion models with LLaVA to efficiently generate novel views based on user instructions for diverse tasks.
paper_ai_keywords: multi-view diffusion models LLaVA novel view synthesis viewpoint generation task-specific instruction templates