title: DreamDiffusion: Generating High-Quality Images from Brain EEG Signals
summary: This paper introduces DreamDiffusion, a novel method for generating
  high-quality images directly from brain electroencephalogram (EEG) signals,
  without the need to translate thoughts into text. DreamDiffusion leverages
  pre-trained text-to-image models and employs temporal masked signal modeling to
  pre-train the EEG encoder for effective and robust EEG representations.
  Additionally, the method further leverages the CLIP image encoder to provide
  extra supervision to better align EEG, text, and image embeddings with limited
  EEG-image pairs. Overall, the proposed method overcomes the challenges of using
  EEG signals for image generation, such as noise, limited information, and
  individual differences, and achieves promising results. Quantitative and
  qualitative results demonstrate the effectiveness of the proposed method as a
  significant step towards portable and low-cost ``thoughts-to-image'', with
  potential applications in neuroscience and computer vision.
paper_authors._id: 649e4348d61f393407b23980
paper_authors.name: Yunpeng Bai
paper_authors._id: 649e4348d61f393407b23981
paper_authors.name: Xintao Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-06-30T17:53:08.979Z
paper_authors._id: 60e272ca6c78a8c122b12127
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/60e272ca6c78a8c122b12127/xldEGBzGrU-bX6IwAw0Ie.jpeg
paper_authors.fullname: Xintao Wang
paper_authors.type: user
paper_authors.user: Xintao
paper_authors._id: 649e4348d61f393407b23982
paper_authors.name: Yanpei Cao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-06-30T17:53:34.881Z
paper_authors._id: 638066faf022c8a5803f7eb8
paper_authors.avatarUrl: /avatars/4cfd699c3f6c5461b12b7dc5e3fe183d.svg
paper_authors.fullname: Yanpei Cao
paper_authors.type: user
paper_authors.user: pookiefoof
paper_authors._id: 649e4348d61f393407b23983
paper_authors.name: Yixiao Ge
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-06-30T17:53:54.915Z
paper_authors._id: 640e9762b03f4cd29f58d982
paper_authors.avatarUrl: /avatars/81da37d628163fe3e094b247c7c3a3b5.svg
paper_authors.fullname: Yixiao Ge
paper_authors.type: user
paper_authors.user: yxgeee
paper_authors._id: 649e4348d61f393407b23984
paper_authors.name: Chun Yuan
paper_authors._id: 649e4348d61f393407b23985
paper_authors.name: Ying Shan
paper_title: DreamDiffusion: Generating High-Quality Images from Brain EEG Signals
paper_summary: This paper introduces DreamDiffusion, a novel method for generating
  high-quality images directly from brain electroencephalogram (EEG) signals,
  without the need to translate thoughts into text. DreamDiffusion leverages
  pre-trained text-to-image models and employs temporal masked signal modeling to
  pre-train the EEG encoder for effective and robust EEG representations.
  Additionally, the method further leverages the CLIP image encoder to provide
  extra supervision to better align EEG, text, and image embeddings with limited
  EEG-image pairs. Overall, the proposed method overcomes the challenges of using
  EEG signals for image generation, such as noise, limited information, and
  individual differences, and achieves promising results. Quantitative and
  qualitative results demonstrate the effectiveness of the proposed method as a
  significant step towards portable and low-cost ``thoughts-to-image'', with
  potential applications in neuroscience and computer vision.
paper_ai_summary: DreamDiffusion generates high-quality images from EEG signals using pre-trained text-to-image models and CLIP image encoder for improved alignment and robust representations, overcoming noise and data limitations.
paper_ai_keywords: temporal masked signal modeling EEG encoder CLIP image encoder EEG signals image generation neuroscience computer vision