Answers.Body: <p>They are not the same model, and the text-only model is definitively able to output graphical content via various forms of code; see e.g. <a href="https://arxiv.org/abs/2303.12712" rel="nofollow noreferrer">Sparks of Artificial General Intelligence: Early experiments with GPT-4</a> for a source on this and for some concrete examples.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-04-14T11:42:34.547Z
Answers.LastActivityDate: 2023-04-17T14:23:20.870Z
Answers.LastEditDate: 2023-04-17T14:23:20.870Z
Answers.UserDisplayName: Polytropos
Answers.Body: <p>No chatGPT is a text-model only.</p>
  <p>If you know that HAL 9000 is mainly a red circle, and you know how to generate a red circle in python, you don't need to visually see the image to provide the code.</p>
  <p>There are many examples where the model can generate very impressive instructions, for example how to make 3d with <a href="https://www.blender.org/" rel="nofollow noreferrer">blender</a>. But that does not mean ChatGPT literally opens the program and works with it, before providing instructions.</p>
  <p>It is discussed <a href="https://openai.com/research/image-gpt#from-language-gpt-to-image-gpt" rel="nofollow noreferrer">here</a> by OpenAI the use of GPT technology as an image-model.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-04-14T20:14:36.967Z
Answers.Text: ChatGPT (aka GPT-3.5) has no variant that supports images, so it it already has the capability regarding images, there is less reason to think GPT-4 must have seen images to do that, interesting.
Answers.UserDisplayName: Volker Siegel
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-04-14T20:23:06.103Z
Answers.Text: ImageGPT is something fundamentally different. It has never seen any text, it literally predicts the color of the next individual pixel, row by row top to bottom, left to right.
Answers.UserDisplayName: Volker Siegel
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-04-14T20:25:44.037Z
Answers.Text: ChatGPT does not internally use other models like DALLE-2. And GPT-4 can roughly do what DALLE-2 can do, by itself, in the unpublished model that supports images.
Answers.UserDisplayName: Volker Siegel
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-04-14T11:55:26.690Z
Answers.LastActivityDate: 2023-04-16T07:41:04.463Z
Answers.LastEditDate: 2023-04-16T07:41:04.463Z
Answers.UserDisplayName: nammerkage
Body: <p>Currently, the published GPT-4 can input and output text.
  A version of GPT-4 that can input and output text and images exists, according to the technical report, but is not yet publicly available.</p>
  <p>I suspect that they are the same.
  Only the interface for image input and output is not publicly available.</p>
  <p>That is because GPT-4 shows some understanding of images if you find a way to output them. One way is asking it to write a program that produces the image output. And that actually works, demonstrated below:</p>
  <blockquote>
  <p>Prompt: &quot;Write a program that creates a picture of HAL 9000&quot;<br />
  [System prompt empty]</p>
  </blockquote>
  <p>The result is this image (created by following the directions in the text output):</p>
  <p><a href="https://i.sstatic.net/dSo0T.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/dSo0T.png" alt="Window created by GPT-4 generated program showing HAL9000" /></a></p>
  <p>The following is the unedited GPT-4 output (formatting in the original):</p>
  <hr />
  <p>Creating a visual program requires a programming language with graphics libraries or frameworks, as well as a specific platform or environment to run the code. Here's an example using Python and the Pygame library to create a simple picture of HAL 9000:</p>
  <ol>
  <li>First, you need to install the Pygame library if you haven't already. You can do this by running the following command in your terminal:</li>
  </ol>
  <pre><code>pip install pygame
  </code></pre>
  <ol start="2">
  <li>Next, create a new Python file (e.g. hal9000.py) and paste the following code:</li>
  </ol>
  <pre><code>import pygame
  
  # Initialize Pygame
  pygame.init()
  
  # Set up the display
  width, height = 400, 400
  screen = pygame.display.set_mode((width, height))
  pygame.display.set_caption(&quot;HAL 9000&quot;)
  
  # Define colors
  BLACK = (0, 0, 0)
  RED = (255, 0, 0)
  WHITE = (255, 255, 255)
  
  # Draw HAL 9000
  def draw_hal9000(surface):
      surface.fill(BLACK)
      
      # Draw the outer circle
      pygame.draw.circle(surface, WHITE, (width // 2, height // 2), 100)
      
      # Draw the inner circle
      pygame.draw.circle(surface, RED, (width // 2, height // 2), 80)
  
      # Draw the lens
      pygame.draw.ellipse(surface, BLACK, (width // 2 - 40, height // 2 - 60, 80, 120))
  
      # Draw the lens reflection
      pygame.draw.ellipse(surface, WHITE, (width // 2 - 20, height // 2 - 45, 15, 25), 2)
  
  # Main loop
  running = True
  while running:
      for event in pygame.event.get():
          if event.type == pygame.QUIT:
              running = False
      
      draw_hal9000(screen)
      pygame.display.flip()
  
  # Clean up
  pygame.quit()
  </code></pre>
  <ol start="3">
  <li>Save the file and run the program using the following command in your terminal:</li>
  </ol>
  <pre><code>python hal9000.py
  </code></pre>
  <p>This code will create a simple picture of HAL 9000 using the Pygame library. The main loop will keep the program running until you close the window.</p>
  <p>Note: This example assumes that you have Python and pip (package installer for Python) already installed on your system. If you don't have Python installed, you can download it from the official website: <a href="https://www.python.org/downloads/" rel="nofollow noreferrer">https://www.python.org/downloads/</a></p>
  <hr />
  
Tags: <image-recognition><image-generation><language-model><large-language-models><gpt-4>
Title: Is the GPT-4 for text the same model that can input and output images?
UserDisplayName: Volker Siegel
ThreadText: # Is the GPT-4 for text the same model that can input and output images?
  
  (Asked by: username_0 on 2023-04-13)
  
  Currently, the published GPT-4 can input and output text.
  A version of GPT-4 that can input and output text and images exists, according to the technical report, but is not yet publicly available.
  I suspect that they are the same.
  Only the interface for image input and output is not publicly available.
  That is because GPT-4 shows some understanding of images if you find a way to output them. One way is asking it to write a program that produces the image output. And that actually works, demonstrated below:
  
  Prompt: "Write a program that creates a picture of HAL 9000"
  [System prompt empty]
  
  The result is this image (created by following the directions in the text output):
  
  The following is the unedited GPT-4 output (formatting in the original):
  
  Creating a visual program requires a programming language with graphics libraries or frameworks, as well as a specific platform or environment to run the code. Here's an example using Python and the Pygame library to create a simple picture of HAL 9000:
  
  First, you need to install the Pygame library if you haven't already. You can do this by running the following command in your terminal:
  
  pip install pygame
  
  
  Next, create a new Python file (e.g. hal9000.py) and paste the following code:
  
  import pygame
  
  # Initialize Pygame
  pygame.init()
  
  # Set up the display
  width, height = 400, 400
  screen = pygame.display.set_mode((width, height))
  pygame.display.set_caption("HAL 9000")
  
  # Define colors
  BLACK = (0, 0, 0)
  RED = (255, 0, 0)
  WHITE = (255, 255, 255)
  
  # Draw HAL 9000
  def draw_hal9000(surface):
      surface.fill(BLACK)
      
      # Draw the outer circle
      pygame.draw.circle(surface, WHITE, (width // 2, height // 2), 100)
      
      # Draw the inner circle
      pygame.draw.circle(surface, RED, (width // 2, height // 2), 80)
  
      # Draw the lens
      pygame.draw.ellipse(surface, BLACK, (width // 2 - 40, height // 2 - 60, 80, 120))
  
      # Draw the lens reflection
      pygame.draw.ellipse(surface, WHITE, (width // 2 - 20, height // 2 - 45, 15, 25), 2)
  
  # Main loop
  running = True
  while running:
      for event in pygame.event.get():
          if event.type == pygame.QUIT:
              running = False
      
      draw_hal9000(screen)
      pygame.display.flip()
  
  # Clean up
  pygame.quit()
  
  
  Save the file and run the program using the following command in your terminal:
  
  python hal9000.py
  
  This code will create a simple picture of HAL 9000 using the Pygame library. The main loop will keep the program running until you close the window.
  Note: This example assumes that you have Python and pip (package installer for Python) already installed on your system. If you don't have Python installed, you can download it from the official website: https://www.python.org/downloads/
  
  **Tags:** image-recognition, image-generation, language-model, large-language-models, gpt-4
  
  ---
  
  ## 2 answers
  
  ### Answer by username_1 on 2023-04-14. Score: 1
  
  They are not the same model, and the text-only model is definitively able to output graphical content via various forms of code; see e.g. [Sparks of Artificial General Intelligence: Early experiments with GPT-4](https://arxiv.org/abs/2303.12712) for a source on this and for some concrete examples.
  
  
  ---
  
  ### Answer by username_2 on 2023-04-14. Score: 1
  
  No chatGPT is a text-model only.
  
  If you know that HAL 9000 is mainly a red circle, and you know how to generate a red circle in python, you don't need to visually see the image to provide the code.
  
  There are many examples where the model can generate very impressive instructions, for example how to make 3d with [blender](https://www.blender.org/). But that does not mean ChatGPT literally opens the program and works with it, before providing instructions.
  
  It is discussed [here](https://openai.com/research/image-gpt#from-language-gpt-to-image-gpt) by OpenAI the use of GPT technology as an image-model.
  
  > **Comments:**
  > **username_0**: ChatGPT (aka GPT-3.5) has no variant that supports images, so it it already has the capability regarding images, there is less reason to think GPT-4 must have seen images to do that, interesting.  
  > **username_0**: ImageGPT is something fundamentally different. It has never seen any text, it literally predicts the color of the next individual pixel, row by row top to bottom, left to right.  
  > **username_0**: ChatGPT does not internally use other models like DALLE-2. And GPT-4 can roughly do what DALLE-2 can do, by itself, in the unpublished model that supports images.  
  
  ---
  
  