Answers.Body: <p><strong>Assuming you create the perfect architecture</strong></p>
  <p>What your network will be learning will be correlations between objects/textures/colors and thermal signatures. Not the thermal signatures themselves.</p>
  <p>It will NOT work the same as a thermal camera. It will not give you actual thermal signatures.</p>
  <p>It will not reflect &quot;this is how hot it is&quot;<br />
  It will only highlight those things that naturally change color due to heat.   So it'd be more like a &quot;highlighter&quot; that tells you where something has changed visually due to heat.</p>
  <p>Examples of where it will not work:</p>
  <ul>
  <li><p>It is common to use thermal cameras to find shorts on motherboards/circuits. Usually, the area where the short is occurring will be VERY hot and will show up well on a thermal camera, but not look like anything in the RGB spectrum<br />
  <a href="https://i.sstatic.net/avuG1.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/avuG1.png" alt="enter image description here" /></a></p>
  </li>
  <li><p>Doing heat profiling for insulation in a room. Will not show up visually. Only on thermal bands.</p>
  </li>
  <li><p>Overheated pipes</p>
  </li>
  </ul>
  <p><strong>Why</strong></p>
  <p>RGB, measure red green and blue respones. Thermal is a measurement on its own like Red, Green, or Blue is. RGB are how strong the signal is on various ranges, on the electromagnetic spectrum. Theres a range for red, green, blue, and thermal respectively.<br />
  <a href="https://i.sstatic.net/IXG5W.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/IXG5W.png" alt="enter image description here" /></a></p>
  <p>The thermal range can be found here:</p>
  <p><a href="https://i.sstatic.net/TI2EZ.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/TI2EZ.png" alt="enter image description here" /></a></p>
  <p><strong>what you'll be learning</strong></p>
  <p>Your approach will be using objects and learned properties of those objects as sort of a lookup mapping between objects and thermal signatures.</p>
  <p>This will only work if objects visually reflect heat changes. (which many don't)</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-03-02T13:29:13.863Z
Answers.Text: Thank you for the long answer. I know that it will now work like a ideal thermal camera. I will now use this data for real time training. My project is on sensor fusion with RGB and thermal camera. But there is no dataset for my particular application. But I have RGB images with semantic maps. So, if I can convert these RGB images to IR images, I will have a complete dataset on which I can train any network. For validation and testing I will use the IR images from actual IR camera.
Answers.UserDisplayName: programmer_04_03
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-03-02T14:53:11.973Z
Answers.Text: My prediction is that it will not work like an ideal thermal camera. Not unless there is a correlation between thermal bands and visual bands. There isn't always. The semantic maps can act as a maker revealing whether or not a visual correlation might exist for a given object class, but I think an overall system wouldn't be a good substitute for an actual IR band.   
  
  I'm curious to see how well your system performs. I know that sensor-fusion is a hot topic in some remote-sensing/satellite imagery processing systems. Check for publications in that space.
Answers.UserDisplayName: Conic
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-03-02T15:15:42.133Z
Answers.Text: I am not sure about that either. I have tried some off the shelf GAN networks that I got on github for rgb to thermal image conversion. The results are underwhelming. I am trying with a bit more variation in dataset. I hope that this will work. I will update here with the results that I get.
Answers.UserDisplayName: programmer_04_03
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-02-28T16:54:11.160Z
Answers.LastActivityDate: 2023-02-28T16:54:11.160Z
Answers.UserDisplayName: Conic
Answers.Body: <p>you may check SpeakingFaces paper by ISSAI. They did something similar with CycleGAN, only with faces though.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-03-02T13:29:33.483Z
Answers.Text: ok. I will check it out
Answers.UserDisplayName: programmer_04_03
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-03-02T19:32:01.730Z
Answers.Text: Your answer could be improved with additional supporting information. Please [edit] to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers [in the help center](/help/how-to-answer).
Answers.UserDisplayName: Community
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-03-30T19:24:04.560Z
Answers.Text: This feels like more of a comment than an answer.
Answers.UserDisplayName: David Hoelzer
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-02-28T13:26:45.957Z
Answers.LastActivityDate: 2023-02-28T13:26:45.957Z
Answers.UserDisplayName: Nurkhan Laiyk
Body: <p>I am working on a project where I am planning to convert RGB images to thermal images. I can convert to either near infrared spectrum images or far infrared spectrum image.</p>
  <p>I am planing on using Generative networks for the task, specifically Pix2Pix. For training GAN, there are datasets available with synchronized RGB and thermal image like dataset for MFNet.</p>
  <p>I will be grateful if anyone can tell me if it is even possible or if it is possible how close the generated images will be to actual thermal images, I will be grateful.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-02-20T11:32:59.673Z
Comments.Text: Maybe you should explain more what a "thermal image" is and how different it is from the RGB images.
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-02-20T12:51:45.130Z
Comments.Text: A thermal image captures light waves in spectrum in different electromagnetic wavelength than RGB cameras. This wavelength is particularly emitted by objects based on their respective temperatures. And these wavelengths are captured by thermal cameras. These cameras can be classified based on the wavelengths they capture. Near infrared, far infrared etc. And these images are particularly gray scale(single channel) and in these images, each pixel denotes temperature at that particular point in the image. So, essentially I want to convert RGB images to these gray scale thermal images.
Comments.UserDisplayName: programmer_04_03
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-02-20T12:51:57.930Z
Comments.Text: Hope this helps.
Comments.UserDisplayName: programmer_04_03
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-02-28T13:33:18.670Z
Comments.Text: probably possible with however they do per-pixel object classification. Note there's no relationship between RGB colour and thermal colour, so the model will basically be trying to guess what kind of things are in the picture, then what colour those things are.
Comments.UserDisplayName: Criticize SE actions means ban
Tags: <generative-adversarial-networks><image-processing>
Title: Converting RGB images to Thermal Images
UserDisplayName: programmer_04_03
ThreadText: # Converting RGB images to Thermal Images
  
  (Asked by: username_0 on 2023-02-19)
  
  I am working on a project where I am planning to convert RGB images to thermal images. I can convert to either near infrared spectrum images or far infrared spectrum image.
  
  I am planing on using Generative networks for the task, specifically Pix2Pix. For training GAN, there are datasets available with synchronized RGB and thermal image like dataset for MFNet.
  
  I will be grateful if anyone can tell me if it is even possible or if it is possible how close the generated images will be to actual thermal images, I will be grateful.
  
  **Tags:** generative-adversarial-networks, image-processing
  > **Comments:**
  > **username_1**: Maybe you should explain more what a "thermal image" is and how different it is from the RGB images.  
  > **username_0**: A thermal image captures light waves in spectrum in different electromagnetic wavelength than RGB cameras. This wavelength is particularly emitted by objects based on their respective temperatures. And these wavelengths are captured by thermal cameras. These cameras can be classified based on the wavelengths they capture. Near infrared, far infrared etc. And these images are particularly gray scale(single channel) and in these images, each pixel denotes temperature at that particular point in the image. So, essentially I want to convert RGB images to these gray scale thermal images.  
  > **username_0**: Hope this helps.  
  > **username_2**: probably possible with however they do per-pixel object classification. Note there's no relationship between RGB colour and thermal colour, so the model will basically be trying to guess what kind of things are in the picture, then what colour those things are.  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_3 on 2023-02-28. Score: 1
  
  **Assuming you create the perfect architecture**
  
  What your network will be learning will be correlations between objects/textures/colors and thermal signatures. Not the thermal signatures themselves.
  
  It will NOT work the same as a thermal camera. It will not give you actual thermal signatures.
  
  It will not reflect "this is how hot it is"  
  It will only highlight those things that naturally change color due to heat. So it'd be more like a "highlighter" that tells you where something has changed visually due to heat.
  
  Examples of where it will not work:
  
  * It is common to use thermal cameras to find shorts on motherboards/circuits. Usually, the area where the short is occurring will be VERY hot and will show up well on a thermal camera, but not look like anything in the RGB spectrum  
    [![enter image description here](https://i.sstatic.net/avuG1.png)](https://i.sstatic.net/avuG1.png)
  * Doing heat profiling for insulation in a room. Will not show up visually. Only on thermal bands.
  * Overheated pipes
  
  **Why**
  
  RGB, measure red green and blue respones. Thermal is a measurement on its own like Red, Green, or Blue is. RGB are how strong the signal is on various ranges, on the electromagnetic spectrum. Theres a range for red, green, blue, and thermal respectively.  
  [![enter image description here](https://i.sstatic.net/IXG5W.png)](https://i.sstatic.net/IXG5W.png)
  
  The thermal range can be found here:
  
  [![enter image description here](https://i.sstatic.net/TI2EZ.png)](https://i.sstatic.net/TI2EZ.png)
  
  **what you'll be learning**
  
  Your approach will be using objects and learned properties of those objects as sort of a lookup mapping between objects and thermal signatures.
  
  This will only work if objects visually reflect heat changes. (which many don't)
  
  > **Comments:**
  > **username_0**: Thank you for the long answer. I know that it will now work like a ideal thermal camera. I will now use this data for real time training. My project is on sensor fusion with RGB and thermal camera. But there is no dataset for my particular application. But I have RGB images with semantic maps. So, if I can convert these RGB images to IR images, I will have a complete dataset on which I can train any network. For validation and testing I will use the IR images from actual IR camera.  
  > **username_3**: My prediction is that it will not work like an ideal thermal camera. Not unless there is a correlation between thermal bands and visual bands. There isn't always. The semantic maps can act as a maker revealing whether or not a visual correlation might exist for a given object class, but I think an overall system wouldn't be a good substitute for an actual IR band.     I'm curious to see how well your system performs. I know that sensor-fusion is a hot topic in some remote-sensing/satellite imagery processing systems. Check for publications in that space.  
  > **username_0**: I am not sure about that either. I have tried some off the shelf GAN networks that I got on github for rgb to thermal image conversion. The results are underwhelming. I am trying with a bit more variation in dataset. I hope that this will work. I will update here with the results that I get.  
  
  ---
  
  ### Answer by username_4 on 2023-02-28. Score: -1
  
  you may check SpeakingFaces paper by ISSAI. They did something similar with CycleGAN, only with faces though.
  
  > **Comments:**
  > **username_0**: ok. I will check it out  
  > **username_5**: Your answer could be improved with additional supporting information. Please [edit] to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers [in the help center](/help/how-to-answer).  
  > **username_6**: This feels like more of a comment than an answer.  
  
  ---
  
  