Answers.Body: <p>This question a bit similar to <a href="https://video.stackexchange.com/questions/22607/im-trying-to-add-guitar-tabs">I&#39;m trying to add guitar tabs to a music video, do you know any software to do it?</a></p>
  
  <p>At first, in which format your data, which protocol used from your sensors? I guess, here some closed-source software and proprietary protocol for there sensors? Obviously, you need software which can read this data format/protocol. If you creating own system with micro-controllers, try to find usable and open protocol for data. It very hard to find open protocols with needed functionality.</p>
  
  <p>At this point I guess, you found any appropriate protocol. For example, JSON with custom fields. Now try to find/write software, which will be use it.</p>
  
  <p>One possible way: write web-application, which will display video stream in your browser, loads data from sensors and display funny overlay. Then just grab video from your screen and broadcast to youtube or any other service. This is "dirty" way, but it will work.</p>
  
  <p>Other way: write custom filter to ffmpeg. You can use ass-renderer for reference how to add text on image. Just add reading data from your sensors and your task completed.</p>
  
  <hr>
  
  <h1>Writing fast and dirty video filters</h1>
  
  <p>Sometimes we need to somehow filter the video, maybe put some text or graphic on top. Write a full application that decompresses the video stream, imbues the image with new elements or changes the image itself, and then compresses the stream anew, somewhat difficult. Today we will learn how to quickly and dirty make such applications without knowing how to encode the video in general.</p>
  
  <p>To begin with, remember that the images consist of rows and columns of points, which are called pixels. Each point consists of 3 or 4 components, usually Red, Green and Blue signals, and sometimes there is an Alpha channel. Therefore, to change 1 point of the image, we need to change 3-4 components (this is usually 3-4 bytes). So, to change the entire image, we need to change all the rows and columns, in each of 3-4 elements and somehow modifying them, save back. For example, if we have a picture of 256x192 pixels in RGB24 format, then it will have 49152 pixels and 147456 color components (bytes).</p>
  
  <p>Therefore, we just need to read pieces of 147456 bytes, somehow modify them and write them back! It's all! No complicated libraries or algorithms are required! Where do we get these bytes? And we can get them with wonderful utility ffmpeg and its equally wonderful rawvideo mode, the exhaust of which we will give to stdout and receive in stdin your application!</p>
  
  <p>Let's write the simplest application that will slightly change the image:</p>
  
  <pre><code>#include &lt;stdio.h&gt;
  #include &lt;stdint.h&gt;
  #include &lt;stdlib.h&gt;
  
  int main () {
  
  // allocate variables and buffer
  
  uint8_t *pixels = malloc (147456);
  uint8_t *pix;
  int q;
  
  while (1) {
  
  // read frame
  if (fread (pixels, 1,147456, stdin) &lt;= 0) {break;}
  
  // process image
  pix = pixels;
  for (q = 0; q &lt;49152; q ++) {
  *pix++ = *pix * 2; // multiple red to 2
  *pix++ = *pix + 120; // shift green channel
  *pix++ = *pix + q / 10; // lines in blue channel
  }
  
  // write frame back
  fwrite (pixels, 1,147456, stdout);
  }
  
  return EXIT_SUCCESS;
  }
  </code></pre>
  
  <p>Yes, that's all! Of course, here all the constants are hardcoded, never do so in real projects!</p>
  
  <p>Now, run this and upload a video here:</p>
  
  <pre><code>ffmpeg -i video.mp4 -s 256x192 -f rawvideo -pix_fmt rgb24 - | ./a.out | ffmpeg -s 256x192 -f rawvideo -pix_fmt rgb24 -i - -y out.mp4
  </code></pre>
  
  <p>And we get the following result:</p>
  
  <p><a href="https://i.sstatic.net/KwRtE.gif" rel="nofollow noreferrer"><img src="https://i.sstatic.net/KwRtE.gif" alt="enter image description here"></a>
  <a href="https://i.sstatic.net/4gVsn.gif" rel="nofollow noreferrer"><img src="https://i.sstatic.net/4gVsn.gif" alt="2.gif"></a></p>
  
  <p>Let's analyze what parameters in this line participated:</p>
  
  <pre><code>ffmpeg -i video.mp4 -s 256x192 -f rawvideo -pix_fmt rgb24 -
  </code></pre>
  
  <p>Here we read from the file <code>video.mp4</code>, resize the picture to <code>256x192</code> (if your video is already 256x192 in size, then you can not do this), then we specify that it should be converted to <code>rgb24</code> color space (if it is not already in it, what for can spend quite a lot of resources), and the result we need in the form of uncompressed video <code>rawvideo</code>, the result we record in <code>-</code>, i.e. in <code>stdout</code>.</p>
  
  <p><code>| ./a.out |</code></p>
  
  <p>And this is our application. Vertical "tubes" mean that it will take data from the previous application and transfer it to the next</p>
  
  <p><code>ffmpeg -s 256x192 -f rawvideo -pix_fmt rgb24 -i - -y out.mp4</code></p>
  
  <p>Here we indicate that the input of this ffmpeg will be uncompressed video in <code>rawvideo</code> format, size of <code>256x192</code> and in the <code>rgb24</code> color space. It is also may need to specify fps, as it is lost from such conversion. We will write output to <code>out.mp4</code>, and even if such file already exists, the <code>-y</code> parameter is responsible for this</p>
  
  <p>Here is a very simple example of changing the image, but you can overlay text, other pictures and anything else. Of course, it's better to write a full-fledged application, or at least patch the already existing filter to the desired state, this will be a more correct solution. But if you need a working solution, while you do not want to write a lot of code, this is the solution for you.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-11-18T08:11:45.447Z
Answers.Text: I can use JSON or XML to pass my sensor data.. or any other format. The problem I have is only how to pass data to ffmpeg..
Answers.UserDisplayName: Ferguson
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-11-19T10:31:43.277Z
Answers.Text: have found here the example: https://s55ma.radioamater.si/2017/08/04/foscam-camera-rtsp-to-youtube-live-streaming-with-ffmpeg-drawtext-overlay/
Answers.UserDisplayName: Ferguson
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-11-19T12:09:09.530Z
Answers.Text: Here you must write your text in file every... frame! Yes, this can work, but it pretty dirty for me. I don't like such tmp files and wasting cpu. So, I suggest write a simple filter for it.
Answers.UserDisplayName: bukkojot
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-11-19T19:06:42.360Z
Answers.Text: Have tested with Java. Have made a simple Thread process which writes some values into file.. When the file is closed I execute Files.move method with StandardCopyOption.ATOMIC_MOVE.. I know it's dirty but have no much experience on how to make some plugin (If you have some good reference (link) please help me ..  The other option is to use OBS studio but I must check If can be executed from command line (terminal) (without GUI)..
Answers.UserDisplayName: Ferguson
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-11-20T01:22:42.200Z
Answers.Text: You have any experience with C or C++? You can write own filter on pipes, like `ffmpeg -i _SOURCE_ -f rawvideo -pix_fmt rgb32 - | ./your_app | ffmpeg -f rawvideo -pix_fmt rgb32 -i - rtmp://broadcast.to.server/`. This will pass frames to your app via stdin, then you can draw on it and pass bitmaps to stdout, where new instance of ffmpeg will encode it and save to file/broadcast.
Answers.UserDisplayName: bukkojot
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-11-20T07:11:59.763Z
Answers.Text: Will like to try but first need to see some example or blog about it. (Have done some project also in C/C++ (past)).. ;)
Answers.UserDisplayName: Ferguson
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-11-20T09:10:50.870Z
Answers.Text: Today-tomorrow I will write one
Answers.UserDisplayName: bukkojot
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-11-20T13:25:29.097Z
Answers.Text: Thank you!  please inform me so I can check your code.. For now the only way I have is to use FFmpeg with drawtext from file which is updated from java.. Or to use OBS studio (https://obsproject.com/download) but need to check if I can run it from terminal window (without gui)..
Answers.UserDisplayName: Ferguson
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-11-20T20:06:31.563Z
Answers.Text: I created example with C, it very simple.
Answers.UserDisplayName: bukkojot
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-11-18T02:52:47.600Z
Answers.LastActivityDate: 2017-11-21T09:24:16.817Z
Answers.LastEditDate: 2017-11-21T09:24:16.817Z
Answers.UserDisplayName: bukkojot
Body: <p>I'am using external camera with ffmpeg to acquire underwater live stream. Now I want to add some dynamic values (text overlay) over video stream and restream that to some external provider like youtube live... My values are from different sensors (temperature, oxygen, salinity etc..) and those values must be embedded into video stream.</p>
  
  <p>What is the righ or "best" or "right" way to do this?</p>
  
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-11-18T07:36:34.367Z
Comments.Text: Since this has to be done on a live stream, use multiple drawtext filters reading from individual (updated) textfiles with the filter set to reload those files.
Comments.UserDisplayName: Gyan
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-10-17T18:50:24.017Z
Comments.Text: Why doing it on the hard way 'I didn't test this method yet' but you can use `.m3u8` file contins lets say two picture that will force `ffmpeg` to continusly load these picture , so you can change/replace/update these pictures file and `ffpmeg` will load them fast
Comments.UserDisplayName: Salem F
Tags: <video><ffmpeg><streaming>
Title: FFmpeg text overlay add sensor values into video stream
UserDisplayName: Ferguson
ThreadText: # FFmpeg text overlay add sensor values into video stream
  
  (Asked by: username_0 on 2017-11-17)
  
  I'am using external camera with ffmpeg to acquire underwater live stream. Now I want to add some dynamic values (text overlay) over video stream and restream that to some external provider like youtube live... My values are from different sensors (temperature, oxygen, salinity etc..) and those values must be embedded into video stream.
  
  What is the righ or "best" or "right" way to do this?
  
  **Tags:** video, ffmpeg, streaming
  
  > **Comments:**
  > **username_1**: Since this has to be done on a live stream, use multiple drawtext filters reading from individual (updated) textfiles with the filter set to reload those files.  
  > **username_2**: Why doing it on the hard way 'I didn't test this method yet' but you can use `.m3u8` file contins lets say two picture that will force `ffmpeg` to continusly load these picture , so you can change/replace/update these pictures file and `ffpmeg` will load them fast  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_3 on 2017-11-18. Score: 1 | Accepted answer
  
  This question a bit similar to [I'm trying to add guitar tabs to a music video, do you know any software to do it?](https://video.stackexchange.com/questions/22607/im-trying-to-add-guitar-tabs)
  
  At first, in which format your data, which protocol used from your sensors? I guess, here some closed-source software and proprietary protocol for there sensors? Obviously, you need software which can read this data format/protocol. If you creating own system with micro-controllers, try to find usable and open protocol for data. It very hard to find open protocols with needed functionality.
  
  At this point I guess, you found any appropriate protocol. For example, JSON with custom fields. Now try to find/write software, which will be use it.
  
  One possible way: write web-application, which will display video stream in your browser, loads data from sensors and display funny overlay. Then just grab video from your screen and broadcast to youtube or any other service. This is "dirty" way, but it will work.
  
  Other way: write custom filter to ffmpeg. You can use ass-renderer for reference how to add text on image. Just add reading data from your sensors and your task completed.
  
  ---
  
  # Writing fast and dirty video filters
  
  Sometimes we need to somehow filter the video, maybe put some text or graphic on top. Write a full application that decompresses the video stream, imbues the image with new elements or changes the image itself, and then compresses the stream anew, somewhat difficult. Today we will learn how to quickly and dirty make such applications without knowing how to encode the video in general.
  
  To begin with, remember that the images consist of rows and columns of points, which are called pixels. Each point consists of 3 or 4 components, usually Red, Green and Blue signals, and sometimes there is an Alpha channel. Therefore, to change 1 point of the image, we need to change 3-4 components (this is usually 3-4 bytes). So, to change the entire image, we need to change all the rows and columns, in each of 3-4 elements and somehow modifying them, save back. For example, if we have a picture of 256x192 pixels in RGB24 format, then it will have 49152 pixels and 147456 color components (bytes).
  
  Therefore, we just need to read pieces of 147456 bytes, somehow modify them and write them back! It's all! No complicated libraries or algorithms are required! Where do we get these bytes? And we can get them with wonderful utility ffmpeg and its equally wonderful rawvideo mode, the exhaust of which we will give to stdout and receive in stdin your application!
  
  Let's write the simplest application that will slightly change the image:
  
  ```
  #include <stdio.h>
  #include <stdint.h>
  #include <stdlib.h>
  
  int main () {
  
  // allocate variables and buffer
  
  uint8_t *pixels = malloc (147456);
  uint8_t *pix;
  int q;
  
  while (1) {
  
  // read frame
  if (fread (pixels, 1,147456, stdin) <= 0) {break;}
  
  // process image
  pix = pixels;
  for (q = 0; q <49152; q ++) {
  *pix++ = *pix * 2; // multiple red to 2
  *pix++ = *pix + 120; // shift green channel
  *pix++ = *pix + q / 10; // lines in blue channel
  }
  
  // write frame back
  fwrite (pixels, 1,147456, stdout);
  }
  
  return EXIT_SUCCESS;
  }
  
  ```
  
  Yes, that's all! Of course, here all the constants are hardcoded, never do so in real projects!
  
  Now, run this and upload a video here:
  
  ```
  ffmpeg -i video.mp4 -s 256x192 -f rawvideo -pix_fmt rgb24 - | ./a.out | ffmpeg -s 256x192 -f rawvideo -pix_fmt rgb24 -i - -y out.mp4
  
  ```
  
  And we get the following result:
  
  [![enter image description here](https://i.sstatic.net/KwRtE.gif)](https://i.sstatic.net/KwRtE.gif)
  [![2.gif](https://i.sstatic.net/4gVsn.gif)](https://i.sstatic.net/4gVsn.gif)
  
  Let's analyze what parameters in this line participated:
  
  ```
  ffmpeg -i video.mp4 -s 256x192 -f rawvideo -pix_fmt rgb24 -
  
  ```
  
  Here we read from the file `video.mp4`, resize the picture to `256x192` (if your video is already 256x192 in size, then you can not do this), then we specify that it should be converted to `rgb24` color space (if it is not already in it, what for can spend quite a lot of resources), and the result we need in the form of uncompressed video `rawvideo`, the result we record in `-`, i.e. in `stdout`.
  
  `| ./a.out |`
  
  And this is our application. Vertical "tubes" mean that it will take data from the previous application and transfer it to the next
  
  `ffmpeg -s 256x192 -f rawvideo -pix_fmt rgb24 -i - -y out.mp4`
  
  Here we indicate that the input of this ffmpeg will be uncompressed video in `rawvideo` format, size of `256x192` and in the `rgb24` color space. It is also may need to specify fps, as it is lost from such conversion. We will write output to `out.mp4`, and even if such file already exists, the `-y` parameter is responsible for this
  
  Here is a very simple example of changing the image, but you can overlay text, other pictures and anything else. Of course, it's better to write a full-fledged application, or at least patch the already existing filter to the desired state, this will be a more correct solution. But if you need a working solution, while you do not want to write a lot of code, this is the solution for you.
  
  > **Comments:**
  > **username_0**: I can use JSON or XML to pass my sensor data.. or any other format. The problem I have is only how to pass data to ffmpeg..  
  > **username_0**: have found here the example: https://s55ma.radioamater.si/2017/08/04/foscam-camera-rtsp-to-youtube-live-streaming-with-ffmpeg-drawtext-overlay/  
  > **username_3**: Here you must write your text in file every... frame! Yes, this can work, but it pretty dirty for me. I don't like such tmp files and wasting cpu. So, I suggest write a simple filter for it.  
  > **username_0**: Have tested with Java. Have made a simple Thread process which writes some values into file.. When the file is closed I execute Files.move method with StandardCopyOption.ATOMIC_MOVE.. I know it's dirty but have no much experience on how to make some plugin (If you have some good reference (link) please help me ..  The other option is to use OBS studio but I must check If can be executed from command line (terminal) (without GUI)..  
  > **username_3**: You have any experience with C or C++? You can write own filter on pipes, like `ffmpeg -i _SOURCE_ -f rawvideo -pix_fmt rgb32 - | ./your_app | ffmpeg -f rawvideo -pix_fmt rgb32 -i - rtmp://broadcast.to.server/`. This will pass frames to your app via stdin, then you can draw on it and pass bitmaps to stdout, where new instance of ffmpeg will encode it and save to file/broadcast.  
  > **username_0**: Will like to try but first need to see some example or blog about it. (Have done some project also in C/C++ (past)).. ;)  
  > **username_3**: Today-tomorrow I will write one  
  > **username_0**: Thank you!  please inform me so I can check your code.. For now the only way I have is to use FFmpeg with drawtext from file which is updated from java.. Or to use OBS studio (https://obsproject.com/download) but need to check if I can run it from terminal window (without gui)..  
  > **username_3**: I created example with C, it very simple.  
  
  ---
  
  