Answers.Body: <p>If it is Windows then why not to use dshow instead of gdigrab?</p>
  <p>Please look at the
  <a href="https://github.com/rdp/screen-capture-recorder-to-video-windows-free" rel="nofollow noreferrer">https://github.com/rdp/screen-capture-recorder-to-video-windows-free</a></p>
  <p>If both video and audio would be via dshow then you could use ffmpeg like this:</p>
  <pre><code>ffmpeg -f dshow -i video=&quot;VIDEO_DEVICE&quot;:audio=&quot;FIRST_AUDIO_DEVICE&quot; -f dshow -i audio=&quot;ANOTHER_AUDIO_DEVICE&quot; ... 
  </code></pre>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-05-11T13:49:13.287Z
Answers.LastActivityDate: 2021-05-11T13:49:13.287Z
Answers.UserDisplayName: wwk
Answers.Body: <p>After doing what you describe with separate files for a while, I recently started using embedded streams instead. It's late and I'm tired, but you should be able to see how it works, and I can elaborate later if you need me to.</p>
  <h1>using separate output files, just so you can see how I have that set up...for reference.</h1>
  <pre><code>ffmpeg -hide_banner \
      -re \
      $videoDecodeMode \
      -f pulse \
          -i $audioOut \
      -f pulse \
          -i $audioMic \
      -f x11grab \
          -r $frameRate \
          -s $resolution \
          -i :0.0 \
          -g 60 \
      $videoEncodeMode \
      -map 2:0 \
          -metadata handler=&quot;video:1920x1080@60&quot; \
          &quot;${exportName}&quot; \
      -map 0:0 \
          -metadata title=&quot;audio:out&quot; \
          &quot;${exportName%.*}_audioOut.wav&quot; \
      -map 1:0 \
          -metadata title=&quot;audio:microphone&quot; \
          &quot;${exportName%.*}_audioMic.wav&quot; &lt;/dev/null &amp;
  </code></pre>
  <h1>using embedded streams</h1>
  <pre><code>ffmpeg -hide_banner \
      -re \
      $videoDecodeMode \
      -f pulse \
          -thread_queue_size 1024 \
          -i $audioOut \
      -f pulse \
          -thread_queue_size 1024 \
          -i $audioMic \
      -f x11grab \
          -r $frameRate \
          -s $resolution \
          -thread_queue_size 1024 \
          -i :0.0 \
      -g 60 \
      -map 2:v \
      -map 0:a \
      -map 1:a \
      -c:a aac -ac 2 -b:a 128k -r:a 48000 \
      -metadata:s:v:0 handler=&quot;video:1920x1080@60&quot; \
      -metadata:s:a:2 handler=&quot;audio:out&quot; \
      -metadata:s:a:3 handler=&quot;audio:microphone&quot; \
      $videoEncodeMode \
      &quot;${exportName}&quot;  &lt;/dev/null &amp;
  </code></pre>
  <h1>embedded streams, companded mic to clean up my line-noise problem a bit, and some stream mixing to put output and mic together on their own stream</h1>
  <pre><code>ffmpeg -hide_banner \
      -re \
      $videoDecodeMode \
      -f pulse \
          -thread_queue_size 1024 \
          -i $audioOut \
      -f pulse \
          -thread_queue_size 1024 \
          -i $audioMic \
      -f x11grab \
          -r $frameRate \
          -s $resolution \
          -thread_queue_size 1024 \
          -i :0.0 \
      -g 60 \
      -filter_complex \
          &quot;[1:a]aformat=channel_layouts=stereo,asplit=2[micOrig][micNew]\
          ;[micNew]\
              compand=0:0.2:-26/-900|-16/-16|0/-10|10/-900:6:0:0:0\
          [micCleaned]\
          ;[0:a][micCleaned]amix=inputs=2:duration=first[allAudio]&quot; \
      -map 2:v -map 0:a \
      -map [allAudio] -map [micOrig] \
      -c:a aac -ac 2 -b:a 128k -r:a 48000 \
      -metadata:s:v:0 handler=&quot;video:1920x1080@60&quot; \
      -metadata:s:a:2 handler=&quot;audio:out&quot; \
      -metadata:s:a:1 handler=&quot;audio:combined&quot; \
      -metadata:s:a:3 handler=&quot;audio:microphone&quot; \
      $videoEncodeMode \
      &quot;${exportName}&quot;  &lt;/dev/null &amp;
  </code></pre>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-08-13T12:49:22.963Z
Answers.LastActivityDate: 2020-08-13T12:58:17.000Z
Answers.LastEditDate: 2020-08-13T12:58:17.000Z
Answers.UserDisplayName: BETLOG
Body: <p>I am using FFmpeg's built-in <strong><em>gdigrab</em></strong> &amp; <strong><em>dshow</em></strong> to record the display &amp; audio of my system. I am encoding to uncompressed H.264/AVC video &amp; uncompressed PCM/WAV audio.</p>
  
  <p>I had been having two problems with audio/video synchronization:</p>
  
  <ol>
  <li>Video &amp; audio not aligned (video &amp; audio do not start at same time).</li>
  <li>Audio drift (audio progressively becomes out of sync with video).</li>
  </ol>
  
  <p>Originally I was using a single command for capture/encode. Something like the following:</p>
  
  <pre><code>ffmpeg -hide_banner -rtbufsize 1000M -f gdigrab -framerate 60 -draw_mouse 0 \
    -i "title=&lt;window_name&gt;" -f dshow -i audio="&lt;sys_audio/mic&gt;" -c:v libx264 \
    -preset ultrafast -qp 0 -x264opts keyint=1 -c:a pcm_s16le -ac 1 "&lt;out_file&gt;.mkv"
  </code></pre>
  
  <p>But I was able to solve my second problem (drifting) by capturing video &amp; audio in separate FFmpeg processes. Note that I am using a UNIX-like shell (<a href="https://www.msys2.org/" rel="nofollow noreferrer">MSYS2</a> with BASH). The following is an example of a shell script I am running:</p>
  
  <pre><code># capture audio &amp; get PID
  ffmpeg -hide_banner -rtbufsize 500M -f dshow -i audio="&lt;sys_audio/mic&gt;" \
    -c:a pcm_s16le -ac 1 "&lt;out_file&gt;-audio.wav" &amp; APID=$!
  
  # capture video
  ffmpeg -hide_banner -rtbufsize 1000M -f gdigrab -framerate 60 -draw_mouse 0 \
    -i "&lt;window_name&gt;" -c:v libx264 -preset ultrafast -qp 0 -x264opts keyint=1 \
    "&lt;out_file&gt;-video.mkv"
  
  # get exit code of video process
  VIDRET=$?
  
  # send interrupt signal to audio process after video process exits
  kill -s SIGINT ${APID}
  
  # mux video &amp; audio if video process exited okay
  if [ "${VIDRET}" -eq "0" ]; then
      ffmpeg -hide_banner -i "${OUTVID}" -i "${OUTAUD}" -map 0:0 -map 1:0 \
        -c copy "&lt;out_file&gt;.mkv"
  
      # delete temp video &amp; audio streams if muxing succeeded
      if [ "$?" -eq "0" ]; then
          rm "${OUTVID}" "${OUTAUD}"
      fi
  fi
  </code></pre>
  
  <p>So the problem left is that video &amp; audio data do not start aligned correctly. The audio is generally somewhere between 0ms-300ms ahead of the video.</p>
  
  <p>It can be easily solved by running the output file through FFmpeg again, or a separate program such as <a href="http://avidemux.sourceforge.net/" rel="nofollow noreferrer">Avidemux</a>, to adjust the audio delay. This can be done without re-encoding in Avidemux (not sure about FFmpeg).</p>
  
  <p>However, I would prefer to solve this issue within the capture process/script so I can avoid the extra step of aligning the data manually.</p>
  
  <p>Recently I have done a system cleanup by removing junk files, making sure fragmentation on my drive is low, &amp; turning off unnecessary background processes. But audio/video alignment is still often off.</p>
  
  <p>So finally, to the plain-&amp;-simple question: Is there a way to get the two FFmpeg processes to begin capture at the same time to get video &amp; audio as close to being synchronized as possible? Can I use the system clock as a seed?</p>
  
  <p>Or, is there a better method than I am using right now? E.g., a single FFmpeg process that will capture both while keeping data aligned &amp; avoiding audio drift.</p>
  
  <p>It appears to me that the problem is simply that the audio process is starting before the video process. I've heard about piping FFmpeg commands, but am not sure of the correct way to do it. I have found some information on piping that I am trying to figure out:</p>
  
  <ul>
  <li><a href="https://trac.ffmpeg.org/wiki/Creating%20multiple%20outputs#Pipedprocesses" rel="nofollow noreferrer">https://trac.ffmpeg.org/wiki/Creating%20multiple%20outputs#Pipedprocesses</a></li>
  <li><a href="https://superuser.com/questions/1057388/pipe-ffmpeg-output-to-other-process">https://superuser.com/questions/1057388/pipe-ffmpeg-output-to-other-process</a></li>
  </ul>
  
  <p>FFmpeg information: Official static 64-bit build from <a href="https://ffmpeg.zeranoe.com/builds/" rel="nofollow noreferrer">Zeranoe</a>.</p>
  
  <pre><code>ffmpeg version N-92511-g0279cb4f69 Copyright (c) 2000-2018 the FFmpeg developers
      built with gcc 8.2.1 (GCC) 20181017
      configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-fontconfig
          --enable-gnutls --enable-iconv --enable-libass --enable-libbluray
          --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb
          --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-libshine
          --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame
          --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264
          --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib
          --enable-gmp --enable-libvidstab --enable-libvorbis --enable-libvo-amrwbenc
          --enable-libmysofa --enable-libspeex --enable-libxvid --enable-libaom
          --enable-libmfx --enable-amf --enable-ffnvcodec --enable-cuvid --enable-d3d11va
          --enable-nvenc --enable-nvdec --enable-dxva2 --enable-avisynth
  </code></pre>
  
  <p>These are my system specs:</p>
  
  <blockquote>
    <ul>
    <li><strong>Model:</strong> <a href="https://support.hp.com/us-en/document/c05381021" rel="nofollow noreferrer">HP Pavilion 24-b217c</a></li>
    <li><strong>OS:</strong> Windows 10 64-bit</li>
    <li><strong>CPU:</strong> <a href="https://ark.intel.com/products/97184" rel="nofollow noreferrer">Intel Core i5-7400T @ 2.40GHz, 2401 MHz, 4 cores</a></li>
    <li><strong>RAM:</strong> 12GB</li>
    <li><strong>GPU(s):</strong>
    
    <ul>
    <li><a href="https://www.geforce.com/hardware/notebook-gpus/geforce-930mx/specifications" rel="nofollow noreferrer">nVidia GeForce 930MX</a>
    
    <ul>
    <li><strong>RAM:</strong> 2GB dedicated</li>
    </ul></li>
    <li><a href="https://www.intel.com/content/www/us/en/support/products/98909/graphics-drivers/graphics-for-7th-generation-intel-processors/intel-hd-graphics-630.html" rel="nofollow noreferrer">Intel HD Graphics 630</a></li>
    </ul></li>
    </ul>
  </blockquote>
  
  <p><strong>-- Edit --</strong></p>
  
  <p>I did figure out a pipe command, but the result is the same. Audio is slightly ahead:</p>
  
  <pre><code>ffmpeg -hide_banner -rtbufsize 500M -f dshow -ac 1 -i audio="&lt;sys_audio/mic&gt;" \
    -c:a pcm_s16le -f s16le pipe: | ffmpeg -y -hide_banner -rtbufsize 1500M -f gdigrab \
    -thread_queue_size 128 -framerate 60 -draw_mouse 0 -i title="&lt;window_name&gt;" -f s16le \
    -thread_queue_size 128 -i pipe: -map 0:0 -map 1:0 -c:v libx264 -preset ultrafast -qp 0 \
    -x264opts keyint=1 -c:a copy "&lt;out_file&gt;.mkv"
  </code></pre>
  
Tags: <video><ffmpeg><audio><sync>
Title: FFmpeg: Align Video & Audio of Desktop Capture
UserDisplayName: AntumDeluge
ThreadText: # FFmpeg: Align Video & Audio of Desktop Capture
  
  (Asked by: username_0 on 2019-02-06)
  
  I am using FFmpeg's built-in ***gdigrab*** & ***dshow*** to record the display & audio of my system. I am encoding to uncompressed H.264/AVC video & uncompressed PCM/WAV audio.
  
  I had been having two problems with audio/video synchronization:
  
  1. Video & audio not aligned (video & audio do not start at same time).
  2. Audio drift (audio progressively becomes out of sync with video).
  
  Originally I was using a single command for capture/encode. Something like the following:
  
  ```
  ffmpeg -hide_banner -rtbufsize 1000M -f gdigrab -framerate 60 -draw_mouse 0 \
    -i "title=<window_name>" -f dshow -i audio="<sys_audio/mic>" -c:v libx264 \
    -preset ultrafast -qp 0 -x264opts keyint=1 -c:a pcm_s16le -ac 1 "<out_file>.mkv"
  
  ```
  
  But I was able to solve my second problem (drifting) by capturing video & audio in separate FFmpeg processes. Note that I am using a UNIX-like shell ([MSYS2](https://www.msys2.org/) with BASH). The following is an example of a shell script I am running:
  
  ```
  # capture audio & get PID
  ffmpeg -hide_banner -rtbufsize 500M -f dshow -i audio="<sys_audio/mic>" \
    -c:a pcm_s16le -ac 1 "<out_file>-audio.wav" & APID=$!
  
  # capture video
  ffmpeg -hide_banner -rtbufsize 1000M -f gdigrab -framerate 60 -draw_mouse 0 \
    -i "<window_name>" -c:v libx264 -preset ultrafast -qp 0 -x264opts keyint=1 \
    "<out_file>-video.mkv"
  
  # get exit code of video process
  VIDRET=$?
  
  # send interrupt signal to audio process after video process exits
  kill -s SIGINT ${APID}
  
  # mux video & audio if video process exited okay
  if [ "${VIDRET}" -eq "0" ]; then
      ffmpeg -hide_banner -i "${OUTVID}" -i "${OUTAUD}" -map 0:0 -map 1:0 \
        -c copy "<out_file>.mkv"
  
      # delete temp video & audio streams if muxing succeeded
      if [ "$?" -eq "0" ]; then
          rm "${OUTVID}" "${OUTAUD}"
      fi
  fi
  
  ```
  
  So the problem left is that video & audio data do not start aligned correctly. The audio is generally somewhere between 0ms-300ms ahead of the video.
  
  It can be easily solved by running the output file through FFmpeg again, or a separate program such as [Avidemux](http://avidemux.sourceforge.net/), to adjust the audio delay. This can be done without re-encoding in Avidemux (not sure about FFmpeg).
  
  However, I would prefer to solve this issue within the capture process/script so I can avoid the extra step of aligning the data manually.
  
  Recently I have done a system cleanup by removing junk files, making sure fragmentation on my drive is low, & turning off unnecessary background processes. But audio/video alignment is still often off.
  
  So finally, to the plain-&-simple question: Is there a way to get the two FFmpeg processes to begin capture at the same time to get video & audio as close to being synchronized as possible? Can I use the system clock as a seed?
  
  Or, is there a better method than I am using right now? E.g., a single FFmpeg process that will capture both while keeping data aligned & avoiding audio drift.
  
  It appears to me that the problem is simply that the audio process is starting before the video process. I've heard about piping FFmpeg commands, but am not sure of the correct way to do it. I have found some information on piping that I am trying to figure out:
  
  * <https://trac.ffmpeg.org/wiki/Creating%20multiple%20outputs#Pipedprocesses>
  * <https://superuser.com/questions/1057388/pipe-ffmpeg-output-to-other-process>
  
  FFmpeg information: Official static 64-bit build from [Zeranoe](https://ffmpeg.zeranoe.com/builds/).
  
  ```
  ffmpeg version N-92511-g0279cb4f69 Copyright (c) 2000-2018 the FFmpeg developers
      built with gcc 8.2.1 (GCC) 20181017
      configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-fontconfig
          --enable-gnutls --enable-iconv --enable-libass --enable-libbluray
          --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb
          --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-libshine
          --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame
          --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264
          --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib
          --enable-gmp --enable-libvidstab --enable-libvorbis --enable-libvo-amrwbenc
          --enable-libmysofa --enable-libspeex --enable-libxvid --enable-libaom
          --enable-libmfx --enable-amf --enable-ffnvcodec --enable-cuvid --enable-d3d11va
          --enable-nvenc --enable-nvdec --enable-dxva2 --enable-avisynth
  
  ```
  
  These are my system specs:
  
  > * **Model:** [HP Pavilion 24-b217c](https://support.hp.com/us-en/document/c05381021)
  > * **OS:** Windows 10 64-bit
  > * **CPU:** [Intel Core i5-7400T @ 2.40GHz, 2401 MHz, 4 cores](https://ark.intel.com/products/97184)
  > * **RAM:** 12GB
  > * **GPU(s):**
  >   + [nVidia GeForce 930MX](https://www.geforce.com/hardware/notebook-gpus/geforce-930mx/specifications)
  >     - **RAM:** 2GB dedicated
  >   + [Intel HD Graphics 630](https://www.intel.com/content/www/us/en/support/products/98909/graphics-drivers/graphics-for-7th-generation-intel-processors/intel-hd-graphics-630.html)
  
  **-- Edit --**
  
  I did figure out a pipe command, but the result is the same. Audio is slightly ahead:
  
  ```
  ffmpeg -hide_banner -rtbufsize 500M -f dshow -ac 1 -i audio="<sys_audio/mic>" \
    -c:a pcm_s16le -f s16le pipe: | ffmpeg -y -hide_banner -rtbufsize 1500M -f gdigrab \
    -thread_queue_size 128 -framerate 60 -draw_mouse 0 -i title="<window_name>" -f s16le \
    -thread_queue_size 128 -i pipe: -map 0:0 -map 1:0 -c:v libx264 -preset ultrafast -qp 0 \
    -x264opts keyint=1 -c:a copy "<out_file>.mkv"
  
  ```
  
  **Tags:** video, ffmpeg, audio, sync
  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_1 on 2021-05-11. Score: 0
  
  If it is Windows then why not to use dshow instead of gdigrab?
  
  Please look at the
  <https://github.com/rdp/screen-capture-recorder-to-video-windows-free>
  
  If both video and audio would be via dshow then you could use ffmpeg like this:
  
  ```
  ffmpeg -f dshow -i video="VIDEO_DEVICE":audio="FIRST_AUDIO_DEVICE" -f dshow -i audio="ANOTHER_AUDIO_DEVICE" ... 
  
  ```
  
  
  ---
  
  ### Answer by username_2 on 2020-08-13. Score: 0
  
  After doing what you describe with separate files for a while, I recently started using embedded streams instead. It's late and I'm tired, but you should be able to see how it works, and I can elaborate later if you need me to.
  
  # using separate output files, just so you can see how I have that set up...for reference.
  
  ```
  ffmpeg -hide_banner \
      -re \
      $videoDecodeMode \
      -f pulse \
          -i $audioOut \
      -f pulse \
          -i $audioMic \
      -f x11grab \
          -r $frameRate \
          -s $resolution \
          -i :0.0 \
          -g 60 \
      $videoEncodeMode \
      -map 2:0 \
          -metadata handler="video:1920x1080@60" \
          "${exportName}" \
      -map 0:0 \
          -metadata title="audio:out" \
          "${exportName%.*}_audioOut.wav" \
      -map 1:0 \
          -metadata title="audio:microphone" \
          "${exportName%.*}_audioMic.wav" </dev/null &
  
  ```
  
  # using embedded streams
  
  ```
  ffmpeg -hide_banner \
      -re \
      $videoDecodeMode \
      -f pulse \
          -thread_queue_size 1024 \
          -i $audioOut \
      -f pulse \
          -thread_queue_size 1024 \
          -i $audioMic \
      -f x11grab \
          -r $frameRate \
          -s $resolution \
          -thread_queue_size 1024 \
          -i :0.0 \
      -g 60 \
      -map 2:v \
      -map 0:a \
      -map 1:a \
      -c:a aac -ac 2 -b:a 128k -r:a 48000 \
      -metadata:s:v:0 handler="video:1920x1080@60" \
      -metadata:s:a:2 handler="audio:out" \
      -metadata:s:a:3 handler="audio:microphone" \
      $videoEncodeMode \
      "${exportName}"  </dev/null &
  
  ```
  
  # embedded streams, companded mic to clean up my line-noise problem a bit, and some stream mixing to put output and mic together on their own stream
  
  ```
  ffmpeg -hide_banner \
      -re \
      $videoDecodeMode \
      -f pulse \
          -thread_queue_size 1024 \
          -i $audioOut \
      -f pulse \
          -thread_queue_size 1024 \
          -i $audioMic \
      -f x11grab \
          -r $frameRate \
          -s $resolution \
          -thread_queue_size 1024 \
          -i :0.0 \
      -g 60 \
      -filter_complex \
          "[1:a]aformat=channel_layouts=stereo,asplit=2[micOrig][micNew]\
          ;[micNew]\
              compand=0:0.2:-26/-900|-16/-16|0/-10|10/-900:6:0:0:0\
          [micCleaned]\
          ;[0:a][micCleaned]amix=inputs=2:duration=first[allAudio]" \
      -map 2:v -map 0:a \
      -map [allAudio] -map [micOrig] \
      -c:a aac -ac 2 -b:a 128k -r:a 48000 \
      -metadata:s:v:0 handler="video:1920x1080@60" \
      -metadata:s:a:2 handler="audio:out" \
      -metadata:s:a:1 handler="audio:combined" \
      -metadata:s:a:3 handler="audio:microphone" \
      $videoEncodeMode \
      "${exportName}"  </dev/null &
  
  ```
  
  
  ---
  
  