Answers.Body: <p><strong>nadermx/backgroundremover</strong></p>
  <p><a href="https://github.com/nadermx/backgroundremover" rel="nofollow noreferrer">https://github.com/nadermx/backgroundremover</a></p>
  <p>This is now the top Google hit, I installed version 0.2.9 successfully on Ubuntu 24.10 with:</p>
  <pre><code>pipx install backgroundremover
  </code></pre>
  <p>Unfortunately, there's a bug mentioned at <a href="https://github.com/nadermx/backgroundremover/issues/116" rel="nofollow noreferrer">https://github.com/nadermx/backgroundremover/issues/116</a> which prevents the most common use cases from working (mp4 -&gt; mp4 with background replaced by image or another video), but the author replied there and might fix it.</p>
  <p>The following did work <a href="https://github.com/cirosantilli/media/raw/master/Ciro_Santilli_selfie_in_cycling_kit_with_old_cottages.mp4" rel="nofollow noreferrer">with this test video (~1 MB, 5.7 s, 1024x768, 162 frames)</a>:</p>
  <pre><code>time backgroundremover -i in.mp4 -tv -o out.mov
  </code></pre>
  <p>which you can download with:</p>
  <pre><code>wget -O in.mp4 https://github.com/cirosantilli/media/raw/master/Ciro_Santilli_selfie_in_cycling_kit_with_old_cottages.mp4
  </code></pre>
  <p>but it is not very useful (as a final use case) as it produces a huge .mov file (238 MB for an 1 MB input file! 2m25s conversion time on Lenovo ThinkPad P14s amd), and you can't output formats like .mp4 directly like that because .mp4 does not support transparency unlike .mov: <a href="https://github.com/nadermx/backgroundremover/issues/92" rel="nofollow noreferrer">https://github.com/nadermx/backgroundremover/issues/92</a> and I don't know how to use that .mov later on to e.g. replace transparency by a fixed color, though I'm sure there's a way as usual with ffmpeg.</p>
  <p>Another thing that worked was their GIF output:</p>
  <pre><code>backgroundremover -i in.mp4 -tg o out.gif
  </code></pre>
  <p>Quality wise, there's are a few glaring artifacts on my simple test case, so it's not perfect. But new better models can always come along later on, currently you can choose between <code>u2net</code>, <code>u2netp</code>, <code>u2net_human_seg</code>. So we understand that this is the base model currently supported: <a href="https://github.com/xuebinqin/U-2-Net" rel="nofollow noreferrer">https://github.com/xuebinqin/U-2-Net</a> , and the exact task that it attempts to solve is &quot;Salient Object Detection&quot;, because e.g. it attempts to select &quot;objects held by people&quot; as part of the foreground just like the person.</p>
  <p><a href="https://i.sstatic.net/etQXW.gif" rel="nofollow noreferrer"><img src="https://i.sstatic.net/etQXW.gif" alt="enter image description here" /></a></p>
  <p><a href="https://i.sstatic.net/h0pvW.gif" rel="nofollow noreferrer"><img src="https://i.sstatic.net/h0pvW.gif" alt="enter image description here" /></a></p>
  <p>Related:</p>
  <ul>
  <li><a href="https://askubuntu.com/questions/651309/software-to-automatically-remove-background-from-images">https://askubuntu.com/questions/651309/software-to-automatically-remove-background-from-images</a></li>
  <li><a href="https://www.reddit.com/r/linux/comments/oso31y/remove_background_from_video_and_images_with_a/" rel="nofollow noreferrer">https://www.reddit.com/r/linux/comments/oso31y/remove_background_from_video_and_images_with_a/</a></li>
  <li><a href="https://stackoverflow.com/questions/55916977/remove-random-background-from-video-using-ffmpeg-or-python">https://stackoverflow.com/questions/55916977/remove-random-background-from-video-using-ffmpeg-or-python</a></li>
  </ul>
  <p>Related: <a href="https://stackoverflow.com/questions/41923492/cant-import-moviepy-editor">https://stackoverflow.com/questions/41923492/cant-import-moviepy-editor</a></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-05-14T07:43:01.163Z
Answers.Text: I find it hard to believe that every video conferencing app I have used (Teams, Meet, etc.) all seem to have this functionality which works very well even with a moving background, and yet this is the best that is available?
Answers.UserDisplayName: Michael
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-05-14T07:46:40.973Z
Answers.Text: @Michael these things are hard to benchmark, I wonder how the proprietary models would perform for my input. My input was also particularly challenging compared to a typical user sitting down facing forward on conference. Do ping me if you find a better one.
Answers.UserDisplayName: Ciro Santilli OurBigBook.com
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-05-14T08:55:22.467Z
Answers.Text: I haven't been able to confirm this, but I suspect that some webcam hardware such as on Macbooks and iPads may have some kind of IR depth sensors, which would greatly reduce the complexity of filtering out the background.  However, I haven't been able to find how one could access this information from something like ffmpeg to do this filtering oneself.
Answers.UserDisplayName: Michael
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-05-14T09:23:34.040Z
Answers.Text: @Michael OK, could be. Some of them have IR for sure, here I show a ffmpeg example: https://askubuntu.com/questions/348838/how-to-check-available-webcams-from-the-command-line/848390#848390
Answers.UserDisplayName: Ciro Santilli OurBigBook.com
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-05-14T10:46:34.157Z
Answers.Text: Thanks!  Sadly, it does not appear that IR/depth information on MacOS (e.g. from TrueDepth cameras) is available using ffmpeg...
Answers.UserDisplayName: Michael
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-04-20T22:42:50.143Z
Answers.LastActivityDate: 2024-12-28T17:03:46.997Z
Answers.LastEditDate: 2024-12-28T17:03:46.997Z
Answers.UserDisplayName: Ciro Santilli OurBigBook.com
Answers.Body: <p>There is a reason why monotone background are used, they allow one to determine where the object is, and where is the background. As far as i know no took exists, which can always determine what is the background, and what is the object that is actually being filmed.</p>
  <p>If the background is really static (not moving in the frame, rather than simply always being behind the person), then you could attempt to extract frames and determine the pixels that keep their value above some arbitrary percentage of frames. You then can create a mask based on if the pixels of the specific frame are the same as the value of the 'stable pixels', and blur in that mask.</p>
  <p>The the background is moving in the frame due to camera movement or changes in some manner, this becomes very difficult, and i may even guess impossible in some situations. Also even if it will work, it will need to be double checked to see that you won't get false positives. You will definitely get false negatives near the person's head.</p>
  <p>In the even you have resources for development and really need it to be done, you could go neural network route. Be prepared to provide neural net lots and lots of training data. For that you would need to film akin to the videos that you will be processing, but doing that in front of the green screen. This will allow you to determine where the head truly is in each frame, because once you put in the background it will be the job of the neural net to figure out where the head is, and your job to tell it whether it is correct or not during the training, and to be able to do that in testing to see how well it has learnt to identify the human head.</p>
  <p>Good luck.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-03-18T11:38:04.813Z
Answers.Text: But then, how do Zoom, Teams etc do to separate you from the background in a live call?  I would like to use that same type of software and apply to some training videos we have where the background is distracting
Answers.UserDisplayName: kxtronic
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-12-11T06:13:14.017Z
Answers.LastActivityDate: 2021-12-11T06:13:14.017Z
Answers.UserDisplayName: v010dya
Body: <p>I'm looking for an ffmpeg-like tool to blur/remove/change a static background from a &quot;talking head&quot; video, on the command line with on-device processing.</p>
  <p>Essentially,</p>
  <pre><code>ffmpeg-like-tool [--blur-background | --remove-background | --change-background] --background image.jpg --in input.mp4 --out output.mp4
  </code></pre>
  <p>Does such a tool exist yet?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-05-03T15:01:27.293Z
Comments.Text: Related: https://stackoverflow.com/questions/55916977/remove-random-background-from-video-using-ffmpeg-or-python
Comments.UserDisplayName: Hermann
Tags: <ffmpeg><software-recommendation><greenscreen><command-line>
Title: How to remove background from video without green screen on the command line?
UserDisplayName: Marek Kowalczyk
ThreadText: # How to remove background from video without green screen on the command line?
  
  (Asked by: username_0 on 2021-03-27)
  
  I'm looking for an ffmpeg-like tool to blur/remove/change a static background from a "talking head" video, on the command line with on-device processing.
  
  Essentially,
  
  ```
  ffmpeg-like-tool [--blur-background | --remove-background | --change-background] --background image.jpg --in input.mp4 --out output.mp4
  
  ```
  
  Does such a tool exist yet?
  
  **Tags:** ffmpeg, software-recommendation, greenscreen, command-line
  
  > **Comments:**
  > **username_1**: Related: https://stackoverflow.com/questions/55916977/remove-random-background-from-video-using-ffmpeg-or-python  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_2 on 2024-04-20. Score: 1
  
  **nadermx/backgroundremover**
  
  <https://github.com/nadermx/backgroundremover>
  
  This is now the top Google hit, I installed version 0.2.9 successfully on Ubuntu 24.10 with:
  
  ```
  pipx install backgroundremover
  
  ```
  
  Unfortunately, there's a bug mentioned at <https://github.com/nadermx/backgroundremover/issues/116> which prevents the most common use cases from working (mp4 -> mp4 with background replaced by image or another video), but the author replied there and might fix it.
  
  The following did work [with this test video (~1 MB, 5.7 s, 1024x768, 162 frames)](https://github.com/cirosantilli/media/raw/master/Ciro_Santilli_selfie_in_cycling_kit_with_old_cottages.mp4):
  
  ```
  time backgroundremover -i in.mp4 -tv -o out.mov
  
  ```
  
  which you can download with:
  
  ```
  wget -O in.mp4 https://github.com/cirosantilli/media/raw/master/Ciro_Santilli_selfie_in_cycling_kit_with_old_cottages.mp4
  
  ```
  
  but it is not very useful (as a final use case) as it produces a huge .mov file (238 MB for an 1 MB input file! 2m25s conversion time on Lenovo ThinkPad P14s amd), and you can't output formats like .mp4 directly like that because .mp4 does not support transparency unlike .mov: <https://github.com/nadermx/backgroundremover/issues/92> and I don't know how to use that .mov later on to e.g. replace transparency by a fixed color, though I'm sure there's a way as usual with ffmpeg.
  
  Another thing that worked was their GIF output:
  
  ```
  backgroundremover -i in.mp4 -tg o out.gif
  
  ```
  
  Quality wise, there's are a few glaring artifacts on my simple test case, so it's not perfect. But new better models can always come along later on, currently you can choose between `u2net`, `u2netp`, `u2net_human_seg`. So we understand that this is the base model currently supported: <https://github.com/xuebinqin/U-2-Net> , and the exact task that it attempts to solve is "Salient Object Detection", because e.g. it attempts to select "objects held by people" as part of the foreground just like the person.
  
  [![enter image description here](https://i.sstatic.net/etQXW.gif)](https://i.sstatic.net/etQXW.gif)
  
  [![enter image description here](https://i.sstatic.net/h0pvW.gif)](https://i.sstatic.net/h0pvW.gif)
  
  Related:
  
  * <https://askubuntu.com/questions/651309/software-to-automatically-remove-background-from-images>
  * <https://www.reddit.com/r/linux/comments/oso31y/remove_background_from_video_and_images_with_a/>
  * <https://stackoverflow.com/questions/55916977/remove-random-background-from-video-using-ffmpeg-or-python>
  
  Related: <https://stackoverflow.com/questions/41923492/cant-import-moviepy-editor>
  
  > **Comments:**
  > **username_3**: I find it hard to believe that every video conferencing app I have used (Teams, Meet, etc.) all seem to have this functionality which works very well even with a moving background, and yet this is the best that is available?  
  > **username_2**: @username_3 these things are hard to benchmark, I wonder how the proprietary models would perform for my input. My input was also particularly challenging compared to a typical user sitting down facing forward on conference. Do ping me if you find a better one.  
  > **username_3**: I haven't been able to confirm this, but I suspect that some webcam hardware such as on Macbooks and iPads may have some kind of IR depth sensors, which would greatly reduce the complexity of filtering out the background.  However, I haven't been able to find how one could access this information from something like ffmpeg to do this filtering oneself.  
  > **username_2**: @username_3 OK, could be. Some of them have IR for sure, here I show a ffmpeg example: https://askubuntu.com/questions/348838/how-to-check-available-webcams-from-the-command-line/848390#848390  
  > **username_3**: Thanks!  Sadly, it does not appear that IR/depth information on MacOS (e.g. from TrueDepth cameras) is available using ffmpeg...  
  
  ---
  
  ### Answer by username_4 on 2021-12-11. Score: 0
  
  There is a reason why monotone background are used, they allow one to determine where the object is, and where is the background. As far as i know no took exists, which can always determine what is the background, and what is the object that is actually being filmed.
  
  If the background is really static (not moving in the frame, rather than simply always being behind the person), then you could attempt to extract frames and determine the pixels that keep their value above some arbitrary percentage of frames. You then can create a mask based on if the pixels of the specific frame are the same as the value of the 'stable pixels', and blur in that mask.
  
  The the background is moving in the frame due to camera movement or changes in some manner, this becomes very difficult, and i may even guess impossible in some situations. Also even if it will work, it will need to be double checked to see that you won't get false positives. You will definitely get false negatives near the person's head.
  
  In the even you have resources for development and really need it to be done, you could go neural network route. Be prepared to provide neural net lots and lots of training data. For that you would need to film akin to the videos that you will be processing, but doing that in front of the green screen. This will allow you to determine where the head truly is in each frame, because once you put in the background it will be the job of the neural net to figure out where the head is, and your job to tell it whether it is correct or not during the training, and to be able to do that in testing to see how well it has learnt to identify the human head.
  
  Good luck.
  
  > **Comments:**
  > **username_5**: But then, how do Zoom, Teams etc do to separate you from the background in a live call?  I would like to use that same type of software and apply to some training videos we have where the background is distracting  
  
  ---
  
  