Answers.Body: <p>If I understand you correctly, then most of the restoration goals will be impossible to be reached. While my knowledge of existing videos tools is not extensive, I am still pretty sure such a software does not exist. As a software developer I have a good grasp of what is impossible and what is possible and what effort you need to do that.</p>
  <p>When details are lost, they are lost. How should an AI restore these? Well, any AI has to be trained first and this needs to be done with pairs of a master video in highest quality and its heavily-compressed export in the target codec. The problem is that the export video has less information than the master and there is no way to define a sensible bidirectionaly mapping. Imagine the master contains a scene with a machine with little screws. Another scene shows a machine with rivets. But after a heavily compression you cannot see a difference between screws and rivets in the export video. Due to loss of detail, both screws and rivets now look like grey smudges. How should you decide what is correct if you only have the compressed video? You can only make a guess. And that's what the AI would do. This means you get a restored video that often shows quite different details than the original video, however now and then the AI will make correct guesses too. This is of course theoretical, I highly doubt there is an AI based software for this job as it would be extremely expensive in development and with only a few potentional customers the price tag would be enormous.</p>
  <p>Discolouration is also hard to solve because often there's no easy way to differentiate between intended de-saturated areas and compression artifacts.</p>
  <p>Macro-blocking can be fixed in theory by applying a blur effect. But you have to sacrifice sharpness. Fine artifical noise can be added to gain a bit of sharpness again. Anyway, this can be done and such a function is probably available in some tools.</p>
  <p>Colour bending can be solved in theory without too much effort but I don't know if anybody does offer such a thing.</p>
  <p><strong>Update</strong>: <em>As has been pointed out in the comments, DaVinci Resolve can do de-bending. Both Premiere and Resolve use some AI based tools to restore a video to fix dust, scratches, de-interlace. But if I infer correctly this does not help much with your problem of restoring a heavily compressed video.</em></p>
  <p>All in all, you cannot undo the artifacts of heavy compression.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-08-17T00:36:01.837Z
Answers.Text: That's why the ai/machine-learning tech is what I'm looking for.  As I understand it, they can be trained by providing a highly compressed video, and the original HD version, which means the results would be much more accurate than a simple algorithm, depending on how many examples its been trained with.
  
  This kind of thing exists for JPEG artifacts on images, and it works pretty well.  Of course you can't expect it to be 100% accurate (what it would have looked like naturally) but something like that would be quite helpful when complete accuracy isn't necessary.  Thanks for the info, though
Answers.UserDisplayName: Rylan
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-01-09T06:14:30.063Z
Answers.Text: This answer is simply incorrect.  DaVinci Resolve and Premiere both already use machine learning approaches for image restoration.  They both use it for object removal.  Resolve uses it to fix dust and scratches.  Resolve has a de-banding and de-flickering algorithm, although I don't think that one uses machine learning, in particular.  Resolve also uses ML to de-interlace.  The tools are out there, and they'll only get better with time, they're just not as automatic as some would expect, and still require guidance from the user.
Answers.UserDisplayName: Jason Conrad
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-01-11T21:09:20.497Z
Answers.Text: @Jason Conrad You have a point and I'll update my answer a bit. But I'm not entirely convinced. Restoring a heavy compressed video such that it looks almost the same as the master is an incredible complex task. If it can be done, please write an answer where you explain how you do it. However, if it's impossible now but might be in the far future, this does not help the OP much. From what I know, people working on AI keep promising too much (and have done so for decades) and generally fall far behind their promises. So I'm not as optimistic as you.
Answers.UserDisplayName: Matt
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-02-08T04:24:50.747Z
Answers.Text: https://idle.slashdot.org/story/21/02/07/0626249/can-artificial-intelligence-restore-85-year-old-popeye-cartoons
Answers.UserDisplayName: Jason Conrad
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-02-08T07:58:34.227Z
Answers.Text: Thanks for the link. Looking of one of the restored cartoon I clearly see the improvement and even have an idea how they have done it in general. However, the difficult challenge wasn't tackled: there is still a lot of dirt/dust visible. The images are shaky, but I guess this can be mitigated with already existing stabilizing effects. Still wondering why they haven't done it. Nevertheless, it's a progress. But I'm sceptical if this works on non-cartoon video with digital artefacts resulting from high compression.
Answers.UserDisplayName: Matt
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-05-21T22:57:09.333Z
Answers.Text: Something many don't realize is all the state of the art machine learning models related to this are released for free. You just have to be pretty tech savvy to use them. AI can do all of those things and it's not expensive at all to train a model that does this. The only point I agree with is if .your compression is so high that even your imagination can't undo it, then an AI model won't correctly do it either. Just depends on how bad it is
Answers.UserDisplayName: Frobot
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-08-16T10:57:45.827Z
Answers.LastActivityDate: 2021-01-11T21:15:24.720Z
Answers.LastEditDate: 2021-01-11T21:15:24.720Z
Answers.UserDisplayName: Matt
Body: <p>I have been researching this for the past couple of days, and all I could find on the subject were a bunch of research papers, a demonstration video of a model that isn't publicly available, and a software that you have to pay hundreds for to make it useful, on a poorly designed and shady looking website.  Everything else is for JPEG artifacts, de-noising (a completely separate issue), and video up-scaling.  Either that, or it's just questions/answers on how to solve the issue from the source (camera, recording software, or editing software) rather than how to fix existing footage.</p>
  <p>I am looking for a software, preferably AI based, that can remove and repair highly compressed videos their compression artifacts. (Discoloration, macro-blocking, banding, loss of detail, etc...) without a serious loss of existing detail or flickering between frames (Caused by frame-by-frame editing).</p>
  <p>Does this actually exist?  It really seems like it doesn't, and relevant information is hard to find.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-01-05T19:21:44.007Z
Comments.Text: Pherhaps this could help? I Haven't tested it by myselfe. https://cinematography.com/index.php?/topic/73521-video-artifact-editor-to-remove-compression-artifacts-and-noise/
Comments.UserDisplayName: Guest
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-01-31T05:22:08.667Z
Comments.Text: topaz labs has AI upscaling that is very good at removing artifacts and sharpening and restoring detail
Comments.UserDisplayName: ExposureForDinner
Tags: <video><mp4><h.264><compression>
Title: Video artifact removal and restoration - How can it be done? (Really?)
UserDisplayName: Rylan
ThreadText: # Video artifact removal and restoration - How can it be done? (Really?)
  
  (Asked by: username_0 on 2020-08-16)
  
  I have been researching this for the past couple of days, and all I could find on the subject were a bunch of research papers, a demonstration video of a model that isn't publicly available, and a software that you have to pay hundreds for to make it useful, on a poorly designed and shady looking website. Everything else is for JPEG artifacts, de-noising (a completely separate issue), and video up-scaling. Either that, or it's just questions/answers on how to solve the issue from the source (camera, recording software, or editing software) rather than how to fix existing footage.
  
  I am looking for a software, preferably AI based, that can remove and repair highly compressed videos their compression artifacts. (Discoloration, macro-blocking, banding, loss of detail, etc...) without a serious loss of existing detail or flickering between frames (Caused by frame-by-frame editing).
  
  Does this actually exist? It really seems like it doesn't, and relevant information is hard to find.
  
  **Tags:** video, mp4, h.264, compression
  
  > **Comments:**
  > **username_1**: Pherhaps this could help? I Haven't tested it by myselfe. https://cinematography.com/index.php?/topic/73521-video-artifact-editor-to-remove-compression-artifacts-and-noise/  
  > **username_2**: topaz labs has AI upscaling that is very good at removing artifacts and sharpening and restoring detail  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_3 on 2020-08-16. Score: 3 | Accepted answer
  
  If I understand you correctly, then most of the restoration goals will be impossible to be reached. While my knowledge of existing videos tools is not extensive, I am still pretty sure such a software does not exist. As a software developer I have a good grasp of what is impossible and what is possible and what effort you need to do that.
  
  When details are lost, they are lost. How should an AI restore these? Well, any AI has to be trained first and this needs to be done with pairs of a master video in highest quality and its heavily-compressed export in the target codec. The problem is that the export video has less information than the master and there is no way to define a sensible bidirectionaly mapping. Imagine the master contains a scene with a machine with little screws. Another scene shows a machine with rivets. But after a heavily compression you cannot see a difference between screws and rivets in the export video. Due to loss of detail, both screws and rivets now look like grey smudges. How should you decide what is correct if you only have the compressed video? You can only make a guess. And that's what the AI would do. This means you get a restored video that often shows quite different details than the original video, however now and then the AI will make correct guesses too. This is of course theoretical, I highly doubt there is an AI based software for this job as it would be extremely expensive in development and with only a few potentional customers the price tag would be enormous.
  
  Discolouration is also hard to solve because often there's no easy way to differentiate between intended de-saturated areas and compression artifacts.
  
  Macro-blocking can be fixed in theory by applying a blur effect. But you have to sacrifice sharpness. Fine artifical noise can be added to gain a bit of sharpness again. Anyway, this can be done and such a function is probably available in some tools.
  
  Colour bending can be solved in theory without too much effort but I don't know if anybody does offer such a thing.
  
  **Update**: *As has been pointed out in the comments, DaVinci Resolve can do de-bending. Both Premiere and Resolve use some AI based tools to restore a video to fix dust, scratches, de-interlace. But if I infer correctly this does not help much with your problem of restoring a heavily compressed video.*
  
  All in all, you cannot undo the artifacts of heavy compression.
  
  > **Comments:**
  > **username_0**: That's why the ai/machine-learning tech is what I'm looking for.  As I understand it, they can be trained by providing a highly compressed video, and the original HD version, which means the results would be much more accurate than a simple algorithm, depending on how many examples its been trained with.  This kind of thing exists for JPEG artifacts on images, and it works pretty well.  Of course you can't expect it to be 100% accurate (what it would have looked like naturally) but something like that would be quite helpful when complete accuracy isn't necessary.  Thanks for the info, though  
  > **username_4**: This answer is simply incorrect.  DaVinci Resolve and Premiere both already use machine learning approaches for image restoration.  They both use it for object removal.  Resolve uses it to fix dust and scratches.  Resolve has a de-banding and de-flickering algorithm, although I don't think that one uses machine learning, in particular.  Resolve also uses ML to de-interlace.  The tools are out there, and they'll only get better with time, they're just not as automatic as some would expect, and still require guidance from the user.  
  > **username_3**: @username_4 You have a point and I'll update my answer a bit. But I'm not entirely convinced. Restoring a heavy compressed video such that it looks almost the same as the master is an incredible complex task. If it can be done, please write an answer where you explain how you do it. However, if it's impossible now but might be in the far future, this does not help the OP much. From what I know, people working on AI keep promising too much (and have done so for decades) and generally fall far behind their promises. So I'm not as optimistic as you.  
  > **username_4**: https://idle.slashdot.org/story/21/02/07/0626249/can-artificial-intelligence-restore-85-year-old-popeye-cartoons  
  > **username_3**: Thanks for the link. Looking of one of the restored cartoon I clearly see the improvement and even have an idea how they have done it in general. However, the difficult challenge wasn't tackled: there is still a lot of dirt/dust visible. The images are shaky, but I guess this can be mitigated with already existing stabilizing effects. Still wondering why they haven't done it. Nevertheless, it's a progress. But I'm sceptical if this works on non-cartoon video with digital artefacts resulting from high compression.  
  > **username_5**: Something many don't realize is all the state of the art machine learning models related to this are released for free. You just have to be pretty tech savvy to use them. AI can do all of those things and it's not expensive at all to train a model that does this. The only point I agree with is if .your compression is so high that even your imagination can't undo it, then an AI model won't correctly do it either. Just depends on how bad it is  
  
  ---
  
  