The present invention relates generally to digital image processing and in particular to interpolation methods used for scaling video sequences spatially and/or in time.
Digital video is typically represented as a series of images or frames, each of which contains an array of pixels. Each pixel includes information, such as intensity and/or color information. When scaling the image frames in space, in time or both in time and space, information for new pixels is missing and values for such pixels have to be interpolated from input pixel values.
In various super-resolution image processors, e.g. as disclosed in WO 2007/115583 A1, a set of candidate directions is considered for each output pixel. Each candidate direction is associated with a candidate directional interpolator used to compute a loss value based on input pixel values in the vicinity of the output pixel. A standard decision for deriving the final pixel value consists in selecting the direction with the smallest loss value. The output pixel value is then computed using the directional interpolator associated with the selected direction.
Such a standard implementation has drawbacks.
For example, in corner cases where substantially different directions are competing with loss values of similar magnitude, there might be abrupt switches from one decision to a different one. This causes discontinuities in the output images to appear. Structure that was not present in the input images may also become apparent and be perceived by the viewer as artifacts. This kind of problem can exist in 2D (still) images or in 3D (video) sequences.
Another kind of problem encountered with standard interpolation techniques in the case of video sequences having fade-in or fade-out effects. The interpolation processor can take inconsistent decisions along the fade-in or fade-out time, and change from a decision to a different one. In this case, the shape of the interpolated image can change simply with a change of contrast, which appears strange to the viewer.
There is thus a need for interpolation methods which avoid various artifacts which can be seen with existing methods.