Conventionally, a compressively coding/decoding apparatus on video signals used to implement coding processing (encoding) and decoding processing (decoding) with dedicated hardware for video processing. This is called hardware codec (for example, see Patent Document 1).
In contrast, accompanied by developments of recent computer technologies, coding processing and decoding processing having been described above has been practicalized by executing video processing program on a general-purpose CPU (Central Processing Unit) (this is called software codec).
Unlike a hardware codec, such a software codec is advantageous for its capability of easily resolving internationally standardized compressively coding system or other various compressively coding method such as MPEG (Moving Picture Experts Group) 1/2/4 and JPEG 2000 and the like only by changing a video processing program. In addition, in the case where a new compressively coding method has been developed, only updating a video processing program in accordance herewith can easily resolve the latest compressively coding method.
Patent Document 1 Japanese Patent Laid-Open No. 2003-61097
In software encoding, a CPU processes video coding processing (video coding thread or process) and audio coding processing (audio coding thread or process) alternately in a time-sharing fashion. As for audio, its delay or lack is apt to be perceived by users, and therefore, in general, audio coding thread undergoes processing preferentially. For example, audio covering one frame undergoes processing during one frame period (1/30 second) and thereafter one frame of video undergoes processing.
FIG. 8 shows a processing state of software encoding, and video data as well as audio data inputted from outside are tentatively stored in video input buffer as well as audio input buffer respectively. The CPU at first executes audio coding thread and reads from audio input buffer, compressively codes and thereafter outputs audio data covering one frame. Subsequently, the CPU executes video coding thread, reads from video input buffer, compressively codes and thereafter outputs one frame of video data. Thus, the CPU proceeds with coding audio data as well as video data sequentially on a frame period basis in a time-sharing fashion.
Here, in compressively coding processing on audio, processing time thereof does not change much, but in contrast, in compressively coding processing on video, processing time thereof changes dramatically in accordance with the level of complication on input video (picture or motion and the like) intended to undergo coding.
For example, processing time is short on video that are abundant in comparatively flat parts and less abundant in movement and, in contrast, processing time increases on video including pictures that are complicated and movements taking place far, such as sports broadcasting, that undergoes panning and zoom at an moderate speed. Due to this increase in processing time, there is a case where it becomes impossible to complete coding processing within one frame period. Such processing delay is called overrun.
That is, in FIG. 8, the N-th frame is extremely complicated and coding is regarded to have failed in completion of coding within one frame period (occurrence of an overrun). In this case, completion, by the CPU, of coding processing on the N-th frame subject to occurrence of an overrun will end in cutting into the time to code the N+1-th frame. Here, switching of respective coding threads on audio is normally implemented by context switching by an OS (Operating System).
Here, this frame period is a period to process the N+1-th frame originally, but the frame subject to occurrence of an overrun has undergone processing, the N+1-th frame cannot undergo processing. Therefore, the CPU skips coding processing on the relevant N+1-th frame to implement coding processing from the next N+2-th frame.
Thus, conventional software encoding is designed to-skip coding on the subsequent frame in the case where an overrun has occurred in video coding processing. However, occurrence of such a frame skip will make displayed video in a state of frame-by-frame advance, giving rise to such a problem that display quality of video is harmed.
In order to avoid such an overrun, parallel operation of a plurality of CPUs and increase in clock frequency of a CPU and the like can be considered, but there is a problem that this will make the entire apparatus complicated and will increase in power consumption.