Information retention is one of the key functions of modern computing. Digital data is stored so it can be used or accessed again at some later point in time. There are many different storage technologies which are used for long term storage of data files including disk drives, flash memory, optical discs, tape drives, as well as others. Modern digital video processing systems use these types of storage mechanisms for the storage of video. In addition to other functions, the systems must manage the inventory of video which resides on a storage device as files are added, changed, deleted, or replaced.
Like most computer systems, many video processing systems typically make use of some type of file index to keep track of all the information pertaining to where files, and portions of files, are located on the storage device. A digital video file, like other computer files, may be split up when stored. The different pieces of the file may get stored in many different locations scattered across the storage device wherever unused space is available. Video processing systems typically use some sort of file index to be able to locate the various pieces of the video file in order to be able to reassemble the pieces correctly when the file is retrieved.
Computer systems and computer operating systems manage the inventory of information which resides on a storage device. In the ideal situation, the first data file written to a storage device starts at the first address location and the second data file written to the storage device starts at the address location immediately following the end of the first data file and the sequence repeats for subsequent data files. This simplistic system only works if data files are only written once and never changed, erased, or replaced. In reality, the distribution of data files across the storage device becomes much more complex after data files are changed, erased, or replaced. Video processing systems used to view and manipulate video data files operate in a similar manner and face similar challenges with respect to the management of data files.
When a data file is erased, an empty slot of storage space becomes available on the storage device. However, this slot is the size of the data file just erased and a new data file to be stored which is larger than the slot cannot be stored, in its entirety, in the slot. Similarly, a data file which was previously stored in a slot on the storage device and is now modified in a manner which makes the file bigger will no longer fit into its original storage slot. Some or all of the data file must now be relocated to another area of the storage device. For these reasons, as well as others, most rewritable storage devices are not sequentially populated with complete data files from beginning to end. The storage device eventually ends up being a continuously changing patchwork of used and unused storage locations.
This situation is further complicated by the fact that data files exist in all sizes and the patchwork of unused space on a storage device contains spaces of many different sizes. While it is ideal to store a data file in a single segment of contiguous storage space on the storage device, this is not always possible because the available slots of unused storage space are often smaller than the data file itself. Therefore, data files are often broken into segments when they are stored in order to make used of the smaller slots of unused storage space. There is a significant management task associated with keeping track not only of which data files are on the storage device and where those data files are located on the storage device but also keeping track of where each of the pieces of a data file resides if the data file has been broken into segments during the storage process.
Most computer systems and computer operating systems make use of some type of file index to keep track of all the information pertaining to where files, and portions of files, are located on the storage device. One type of file index which is commonly used is a file allocation table (FAT). A file index serves a function similar to that of a table of contents in providing detailed information about what data files currently reside in the storage device and where they are located. Because the data files may be scattered around the storage device in many different places and pieces, the file index is critical to finding data files on the storage device and retrieving them. Digital video files, like other computer files, may be split up and scattered around in a similar manner. A video processing system must also use some sort of file index to be able to locate and retrieve the various pieces of a video file and assemble them correctly.
In many systems, when a data file is erased the actual data bits of information on the storage device are not actually erased or overwritten. Instead, the reference to the data file is simply removed from the file index. Without the information about where the pieces of the data file are located, it is erased in the functional sense even though all of the bits of data still exist on the storage device. The actual data bits associated with the data file may only get changed or overwritten when that slot on the storage device is used to store a different data file. Similarly, formatting a storage device often only involves resetting or erasing the file index and the data files themselves are unchanged. However, these data files are virtually inaccessible because the file index, the map which explains where all the segments of data are and how they fit together, is gone.
A similar situation may result if some portion of a storage device is damaged, erased, corrupted, or no longer works properly. If any of these circumstances affect the file index or the tools which allow the file index to perform its function, it may appear that some or all of the data files have been erased. In reality, the data files may still exist on the storage device but they are inaccessible because the map to where the pieces of the data files are located within the storage device is damaged, corrupted, or erased.
In these situations, it is desirable to have another means of recovering and reconstructing data files which may still exist on the storage device. While the file index is effective as a single, centralized source of information regarding the organization of the data files on the storage device, it is also a central point of risk or failure which impacts all of the data on the storage device. For these reasons, it is desirable to also have information located with the data files or segments themselves which could aid in the reconstruction of the video data file if the file index could not be used.