1. Field of the Invention
The invention relates to database replication. Particularly, the invention relates to ensuring a low-latency read of log records from a Database Management System (“DBMS”) in asynchronous log-based database replication capture from a blocking log read Application Programming Interface (“API”).
2. Description of the Related Art
Replication for a DBMS is a distributed technology that refers to the process of maintaining one or more copies of the same data, with or without transformations. The origin of the data is known as the “source,” and the receiver of the copy is the “target.” Asynchronous replication is the most common technique for database replication and is a process where at any given time, the source data is out of sync with the target data, but both are transactionally consistent.
A DBMS often provides an API to access the log records that it maintains for transactional recovery. A replication program may use these log records to capture the changes that occurred in the database and replicate them. Typically, log-based asynchronous database replication systems have 3 components: 1) a capture component to capture database changes at a source, by reading the DBMS recovery log; 2) a staging area where captured changes can be stored or sent to the target through various mechanisms; and 3) an “apply component” which receives these changes and commits them to the target.
The replication delay (or latency) is the time it takes for a change committed at the source database to be committed at the target database. Delays can be introduced at many steps during the replication process. However, a predictable and constant latency is expected for all transactions that are logically related. The latency corresponds to the age of the data at the target, which in turn affects business decisions. An application might not be able to make a business decision when some data is older than a certain age. The tolerable age or staleness of the data corresponds to the maximum replication latency.
In general, the latency is expected to be near real-time; however, latencies that are still acceptable for business decisions sometimes range from a fraction of a second, to a few seconds, a few minutes, or even hours. Transactions that are related by a common logical dependency must be replicated within a common maximum latency. For example, a transaction updating an ORDERS table will rely on the transactions updating the PARTS and PRICES catalogs to be replicated within the same tolerance, because the order needs the part number and prices to be up-to-date. An application might check replication latency at the target system to determine if the data is sufficiently up-to-date for proceeding with a business transaction involving the ORDERS, PARTS, and PRICES tables.
Because state-of-the art DBMS replication systems, such as IBM WebSphere Replication Server, replay (non-conflicting) transactions in parallel at the target DBMS to maximize throughput, replication latency will not be identical for all transactions. This is not a problem, as long as all logically related transactions are replicated within the same tolerance.
A database replication system must ensure that replication latency does not exceed the maximum tolerable latency, even during periods of unusual activity, low or high. Challenges regarding latency include the variable sizes of each transaction, inter-dependencies between these transactions, the performance of the DBMS, and the components of the replication system itself.