Preventing concurrent access of unprocessed data by multiple threads in multiple processes is a common problem. Along with this problem is a related requirement of having custom error-handling if the processing of the data generates an error.
The invention relates to methods, systems, and computer program products for implementing a database-based or a transactional data-source based scalable, fault-tolerant and robust mechanism to prevent concurrent processing of data by multiple processing entities. The locking mechanism presented in this invention also provides simple load-balancing capabilities. Processing entities often need to access one or more resources to perform work in a computing system. Examples of such entities include processes, tasks, and threads (hereinafter referred to without limitation as either a “process” or “thread”). In modern computing systems, there may be a large number of concurrent entities that seek to access resources in a system.
To prevent concurrent access, various mechanisms may be implemented to manage the type, number, and/or ordering of accesses that are permitted to resources in the system. A common mechanism that is used to synchronize and manage access to resources in computing and database systems is referred to as a “lock”. A lock is a data structure that indicates whether or which particular entities have been granted rights to a resource. An entity must acquire a lock to a resource before the entity is permitted to access the resource.
The scope of possessory or access rights granted to an entity for a particular resource is often related to the type of work that the entity intends to perform upon that resource. For example, an “exclusive lock” could be granted to an entity that seeks to access a data item in a way that is incompatible with concurrent access by other entities, e.g., to modify, write or delete the data item. The exclusive lock therefore grants exclusive access to the data item, which prevents other entities from being able to concurrently access the same data item at the same time. This type of lock essentially serializes access to its corresponding resource. A “shared lock” could be granted if an entity wishes to perform activities upon a resource which can also be concurrently performed with activities by other entities upon the same resource without introducing conflicts or inconsistencies to the data, e.g., to read a data item. Therefore, the shared lock can be concurrently granted to multiple entities for the same resource at the same time. Depending upon the exact configuration of the computing or database system, other types of locks and lock scopes can be implemented to manage access to data.
As noted above, using locks is a simple solution to avoid concurrent processing of data in a multi-process environment. However, the database locks have a drawback. The database locks are acquired as a part of a transaction. If an error happens in processing the resource the processing entity will have to rollback the transaction. The transaction rollback will release the lock resulting in other entities picking up the errored out resource for processing. In addition, the transaction-based database locks are not a useful solution if the processing time for the resource is large. A transaction with a large time will also place an unnecessary load on the database system.
The present scalable, high-performance, fault-tolerant locking mechanism can be applied to numerous contexts. For example, locking may be employed in the context of a resequencer. A resequencer is a product that can order messages to be delivered to a “message” consumer. Almost all types of computing systems use and implement the concept of messages. A message contains information which is sent from a source location or entity to a receiver location or entity. “Message passing” refers to a type of communications used by computing systems to send and exchange messages from sources to destinations. When messages are sent from a source to a destination, it is possible that the messages may be delivered out of order. A resequencer is used to order messages into the correct sequence for delivery. This locking mechanism may also be used to implement an asynchronous store and forward message passing.
Embodiments of the present invention provide an improved approach for handling access by multiple processing entities to resources in a computing system. According to some embodiments, a “heartbeat” infrastructure is employed to create and maintain process identifiers with respect to resources. The heartbeat infrastructure provides fault-tolerance when a process that has locked data items fails. The present inventive approach may be applied to any computing context, e.g., to implement a resequencer. Other and additional objects, features, and advantages of the invention are described in the detailed description, figures, and claims.