Repetitive sequential human activity includes repeated events, each of which is a combination of sub-actions (primitives) with certain spatial and temporal constraints. Such activities are often observed in workplaces where repeated tasks need to be performed, in which each task can also include a specific set of ordered steps. For instance, in a grocery store, a characteristic sequential action performed by a cashier includes obtaining an item from the lead-in belt, presenting the item to the barcode scanner for pricing and depositing the item onto the take-away belt for bagging. Another example can include an assembly line at a plant (for example, an automobile plant) where a worker repeatedly integrates multiple parts in order before passing the assemblage to the next process in the chain.
Effective analysis of repetitive sequential activities can have broad applications in many contexts, such as workplace safety, retail fraud detection and product quality assurance. In an assembly line example, defective products are often the result of incorrect order of assembly. In such a case, accurate recognition of worker activities can assist in the quality assurance process. In another example, there is a prevalent type of fraud in retail stores that is the direct result of improper behavior on the part of the cashier. In such a situation, fraud occurs when the cashier passes an item through the checkout lane without actually registering it in the purchase list. These actions can be called fake scans and are also referred to as sweethearting. Sweethearting is a serious problem in the retail industry and can cause significant revenue shrinkage.
Existing approaches for human activity recognition are primarily based on graphical models such as, for example Finite State Machines (FSM), Hidden Markov Models (HMM), Context-Free Grammar (CFG) and Dynamic Bayesian Networks (DBN). However, such approaches cannot handle the issue of overlap between primitives. Some approaches for detecting fake scans include validation of motion flow in the transaction area using temporal constraints. Such approaches, however, result in a high false positive rate.