The present invention relates generally to the field of databases, and more particularly to archiving techniques to facilitate the analysis of performance monitoring data stored within a live database.
Business intelligence (BI) is the set of techniques and tools for the transformation of raw data into meaningful and useful information for business analysis purposes. Common functions of business intelligence technologies are reporting, online analytical processing, analytics, data mining, process mining, business performance management, benchmarking, text mining, predictive analytics, and prescriptive analytics. Stakeholders utilizing BI operations may experience performance degradation of a system providing BI results, and the extensive use of activity or event logging, also referred to as performance monitoring data, on the system is often used to investigate performance issues.
If an issue requires performance monitoring data at a more granular level than is currently being monitored, a system typically requires a restart to change the logging level, and the system performance may degrade further for some period of time while more detailed logging is enabled. Real-time monitoring data is often collected and stored within the live database, or the same database to which the performance monitoring is directed. The real-time monitoring data is held for a defined retention period, then removed from current tables and stored in a separate database, after the retention period has passed. Retention is often limited, due to the rapid growth of performance monitoring data, which may consume an undesirable volume of storage in the live database. For that reason, historical performance monitoring data is typically moved to a separate database, which increases the difficulty in doing trend analysis between current and historical data when the data is stored into two separate databases. In some cases the data is stored in tapes and it will take a significant amount of time to restore the data for analysis.
When the applicable performance monitoring data, often containing transaction event log files, has been identified, a BI administrator may need to review multiple generated log files to assemble enough applicable information to understand the source of the problem and take action. In a typical distributed environment, a BI administrator may need to review hundreds or even thousands of log files that may be located across different servers. An investigation of such magnitude is very time consuming and technically challenging, because it requires linking disparate information from different server systems to generate a complete picture of the BI events, and hopefully sort through and pin point the problem.