In any dynamic system, interactions occur between external agents and the system. There are many examples of this, such as humans, as external agents, interacting with the ecological system of the planet, humans interacting with a traffic management system, humans interacting with the Internet and so on.
The way external agents interact with systems is a great source of interest since the systems are typically of technical, economic or commercial value. By looking at the chain or stream of interactions that can occur between the individual external agents and the system in question, and in particular by looking at particular events of interest in that chain, it is possible to model the behaviour of the external agents and/or the effects on the system and take decisions regarding changes to or operation of the system. In this way, it is possible to learn the relationship between cause and effect, with analytical models of past interactions being built and then predictive models applied to drive future decision making.
In order to study the effects that are happening consequent to the interactions occurring between the external agents and the system, and to study the behaviour of both the external agent and the system as mentioned above, the interactions that are occurring need to be monitored and the data on the interactions recorded and stored for analysis.
One problem with this is that huge volumes of data need to be stored. Another problem is that the huge volumes of data need to be accessible and retrievable by a query in a manner that facilitates analysis in a useful and time effective manner. Yet another problem is that during the interactions occurring between the external agents and a system, it is not known at that time which of those interactions may result in an event of interest that is deemed important or significant for subsequent analysis. Thus, the storage and retrieval of the data must be flexible so that the analysis can take place around differing and subsequently selected events of interest amongst the plurality of potential events of interest in the chain of interactions occurring, and all in a reasonable time frame and within the current constraints on computer processing power.
The memory capacity that is available today facilitates the storage of vast amounts of data and it is now possible to monitor and store data concerning many interactions. However, whilst memory capacity continues to grow apace, the speed of access to and the ability to query, retrieve and analyse that data lags far behind. There is therefore a need to find improvements in the way that the vast amounts of data are stored for retrieval and analysis in order that useful information can be available within reasonable time constraints.
It is an object of the present invention to provide a method of storing and analysing data whereby large amounts of data can be monitored and stored in a manner enabling versatile and flexible analysis of the data without recourse to centralised massive computing power.