Automated processes generate a variety of unstructured and semi-structured data. For example, complex software and products and hardware systems and devices typically generate “logs” of operating data on a regular basis, e.g., hourly, daily, etc., or in the case of streams continuously, in real time. Logs may include machine data generated by an application or the software and/or hardware infrastructure used to run the application. Logs may comprise a record of all the events that happened in an application and/or other system. They typically include unstructured or semi-structured information and often contain common parameters such as a time stamp, a serial number or other identifier, and IP or other address, process identifiers, etc. Logs may be provided in the form of files, streams, bundles, emails, messages, or other forms.
For manufacturers and/or users of a large installed base, it may be difficult to receive, process, store, and make use of log data, e.g., to detect, analyze, and predict bugs or other failures; to analyze performance; etc.