While monitoring has a conventional place in the security the associated tools may specialist’s toolbox, not be sized properly for Big Data systems. For example, in the cloud setting, the following is argued: “Monitoring demonstrates several challenges including gathering metrics from a variety of layers (infrastructure, platform, application), the need for fast processing of this data to enable efficient elasticity and the proper management of this data in order to facilitate analysis of current and past data and future predictions. In this work, we classify monitoring as a big data problem and propose appropriate solutions in a layered, pluggable and extendable architecture for a monitoring component.” (Zareian, Fokaefs, Khazaei, Litoiu, & Zhang, 2016) Big Data security and privacy support for audit and logging for monitoring and management is critical, but security operations must be able to scale along with associated Big Data applications. In addition, monitoring must be appropriate for both the utility, domain, and application models involved. This requires a close collaboration between application designers and security teams that is often not achieved.
While monitoring has a conventional place in the security specialist’s toolbox, the associated tools may not be sized properly for Big Data systems. For example, in the cloud setting, the following is argued: “Monitoring demonstrates several challenges including gathering metrics from a variety of layers (infrastructure, platform, application), the need for fast processing of this data to enable efficient elasticity and the proper management of this data in order to facilitate analysis of current and past data and future predictions. In this work, we classify monitoring as a big data problem and propose appropriate solutions in a layered, pluggable and extendable architecture for a monitoring component.” (Zareian, Fokaefs, Khazaei, Litoiu, & Zhang, 2016) Big Data security and privacy support for audit and logging for monitoring and management is critical, but security operations must be able to scale along with associated Big Data applications. In addition, monitoring must be appropriate for both the utility, domain, and application models involved. This requires a close collaboration between application designers and security teams that is often not achieved.