RePIDS: a Multi Tier Real-Time Payload-Based Intrusion Detection System Aruna Jamdagnia,b, Zhiyuan Tana,b, Xiangjian Hea,*, Priyadarsi Nandaa and Ren Ping Liub aCentre for Innovation in IT Services and Applications (iNEXT), University of Technology, Sydney, Australia bCSIRO, ICT Centre, Australia Arunaj@it.uts.edu.au {Zhiyuan.Tan, xiangjian.he, Priyadarsi.Nanda}@uts.edu.au ren.liu@csiro.au ABSTRACT Intrusion Detection System (IDS) deals with huge amount of network traffic and uses large feature set to discriminate normal pattern and intrusive pattern. However, most of existing systems lack the ability to process data for real-time anomaly detection. In this paper, we propose a 3-Tier Iterative Feature Selection Engine (IFSEng) for feature subspace selection. Principal Component Analysis (PCA) technique is used for the pre-processing of data. Mahalanobis Distance Map (MDM) is used to discover hidden correlations between the features and between the packets. We also propose a novel Real-time Payload-based Intrusion Detection System (RePIDS) that integrates a Tier IFSEng and the MDM approach. Mahalanobis Distance (MD) dissimilarity criterion is used to classify each packet as either a normal or an attack packet. The effectiveness of the proposed RePIDS is evaluated using DARPA 99 dataset and Georgia Institute of Technology attack dataset. The traffic for Web-based application is considered for validating our model. a F-Value, criterion, is used to evaluate the detection performance of RePIDS. Experimental results show that RePIDS achieves better performance (a high 0.9958 for DARPA dataset and 0.976 for Georgia Institute of Technology F-Value, attack dataset respectively, with only 0.85% false alarm rate) and lower computational complexity when compared against two state-of-the-art payload-based intrusion detection systems. Additionally, it has 1.3 time higher throughput in comparison with real scenario of medium size enterprise network. Intrusion detection, Data pre-processing, Principal Component Analysis, Mahalanobis Distance Map, Keywords: Principal Components, Iterative feature selection
RePIDS: a Multi Tier Real-Time Payload-Based Intrusion Detection System Aruna Jamdagnia,b, Zhiyuan Tana,b, Xiangjian Hea,*, Priyadarsi Nandaa and Ren Ping Liub aCentre for Innovation in IT Services and Applications (iNEXT), University of Technology, Sydney, Australia bCSIRO, ICT Centre, Australia Arunaj@it.uts.edu.au {Zhiyuan.Tan, xiangjian.he, Priyadarsi.Nanda}@uts.edu.au ren.liu@csiro.au ABSTRACT Intrusion Detection System (IDS) deals with huge amount of network traffic and uses large feature set to discriminate normal pattern and intrusive pattern. However, most of existing systems lack the ability to process data for real-time anomaly detection. In this paper, we propose a 3-Tier Iterative Feature Selection Engine (IFSEng) for feature subspace selection. Principal Component Analysis (PCA) technique is used for the pre-processing of data. Mahalanobis Distance Map (MDM) is used to discover hidden correlations between the features and between the packets. We also propose a novel Real-time Payload-based Intrusion Detection System (RePIDS) that integrates a IFSEng and the MDM approach. Mahalanobis Distance (MD) dissimilarity criterion is used to classify each packet as either a normal or an attack packet. The effectiveness of the proposed RePIDS is evaluated using DARPA 99 dataset and Georgia Institute of Technology attack dataset. The traffic for Web-based application is considered for validating our model. F-Value, a criterion, is used to evaluate the detection performance of RePIDS. Experimental results show that RePIDS achieves better performance (a high F-Value, 0.9958 for DARPA dataset and 0.976 for Georgia Institute of Technology attack dataset respectively, with only 0.85% false alarm rate) and lower computational complexity when compared against two state-of-the-art payload-based intrusion detection systems. Additionally, it has 1.3 time higher throughput in comparison with real scenario of medium size enterprise network. Keywords: Intrusion detection, Data pre-processing, Principal Component Analysis, Mahalanobis Distance Map, Principal Components, Iterative feature selection Tier
FinalRevisedJournalpaper_18May12.docx