text:
\section{Introduction}

With the rapid growth of digital technology and communications, we are overwhelmed by network data traffic, which are diverse for media type (e.g. video, voice, text, sensory, etc.), and originate from (and are transported through) a broad range of sources (e.g. mobile networks, cloud infrastructures, Internet of Things, etc.). Consequently, we handle high-dimensionality data, calling for increasingly more sophisticated classification methods \cite{Camastra1,Camastra2}.

Typically, we refer to high dimensionality when we deal with data whereby a large number of features may be extracted, to the point that the features may even exceed the number of observations. This leads to major issues, particularly the massive increase in training times.

To this end, Feature Selection (FS) is a promising research direction, looking at ways to reduce the feature space in order to pinpoint only the most significant features. As a fundamental pre-processing step in machine learning, FS is gaining prominence in network management and, specifically, for the purposes of network intrusion detection and network traffic classification problems \cite{granville1,granville2,stadler,boutaba}.

More generally, FS finds an even much broader applicability in field as diverse as bioinformatics \cite{bioinf0,bioinf1,bioinf2,bioinf3},image recognition/retrieval \cite{image_recog1,image_recog2,image_recog3,image_retr1,image_retr2,image_recog4,image_recog5}, fault diagnosis \cite{fault_diag1,fault_diag2}, text mining \cite{text_min1,text_min2,text_min3} and, interestingly, in network traffic analysis/classification, whose pertinent bibliography will be covered more closely in the following.

Machine learning engines can be easily embedded in network intrusion detection systems (IDS), which represent an essential part of network infrastructures to guarantee security \cite{nids1,nids3,nids4} and availability \cite{ava_1,ava_2,ava_3,ava_4,esrel15,esrel17,tnsmava,nomsmdm1,nomsmdm2}.
Specifically, modern NIDS can be equipped with software probes in charge of analyzing network traffic on the basis of some characterizing features such as: distribution of inter-arrival times, distribution of packet sizes, presence of specific TCP/IP flags, percentage of forward/backward flows. Such statistical information is instrumental to revealing anomalous traffic, which is often behind Distributed Denial of Service (DDoS) attacks \cite{ddos1,ddos2}, covert Voice-over-IP sessions \cite{dimvoip}, threat diffusion \cite{dimkend} and Peer-2-Peer traffic \cite{cerroni1,cerroni2,cerroni3}. In many cases, these flows would pass unobserved through other conventional signature-based analyses.

In principle, a larger number of features would allow to perform more granular analyses; yet, two main drawbacks emerge. First, a proliferation of correlated features leads to levels of redundancy that are useless, while resulting in increasingly longer training times. Also, not all features are valuable in characterizing traffic, incurring bias during the classification step.

If we consider, as an example, some novel probes such as ISCXFlowMeter \cite{cicflowmeter}, these can actually generate more than $80$ features to characterize data traffic, which makes it extremely difficult for the network analyst to deal with this vast amount of information. This is where an FS pre-processing step becomes invaluable.

Across the scientific literature, many works are devoted to surveying machine learning algorithms with applications to internet traffic classification. Yet, few attempts have been made in the field of FS techniques, where two main shortcomings emerge:  \textit{i)} many works analyze and compare FS algorithms applied to non-specific datasets \cite{survey_general_1,survey_general_2,survey_general_3}; \textit{ii)} are either based on outdated datasets, or lack the experimental dimension. Overall, not much novel algorithms have been compared (refer to details provided in Table $1$, discussed further ahead).
Aimed at filling these gaps, our paper provides the main following contributions:
\begin{itemize}
\item We perform an experimental analysis of more recent datasets (classic literature focuses on the $20$-year old KDD$99$ dataset) by means of our designed-from-scratch Python based routine. This allows us to \textit{i)} perform cleaning, re-balancing, and data mixing operations, and \textit{ii)} automatically interact with specific ML-based engines.
\item We present a variety of experimental results (including: feature correlation, time complexity, performance) aimed at critically comparing selected families of FS algorithms, ranging from classic ones (rank search, linear forwarding selection) to modern bio-inspired algorithms (genetic search, ant colonies, multi-objective evolutionary), thus going well beyond a conventional literature survey.
\end{itemize}
\noindent
Our experimental-based, comparative assessment finds fruitful applications in the field of network/security management, where machine learning techniques are proved to offer a precious support to intrusion detection tasks, and where critical trade-offs between performance classification and resource consumption arise.

The paper is organized as follows: Section $2$ offers a general perspective on FS methods, in line with other accredited taxonomies. In Section $3$ we analyze how diverse FS techniques have been applied in the literature, often based on the old KDD$99$ dataset. Section $4$ proposes an {\em excursus} of popular FS algorithms (classic and modern) along with some necessary technical details. In Section $5$ we analyze the exploited novel datasets by grouping the most relevant features in families. In Section $6$ we perform an experimental-based comparative analysis of prominent FS algorithms, providing performance, feature correlation, and time-complexity figures. Finally, Section $7$ draws conclusions and provides future-direction indications.

\section{Overview of Feature Selection}

Feature Selection refers to that set of techniques and strategies that allow to optimize the feature space, namely, an $n$-dimensional space where each sample is represented as a point. When dealing with large feature spaces, the analysis of data, which starts from their representative features, can be tremendously time and resource consuming. Hence, the need to devise suitable FS strategies aimed at eliminating $i)$ irrelevant features, namely, those features that are not actually needed to build an optimal feature subset; and $ii)$ redundant features, namely, those features that strongly depend on other features \cite{relred}.

Feature selection can be considered as a special case of feature extraction methods \cite{Camastra2008}. The latter refer to a set of techniques (e.g. PCA, Single Value Decomposition, Linear Discriminant Analysis and others) useful to transform the original feature space in a new one aimed at alleviating the effects of the notorious {\em curse of dimensionality} problem \cite{blum,dash}. It is important to note that feature extraction comes with the critical risk of obtaining a transformed feature space that could lose its original physical meaning, whereas classic FS aims at preserving it \cite{sel_vs_extr,sel_vs_extr2}.

 A more formal definition of the FS problem follows. Given a feature set $X=\{x_i: i=1,\dots, N \}$, find a subset $S_K=\{x_{i_1}, x_{i_2}, \dots, x_{i_K}\}$ with $K<N$, where an objective function Y($\cdot$) is optimized, namely:
 \begin{equation}
 \{x_{i_1}, x_{i_2}, \dots, x_{i_K}\}=\argmax_{K,i_k}[Y\{x_i: i=1,\dots, N \}].
 \end{equation}
 Unless otherwise stated, the problem is to select the optimal subset of features (according to a specific criterion) from the initial set, where two steps are typically performed: the first one involves a search strategy to pinpoint candidate subsets; the second one involves an objective function to evaluate the selected candidate subsets. The latter can be split in two types \cite{filters_wrappers,filters_wrappers_2,filters_wrappers_3,filters_wrappers_4}: $i)$ {\em filters}, referring to objective functions that rely on properties of the data by evaluating the information content (e.g. correlation measures, inter-class distance); $ii)$ {\em wrappers}, referring to objective functions that exploit training models by starting from a subset of features and, then, adding or removing features based on the previous model.

A commonly accepted taxonomy of FS methods is presented in \cite{taxonomy}, where three classic approaches have been identified as \textit{supervised}, \textit{unsupervised}, and \textit{semi-supervised}. According to the supervised approach, labeled data are exploited to single out a feature subset, considering specific criteria for measuring the features importance.
Conversely, unsupervised techniques seek to unveil the intrinsic data structure to select the most significant features, without assuming any {\em a priori} knowledge \cite{tnsm1}.

Finally, the semi-supervised approach is based on a mixed strategy, striving to enrich an unlabeled set with some labeled data, so as to improve the FS phase.
Both the unsupervised and the semi-supervised approaches exhibit the drawback of neglecting potential correlations among features, resulting in the analysis of sub-optimal sets. This may prove  critical when dealing with traffic analysis, where we need to take into account statistical-based features (e.g. inter-arrival times variance, average packet length, etc.) and deterministic ones (e.g. IP addresses, port numbers, etc.).
Let us consider, for example, a particular kind of traffic directed towards a fixed destination port for a certain period of time. An unsupervised approach could lead to a subset of features which does not include the destination port. Thus, a crucial (as well as deterministic) piece of information gets lost.

On the other hand, a supervised approach can offer optimal results, provided that the data are correctly labeled. This case typically occurs in a controlled network environment, where, with the help of network analyzers, it is possible to automatically label the type of passing data traffic.

Because of the great potential of supervised methods, and thanks to the availability of suitable labeled datasets, we have decided to focus our experimental-based comparative evaluation on supervised FS methods.

\section{Related Work on Feature Selection applied to ML-based Intrusion Detection}

\begin{table*}[pos=ht]
\centering
	\caption{Prominent related work surveying FS techniques applied to Network Intrusion Detection.}
	\small
	\renewcommand{\arraystretch}{1.1}
	\begin{tabular}{|p{2.5cm}|p{4cm}|p{3.1cm}|p{6cm}|}
		\hline
		\textbf{Authors} &  \textbf{Experiments} &  \textbf{Single/Multi Class} &  \textbf{Description} \\  \hline
		Wang et al. \cite{table_wang} & Performance analysis & Single Class & Feature Selection performed on the KDD99 dataset, by applying C4.5 and Bayesian Network algorithms \\ \hline

		Janarthanan et al. \cite{table_jana} & Performance analysis & Single/Multi Class & Empirical selection of features by performing tests on KDD99 and UNSW-NB15 datasets by means of Random Forest algorithm \\ \hline

		El-Khatib \cite{table_khatib} & Performance analysis & Single/Multi Class & Feature selection combining Filter/Wrapper methods for Wireless IDS. Tests have been performed on a WLAN environment \\ \hline

		Chen et al. \cite{table_chen} & Performance analysis, Time analysis & Single Class & Correlation-based Feature selection using the KDD99 dataset \\ \hline

		Nisioti et al. \cite{table_nisioti} & N/A & N/A & Classic survey on ML-based techniques (including FS) with pointers to detailed sources, but with no experiments  \\ \hline

		Iglesias et al. \cite{table_iglesias} & Performance analysis & Single/Multi Class & Comparison among $4$ FS algorithms on the NSL-KDD dataset \\ \hline

		Singh et al. \cite{table_singh} & Performance analysis, Time analysis & Single Class & Comparison among various FS algorithms on the KDD99 dataset \\ \hline

		Bahrololum et al. \cite{table_bahrololum} & Performance analysis & Single/Multi Class & Comparison among PSO, Decision Tree, Flexible neural tree algorithms on the KDD99 dataset \\ \hline

		Dhote et al. \cite{table_dhote} & N/A & N/A & Limited survey of FS  techniques applied to network traffic, with pointers to detailed sources, but with no experiments  \\ \hline

		This work & Performance analysis, Feature Correlation analysis, Time analysis & Single/Multi Class & Feature Selection performed on the CIC-IDS-2017/2018 dataset, by considering $9$ FS algorithms from classic (Rank, Scatter) to modern (Genetic, Multi-Objective Evolutionary) \\ \hline
	\end{tabular}
\end{table*}

Most scientific literature involving ML approaches is typically focused on the proposal of algorithms or techniques for classification/detection of specific network traffic (\cite{classif_buczak,classif_mishra,classif_dimauro, classif_dimauro2, classif_dimauro3,classif_liotta,classif_nn,classif_dimauro4}). Unfortunately, the straight application of machine learning to network traffic analysis is not always feasible, due to the broad variety of traffic types, which leads to unmanageable feature spaces. Intuitively, in fact, a highly diversified traffic (multimedia, asynchronous, bursty, etc.) requires a large set of features able to capture the variegated ``nature" of the different flows. That is why appropriate pre-processing steps (i.e. FS) play a crucial role, thus a significant portion of ML-based literature has shown interest in FS techniques.

For instance, to improve the performance of IDS frameworks, the authors of \cite{ids_18_1} propose a mixed strategy involving Principal Component Analysis and fuzzy clustering with KNN-based FS techniques.
A correlation-based FS approach coupled with a Support Vector Machine (SVM) classifier is proposed in \cite{ids_18_2} to build a cloud-based IDS.
An IDS based on deep learning methods, along with a filter-based FS algorithm, is introduced in \cite{ids_19}. Similarly, a Convolutional Neural Network (CNN) approach is exploited in \cite{ids_18_3} to select traffic features from raw data sets, improving the accuracy of an intrusion detector.
Yu and Liu propose a mutual information-based algorithm that can analytically select optimal features for classification, by handling  linearly and non-linearly dependent data \cite{ids_16}. Again, FS is exploited jointly with Artificial Neural Networks \cite{ids_ann_19} and Deep Neural Networks \cite{ids_dnn_19}, respectively, to improve IDS performance. Furthermore, authors in \cite{ids_2011} embed in an IDS two FS algorithms that are compared against mutual information-based methods.

A major shortcoming of these works is that methods are validated and compared through the outdated KDD$99$ dataset. This contains information about older network attacks (that have been mitigated by now) or, in some cases, adopts an updated version of KDD$99$, namely NSL-KDD \cite{nsl-kdd}. Yet, although NSL-KDD adds some improvements onto KDD$99$ (e.g. no redundant records, better balancing between training and test set, etc.), it does not take into account features that characterize novel cyber attacks.
Other works in the field of machine learning applied to intrusion detection rely on more recent datasets such as  UNSW-NB15 \cite{unsw1,unsw2,unsw3,unsw4}. Although quite recent, such a dataset has two limitations: first,  the traffic has been collected in a reduced testbed; and secondly,  the number of features is limited to $49$, which is too small to appreciate the effectiveness of feature selection techniques.
In contrast, in our work we rely on an up-to-date dataset from the Canadian Institute for Cybersecurity \cite{cic} which has the following benefits: $i)$ it contains data traffic gathered over a vast network area; and $ii)$ it accounts for about $80$ features, allowing to extensively test the feature selection algorithms. Additional details about this dataset are provided in Section \ref{sec:dataset}.

Going more specifically into the set of works that share with this paper the aim to compare or survey FS methods for intrusion detection, we have collected the significant papers in Table $1$. In it, we have identified the material covered in the literature, which helps appreciating the contributions of our paper. The first column of Table $1$ points to the source; the second column highlights the type of experimental analysis (if any); the third column pinpoints the type of datasets utilized (single or multi-class); the last column provides a concise description of the surveyed material.

\section{Review of Feature Selection Algorithms under Scrutiny}

In this section, we briefly describe the algorithms under scrutiny, which belong to different families of FS techniques, ranging from classic rank-guided (Rank, Linear Forward Selection) and meta-heuristic (Tabu, Scatter, Particle Swarm) ones, to nature-inspired algorithms (Ant, Cuckoo), and up to modern techniques (Genetic, Multi-Objective Evolutionary).

For each algorithm, we provide a brief recap along with its pertinent application in network traffic analysis and security in literature.

\subsection{Rank-based Feature Selection}

Algorithms belonging to this family follow an approach based on two macro-steps: in the first one, the features are \textit{ranked} according to a certain statistical measure, whereas in the second step the algorithm chooses the top ranked features (eventually partitioned in clusters). We investigate and put to test two representative algorithms of this family: Rank Search and Linear Forward Selection.

\subsubsection{Rank Search}
The Rank Search technique refers not only to a single algorithm, but, to an umbrella of methods able to produce a list of attributes ranked with some criteria. One of the most applied rank-based techniques in FS relies on the Information Gain (IG) concept \cite{rank}. Given an attribute {\em A} and a class {\em C}, the {\em entropy} of the class without and with prior observation of the attribute are, respectively:

\begin{equation}
H(C)=-\sum_{c \in C}p(c)log_2p(c),
\label{eq:entr_noattr}
\end{equation}
and
\begin{equation}
H(C|A)=-\sum_{a \in A}p(a)\sum_{c \in C}p(c|a)log_2p(c|a).
\label{eq:entr_withattr}
\end{equation}

The IG is given by $H(C)-H(C|A_i)$, and represents the amount of the class entropy decreasing due to the {\em a priori} knowledge introduced by $i$-th attribute.

Over network traffic analysis literature, the rank search method has been widely exploited in conjunction with standard machine learning algorithms typically used during the classification step. In \cite{rank_fuzzy}, having the NSL-KDD dataset as input, the proposed algorithm $i)$ evaluates the information gain value, $ii)$ applies fuzzy rules to remove unwanted features, $iii)$ calculates the mean value of IG across the new subset of features, $iv)$ refines the feature subsets by applying an algorithm based on conditional probability evaluation.
Authors in \cite{rank_svm} propose a detection model able to cope with network attacks based on the feature IG ranking; once obtained a satisfying feature set, a triangle area based KNN, combining both SVM and greedy techniques, is exploited to single out even more discriminative and useful features.
A combination between an IG-based feature selection method and a C4.5-based classifier is advanced in \cite{rank_c45}, where the resulting algorithm has been optimized (in terms of power consumption) to reveal DoS attacks in ad hoc networks. Similar ensemble has been exploited in \cite{rank_supervised}, where authors consider a broader set of classifiers (C4.5, Random Forest, Naive Bayes, etc.) and where the analysis focuses on generic malware detection. Again, a mixed Genetic/KNN approach for intrusion detection presented in \cite{rank_knn} benefits from a feature reduction procedure obtained applying ID3 algorithm to find higher IG.
\subsubsection{Linear Forward Selection}
Such a technique to reduce the feature space dimensionality has been presented in \cite{lfs}. Linear Forward Selection (LFS) can be considered as an improvement of Sequential Forward Selection (SFS) method which starts with an empty set of features and sequentially adds one feature at a time. In order to face the $\mathcal{O}(N^2)$ complexity of SFS, in LFS the number of considered features at each step does not exceed a certain user-specified constant, thus, the resulting performance is impressively ameliorated. Two methods for limiting the number of features have been implemented in LFS algorithm: $i)$ {\em Fixed Set}, where a score obtained by means of a wrapper evaluator is used to choose the top-$k$ ranked attributes, thus, the maximum number of evaluations reduces to $k/2(k+1)$ ; $ii)$ {\em Fixed Width}, where, at each forward selection step, the number of features is increased by one, such that the set of candidate expansions
consists of the best $k$ features not been selected so far during the search. In such case, the maximum number of evaluations amounts to $N \cdot k-k/2(k+1)$.
Forward Selection-based techniques have been exploited across the traffic classification domain for detection of malicious application on Android \cite{lfs_android}, detection of {\em zero day} attacks \cite{lfs_zero}, designing novel anomaly detection systems \cite{lfs_ids}.

\subsection{Meta-heuristic Feature Selection}

Techniques belonging to this family rely on the principle that it is possible to select \textit{heuristics}, namely,  approximate algorithms searching for a sufficiently good solution to an optimization problem, useful when some constraints arise (e.g. limited computational resources, incomplete information). We select three representative algorithms of this family: Tabu Search, Scatter Search, and Particle Swarm Optimization.

\subsubsection{Tabu Search}

Tabu Search (TS) algorithm has been originally proposed in \cite{glover_article}, and, then, it has been extended and exploited to solve practical optimization problems \cite{glover_book,rego_book,tabu}. TS is based on a {\em metaheuristic} method able to pilot a local heuristic search in exploring
the space of solutions beyond the local optimality. Two main features characterize TS: \textit{adaptive memory} and \textit{responsive exploration}. The former allows to perform local choices guided by information collected during the search, whereas, the latter allows to make strategic choices. In other words, TS exploits a local search procedure combined with memory-based strategies, thus, the issue of getting trapped in local optimal solutions is avoided. To implement the memory-based mechanism, TS builds a map of recently visited solutions called Tabu List (TL).
A simplified TS algorithm is illustrated below:

\begin{megaalgorithm}
	\SetAlgoLined
	\caption{Tabu Search}
	\label{alg_tabu}
	1. Given a function $f(x)$ to be optimized over a set $\mathcal{X}$, start from initial solution $x_0 \in \mathcal{X}$, initialize Tabu List (TL), and initialize a counter $i=0$. \\
	2. Given $N(x_i) \subset \mathcal{X}$ the {\em neighborhood} of $x_i$, $N(x_i)$ can be reached by $x_i$ by means of a {\em move} operation. Thus, generate a neighborhood move list $M(X_i$).\\
	3. Given $x_{i+1}$ the best solution in $M(X_i$), update TL. \\
	4. In case stopping conditions are met, terminate. Otherwise, repeat Step $2$.
\end{megaalgorithm}

\noindent It is interesting to notice that, the memory structures characterizing TS method operate according four {\em dimensions} \cite{glover_book}:
\begin{itemize}
	\item {\em Recency}: concerns the ability of keeping track of solutions attributes that have changed across the recent past.
	\item {\em Frequency}: involves the mechanism exploited to broad the foundation for selecting preferred moves.
	\item {\em Quality}: pertains to the capacity of discriminating the merit of solutions visited during the search operation.
	\item {\em Influence}: takes into account the impact of choices performed during the search.
\end{itemize}

In the field of network traffic classification, TS has been exploited in \cite{tabu_fuzzy} jointly with fuzzy techniques aimed at optimizing the exploration of feature search space in intrusion detection problems. A combination of TS technique and KNN is presented in \cite{tabu_knn}, where KNN is initially exploited to generate a subset of non-redundant features, whereas, TS is used to refine the obtained subset. Again, authors in \cite{tabu_c45} propose a {\em wrapper} FS algorithm, dubbed GATS-C4.5, that embeds an hybrid Genetic and Tabu-based method as feature selection strategy and a supervised ML algorithm (C4.5) as the evaluation function. Similar combinations between TS and Genetic techniques have been used in \cite{tabu_genetic} where some tests (vs pure Genetic algorithms) across DARPA dataset have been carried out, and in \cite{tabu_genetic2} where SVM has been adopted as a classification criterion.

\subsubsection{Scatter Search}

Scatter Search is a metaheuristic algorithm involving memory-based mechanisms similar to those exploited in Tabu Search \cite{scatter_source}. The main strategy relies on an iterative process that organizes high-quality optimal solutions into subsets, and where five ``methods" emerge: $i)$ {\em Diversification}, to create a set of different trial solutions by exploiting a seed solution; $ii)$ {\em Improvement}, to convert a trial solution in one or more improved solutions; $iii)$ {\em Reference Set Update}, to create and keep update a reference set of best solutions; $iv)$ {\em Subset Generation}, to manipulate the reference set aimed at deriving a subset of solutions; $v)$ {\em Solution Combination}, to linearly re-combine the solutions on the basis of subsets obtained with the previous methods.

As an effective FS procedure, Scatter Search has been exploited together with NLP solvers for global optimization \cite{scatter_nlp}, with rough sets for the implementation of credit scoring mechanisms \cite{scatter_roughset}, or in a parallelized fashion to improve the feature subset selection problem \cite{scatter_parallel}. Again, Scatter Search has been profitably exploited in issues involving credit cards fraud detection \cite{scatter_credit}, and software security characterization \cite{scatter_sec_sw}.

\subsubsection{PSO Search}

The Particle Swarm Optimization (PSO) algorithm has been originally discovered in \cite{pso_scource}, through simulations carried out across a simplified social model aimed at reproducing the behavior of birds flocking.

Then, the population of agents became more similar to a swarm than flock, and single individuals were named {\em particles} that represent the candidate solutions.
In a mathematical form, given $\mathcal{A} \subset \mathbf{R}^n$ the search space, and $f:\mathcal{A}\xrightarrow{}Y\subseteq\mathbf{R}$, the swarm is defined by a set $\mathbf{S}=\{x_1,x_2,\dots,x_N\}$ of $N$ particles, where $x_i=(x_{i1},x_{i2},\dots,x_{in})^T \in \mathcal{A}$ with $i=1,2,\dots,N$. It is assumed that particles are able to iteratively move within search space $\mathcal{A}$ by means of a velocity parameter defined as $v_i=(v_{i1},v_{i2},\dots,v_{in})^T$ with $i=1,2,\dots,N$.
In PSO method, it is also defined a memory set $\mathbf{P}=\{p_1,p_2,\dots,p_N\}$ containing the best positions $p_i=(p_{i1},p_{i2},\dots,p_{in})^T$ (with $i=1,2,\dots,N$) visited by each particle. Given $t$ the time counter, the current position and velocity for particle $i$ are, respectively, $x_i(t)$ and $v_i(t)$, whereas $p_i(t)=\argmin_{t} f_i(t)$.
Accordingly, PSO is defined by the following equations:

\begin{eqnarray}
v_{ij}(t+1)&=&v_{ij}(t)+\gamma_1 r_1 [p_{ij}(t)-x_{ij}(t)]
\\ \nonumber
&+&\gamma_2 r_2 [g(t)-x_{ij}(t)],
\label{eq:pso_vel}
\\
x_{ij}(t+1)&=&x_{ij}(t)+v_{ij}(t+1),
\label{eq:pso_pos}
\end{eqnarray}
where: $r_1$ and $r_2$ denote random variables uniformly distributed in [0,1], $\gamma_1$ is the {\em cognitive} parameter which affects the step size that the particle takes towards its best candidate solution $p_{ij}(t)$, and $\gamma_2$ is the {\em social} parameter which affects the step size that the particle takes towards the swarm's best solution $g(t)$. At each iteration, best positions $p_{i}(t+1)$ are updated as well, namely

\begin{equation}
\label{eq:pso_eq}
\centering
p_{i}(t+1)=
\left\{
\begin{array}{l}
{\begin{array}{ll}
	\hspace{-0.2cm} x_i(t+1) \;\;\;\;\; \text{if } \; f(x_i(t+1))\leq f(p_i(t)),
	\end{array}}
\\
\\
p_{i}(t) \;\;\;\;\;\text{otherwise.}
\end{array}
\right.
\end{equation}

In the field of FS applied to network traffic analysis, PSO has been profitably exploited jointly with classification techniques. It is the case of \cite{pso_svm,pso_svm2}, where a particle swarm selection method has been exploited with SVM-based classifiers. Again, a hybrid FS model based on PSO and Random Forest has been exploited in \cite{pso_rf}, where  independent measures and a learning algorithm are exploited to evaluate feature subsets.
Interesting is also an evolution of classic PSO advanced in \cite{pso_bigdata}, and dubbed Accelerated PSO, amenable to deal with FS on Big Data streams.

\subsection{Nature-inspired Feature Selection}

Although relying on meta-heuristic concepts, this family of algorithms takes inspiration from the nature ecosystem, where many animal species exhibit impressive behaviors aimed at optimizing their life cycle. We assess Ant Optimization and Cuckoo search as representative algorithms of such family.

\subsubsection{Ant Search}
This technique, originally proposed in \cite{ant}, is inspired by ants colonies behavior, where the optimization problem is solved by a ``colony" of cooperating agents. Analyses carried out by ethologists showed that, following pheromone trails, each ant is able to follow a preceding ant which releases such substance, thus, a whole ant colony is able to self-organize itself. The emerging collective behavior relies on a positive feedback loop: the probability which an ant chooses a certain path increases with the number of ants that choosing the same path, since the trail is continuously reinforced with new pheromone.
The Ant algorithm can be profitably exploited in feature selection problems by evaluating the probability $p_{ij}^k$ that the $k$-th ``ant" could arrive to feature $j$ by starting from feature $i$, namely,

\begin{equation}
\label{eq:ant_eq}
\centering
p_{ij}^k=
\left\{
\begin{array}{l}
{\begin{array}{ll}
	\hspace{-0.2cm} \frac{[\tau_{ij}]^\alpha \cdot [\eta_{ij}]^\beta}{\sum_{k \in \mathscr{U}_k} [\tau_{ik}]^\alpha \cdot [\eta_{ik}]^\beta } \;\;\;\;\; \text{if } \; j \in \mathscr{U}_k,
	\end{array}}
\\
\\
0 \;\;\;\;\;\text{otherwise,}
\end{array}
\right.
\end{equation}
where, $\mathscr{U}_k$ is the set of feasible attributes (not visited yet), $\tau_{ij}$ is the amount of pheromone across the $ij$ path, $\eta_{ij}$ represents the heuristic information for the selected attribute $j$, $\alpha$ and $\beta$ are parameters in charge of controlling pheromone trials and heuristic information, respectively.

As regards the FS problem in network traffic analysis, Ant-based methods have been used in: \cite{ant_svm} where a SVM classifier is adopted on KDD99 dataset, after a feature reduction obtained through ACO (Ant Colony Optimization) technique; in \cite{ant_stream} where an ACO-based feature selection method allows to deal with big streamed data; \cite{ant_faco} where an improved ACO-based algorithm (named FACO) has been designed and tested across the classic KDD99 dataset.

\subsubsection{Cuckoo Search}

This technique is inspired to the {\em brood parasitism} strategy characterizing some cuckoos species. In particular, such species lay their eggs in the nests of other birds (hosts). Since host birds can engage a conflict as they recognize alien eggs, particular cuckoo species have evolved in such a way that females are able in mimicking colour and size of eggs of some host species, thus the hosts are cheated, and the probability of cuckoos reproductivity grows. Considering an egg in a nest as a solution, three idealized rules emerge in Cuckoo Search procedure \cite{cuckoo_book}: $i)$ each cuckoo lays one egg at time, in a randomly chosen nest; $ii)$ the bests nests (having high-quality eggs) are candidate to carry over next generations; $iii)$ the number of nests is fixed and, as a host bird discovers alien (cuckoo) eggs with a probability $p_d$, it gets rid of it. The aim of the algorithm is to exploit new (and eventually better) solutions in place of not-so-good solutions in the nest.
New solutions $\mathbf{x}_i^{t+1}$ for cuckoo $i$ are obtained through the following expression which represents the stochastic equation for random walk, namely,

\begin{equation}
\mathbf{x}_i^{t+1}=\mathbf{x}_i^t + \alpha \otimes \mathscr{L}(\lambda),
\label{eq:cuckoo}
\end{equation}
where, $\alpha>0$ is a scale factor, the product $\otimes$ refers to entrywise multiplications, whereas $\mathscr{L}$ is the L\'evy distribution with ($1<\lambda\leq 3$).

Cuckoo search method has been exploited in network traffic analysis paired with various techniques and technologies. In \cite{cuckoo_pca}, authors propose an algorithm that uses PCA and Cuckoo Search to reduce the feature space and to optimize the clustering center selection. A Cuckoo-based FS algorithm is proposed in \cite{cuckoo_ids} to preprocess network data aimed at improving the IDS detection accuracy in cloud environments. A Cuckoo search strategy has been also used in \cite{cuckoo_ann} to optimize Artificial Neural Networks when dealing with traffic anomaly detection issues. Again, coupled with SVM, Cuckoo search has been adopted in FS to deal with problem of phishing mail detection \cite{cuckoo_svm}.
Recently, extended versions of Cuckoo Search algorithm have been advanced to cope with classification of {\em tweets} in sentiment analysis \cite{cuckoo_sentiment}, or to defeat attacks in Software Defined Network infrastructures \cite{cuckoo_sdn}.

\subsection{Evolutionary Feature Selection}
\vspace{0.2cm}
Such family of algorithms is inspired by natural selection theory, claiming that living organisms survived across millions of years thanks to an adaptation process. In a similar way, this aptitude can be translated in search for optimal solutions to a problem. Two exemplary tested algorithms are: Genetic search and Multi Objective Evolutionary search.

\vspace{0.4cm}
\subsubsection{Genetic Search}

Genetic Algorithms (GAs) have been designed around the mid-1950s, when biologists started to perform computer-based simulations aimed at analyzing more in deep the evolution of genetic processes \cite{weise}. Then, GAs have been extended to face problems ranging from neural networks weight estimation \cite{genetic1} to inequalities-based problems \cite{genetic2}. A pioneering work in this field has been carried out by Holland \cite{holland_1962,holland_1992}, and, today, many variants of GAs exist \cite{genetic_survey} and are applied in economy, computer science, sociology.

The basic skeleton of a GA includes three operators \cite{Goldberg:1989:GAS:534133}: {\em Reproduction, Crossover} and {\em Mutation}.

Reproduction refers to a process in charge of evaluating the ability of an individual to be selected (among others) for reproduction, on the basis of a {\em fitness} score.

Crossover concerns the capability of a genetic operator in recombining information to create new offspring. Typically, offspring is generated by exchanging genes of parents until a {\em crossover point} is reached.

Mutation pertains to the probability that some offspring genes could be modified or altered.

Genetic-based feature selection in network traffic analysis has been used in conjunction with many ML-based methods. Authors in \cite{ga_ann} exploit a GA-based FS approach to optimize network traffic data before applying an artificial neural network to perform attacks detection across cloud infrastructures. A combination of a genetic FS method and a supervised classifier based on J48 algorithm is proposed in \cite{ga_j48}. More frequent across the scientific literature is the coupling between genetic FS and SVM classifiers applied to network traffic classification problems (see \cite{ga_svm,ga_svm2,ga_svm3}).

When dealing with FS problems, GAs allow to explore the solution space by selecting the most promising regions, thus, avoiding a costly exhaustive search. In our domain, the initial population is represented by the whole feature space and the fitness function relies on the correlation among features and expressed by means of a {\em merit} indicator defined further ahead in eq. (\ref{eq:merit}).

Once entered the cycle represented in Fig. \ref{fig:GA}, the algorithm calculates the fitness of each candidate solution per iteration, selects individuals to reproduce, and generates a new population by taking into account crossover (feature recombination with a certain probability), and mutation (one feature can be turned into another feature with a certain probability).

\begin{figure}[pos=ht]
	\centering
	\captionsetup{justification=centering}
	\includegraphics[scale=0.3,angle=90]{Images/genetic_cycle.eps}
	\caption{Genetic Algorithms life cycle.}
	\label{fig:GA}
\end{figure}

\subsubsection{Multi-Objective Evolutionary Search}

The family of solutions concerning a multiobjective optimization problem (MO) includes all the elements of the search space whose objective vectors cannot be simultaneously improved (Pareto optimality concept) \cite{moea_source}. The set of such objective vectors is said non-dominated.

More formally, a MO problem can be formulated as follows: given a vector of $n$ objective functions $\mathbf{f}$ of a vector variable $\mathbf{x}$ in a domain $\mathcal{D}$ defined as
\begin{equation}
\mathbf{f}(\mathbf{x})=(f_1(\mathbf{x}), f_2(\mathbf{x}), \dots, f_n(\mathbf{x}),
\end{equation}
a decision vector $\mathbf{x}_h \in \mathcal{D}$ is Pareto-optimal {\em iff} there is no $\mathbf{x}_k \in \mathcal{D}$ such that:

\begin{align}
\label{eq:reward_het_col}
\left\{
{\begin{array}{ll}
	\hspace{-0.2cm} \;\;\ \forall i \in \{1,\dots,n\}, k_i \leq h_i  \\  \\
	\;\;\;\;\;\;\;\;\; \;\;\; \;\;\;    \wedge  \\    \\
	\exists i \in \{1,\dots,n\} : k_i < h_i.
	\end{array}}
\right.
\end{align}

On the other hand, Evolutionary Algortihms (EAs) can be profitably exploited in MO-based problems since many  ``individuals" can search in parallel for multiple solutions, with the possibility of taking advantage of similarities among solutions belonging to the same family. Possible implementations of MO-EA techniques are ENORA (Evolutionary NOn-dominated Radial slots based Algorithm), and NSGA (Non-dominated Sorted Genetic Algorithm) compared in \cite{moea}.

Applied to the FS problem, the purpose of a multi-objective search algorithm is to discover a subset of features (a family of solutions) being a good approximation of the Pareto front.
The MO-EA approach has been exploited in network traffic classification jointly with ensemble ML-based methods \cite{moea_ensemb}, where some objectives such as maximizing true positive rate, maximizing classification accuracy, minimizing feature number, and minimizing false positive rate, are satisfied simultaneously and with no conflicts.
An NSGA-based approach to ameliorate the performance classification of IDS platforms has been adopted in \cite{moea_nsga1} and in \cite{moea_nsga2}.
Again, a MO-EA technique jointly with fuzzy classifiers for coping with the traffic classification problem has been introduced in \cite{moea_fuzzy}.

\section{The considered Datasets}
\label{sec:dataset}

One of the great issues when dealing with supervised FS approaches in traffic analysis, is finding training sets that are both recent and labeled.
As remarked in Sect. $3$, many works rely on the obsolete KDD$99$ dataset \cite{kdd99}. Created about $20$ years ago, KDD$99$ has been broadly employed to validate machine learning algorithms, particularly to differentiate malicious data traffic from benign one. However, the KDD$99$ dataset does no longer reflect the characteristics of modern data traffic, which has sensibly changed across time. This is very much the case of multimedia traffic (e.g. voice, video), being the principle revenue-making stream for service providers but also the vehicle of covert malicious data.
\textit{Per contra}, in this work we consider more recent datasets obtained by means of CICFlowMeter \cite{cicflowmeter,icissp-dataset}, an open-source engine that can both gather and label network traffic in a controlled environment.
Each dataset contains records labeled either as Benign or Malicious (malicious traffic is often split in sub-labeled traffic representing different kinds of attacks).
Specifically, we consider the following datasets:
\begin{itemize}
	\item \textit{DDoS} which contains traffic relating to distributed denial of service attacks, aimed at saturating the network resources of specific targets;
	\item \textit{Portscan} which contains traffic relating to Portscan attacks, aimed at discovering open, network device ports;
	\item  \textit{Webattack}, including malicious traffic which implements various web-based attacks such as Brute Force, Cross-Site Scripting, and Sql Injection;
	\item \textit{TOR} which includes traffic passing over the TOR network, an anonymous and private data circuit often exploited to carry dangerous information or malicious encrypted traffic;
	\item \textit{Android} which embeds various families of Android-based threats (adwares, ransomwares, etc.).
\end{itemize}
We want to notice that the first four datasets are exploited into the single class analysis, whereas the \textit{Android} dataset is exploited in the forthcoming multi-class analysis. In this latter analysis, we build a new dataset called \textit{MultiAndroid} (see Sect. $6.2$), obtained by selecting the most relevant mobile threats from the \textit{Android} dataset mixed with some benign traffic.
In order to avoid the issue of bias that typically arises during classification in imbalanced datasets, we have first re-arranged the datasets. We have achieved well-balanced benign/malicious features, spanning across about $50k$ instances for each dataset. Each one contains up to $78$ features, except for the TOR dataset including 30 features.
For the sake of convenience, we found it useful to group the features in $5$ macro-classes (the  complete list of features can be found in \cite{icissp-dataset2}):
\begin{itemize}
	\item \textbf{Time-based features:} Forward/Backward inter-arrival times (IAT) between two flows, duration of active flow (min, max, mean, std), duration of idle flow (min, max, mean, std), etc.;
	\item \textbf{Byte-based features:} Forward/Backward number of bytes in a flow, Forward/Backward number of bytes used for headers, etc.;
	\item \textbf{Packet-based features:} Forward/Backward number of packets in a flow, Forward/Backward length of packets in a flow (min, max, mean, std), etc.;
	\item \textbf{Flow-based features:} Length of a flow (mean, max, etc.);
	\item \textbf{Flag-based features:} Number of packets with active TCP/IP flags (FIN, SYN, RST, PUSH, URG, etc.).
\end{itemize}

\section{Experimental Results}

\begin{figure*}[pos=htp]
	\centering
	\begin{tabular}{cccc}
				\subfloat[Ant (21 fts)]{\includegraphics[width=2.3in]{Images/corr_ant2}} \hfil
				\subfloat[Scatter (4 fts)]{\includegraphics[width=2.3in]{Images/corr_scatter2}} \hfil
				\subfloat[MO-EA (5 fts)]{\includegraphics[width=2.1in]{Images/corr_moea2}} \hspace{10mm} \\
				\subfloat[Ranking (10 fts)]{\includegraphics[width=2.3in]{Images/corr_rank2}} \hfil
				\subfloat[Cuckoo (7 fts)]{\includegraphics[width=2.3in]{Images/corr_cuckoo2}}  \hfil
				\subfloat[Tabu/LFS (6 fts)]{\includegraphics[width=2.1in]{Images/corr_tabu2}} \hfil \\
				\subfloat[Genetic (27 fts)]{\includegraphics[width=2.4in]{Images/corr_genetic2}}   \hfil
				\subfloat[PSO (18 fts)]{\includegraphics[width=2.4in]{Images/corr_pso2}}  \hfil

	\end{tabular}
	\caption{Correlation maps for different algorithms - DDoS dataset. In parenthesis is reported the number of features surviving after the FS process.}
	\label{fig:corrmaps_ddos}
\end{figure*}

Herein we describe our analytical study, which required the development of a dedicated Python routine, to normalize/balance the datasets and to automate the comparison among FS algorithms.
We have validated the scrutinized search algorithms using the Correlation-based Feature Selector (CFS) as objective function \cite{cfs1,cfs3}.

The main idea behind CFS is that a good feature subset includes those features that are highly correlated with the class, while being strongly uncorrelated among them. A formal definition is offered in \cite{cfs5}: a feature $X_i$ is relevant {\em iff} there is some $x_i$ and $y$ for which $p(X_i=x_i)>0$ such that

\begin{equation}
p(Y=y|X_i=x_i)\neq p(Y=y).
\end{equation}
Namely, $X_i$ is relevant if $Y$ is conditionally dependent on $X_i$.
Thus, CFS is a filter algorithm that can rank feature subsets according to a correlation-based heuristic function. Precisely, given a subset $S$ including $k$ features, the heuristic {\em merit} $M_{S,k}$ is defined as:

\begin{equation}
M_{S,k}=\frac{k \overline{r_{fc}}}{\sqrt{k+k(k-1)\overline{r_{ff}}}},
\label{eq:merit}
\end{equation}
where $\overline{r_{fc}}$ is the average value of feature/class correlations, and $\overline{r_{ff}}$ is the average value of feature/feature correlations.
The numerator of (\ref{eq:merit}) may be seen as an indicator of how far a set of features is predictive of a class; whereas, the denominator contains information about how much redundancy there is among features.

Our assessment is split into two parts: the first one concerns a \textit{single class} analysis, where we evaluate datasets exhibiting \textit{dichotomous} information (malign/benign); the second one is focused on \textit{multi class} problems, where we evaluate the effectiveness of FS in the presence of multiple classes.

\subsection{Single Class Analysis}

\begin{figure*}[pos=htp]
	\centering
	\begin{tabular}{cc}
		\subfloat{\includegraphics[width=3.2in]{Images/fs_times_single}} \hspace{8mm}
		\subfloat{\includegraphics[width=3.2in]{Images/train_times_single}}
	\end{tabular}
	\caption{FS times - DDoS dataset (a); Training times - DDoS dataset (b).}
	\label{fig:times}
\end{figure*}

Let us consider the Distributed Denial of Service (DDoS) attack which, recently, is also affecting modern SDN-based networks \cite{jsac4,jsac5}. DDoS attacks are designed to overwhelm the target network resources by means of a \textit{botnet}, namely, a network composed of a large number of malicious nodes sending tiny packets towards the target, ultimately coordinated by a \textit{botmaster}.

Let us now analyze the results obtained by pre-processing the DDoS dataset through the set of FS algorithms introduced above. In Fig. \ref{fig:corrmaps_ddos} we report, for each algorithm, the correlation map corresponding to a graphical representation of covariance matrices. This representation embeds three important pieces of information: \textit{i)} the number of features surviving after the FS processing step; \textit{ii)} the type of features; and \textit{iii)} the relationship existing among surviving features. The latter is taken into account by means of a gray scale, in which darker shades indicate higher levels of correlation. Thus, each $(i,j)$ ``pixel" gives the correlation level between feature $i$ and feature $j$.
Accordingly, the pixels on the main diagonal are always black (maximum correlation, corr=$1$), due to the self-correlation.
As was to be expected, higher correlation are found among those features belonging to the same family (Time-based, Flow-based, etc.).

Some interesting considerations about the various correlation maps arise. First, the number of features retained by different algorithms may significantly diverge, which is due to the specific approaches adopted by each algorithm. The Genetic algorithm is the one retaining the most features. This is to be ascribed to the particular strategy of this algorithm, which strives to escape local optima by applying the mutation operator, thus allowing to consider more paths, namely, more features. Second, some common features retained by all the algorithms can be recognized. For instance, the destination port feature is always present since, in a DDoS attack, a target victim is typically reached on a particular exposed TCP/UDP port. Moreover, since DDoS attacks are characterized by a large amount of small-size packets, features embodying information about packet lengths are retained. The difference is that, some algorithms (e.g. Scatter, MO-EA, Cuckoo, Tabu, LFS) just keep the essential features related to packet length (e.g. total packet length, total number of bytes sent in initial window); whereas, other algorithms (e.g. Ranking, Genetic, PSO, Ant) prefer to retain more features belonging to the same family. DDoS is also characterized by some kind of synchronization among the bots, which are coordinated to launch an almost-simultaneous attack. This means that time-related features will often provide useful information to detect DDoS. Interestingly, the Genetic algorithm retains $5$ features relating to the inter-arrival flow times, resulting in a dark gray cluster at the center of the correlation map (Fig. \ref{fig:corrmaps_ddos}(g)).

\begin{figure*}[pos=ht]
	\centering
	\begin{tabular}{cccc}
		\subfloat[]{\includegraphics[scale=0.48]{Images/distributed2}}  \hspace{8mm}
		\subfloat[]{\includegraphics[scale=0.48]{Images/portscan2}}  \hspace{8mm} \\
		\subfloat[]{\includegraphics[scale=0.48]{Images/webattack2}}  \hspace{8mm}
		\subfloat[]{\includegraphics[scale=0.48]{Images/tor2}} \hspace{8mm}
	\end{tabular}
	\caption{Performance in terms of Accuracy/F-Measures for different single class datasets: DDoS (a), Portscan (b), WebAttack (c), TOR (d).}
	\label{fig:perf_single}
\end{figure*}

It is also possible for DDoS attacks to be even more effective through the modification of the IP flags (e.g. SYN/RST flooding). Accordingly, features embodying information about IP flags (e.g. RST-SYN-URG flag count) are retained by algorithms such as Ant (Fig. \ref{fig:corrmaps_ddos}(a)), MO-EA (Fig. \ref{fig:corrmaps_ddos}(c)), Cuckoo (Fig. \ref{fig:corrmaps_ddos}(e)), Genetic (Fig. \ref{fig:corrmaps_ddos}(g)), and PSO (Fig. \ref{fig:corrmaps_ddos}(h)).
Let us note that many algorithms opt for selecting features that are uncorrelated among them (few dark gray or black clusters are present) since they convey more variegated information.

\begin{figure*}[pos=ht]
	\centering
	\begin{tabular}{cccc}
		\subfloat[Ant (22 fts)]{\includegraphics[width=2.4in]{Images/corr_ant_multi2}} \hfil
		\subfloat[Scatter/Tabu (9 fts)]{\includegraphics[width=2.4in]{Images/corr_scatter_multi2}} \hfil
		\subfloat[MO-EA (6 fts)]{\includegraphics[width=2.2in]{Images/corr_moea_multi3}} \hspace{10mm} \\
		\subfloat[Ranking (28 fts)]{\includegraphics[width=2.3in]{Images/corr_rank_multi2}} \hfil
		\subfloat[Cuckoo (17 fts)]{\includegraphics[width=2.3in]{Images/corr_cuckoo_multi2}}  \hfil
		\subfloat[LFS (9 fts)]{\includegraphics[width=2.1in]{Images/corr_lfs_multi3}} \hfil \\
		\subfloat[Genetic (31 fts)]{\includegraphics[width=2.6in]{Images/corr_genetic_multi2}}   \hfil
		\subfloat[PSO (23 fts)]{\includegraphics[width=2.6in]{Images/corr_pso_multi2}}  \hfil
	\end{tabular}
	\caption{Correlation maps - MultiAndroid dataset. In parenthesis is reported the number of features surviving after the FS process.}
	\label{fig:corrmapsmulti}
\end{figure*}

Let us now analyze some findings obtained from the time-complexity evaluation. To this aim, we use a PC equipped with Intel Core\textsuperscript{TM} i5-7200U CPU@ 2.50GHz CPU and 16 GB of RAM.
In Fig. \ref{fig:times}(a), we show how the FS time varies with training size, for the DDoS dataset. No dramatic differences are observed across the various algorithms, even more significantly as the training size grows. Considering a relatively large training size (with $5\cdot10^4$ training instances), FS times range from about $10$ seconds (Scatter algorithm) to almost $26$ seconds (MO-EA algorithm). Surprisingly, the FS times are rather uniform, in spite of the broad variation in number of retained features (by each of the algorithms). For instance, remaining in the case of $5\cdot10^4$ training instances, Scatter retains the minimum number of features (4), while Genetic retains the maximum number of features (27); yet FS times are comparable ($16.19$ and $10.18$ seconds, respectively). Although it is legitimate to expect that higher FS time could be justified to produce a more reduced feature space, the scarce correlation between such observables is due to the particular logic implemented in each FS algorithm.

On the other hand, Fig. \ref{fig:times}(b) provides the training times obtained by applying the J48 benchmark algorithm, downstream of the FS processing step. Here, the black line (with empty circles) gives the training times obtained when no FS processing is employed. We can observe how FS leads to significant improvements, in terms of both times and trends. The black (benchmark) line grows rapidly to almost 80 seconds, while most algorithms peak to almost 5 seconds, with the exception of the Genetic algorithm (yellow line) and the Particle Swarm algorithm (light blue line) that take over 10 seconds to complete.
This indicates that the FS process, on the whole, brings gains in the range of about one order of magnitude, which may become even more significant as the dataset grows.

 Let us now analyze the performance of the proposed FS algorithms in terms of Accuracy and F-Measure. These two metrics, widely used in the field of traffic classification \cite{accuracy1,accuracy2}, are defined as follows:
\begin{itemize}
\item \textbf{Accuracy}: the ratio of the correctly predicted observations to the total observations. This is the most intuitive indicator.
\item \textbf{F-Measure}: the weighted average of precision (ratio of correctly classified flows over all predicted flows in a class) and recall (ratio of correctly classified flows over all ground truth flows in a class). This is an indicator of a per-class performance.
\end{itemize}

\begin{figure*}[pos=ht]
	\centering
	\begin{tabular}{cc}
		\subfloat[]{\includegraphics[width=3.2in]{Images/fs_times_multi.eps}} \hspace{8mm}
		\subfloat[]{\includegraphics[width=3.2in]{Images/train_times_multi.eps}}
	\end{tabular}
	\caption{FS times - MultiAndroid dataset (a); Training times - MultiAndroid dataset (b).}
	\label{fig:multitimes}
\end{figure*}

To verify that the effectiveness of the FS algorithms is not linked to specific datasets, we have considered the $4$ different datasets introduced in Sect. $5$ (DDoS, Portscan, WebAttack, and TOR), reporting our findings in Fig. \ref{fig:perf_single}.
Just like for the previous experiments, we have used the tree-based J$48$ algorithm as a benchmark.
We have adopted a $10$-fold cross-validation which is typical in applied ML, and offers a good trade-off between training time and robustness. Noticeably, all FS algorithms perform satisfactorily (both in accuracy and F-measure) in comparison to the benchmark (first bars in all the histograms, labeled as ``NO F.S.'') for the four datasets.

In some instances the FS algorithms performed even better than the benchmark (e.g., Rank and Genetic algorithms in the WebAttack dataset). This can be explained by a phenomenon that is well-known in ML, whereby models based on too many features may lead to biased classification. On the other hand, when FS manages to retain a sufficiently high number of meaningful features, there is a positive effect on accuracy. This is the case of the Genetic algorithm applied to the TOR dataset (Fig. \ref{fig:perf_single}(d)) that performs better than the other methods.

\subsection{Multi Class Analysis}

Another fruitful analysis is aimed at evaluating FS algorithms when multi-instance datasets are considered. This turns out to be particularly useful when it is not possible to discern different types of data traffic via some pre-processing filter (e.g. IP/Port-based filtering).
To assess this case, we consider two datasets: the \textit{MultiAndroid} dataset, containing benign traffic mixed up with five different types of Android-based threats; and the \textit{DDoS/Portscan} dataset, including a mix of DDoS, Portscan, and benign traffic.
The MultiAndroid dataset, includes the following types of malign traffic:

\begin{itemize}
\item {\em FakeApp.AL}: a scareware hidden inside a fake {\em Minecraft} application, one of the most popular game applications;
\item {\em Android Defender}: a malware which, once accidentally downloaded and installed, raises some fake alerts;
\item {\em Gooligam}: an insidious malware that has already infected more than $1$ million Android-based devices, aimed at stealing Google accounts for Drive, Docs, Gmail, etc.;
 \item {\em Feiwo}: belonging to the adware family, it acts by showing advertisements in the system notification bar, and by sending device GPS coordinates to a remote server;
 \item {\em Charger}: a ransomware hidden in some Google Play applications, which gains root privileges and steals contacts before asking for a ransom.
\end{itemize}

Let us analyze how FS algorithms impact on the MultiAndroid dataset in terms of feature correlation referring first to the panels of Fig. \ref{fig:corrmapsmulti}.
Comparing these results with the ones of Fig. \ref{fig:corrmaps_ddos}, an interesting difference emerges: all FS algorithms retain more features w.r.t. the single-class case. This behavior is coherent with the fact that, to deal with different types of threats (ransomware, adware, malware) we need more features, to be able to capture this higher variability. This effect is even more evident in time-based features (mainly inter-arrival times) and in size-based features (mainly packet lengths).

Looking at DDoS, we observe a difference between single- and multi-class analysis. In the latter, the destination port is not retained as a crucial feature. This is possibly because malwares exploit different mechanisms to create damage: rather than directly overwhelming a particular target port, they first act in the background (e.g. by stealing privacy data) and then produce malicious traffic in egress. On the other hand, DDoS attacks generate ingress traffic from the infected device.

It is worth noticing that, when applied to multi-class problems, all algorithms have preserved their original logic. For instance, with 31 surviving features, the Genetic algorithm is still the algorithm that saves more features, thanks to the role played by the mutation operator. Another example is the MO-EA algorithm that, just like in the single-class experiment, retains the smallest number of features ($6$). This is mainly due to the diversity-preservation mechanism, which forces the selection of a representative subset of the whole Pareto front. It optimizes conflicting objective functions, thus few solutions survive.

The time-complexity evaluation is reported in Fig. \ref{fig:multitimes}, which evaluates the usual FS algorithms onto the MultiAndroid dataset. FS times exhibit the same order of magnitude as in single-class analysis (Figs.\ref{fig:times}(a)). For a training size amounting to $5\cdot10^4$ instances, the fastest algorithm is Scatter (FS time amounting to $9.541$ seconds); whereas the slowest one is MO-EA (FS time amounting to $24.827$ seconds).

The situation changes dramatically when we consider training times for the J$48$ benchmark algorithm (Fig. \ref{fig:multitimes}(b)). Notably, multi-class algorithms are roughly one order of magnitude slower than their single-class counterpart. For instance, let us consider the Genetic algorithm (yellow curve). For a $10^3$ training size, Genetic FS reduces the training time to $1.861$ seconds, growing to the following (X;Y) points: ($10^4$; $10.731$); ($2*10^4$; $56.748$); ($3*10^4$; $133.346$); ($5*10^4$; $301.997$). The longer training times arise from the process of training multiple classes. Nevertheless, significant gains are still obtained by all FS algorithms compared to the ``NO F.S.'' benchmark, which peaks at $446.329$ secs.

\begin{figure*}[pos=ht]
	\centering
	\begin{tabular}{cccc}
		\subfloat[]{\includegraphics[scale=0.48]{Images/android_multi_accu.eps}}  \hspace{8mm}
		\subfloat[]{\includegraphics[scale=0.48]{Images/android_multi_fmeas.eps}}  \hspace{8mm} \\
		\subfloat[]{\includegraphics[scale=0.48]{Images/multi_accu_ddos_portscan.eps}}  \hspace{8mm}
		\subfloat[]{\includegraphics[scale=0.48]{Images/multi_fmeas_ddos_portscan.eps}} \hspace{8mm}
	\end{tabular}
	\caption{MultiAndroid dataset: Accuracy (a), F-Measure (b); DDoS/Portscan dataset: Accuracy (c), F-Measure (d);}
	\label{fig:perf_multi}
\end{figure*}

Turning now to the performance analysis, in Fig. \ref{fig:perf_multi} we compare the two multi-class datasets, MultiAndroid and DDoS/Portscan, drawing some interesting considerations. It is comparably more difficult to detect Android threats than DDoS/Portscan attacks - MultiAndroid accuracy is below $0.7$ and F-Measure is below $0.5$. However, this issue is not generated by the FS processes, since the ``NO F.S.'' performance is poor too, particularly with the ``Benign'' class. This issue arises from two facts. First, mobile network attacks are often accompanied by activities that do not directly/immediately generate network anomalies. Examples are ransomware and malware, whereby the anomalies arise after the user has downloaded the malicious application. There is typically a lag between infection and anomalies, as the malicious program initially establishes a secret/silent communication with a remote server, and then gradually steals/sends private user data. Another example is adware, where those annoying banners actually incur very little data, thus making it hard to detect from the regular traffic. A second reason for the poor MultiAndroid performance is the strong similarity among different malign classes (e.g., scareware,  adware, ransomware).
Similar considerations hold true in the case in which we consider a dataset including Webattack and TOR traffic (not reported for space constraints), whereby the high similarity between the two classes resulted in poor classification performance.
We should however stress that FS algorithms are still very beneficial, since the time-complexity benefits identified are achieved with no dramatic loss in accuracy.

By contrast, the DDoS/Portscan multi-class case achieves outstanding performance (Figs.\ref{fig:perf_multi}(c) and (d)). This is because these types of attacks are radically distinct in the way they exploit network vulnerabilities: DDoS falls under the umbrella of volumetric attacks; whereas Portscan attacks employ monitoring strategies to unveil possible open ports.
In other words, a peculiar {\em symptom} of a DDoS attack is the presence of an exceptionally large number of connections coming from different nodes and heading towards one network target's port. Conversely, a {\em symptom} of Portscan attacks is the presence of just a single node (or a few nodes in case of simultaneous Portscans) opening a considerably large number of connections towards multiple ports of a certain network target.
Thus it is relatively easier to differentiate between these two attacks.

\subsection{General Remarks}
Overall, we can observe that FS algorithms do lead to an effective reduction in feature space, ranging from $65 \%$ (Single Class, Genetic) to $95 \%$ (Single Class, Scatter) and from $60 \%$ (Multi Class, Genetic) to $92 \%$ (Multi Class, MO-EA). Such feature-space reduction translates into significant computational-time improvements, which become even more remarked as the training size grows. For instance, with a training set of $50k$ samples (single-class DDoS) the MO-EA algorithm takes $24.8$ secs to perform FS, while the training time compared to the benchmark drops from $72.2$ to $5.13$ secs. At the same time, performance is not significantly degraded by the feature reduction process - accuracy drops from $0.9993$ to $0.9971$. Similar considerations hold for all other algorithms.

The performed assessment provides invaluable guidelines for network/security management practitioners dealing with traffic classification problems. Our evaluation framework aims at weighing the practical benefits of the various FS techniques in terms of time-complexity reduction and performance guarantees. For instance, if we aimed at minimizing the overall processing time (i.e., FS plus training times), the Scatter algorithm would be the best choice. This incurs a total processing time amounting to $14.338$ seconds for the single-class case (FS= $10.178$ secs plus training= $4.16$ secs), and to $219.963$ seconds for multi-class (FS= $9.541$ secs plus training= $210.422$ secs). Conversely, the Genetic method would be preferable to maximize performance.

\section{Conclusion and Future Direction}
A prominent research direction for network intrusion detection is the adoption of machine learning methods, particularly for the detection of anomalous (and often malicious) network-traffic flows. Looking at the literature, we find ample examples of network classification problems. Yet, little attention has been turned towards feature selection, which is an essential classification pre-processing step. We argue that the main reason for this overlook is that most studies have been based on the obsolete KDD$99$ dataset, which includes few features, thus making FS irrelevant.
On the other hand, we consider that modern network engines generate much richer features (in fact, hundreds of features), which allow more fine and granular network traffic analyses. However, this extra capability results into impractical ML training times, making it necessary to understand how FS may be realized effectively.

To this end, herein we have carried out an experimental comparative evaluation of prominent methods, with the view to provide insights as to how the different FS algorithms perform in the peculiar context of network-traffic classification. Our assessment shows how few, relevant features are retained, but also that the FS reduction process is virtually lossless, with a significant acceleration of the overall training process.

To sum up, the novelties of our work are:

\textit{i)} we carry out an experimental-based review, considering recent datasets (including DDoS, Portscan, WebAttacks, and Android threats), as opposed to the obsolete KDD$99$ dataset adopted in most literature;

\textit{ii)} we compare and contrast a representative number of alternative FS algorithm types, including classic rank-guided methods (LFS, Ranking), meta-heuristic methods (Particle Swarm, Tabu, Scatter), nature-inspired methods (Ant, Cuckoo), and evolutionary methods (Genetic, MO-EA);

\textit{iii)} we provide actual experimental results, unveiling trade-offs between performance (Accuracy/F-Measure) and computational time, at different scales (training set size).

Ultimately, our analysis shows the benefits linked to embedding the FS process into network analysis, providing a valuable tool for identifying the most useful features out of hundreds of possibilities. This will prove invaluable to the fields of network management, security management, intrusion detection and incident response. We should note that, the purpose of our comparative evaluation was not to claim the predominance of some FS algorithms over others but, rather, to suggest a methodical framework to work with FS.

As a byproduct of our investigation, some interesting open research directions emerge:
\textit{i)} extending the present analysis to \textit{unsupervised} FS techniques, which would be useful to deal with datasets lacking class labels, or with new types of (unknown) malicious traffic - this is the case of so called {zero-day} attacks that have no prior information; \textit{ii)} considering the case of streamed data analysis, which is necessary when dealing with extremely time-variant streams, whereby the FS process should be repeated across time (e.g. by using a mobile time window), so as to periodically update the resulting dataset with the freshest features;
\textit{iii)} designing routines to automatically manage the best FS strategies to be applied in accordance to specific criteria (e.g. accuracy target, latency needs, etc.).
Our investigation goes into the direction of the $6$G paradigm that, according to most network scientists, will be characterized by intelligent resource management, smart adjustments, and automatic service provisioning.

\printcredits

\bibliographystyle{IEEEtran}

redpajama_set_name: RedPajamaArXiv


text:
class GameMaterial : public Material
{
public:
	GameMaterial( char *name, char *path = "./" );
	virtual ~GameMaterial();

	Sound *GetStepSound();

private:
	LinkedList< Sound > *m_stepSounds;
};

#endif

redpajama_set_name: RedPajamaGithub


Soon Seng Truck & Parts Sdn Bhd, acclaimed as one of the reputable authorised Hino dealers in the southern region delivered 20 new trucks to Hitachi Transport System recently.
Hitachi Transport System (M) Sdn Bhd (Hitachi TS) recently took delivery of 10 units of the Hino 300 Series range of light-duty trucks and 10 units of the 500 Series range of medium-duty trucks at its new cold warehouse.
The vehicles were purchased from Soon Seng Truck & Parts Sdn Bhd, acclaimed as one of the reputable authorised Hino dealers in the southern region. It is said that Hitachi TS is impressed by Hino's commitment as a principal in supporting Soon Seng.
Hino Motors Sales (Malaysia) Sdn Bhd (HMSM) is committed to providing Hitachi TS with comprehensive training to its drivers. The wide service network provided by HMSM impressed Hitachi TS, with 110 authorised outlets to provides sales, service and spare-parts to customers. 'Excellent' after-sales service is hailed as one of HMSM's strength since its inception in Malaysia in 1977.
Incorporated in 1988, Hitachi TS currently has an operating fleet of 87 units with the support of 475 dedicated employees. The 5,000m2 new cold warehouse in Nilai, which commenced operations on 1st November, 2018, marks the entry of Hino into its cold chain logistics business.
The newly established facility aims to improve the efficiency in responding to customer's needs and requirements in cold chain distribution. Besides providing temperature controlled warehousing and delivery service that complemented with its security management system, Hitachi TS also aims to create value to customers by introducing SmartLogistics Technology into the logistics service it is providing.
The handing of a mock key by HMSM Managing Director Ken Iwamoto to Hitachi TS Managing Director Akihito Nii was witnessed by the management team and staff from Hitachi TS and HMSM.
With the Hino Total Support Customer Center in Sendayan, he said, the company was going one step further to deliver the best support to customers with the necessary training and safety driving techniques to 'skill up' their drivers and ensure their trucks were looked after well.


text:
package org.dominokit.domino.gwt.client.slots;

import elemental2.dom.*;
import elemental2.dom.EventListener;
import java.util.*;
import java.util.logging.Logger;
import jsinterop.base.Js;
import jsinterop.base.JsPropertyMap;
import org.dominokit.domino.api.client.mvp.slots.ContentSlot;
import org.dominokit.domino.api.client.mvp.slots.InvalidSlotException;
import org.dominokit.domino.api.client.mvp.slots.IsSlot;
import org.dominokit.domino.api.client.mvp.slots.SlotsManager;
import org.dominokit.domino.api.client.mvp.view.HasContent;

public class ElementsSlotsManager implements SlotsManager {

  public static final Logger LOGGER = Logger.getLogger(ElementsSlotsManager.class.getName());

  private static final Map<String, Deque<EventListener>> LISTENERS_SLOTS = new HashMap<>();
  private static final Map<String, Deque<IsSlot>> NORMAL_SLOTS = new HashMap<>();

  public ElementsSlotsManager() {
    DomGlobal.document.addEventListener(
        "domino-slot-added",
        evt -> {
          CustomEvent addEvent = Js.uncheckedCast(evt);
          JsPropertyMap<Object> eventDetails = Js.uncheckedCast(addEvent.detail);
          String slotKey = Js.uncheckedCast(eventDetails.get("slot-key"));
          EventListener slotListener = Js.uncheckedCast(eventDetails.get("slot-listener"));
          initSlotQueueIfNotExists(slotKey);
          if (!LISTENERS_SLOTS.get(slotKey).isEmpty()) {
            EventListener currentListener = LISTENERS_SLOTS.get(slotKey).peek();
            DomGlobal.document.removeEventListener(slotKey + "-slot-event", currentListener);
          }
          LISTENERS_SLOTS.get(slotKey).push(slotListener);
          DomGlobal.document.addEventListener(slotKey + "-slot-event", slotListener);
        });

    DomGlobal.document.addEventListener(
        "domino-slot-removed",
        evt -> {
          CustomEvent addEvent = Js.uncheckedCast(evt);
          JsPropertyMap<Object> eventDetails = Js.uncheckedCast(addEvent.detail);
          String slotKey = Js.uncheckedCast(eventDetails.get("slot-key"));
          EventListener slotListener = Js.uncheckedCast(eventDetails.get("slot-listener"));

          LISTENERS_SLOTS.get(slotKey).remove(slotListener);
          DomGlobal.document.removeEventListener(slotKey + "-slot-event", slotListener);

          if (!LISTENERS_SLOTS.get(slotKey).isEmpty()) {
            EventListener replacementListener = LISTENERS_SLOTS.get(slotKey).peek();
            DomGlobal.document.removeEventListener(slotKey + "-slot-event", replacementListener);
          }
        });
  }

  private void initSlotQueueIfNotExists(String slotKey) {
    if (!LISTENERS_SLOTS.containsKey(slotKey)) {
      LISTENERS_SLOTS.put(slotKey, new LinkedList<>());
    }
  }

  public void registerSlot(String key, IsSlot slot) {

    if (slot instanceof ElementSlot) {
      LOGGER.info(" >> REGISTERING GLOBAL SLOT [" + key + "]");
      if (!LISTENERS_SLOTS.containsKey(key.toLowerCase())) {
        LISTENERS_SLOTS.put(key.toLowerCase(), new LinkedList<>());
      }
      DomGlobal.document.dispatchEvent(slotEvent("domino-slot-added", key, Js.uncheckedCast(slot)));
    } else {
      LOGGER.info(" >> REGISTERING SLOT [" + key + "]");
      if (!NORMAL_SLOTS.containsKey(key.toLowerCase())) {
        NORMAL_SLOTS.put(key.toLowerCase(), new LinkedList<>());
      }
      NORMAL_SLOTS.get(key.toLowerCase()).push(slot);
    }
  }

  public void removeSlot(String key) {
    LOGGER.info(" << REMOVING SLOT [" + key + "]");
    if (LISTENERS_SLOTS.containsKey(key.toLowerCase())) {
      EventListener poppedOut = LISTENERS_SLOTS.get(key.toLowerCase()).pop();
      DomGlobal.document.dispatchEvent(slotEvent("domino-slot-removed", key, poppedOut));
    } else {
      NORMAL_SLOTS.get(key.toLowerCase()).pop();
    }
  }

  @Override
  public void revealView(String slotKey, HasContent view, HasContent.CreateHandler createHandler) {
    if (!LISTENERS_SLOTS.containsKey(slotKey) && !NORMAL_SLOTS.containsKey(slotKey)) {
      throw new InvalidSlotException(slotKey);
    }
    if (LISTENERS_SLOTS.containsKey(slotKey)) {
      HTMLElement viewElement = Js.uncheckedCast(view.getContent(createHandler).get());
      CustomEventInit slotEventDetails = CustomEventInit.create();
      slotEventDetails.setDetail(viewElement);

      CustomEvent slotEvent = new CustomEvent(slotKey + "-slot-event", slotEventDetails);
      DomGlobal.document.dispatchEvent(slotEvent);
    } else {
      ContentSlot slot = (ContentSlot) NORMAL_SLOTS.get(slotKey).peek();
      slot.updateContent(view, createHandler);
    }
  }

  private CustomEvent slotEvent(String eventType, String key, EventListener eventListener) {
    JsPropertyMap<Object> eventDetails = JsPropertyMap.of();
    eventDetails.set("slot-key", key);
    eventDetails.set("slot-listener", eventListener);

    CustomEventInit customEventInit = CustomEventInit.create();
    customEventInit.setDetail(eventDetails);

    return new CustomEvent(eventType, customEventInit);
  }
}


text:
Current: Darksiders 3 First DLC 'The Crucible' is Available Now
Darksiders 3 PS4 The Crucible THQ Nordic
Darksiders 3 First DLC 'The Crucible' is Available Now
Joe Apsey / February 26, 2019
Darksiders 3's first piece of DLC is now available to purchase. The Crucible brings back the titular wave-based arena mode that was present in Darksiders 2. The DLC can be purchased on the PlayStation Store for $6.99/€6.99/£5.99.
The Crucible offers 101 waves of enemies to slay. After every 5 waves you have the option to leave the arena with your rewards, or continue and fight more waves of enemies. After 25 waves you will then fight one of the "real" sins according to a press release. Defeating 100 waves will then offer up a final wave, offering the final challenge of the DLC.
New armor sets are available to earn in this mode, alongside crafting materials and enhancements that have never been seen before. You can check out the launch trailer for the DLC below:
Personally, I consider this to be fairly lacklustre new content for the game. The asking price also seems a little steep considering this will probably be fun for the first run through, but get old very quickly.
Darksiders 3 will also be receiving one more piece of DLC: the "Keepers of the Void" pack. This DLC will see "Fury travel into a rather charming-sounding place called 'The Serpent Holes' to remove the ancient threat that resides therein." We liked Darksiders 3 when it launched back in November. In our review stating the game "does a lot right but suffers from some technical issues that hold it back from greatness."
We produced a bunch of guides for the game, to help you through it. You can check out guides for the Abyssal Armor Puzzle Solution, all of the Humans in the game, the locations of The Chosen, and the locations of all the Weapon Enhancements.
Darksiders 3 and The Crucible DLC is now available on PlayStation 4. What do you think of this DLC? Is it worth the asking price?
PS4 / PS4 Pro
Dreams (PS4)

redpajama_set_name: RedPajamaCommonCrawl


text:
By Sacha Zimmerman Scoblic
Why is the pop star Pink the only artist tackling the tough issues surrounding feminism and alcohol?
It's hard to be a feminist when you're drunk. It's not that drinking isn't ladylike or that women shouldn't drink at all; my point is neither meant to be puritanical nor draconian. But how true to your own ideals, principles, and sense of self can you be if one is drowning that sense of self in booze?Whenever a woman hands her power to someone else--in this case, something else, alcohol--she is less than. "[A] woman exerting her power by making herself, to one degree or another, incapacitated," as a New York article by Alex Morris about young women and alcohol put it, is a disjunction, even if the women Morris interviewed did not want it to be so. And, once to the point of addiction, a woman is a slave to the bottle; what kind of feminist is a slave?
Still, in a pop landscape that turns drunk women into comedic icons, it's easy to see why bingeing might at least appear empowering. Every swilled cosmopolitan on "Sex in the City" is fetishized; indeed, the pink martini has become a kind of symbol of female liberation. (Never mind that actress Kristin Davis, who plays Charlotte, is an alcoholic who has been sober 18 years.) Meanwhile, every booze-soaked character actress on television, from Karen on "Will & Grace" to the ladies of "Ab Fab," makes alcoholism seem downright fun--if anti-intellectual. The gals on "How I Met Your Mother" are constantly drinking beer but are rarely shown drunk--unlike the Woo Girls of a recent episode (so named for their penchant to scream, "Wooo!" every time a drink is poured or a great song comes on). Comics Cheslea Handler and Amy Sedaris have best-selling books that celebrate their inebriation: Are You There, Vodka? It's Me, Chelsea and I Like You: Hospitality Under the Influence. But for every genuine laugh (these women are funny), there's a more insidious message: Getting wasted is part of being a cool, fun chick. The good-time-Sallies of recent pop history may be hilarious caricatures, but they aren't strong women. And the real-life versions are far less funny.
A year and a half ago, the U.S. seemed to reach the apotheosis of a pop culture "celebutante"/"prosti-tot" mania; even Newsweek, in a slimy play for sales, featured a boozy Britney Spears and Paris Hilton on its cover. The sloshed reality wasn't an arch wit swilling a Manhattan and spewing one-liners: It looked like Paris Hilton, who headed off to jail after too many DWIs; like a rail-thin Nicole Richie, tottering under the weight of her Starbucks cup, and piling up arrests; and like Lindsay Lohan, who made two trips to "rehab" in as many months. It was around that time when I first heard the song "Stupid Girls," by the pop star Pink. The video lampooned the heavily accessorized waifs, the surgically enhanced bombshells, and the hyper-sexualized behavior of both. The lyrics asked, "What happened to the dream of a girl president? She's dancing in the video next to 50 Cent. ... Oh where, oh where have the smart people gone?"
This fall, Pink released her fifth album, Funhouse, in which she turns her wicked eye on herself. The comedic sensibility of "Stupid Girls" is reprised in the video for "So What," Pink's tongue-in-cheek look at her own divorce ("So what? I'm still a rock star!"). But the most significant track on Funhouse is the moving ballad "Sober," about the singer's struggle to define herself. Pink's video artfully juxtaposes the person she is when inebriated and the person she really is, capturing the drunk's double identity. The first bars of "Sober" will be familiar to any present or former party girl:
"I don't want to be the girl that laughs the loudest
Or the girl who never wants to be alone
I don't want to be that girl at 4 o'clock in the morning
Cause I'm the only one you know in the world that won't be home"
Pink's video portrays the hackneyed moments of any drunken evening: inexplicable laughter, drunk dialing, stumbling into the room, listing over the toilet bowl ("When it's good, then it's good, it's so good, til it goes bad"). But the song's true subject is more substantial. It's about her struggle to know her sober self, and to resist the siren song of evening: "Night is calling. It whispers to me softly, 'Come and play.'" Scared of the silence left in the absence of a party, but just as scared of staying up until dawn yet again, Pink is left tortured by these halves of herself. Finally, a special effects-aided Pink is seen throwing another Pink onto a bed and kissing her, twin alter-egos with an intense desire for what the other represents: free and fun Pink; smart and savvy Pink.
It is a vivid depiction of what happens when an addict has to mourn the person alcohol made her. "I'm looking for myself sober," Pink belts outs in the chorus. What else can she do? The drunk party-girl in the video is out of control, lost within the party, half-dressed, confused, sick, passed out. It turns out being a party girl isn't all cosmos and snarky asides. Remarkably, Pink is perhaps the only pop-culture figure today who is actually struggling with the question of what it means to drink to excess and still be a powerful woman. Recognizing the impotency that comes with too much alcohol doesn't make Pink no fun; it makes her smart. And in the current climate, it makes her brave, too.
Sacha Zimmerman is a frequent contributor to The New Republic.
By Sacha Zimmerman
Politics, Charlotte, New York, Newsweek, Starbucks, Britney Spears, Amy Sedaris, Alex Morris, Comics Cheslea Handler, Lindsay Lohan, Kristin Davis, Funhouse, in, Nicole Richie, Paris Hilton, Stupid Girls, Will & Grace


screen_blanker.png (57.8 KB) - added by kutspam@… 13 years ago.
screen_blanker_bt.png (65.0 KB) - added by kutspam@… 13 years ago.
Could you provide a serial log ?
I forgot to tell that I had enabled "Fade now when mouse is here" and clicked in the low left corner and also enabled "Don't fade when mouse is here" to the upper right corner. It only crashes when I enable those features. No idea how to provide a serial log I have attached screenshots. How can I help with debugging useing screenshots and gdb?
Tested with today's Haiku image. Works correctly now.
+ Great thing that you have increased the range of pixels in the corners where the screensaver does or doesn't become active! In R5 only the corner pixel would work with the "Fade now when mouse is here" and "Don't fade when mouse is here"


PhillipLong . ZahiraSelket. bighellboy4r. KeishaKaneStar69.
SelenaGlamourNyushaSexyVadickDanielgabyperez .monikathebestVadickDanielJennydollJennylhadyGemini .POKER9Littlefox7Vyvicloeandsam .ISeeYouHoneyurlittleslutSummerSynnUrFEEDERlyka .LANDOBAARJosySummersTracceyAnneRedd .MONIKHOTTSwowlatinosMarianCandyIvyLeisurexo .RiversidexxjesietaylorjesietaylorVadickDaniel .VadickDanielLaurenVeeMAIKOLCUTEGUYkhett .AdalineBankseviedanceMaxLuterAdalineBanks .ShanonMatnaughtymistress32EroticAngelsToriEvans .BiancaWattersonxBarbieLove2CumxAnalToysAndPunisMagdalenaDiablo .
InnerFlameCarollinaBabyFRESHBRITGUYXkristiiiX .CumxForYouXDasModelarianasexiAdalineBanks .BiancaWattersonwitheskinAvaJansonDasModel .arianasexiurlittlesluturlittleslutVyvi .AlizeeMillsCuteFlorinForUNASTYdiamondBOYZahiraSelket .hotlizxxxBoyHornyVsBoyHotAvaJansonLaurieFlemine .JosySummersAngieTaylor1EthanKluma0JohnScordie .Kyle12EroticAngelsBoyHornyVsBoyHot0AnaLPREGGYx4U .ViickyyAnneReddRiversidexxxxSALOMExxx .eviedanceMsPerfectCOCKtsCarollinaBabyCurvyHelena4U .angelashantalMasterAchileTSCOUPLESEXDANYLOCUTEBOY .


To lure skilled educators to Carver Middle School, the district is providing additional pay totaling $70,000 per trainer over the subsequent three years, the largest district incentive ever for lecturers. Now computer systems spy on us and typing speeds betray feelings, it's touching that the smartest college students within the land still think they will offend in non-public. Calicut College: Know how benefit record is made, certificates required for admission, eligibility, courses, which part gets quota and other particulars.
College of Otago fears Govt changes to tertiary training might end in free-for-all. Youngsters who find work on a zero-hours contract after leaving faculty shouldn't be classed as having achieved a optimistic vacation spot" in Scottish Government statistics, Labour has claimed.
Enrollment declines in many Oklahoma rural school programs means a drop in state funding. The government has instructed the university that if the route is not complied with at the earliest, grants-in-support to the colleges will likely be withheld.
Now, faculty-choice advocates wish to problem funding bans in about three dozen states — including Idaho. Proficient Sheffield School college students have won a place within the grand final of a high profile catering and hospitality competitors. Olympic celebrity Cathy Freeman is celebrating 10 years of success together with her Cathy Freeman Basis, which has found great success in helping Indigenous youth acquire new ranges of schooling.
A FUND to help faculties serving poorer areas provide youngsters actions outdoors teaching hours is going through a large reduce. The reply is similar for the query: Do 'public faculties' serve all students?" Image credit score: Snopes. College students who appeared in the UP board exams this 12 months are also apprehensive in gentle of the current fiasco which was Bihar Board intermediate end result.