text:
\section{Introduction}

One of the fundamental topics in statistical learning theory is
the one of the \textit{generalization error} (GE). Given a training set and a hypothesis class, a learning algorithm chooses a hypothesis based on the training set in such a way that it minimizes an empirical loss. This loss, which is calculated on the training set, is also called the training loss and it often underestimates  the expected loss. The GE is the difference between the empirical loss and the expected loss.

There are various approaches in the literature that aim at bounding the GE via the complexity measures of the hypothesis class, such as the VC-dimenension \citep{Vapnik1991,Vapnik1999}, the fat-shattering dimension \citep{Alon1997}, and the Rademacher and the Gaussian complexities \citep{Bartlett2003}. Another line of work provides the GE bounds based on the stability of the algorithms, by measuring how sensitive is the output to the removal or change of a single training sample \citep{Bousquet2002}. Finally, there is a recent work by \cite{Xu2012a} that bounds the GE in terms of the notion of algorithmic robustness.

An important property of the (traditional) GE bounds is that they are distribution agnostic, i.e., they hold for any distribution on the sample space. Moreover, GE bounds can lead to a principled derivation of learning algorithms with GE guarantees, e.g., Support Vector Machine (SVM) \citep{Cortes1995} and its extension to non-linear classification with kernel machines \citep{Hofmann2008}.

However, the design of learning algorithms in practice does not rely only on the complexity measures of the hypothesis class, but it also relies on exploiting the underlying structure present in the data. A prominent example is associated with the field of computer vision where the features and learning algorithms are designed to be invariant to the intrinsic variability in the data \citep{Soatto2016}. Image classification is a particular computer vision task that requires representations that are invariant to various nuisances/transformations such as viewpoint and illumination variations commonly present in the set of natural images, but do not contain ``helpful information'' as to the identity of the classified object. This motivates us to develop a theory for learning algorithms that are invariant to certain sets of transformations.

The GE of invariant methods has been studied via the VC-dimension by \cite{Abu-Mostafa1993}, where it is shown that the subset of an hypothesis class that is invariant to certain transformations is smaller than the general hypothesis class. Therefore, it has a smaller VC-dimension. Yet, the authors do not provide any characterization of  how much smaller the VC-dimension of an invariant method might be. Similarly, group symmetry in data distribution was also explored in the problem of covariance estimation, where it is shown that leveraging group symmetry leads to gains in sample complexity of the covariance matrix estimation \citep{Soloveychik2014,Shah2012a}.

There are various other examples in the literature that aims to understand/leverage the role of invariance in data processing. For example, Convolutional Neural Networks (CNNs) -- which are known to achieve state of the art results in image recognition, speech recognition, and many other tasks \citep{LeCun2015} -- are known to possess certain invariances. The invariance in CNNs is achieved by careful design of the architecture so that it is (approximately) invariant to various transformations such as  rotation, scale and affine deformations \citep{Gens2014,Cohen2016,Dieleman2016}; or by training with augmented training set, meaning the training set is augmented with some transformed versions of the training samples,  so that the learned network is approximately invariant \citep{Simard2003a}. Another example of a translation invariant method is the scattering transform, which is a CNN-like transform based on wavelets and point-wise non-linearities \citep{Bruna2012}. See also \citep{Sifre2013,Wiatowski2015a}. In practice, such learning techniques achieve a lower GE than their ``non-invariant'' counterparts.

\cite{Poggio2012a} and \cite{Anselmi2014a,Anselmi2016}  study biologically plausible learning of invariant representations and connect their results to CNNs. The role of convolutions and pooling in the context of natural images is also studied by \cite{Cohen2016a}.

There are various works that that study the GE of CNNs \citep{Sokolic2016,Shalev-Shwartz2014,Neyshabur2015,Huang2015}, however, they do not establish any connection between the network's invariance and its GE.

Motivated by the above examples, this work proposes a theoretical framework to study the GE of invariant learning algorithms and shows that an invariant learning technique may have a much smaller GE than a non-invariant learning technique. Moreover, our work directly relates the difference in GE bounds to the size of the set of transformations that a learning algorithm is invariant to. Our approach is significantly different from  \citep{Abu-Mostafa1993} because it focuses on the complexity of the data, rather than on the complexity of the hypothesis class.

\subsection{Contributions}

The main contribution of this paper can be summarized as follows:

\vspace{0.cm}
\textit{
We prove that given a learning method invariant to a set of transformations of size $T$, the GE of this method may be up to a factor $\sqrt{T}$ smaller than the GE of a non-invariant learning method.
}
\vspace{0.cm}

Additionally, our other contributions include:
\begin{itemize}
	\item We define notions of stable invariant classifiers and provide GE bounds for such classifiers
	\item We establish a set of sufficient conditions that ensure that the bound of the GE of a stable invariant classifier is much smaller than the GE of a robust non-invariant classifier. We are not aware of any other works in the literature that achieve this;
	\item Our theory also suggests that explicitly enforcing invariance when training the networks should improve the generalization of the learning algorithm. The theoretical results are supported by experiments on the MNIST and CIFAR-10 datasets.
\end{itemize}
\section{Problem Statement}

We start by describing the problem of supervised learning and its associated GE. Then we define the notions of invariance in the classification task and the notion of an invariant algorithm.

\subsection{Generalization Error}

We consider learning a classifier from training samples. In particular, we assume that there is a probability distribution $P$ defined on the sample space $\sZ$ and that we have a training set drawn i.i.d. from $P$  denoted by $S_m =\{ s_i \}_{i=1}^m$, $s_i \in \sZ$, $i = 1, \ldots, m$. A learning algorithm $\sA$ takes the training set $S_m$ and maps it to a learned hypothesis $\sA_{S_m}$. The loss function of an hypothesis $\sA_{S_m}$ on the sample $z \in \sZ$ is denoted by $l(\sA_{S_m}, z)$. The  empirical loss and the expected loss of the learned hypothesis $\sA_{S_m}$ are defined as
\begin{IEEEeqnarray}{l}
	l_{\text{emp}}(\sA_{S_m}) = 1/m \sum_{s_i \in S_m} l \left(\sA_{S_m}, s_i \right) \quad \text{and} \\
	l_{\text{exp}}(\sA_{S_m}) = \E_{s \sim P} \left[ l\left( \sA_{S_m}, s \right) \right],
\end{IEEEeqnarray}
respectively; and the GE is defined as
\begin{IEEEeqnarray}{rCl}
	GE(\sA_{S_m}) = | l_{\text{emp}}(\sA_{S_m}) - l_{\text{exp}}(\sA_{S_m}) | \,. \label{eq:GE}
\end{IEEEeqnarray}

We consider a classification problem, where the sample space $\sZ = \sX \times \sY$ is a product of the input space $\sX$ and the label space $\sY$, where a vector $\bx \in \sX \subseteq \Ro{N}$ represents an observation that has a corresponding class label $y \in \sY = \{1,2,\ldots, N_{\sY}\}$. We will write $z = (\bx, y)$ and $s_i = (\bx_i, y_i)$.

\subsection{Stable Classifier and its Generalization}

The feature extractor (e.g., CNN) used in this work defines the non-linear function $f(\bx, \theta): \Ro{N} \to \Ro{N_\sY}$, where $N_\sY$ represents the number of classes, $N$ represents the dimension of the input signal, and $\theta$ represents the parameters of the feature extractor. The classifier defined by the feature extractor is then given as
\begin{IEEEeqnarray}{rCl}
	\argmax_{i \in [N_\sY]} (f(\bx, \theta))_i \,,
\end{IEEEeqnarray}
where, $(f(\bx, \theta))_i$ is the $i$-th element of $f(\bx, \theta)$. For example, this may correspond to a CNN with a softmax layer at the end. We will often write $f(\bx, \theta) = f(\bx)$, and define its Jacobian matrix as
\begin{IEEEeqnarray}{rCl}
	\bJ(\bx, \theta) = \frac{d f(\bx,\theta)}{d \bx} = \bJ(\bx) \,. \label{eq:jacobian_matrix}
\end{IEEEeqnarray}

A learning algorithm $\sA$ therefore returns a hypothesis, which is a function of the training set $S_m$,
\begin{IEEEeqnarray}{rCl}
	\sA_{S_m}(\bx) = \argmax_{i \in [N_\sY]} (f(\bx, \theta(S_m)))_i \,. \label{eq:classifier}
\end{IEEEeqnarray}
In a classification task, the goal of learning is to find an hypothesis that separates training samples from different classes. To model this we define the score of a training sample, which measures how confident the prediction of a classifier is:
\begin{definition}[Score]
Consider a training sample $s_i = (\bx_i, y_i)$. The score of training sample $s_i$ is defined as
\begin{IEEEeqnarray}{rCl}
o(s_i) &=& \min_{j \neq y_i} \sqrt{2} \left( (f(\bx_i))_{y_i} - (f(\bx_i))_{j} \right) \,. \label{eq:output_score}
\end{IEEEeqnarray}
\end{definition}
Note that a large score of training samples does not imply that the learned hypothesis will have a small GE. In this work we leverage the (non-invariant) GE bounds provided by \cite{Sokolic2016}. Before providing such bounds we define the notion of learning algorithm stability and the notion of covering number that are crucial for the GE bounds.

\begin{definition}[Stable learning algorithm]
Consider the algorithm $\sA$ and the hypothesis $\sA_{S_m}(\bx)$ given in \eqref{eq:classifier}. The learning algorithm $\sA$ is stable if for any training set $S_m$
\begin{IEEEeqnarray}{rCl}
	\max_{\bx \in \Rn} \| \bJ(\bx) \|_2 \leq 1  \,, \label{eq:stability_assumption}
\end{IEEEeqnarray}
where $\| \cdot| \|_2$ denotes the spectral norm.
\end{definition}
Stability of a learning algorithm defined in this way ensures that a learned classifier has a small GE as we shall see in Theorem~\ref{th:GE_noninvariant}.

We also need a measure of complexity/size of the input space $\sX$, which is given by the covering number.
\begin{definition}
Consider a space $\sX$ and a metric $d$. We say that the set $\sC$ is an $\epsilon$-cover of $\sX$ if $\forall \bx \in \sX$, $\exists \bx' \in \sC$ such that $d(\bx, \bx') \leq \epsilon$. The covering number of $\sX$ corresponds to the cardinality of the smallest $\sC$ that covers $\sX$. It is denoted by $\N(\sX; d,\epsilon)$.
\end{definition}
In this work we will assume that $d$ is the Euclidean metric: $	d(\bx, \bx') = \| \bx - \bx' \|_2$

Finally, we can provide the GE bounds for the stable learning algorithm. This is a variation of theorems 2 and 4 by \cite{Sokolic2016}.\footnote{\cite{Sokolic2016} also provide tighter GE bounds. For the sake of simplicity we use the bounds based on the spectral norm of the Jacobian matrix.}
\begin{theorem} \label{th:GE_noninvariant}
	Assume that the learning algorithm $\sA$ is stable and that there exists a constant $\gamma$ such that
	\begin{IEEEeqnarray}{rCl}
		o(s_i) \geq \gamma \quad \forall s_i \in S_m \,.
	\end{IEEEeqnarray}
Assume also that the loss $l(\cdot)$ is the 0-1 loss. Then, for any $\delta > 0$, with probability at least $1- \delta$,
	\begin{IEEEeqnarray}{rCl}
	GE(\sA_{S_m}) &\leq& \sqrt{\frac{2 \log(2) \cdot N_\sY \cdot \N(\sX; d, \gamma / 2 ) }{ m}} \nonumber \\ && + \sqrt{ \frac{2 \log(1/\delta)}{m}}   \,.  \label{eq:GE_bound}
	\end{IEEEeqnarray}
\begin{proof}
The proof is straightforward by the application of theorems 2 and 4 by \cite{Sokolic2016}.
\end{proof}
\end{theorem}
The GE therefore approaches zero with rate $1/\sqrt{m}$ and it depends on the number of classes via $\sqrt{N_\sY}$. Critical is  the dependence on the covering number $\N(\sX; d, \gamma / 2 )$, which is a function of the input space $\sX$ and the margin $\gamma$.

\subsection{Structured Input Space and Invariant Algorithms}

The bound of the GE provided in the previous section depends on the covering of the input space $\sX$. As noted in the introduction, $\sX$ often exhibits symmetries that may reduce its ``effective'' complexity and therefore also reduce the GE. We formalize this intuition in this section.

To capture the additional structure present in the data, we model the input space $\sX$ as a product of a base space $\sX_0$ and a set of transformations $\sT$:
\begin{IEEEeqnarray}{rCl}
	\sX = \sT \times \sX_0  \coloneqq \{t(\bx): t \in \sT, \bx \in \sX_0 \} \,, \label{eq:Xdecomposition}
\end{IEEEeqnarray}
where $\sX_0 \subseteq \Rn$, $\sT = \{ t_0, t_2, \ldots t_{T-1} \}$ and $T$ corresponds to the size of $\sT$.\footnote{Note that the discrete representation of this set is not limiting in practice.} We assume $t_0$ to be the identity, i.e., $t_0(\bx) = \bx$ throughout this work. For example, if $\sX_0$ is a set of images and $\sT$ is a set of translations, then $\sX$ will be the set of images with all possible translations of the images in $\sX_0$. See also Figure~\ref{fig:set_decomposition}.

We assume that  the classification task is invariant to the set of transformations $\sT$, i.e., we are really interested only in the set $\sX_0$ but have access to transformed samples of it, where clearly all of them have the same label. In other words, the class labels of $t(\bx)$ are the same for all $t \in \sT$. In this case, it is reasonable to leverage this by using an invariant learning algorithm.\footnote{Here we define a notion of absolute invariance. It is easy to extend it to approximate invariance, where in $\sX$ we have transformed versions of $\sX_0$ plus small/bounded noise; and  also to extend the GE bounds in a similar manner for approximately invariant learning algorithms.}
\begin{definition}[Invariant algorithm]
A learning algorithm $\sA$ is invariant to the set of transformations $\sT$ if the embedding is invariant:
\begin{IEEEeqnarray}{rCl}
	f(t_i(\bx), S_m) = f(t_j(\bx), S_m) \quad  \forall \bx \in \sX_0,  t_i, t_j \in \sT \,, \nonumber \\ \*
\end{IEEEeqnarray}
for any training set $S_m$. We will denote such learning algorithm  by $\sA_{S_m}^\sT$.
\end{definition}
This leads us to the question that will occupy us throughout this paper:  what is the GE of an invariant learning algorithm.

\begin{figure}[t]
\vspace{-0.1cm}
\centering
	\subfigure[Input space.]{
	\includegraphics[height=0.5\linewidth]{setX.pdf}}
	\subfigure[Input space decomposition.]{
	\includegraphics[height=0.5\linewidth]{setX_partition.pdf}}
	\caption{Theorem 1 shows that the size of the input space $\sX$ determines the GE of a stable learning algorithm. The input space can often be constructed as a product of a simpler base space $\sX_0$ and a set of transformations $\sT$, where the transformations in $\sT$ preserve the class labels. Theorem 2 shows that the GE of an invariant stable learning algorithm is determined by the size of the base space $\sX_0$. The size of the base space $\sX_0$ can be much smaller than the size of the input space $\sX$.} \label{fig:set_decomposition}
\vspace{-0.2cm}
\end{figure}

\section{Generalization Error of Invariant Classifiers}

In this section we provide bounds to the GE of invariant algorithms. The invariance of the learning method induces a possibly more efficient covering of the input space $\sX$, which translates into a lower GE.

The GE of invariant and stable learning algorithms can be bounded as follows:

\begin{theorem} \label{th:GE_invariant}
	Assume that the learning algorithm $\sA$ is stable and invariant to $\sT$ and that there exists a constant $\gamma$ such that
	\begin{IEEEeqnarray}{rCl}
		o(s_i) \geq \gamma \quad \forall s_i \in S_m \,.
	\end{IEEEeqnarray}
Assume also that the loss $l(\cdot)$ is the 0-1 loss. Then, for any $\delta > 0$, with probability at least $1- \delta$,
	\begin{IEEEeqnarray}{rCl}
		GE(\sA_{S_m}^\sT) &\leq& \sqrt{\frac{2 \log(2) \cdot N_\sY \cdot \N(\sX_0; d, \gamma / 2 ) }{ m}} \nonumber \\ && + \sqrt{ \frac{2 \log(1/\delta)}{m}}   \,.  \label{eq:GE_bound_inv}
	\end{IEEEeqnarray}
\begin{proof}
We show that under the assumptions of this theorem the learning algorithms is $(\N(\sX_0; d, \gamma/2), 0)$-robust (see \citep{Xu2012a} or \citep{Sokolic2016}). The GE bound then follows from Theorem 3 and Example 9 by \cite{Xu2012a} (or theorems 1 and 2 by \cite{Sokolic2016}).

We construct a covering as follows. Take the covering that leads to the covering number $\N(\sX_0; d, \gamma/2)$ and denote the subsets of $\sX_0$ by $\sK_i$, $i = 1, \ldots, \N(\sX_0; d, \gamma/2)$. By the definition of $\sX$ in \eqref{eq:Xdecomposition} we can cover $\sX$ by $\N(\sX_0; d, \gamma/2)$ sets of the form $\sT \times \sK_i$, $i = 1, \ldots, \N(\sX_0; d, \gamma/2)$.

Now take $\bx_i$ in the training set and $\bx \in \sX$ such that $\bx_i, \bx \in \sT \times \sK_j$. Due to the invariance of $f$ we have $\| f(\bx_i) - f(\bx) \|_2 < \gamma$ and all $\bx$ will lie in the same decision region as $\bx_i$. This implies that stable and invariant learning algorithm is  $(\N(\sX_0; d, \gamma/2), 0)$-robust. The GE bound follows from Theorem 3 by \cite{Xu2012a}.
\end{proof}
\end{theorem}

Note that the GE bound in Theorem~\ref{th:GE_invariant} is of the same form as the GE bound in Theorem~\ref{th:GE_noninvariant} and the main difference is in the employed covering number. In particular, the ratio between the bounds is
\begin{IEEEeqnarray}{rCl}
	R(\sX_0, \sX; d, \epsilon) = \left( \frac{ \N(\sX_0; d, \epsilon )}{ \N(\sX; d, \epsilon )} \right)^{1/2} \,,
\end{IEEEeqnarray}
where $\epsilon =  \gamma /2$ in our case. We are especially interested in the scenarios where the GE bound of an invariant method is much smaller than the GE bound of a non-invariant method. This happens when \mbox{$R(\sX_0, \sX; d , \epsilon) \ll 1$}. We now establish a set of sufficient conditions on $\sX_0$, $\sT$, $d$ and $\epsilon$ such that \mbox{$R(\sX_0, \sX; d, \epsilon) \ll 1$}.

\begin{theorem} \label{th:ratio_bound}
	Assume that $\sX = \sT \times \sX_0$ and choose $\epsilon < 1$. Then
\begin{IEEEeqnarray}{c}
d(t(\bx), t'(\bx')) > 2 \epsilon \quad \forall \bx, \bx' \in \sX_0, t \neq t' \in \sT  \label{eq:covering_condition} \\
	\text{and} \nonumber \\
d(t(\bx), t(\bx')) \geq d(\bx, \bx') \quad \forall \bx, \bx' \in \sX_0, t  \in \sT \label{eq:isometry_condition} \\
\implies
R(\sX_0, \sX; d, \epsilon) \leq 1/ \sqrt{T} \,, \label{eq:ratio_bound}
	\end{IEEEeqnarray}

where $T$ is the number of elements in $\sT$. On the other hand,
\begin{IEEEeqnarray}{C}
d(t(\bx), t'(\bx)) = 0 \quad \forall \bx \in \sX_0, t \neq t' \in \sT \label{eq:same_condition}  \\
\implies  R(\sX_0, \sX; d, \epsilon) = 1. \label{eq:ratio_bound_2}
\end{IEEEeqnarray}
\begin{proof}
Consider any covering of $\sX_0$ that leads to the covering number $\N(\sX_0; d, \epsilon )$. Denote the metric balls of radius $\epsilon$ that cover $\sX_0$ by $\sC_i$, $i = 1, \ldots, \N(\sX_0; d, \epsilon )$.  Denote the elements of $\sT$ as $t_j$, $j = 1, \ldots, T$ and the transformed sets by $t_j(\sX_0) = \{t_j(\bx) : \bx \in \sX_0 \}$, $j = 1, \ldots, T$.

First, we show that \eqref{eq:covering_condition} implies that any possible metric ball of radius $\epsilon$ can only have non-empty intersection with one of the ``copies'' of $\sX_0$. Denote by $\sB$ an arbitrary metric ball of radius $\epsilon$. Then
\begin{IEEEeqnarray}{rCl}
	\sB \cap t_j(\sX_0) \not= \emptyset \implies 	\sB \cap t_k(\sX_0) = \emptyset \quad \forall k \neq j.
\end{IEEEeqnarray}
To see this, observe that the definition of $\sB$ implies  that $d(\bx, \bx') \leq 2 \epsilon$, $\forall \bx, \bx' \in \sB$. Now take a point $\bx \in \sB \cap t_j(\sX_0)$ and a point $\bx' \in t_k(\sX_0)$, $k \neq j$. Note that by \eqref{eq:covering_condition} $d(\bx, \bx') > 2 \epsilon$, which implies that $\bx' \not \in \sB $ and therefore $\sB \cap t_k(\sX_0) = \emptyset$. This implies that the  covering number of $\sX$ with metric ball of radius $\epsilon$ is
\begin{IEEEeqnarray}{rCl}
	\N(\sX; d, \epsilon) = \sum_{j=1}^T \N(t_j(\sX_0); d, \epsilon) \,.
\end{IEEEeqnarray}
Finally, it remains to be proven that $\N(t_j(\sX_0); d, \epsilon) \geq \N(\sX_0; d, \epsilon)$ $\forall t_j \in \sT$, which is straightforward to establish given the condition  \eqref{eq:isometry_condition}. This proves \eqref{eq:ratio_bound}. Proof of \eqref{eq:ratio_bound_2} is trivial as $\sX_0 =  \sX$ when \eqref{eq:same_condition} holds.
\end{proof}
\end{theorem}

We have shown, via conditions on the geometry of the base space $\sX_0$, and the effect of transformations in $\sT$ on it, that the ratio $R(\sX_0, \sX; d, \epsilon)$ can be smaller or equal to $1/\sqrt{T}$. Note that conditions \eqref{eq:covering_condition} and \eqref{eq:isometry_condition} ensure that the effect of transformations in $\sT$ can not be captured by the metric $d$. Otherwise, the invariant algorithm has no advantage over a non-invariant one (this is illustrated by examples in Section~\ref{sec:examples}):
\begin{itemize}
	\item The sufficient condition in  \eqref{eq:covering_condition} can be stated as follows. Take any pair of vectors in the base space $\sX_0$ and transform them by the two transformations in $\sT$ that are not equal. Then the distance between the pair of vectors must be at least $2 \epsilon$. In other words, the transformation must not make two distinct vectors in the base space $\sX_0$ (distance at least $2 \epsilon$) indistinguishable (distance smaller than $2 \epsilon$). Similarly, any two transformations in $\sT$ that are not equal must make two similar vectors in $\sX_0$ (with distance smaller than $2 \epsilon$) distinct  (distance at least $2 \epsilon$).

	\item The sufficient condition in  \eqref{eq:isometry_condition} ensures that the transformations in $\sT$ are not trivial, i.e., they do not reduce the complexity of the base space $\sX_0$. For example, a transformation that maps any $\bx \in \sX_0$ into itself violates  \eqref{eq:isometry_condition} and leads to a set of the same complexity, as formalized by \eqref{eq:same_condition}.
\end{itemize}

The results of this section can be summarized by the following remark:
\begin{remark}
Given an input space  $\sX$, which is structured according to the assumptions of Theorem~\ref{th:ratio_bound} and the size of transformation set $T$, we have established that the GE of an invariant stable learning algorithm may be up to a factor $\sqrt{T}$ smaller than the GE of a non-invariant stable learning algorithm. To the best of our knowledge, this is the first time such quantitative result is provided for invariant algorithms.

\end{remark}

\subsection{Illustration} \label{sec:examples}
To provide additional intuition related to Theorem 3, we present the following toy example. We consider four images of dimension $N \times N$, with $N = 16$, Figure~\ref{fig:atoms}(a). The sets of transformations that we consider are:
\begin{itemize}
	\item Translation set: The set of pixel-wise cyclic translations in any direction. The size of the set is $N^2$.

	\item Rotation set: The set of image rotations by $90^\circ$. The size of this set is 4 (this may explain why the $90^\circ$ rotation invariance is useful but not as critical as the translation invariance).

	\item Trans-rotation set: A product of the translation and the rotation sets, where the rotation is applied first followed by a translation. The size of this set is $4 \times N^2$.
\end{itemize}
Note that all the transformations above can be implemented by permutation matrices which are orthonormal. This is important as it implies that all the considered sets  satisfy the condition in \eqref{eq:isometry_condition}. Examples of transformed atoms are shown in Figure~\ref{fig:atoms}(b).

We now provide an example of a base space $\sX_0$ and a transformation set $\sT$ for which $R(\sX_0, \sX; d, \epsilon) \leq 1/\sqrt{T}$; and then provide an example of a base space $\sX_0$ and a transformation set $\sT$ for which $R(\sX_0, \sX; d, \epsilon) \not \leq 1/\sqrt{T}$.

\begin{figure}[t]
	\centering
	\subfigure[Atoms.]{
	\includegraphics[height=0.4\linewidth]{shapes.pdf}}
	\hspace{1cm}
	\subfigure[Transformed atoms.]{
	\includegraphics[height=0.4\linewidth]{shapes_trans.pdf}}
	\caption{(a) A set of atoms (\textit{cross}, \textit{circle}, \textit{corner}, \textit{curve}) used to construct the base space. (b) Examples of transformed atoms with a transformation from the trans-rotation set.} \label{fig:atoms}
\vspace{-0.1cm}
\end{figure}

\paragraph{Example for $R(\sX_0, \sX; d, \epsilon) \leq 1/\sqrt{T}$:}

Consider $\sX_0 = \{cross, circle, corner, curve \}$ and $\sT$ to be the translation set. The set $\sX = \sT \times \sX_0$ then contains all possible translations of shapes in $\sX_0$. We have verified numerically that the condition in \eqref{eq:covering_condition} is satisfied for all $\epsilon < 0.375$. Therefore, $R(\sX_0, \sX; d, \epsilon) \leq 1/\sqrt{T}$ for $\epsilon < 0.375$, where $\sqrt{T} = N = 16$ is the dimension of the images. Therefore, a translation invariant learning method can attain a GE with a factor $N$ smaller than the GE of a non-invariant method.

Similarly, if we take $\sX_0 = \{corner, curve \}$ and $\sT$ to be the trans-rotation set,  we can establish  $R(\sX_0, \sX; d, \epsilon) \leq 1/(2 N)$ for $\epsilon < 0.26$.

\paragraph{Examples for $R(\sX_0, \sX; d, \epsilon) \not \leq 1/\sqrt{T}$:}

Now consider $\sX_0 = \{cross, circle \}$ and $\sT$ to be the rotation set. Therefore, $\sX = \sT \times \sX_0$ contains all possible $90^\circ$ rotations of \textit{circle} and \textit{cross} in Figure~\ref{fig:atoms}(a). It is clear that the \textit{circle} and \textit{cross} are already invariant to such rotation, i.e., they corresponds to exactly the same shape. Therefore, the condition in \eqref{eq:same_condition} holds and $R(\sX_0, \sX; d, \epsilon) = 1$. Clearly, in such cases, an invariant learning algorithm is not expected to have a smaller GE than a non-invariant learning algorithm.

\section{Invariant CNNs}

In this section we discuss the implication of our theory on CNNs, which are very popular for classification. Note that this is one particular example and that our theory holds also for other possible classifiers.  We consider two ways for which invariance can be achieved for CNNs: via an appropriate construction of the CNN architecture or by training them to be invariant.

\paragraph{Invariance of the CNN Architecture-}

Given that the set of transformations is a group, averaging a function over a group leads to an invariant representation \citep{Anselmi2014a,Bruna2012,Cohen2016}. For example, in conventional CNNs the pooling operators usually average over the translation group and make the CNNs translation invariant.

\cite{Cohen2016} generalize the notion of convolution over translation group to general groups, which leads to architectures that can be invariant to arbitrary transformations that form discrete groups. A related approach involves normalization of the network input, which eliminates the effect of affine transformations of the input \citep{Jaderberg2015}.

\paragraph{Invariance of the CNN Learning-}

As an alternative to encoding the invariances in the CNN architecture we can train a CNN to become invariant.  This is particularly helpful in the cases that we do not know exactly how to characterize or impose the invariance manually on the network. Such an ``approximate invariance'' is achieved by training CNNs with data augmentation, which involves training the network with the transformed samples of the training examples. This was indicated by \cite{Lenc2015}, who showed that CNNs trained on the ImageNet \textit{implicitly} learn to be invariant to flips, scalings and rotations.

Our theory suggests that enforcing the invariance of the CNN representation \textit{explicitly} should improve the robustness of CNNs and improve their GE. For example, we may train networks with an explicit regularization term of the form
\begin{IEEEeqnarray}{rCl}
	\sum_{t \in \sT} \| f(\bx_i) - f(t(\bx_i)) \|_2^2 \,, \label{eq:invariance_regularizer}
\end{IEEEeqnarray}
which promotes the invariance of the representation. We validate the effectiveness of this regularization in Section~\ref{sec:experiments_cifar}.

\section{Experiments} \label{sec:experiments}

We now demonstrate the theoretical results with experiments on the MNIST and CIFAR-10 datasets.

\subsection{Rotation Invariant CNN}

Here we compare a rotation invariant CNN and a conventional CNN on  rotated MNIST datasetses. The rotated MNIST-$D^\circ$ dataset is constructed by rotating the digits by an angle $r \cdot D^\circ$, $r \in \{0,1,2,\ldots, 360/D - 1\}$, where the index $r$ is chosen randomly for each image in the dataset. We use $D = 180, 90, 45$.

We use a 7 layer CNN architecture: $(32,5,5)$-conv, $(2,2)$-max-pool, $(64,5,5)$-conv, $(2,2)$-max-pool, $(128,5,5)$-conv followed by a global average pooling layer and a softmax layer, where $(k,u,v)$-conv denotes the convolutional layer with $k$ filters of size $u \times v$, and $(p,p)$-max-pool denotes the max-pooling layer with pooling regions of size $p \times p$. The rotation invariant CNN is the same as the conventional CNN, but it includes a cyclic slice layer before the first convolutional layer and a cyclic pool layer before the softmax layer. Both, the cyclic slice layer and the cyclic pool layer were proposed by \cite{Dieleman2016} and together they ensure that the CNN is invariant to rotations. In particular, the cyclic slice layer takes input image $\bx$ and creates copies of $\bx$, each rotated for $r \cdot D^\circ$, $r = 0,1,2,\ldots, 360/D - 1$, where $D$ is the same as in the dataset MNIST-$D^\circ$. The copies are then passed through the CNN independently. At the end of the CNN, before the softmax layer, the outputs of the copies are averaged by a cyclic pool layer to obtain a rotation invariant representation.

The networks are trained using stochastic gradient descent (SGD) with momentum, which was set to $0.9$. The training objective is the standard categorical cross entropy (CCE) loss. Batch size was set to 32 and learning rate was set to $0.01$ and reduced by 10 after 100 epochs. The networks were trained for 150 epochs in total. Weight decay regularization was set to $10^{-4}$. We used training sets of sizes $10^3$, $10^4$, $2\cdot 10^4$, $5\cdot 10^4$.

\begin{figure*}[t]
	\centering
	\subfigure[Test set classification acc.]{
	\includegraphics[height=0.19\paperwidth]{mnist_test_acc.eps}}
	\hspace{-0.2cm}
	\subfigure[Generalization error.]{
	\includegraphics[height=0.19\paperwidth]{mnist_GE.eps}}
	\hspace{-0.2cm}
	\subfigure[Generalization error ratio.]{
	\includegraphics[height=0.19\paperwidth]{mnist_GE_ratio.eps}}
\vspace{-0.2cm}
\caption{(a) Clasification accuracy, (b) the GE of the rot. invariant CNN and the conventional CNN and (c) the ratio of the GEs of the rotation invariant CNN and the conventional CNN on the rotated MNIST datasets. } \label{fig:mnist_ro}
\vspace{-0.1cm}
\end{figure*}

The classification accuracies are reported in Figure~\ref{fig:mnist_ro}(a), the GE is reported in  Figure~\ref{fig:mnist_ro}(b) and the ratio of the GEs of the invariant and the conventional CNNs are shown in Figure~\ref{fig:mnist_ro}(c). We may note that the (explicitly) rotation invariant CNN always has a higher classification accuracy than the conventional CNN. Moreover, the GE of the rotation invariant CNN is much smaller than the GE of the conventional CNN. The difference is most significant when the training set is small, which demonstrates the importance of invariance for the generalization of learning algorithms.

Note also that the GE of the rotation invariant CNNs on different datasets MNIST-$D^\circ$, $D=180,90,45$, is roughly the same, whereas the conventional CNNs have a higher GE on the datasets with a smaller $D$. This can be explained by the fact that the rotated MNIST dataset with a smaller $D$ is more complex due to the larger number of rotations. The sizes of the transformation sets for $D=180,90,45$ are $2,4$ and 8, respectively. Theorem~\ref{th:ratio_bound} predicts that the ratio of the GEs of an invariant and a non-invariant CNNs is equal to $\sqrt{|\sT|}$. The actual ratios are shown in Figure~\ref{fig:mnist_ro}(c). We can observe that the GE ratios obtained empirically roughly follow the theoretical prediction. However, when the training set is small, the conventional CNN generalizes worse than predicted by our theory and when the training set is large, the conventional CNN generalizes better than predicted by our theory. We conjecture that the conventional CNNs learn to be ``partially'' invariant when the number of training samples is large. Moreover, the current theory might not capture the relationship between invariant and non-invariant CNNs entirely, especially when the assumptions of Theorem 3 do not hold.

Finally, we also consider the rotation invariant MNIST dataset, where each image $\bx$ in the dataset is rotated by $r \cdot D^\circ$, $r \in \{0,1,2,\ldots, 360/D - 1\}$ and the $360/D$ copies are averaged to obtain a sample. As our theory suggests, the rotation invariant CNNs in this case do not have a lower GE than a conventional CNN because the dataset itself is rotation invariant. In fact, given the rotation invariant MNIST dataset, the rotation invariant CNN and the conventional CNN are equivalent. This can be easily established by observing that the cyclic slicing layer produces copies of the input that are identical. We have verified empirically that the rotation invariant and the non-invariant CNNs perform the same on the rotation invariant MNIST dataset.

\subsection{Learning the Invariances} \label{sec:experiments_cifar}

Finally, we demonstrate that learning invariances explicitly can lead to a lower GE. We use the CIFAR-10  dataset, which is normalized following \citep{Zagoruyko2016}, and the Wide ResNet \citep{Zagoruyko2016} with 13 layers of width 5.

The networks are trained using SGD and the learning rate is set to 0.01 for the first epoch and then to $0.05$, $0.005$ and $0.0005$, each for 30 epochs. We use $10^3$, $10^4$, $2\cdot 10^4$ and $5\cdot 10^4$ training samples. We have found that using the Jacobian regularization \citep{Sokolic2016} improves performance in all cases and it's factor is set to $0.1$ with smaller training sets (2500, 5000, 10000) and  $0.05$ otherwise. Batch size is set to 128.

SGD batches are constructed as follows: the first half of the batch contains images from the training set and the other half of the mini batch contains transformed versions of the images in the first half of the mini batch where the transformations are chosen at random. The set of transformations contains shifts of $\pm4$ pixels and horizontal flips, as in \citep{Zagoruyko2016}.

We promote the invariance by using the regularizer in \eqref{eq:invariance_regularizer}. We chose to regularize the output of the last global pooling layer instead of the softmax output and use the corresponding pairs from the batch to compute \eqref{eq:invariance_regularizer}. The regularization factor in all experiments is set to $10^{-4}$.

Table~\ref{tab:cifar} reports the standard test accuracy and the accuracy of the predictions averaged over the augmented test set (denoted by + avg.), which are obtained as follows: for each test image we average the softmax outputs for the original image, shifted images ($9 \times 9$ shifts), horizontally flipped image and scaled images (scaling factors are $0.8$ and $1.2$). Note that this method requires approximately 80 forward passes through a network to obtain a prediction.

Classification accuracies on the test set and on the augmented test set for CNNs trained with invariance regularization and for CNNs trained without the invariance regularization are reported in Table~\ref{tab:cifar}. The training set accuracies were \mbox{100\%} or very close to \mbox{100\%} in all cases. First, we observe that invariance regularization leads to a lower GE (a higher accuracy) in all cases. Moreover, testing with the augmented test set is even more robust and leads to a lower GE for both, the regularized and the non-regularized CNNs. Note however, that CNNs trained with explicit invariance regularization  (except when 2500 training samples are used) performs better or on par with a non-regularized network evaluated on the augmented test set, where testing with the augmented test set is approximately 80 times more expensive than conventional testing with a single image. This experiment verifies the hypothesis that enforcing the network invariance explicitly can lead to a smaller GE.

The ratio of the GEs of the CNN trained with data augmentation and the invariance regularization and the CNN trained without data augmentation are between $1.5$ and $2$. Note that the theory from Section~3 does not apply directly as (i) the CNN trained without data augmentation is already (partially) invariant to translations due to its convolutional structure with pooling \citep{Bruna2013b, Boureau2010}; (ii) the CNNs trained with data augmentation and invariance regularization are not perfectly invariant as defined in Definition~4, but only approximately invariant.

\begin{table}[t]
{
\centering
\caption{Classification accuracy $[\%]$ on CIFAR-10. } \label{tab:cifar}
\begin{tabular}{lccccc}
\toprule
& \multicolumn{5}{c}{number of training samples} \\
 & 2500 & 5000 & 10000 & 20000 & 50000   \\

\cmidrule(r){2-6

No reg.  & 68.71 & 76.74 & 85.17 & 87.15 & 93.65   \\

Inv. Reg. & 69.32 & 79.08 & 86.69 & 88.14 & 94.50 \\

\midrule

{\tiny\begin{tabular}{@{}c@{}}No reg. \\ + avg. \end{tabular}} & 70.59 & 78.40 & 86.05 & 88.13 & 94.26 \\

{\tiny\begin{tabular}{@{}c@{}}Inv. Reg. \\ + avg. \end{tabular}} & 70.71 & 79.65 & 86.96 & 88.98 & 94.78   \\
\bottomrule
\end{tabular}
}
\vspace{-0.2cm}
\end{table}

\section{Discussion and Conclusion}

We have formally demonstrated that the GE of an invariant learning algorithm can be much smaller than the GE of a non-invariant learning algorithm, provided that the input space can be  factorized into a product of a transformation set and a base space, where the covering number of the base space is much smaller than the covering number of the input space. This work offers an important foundation for the study of the GE of learning algorithms, such as CNNs and their extensions, that
leverage symmetries in the data.

Our assumption in this work is that the set of transformations $\sT$ is discrete. A more general approach would be to assume that the set of transformations $\sT$ is continuous. We conjecture that current results can be extended to such cases by an appropriate covering of $\sT$. Second, we have assumed that a learning method is perfectly invariant. The notion can be extended to approximately invariant learning methods and bounds of the same form as in \eqref{eq:GE_bound_inv} can be derived for this case.
\subsubsection*{Acknowledgements}
The work of J. Sokoli\'c and M. R. D. Rodrigues was supported in part by EPSRC under grant EP/K033166/1. The work of R. Giryes was supported in part by GIF. The work of G. Sapiro was supported in part by NSF, ONR, ARO, and NGA.

\clearpage
\subsubsection*{References}
\renewcommand{\section}[2]{}%
\bibliographystyle{abbrvnat_jure}

redpajama_set_name: RedPajamaArXiv


Pack Meeting Tuesday, January 31 at 7:00 p.m.
Our next Pack meeting will be Tuesday, January 31, at St. James' Episcopal Church, starting at 7:00 p.m.
Join Pack 163 as we march with other Fremont Cub Scouts, Boy Scouts, and Varsity Scouts, and Venture Scouts and families in the annual Fremont 4th of July parade!
Our staging area is the Fremont Public Library (main branch) where we will meet at 9:00 a.m.
The parade starts at 10:00 a.m. Scouts should be in their full Class A uniforms, and the entire family can join us as we march through Fremont. Be sure to wear plenty of sunscreen, BRING WATER, and wear a hat!
The parade will begin at 10:00 a.m. and start at Paseo Padre Pkwy & Stevenson Boulevardd and finish at Capitol Avenue.
Fremont is preparing for its 18th annual Fremont 4th of July Parade. The parade will roll through downtown with colorful floats, large balloons, marching bands, antique vehicles, dance groups, and much more. This year's theme is "Everyday Heroes" and will have a special tribute to honor our brave men and women of Fremont Police and Fire departments. This year's Grand Marshal is the Candle Lighters organization, honoring their 48 years of serving Fremont. The parade kicks off with the singing of our National Anthem by Irvington High School junior Cathryn Flores followed by a precision flyover by the California Beechboys.
The parade committee has a new, improved website: www.fremont4th.org as well as a phone app with the latest updates including entries, judges, volunteering, sponsoring and more. It is sponsored by Fremont Chevrolet, Northwestern Polytechnic University, Robson Homes, Washington Hospital Healthcare Foundation, and Fremont Police Officers Association to name a few.
The parade committee is still accepting donations and sponsorships, as well as seeking volunteers. Please visit www.fremont4th.org for more information. The Fremont 4th of July Parade is a local, nonprofit 501(c)3 community organization, not affiliated with the city, and responsible for the production and financing of the parade.
Join the Pack 163 Webelos I scouts as they pull invasive weeds to help make our parks beautiful. They will be meeting at the COYOTE HILLS VISITOR CENTER, Saturday, May 28, from 10:00 a.m. to 11:30 a.m.
USS Hornet: 50 Pack 163 scouts and family members took part in the overnight "Live Aboard" program.
Our home for the next day, the USS Hornet.
The USS Hornet is on display in Alameda, CA.
Pack 163 scouts assembled for ship orientation.
Some Pack 163 scouts sitting on a torpedo in the torpedo room.
Pack 163 scouts ride an aerial torpedo.
The USS Horent's huge anchor chains in the forcastle.
The USS Hornet control tower and bridge, seen from the flight deck.
The USS Hornet ship's bell.
Apollo command module CM-011A was flown on the suborbital flight AS-202 mission in 1966 to test the Service Module engines and the re-entry heat shield and parachutes. After splashdown it was recovered by the USS Hornet.
Next Pack Meeting: Tuesday May 31, Saint James Episcopal Church, 7:00 p.m.
Our next pack meeting is Tuesday, May 31, at Saint James Episcopal Church at 7:00 p.m.
Pack 163 Bridging Ceremony & Overnight Campout: Saturday, June 4, Garin/Dry Creek Pioneer Regional Park, 3:00 p.m.
Each family attending the Pack Bridging ceremony should bring a potluck dish.
Overnight family camping with the Pack is a special opportunity and not to be missed! Bring your tent and sleeping bags and enjoy the campfire program!


Fairfax Symphony Orchestra (VA), and former Artistic Director of the American Youth Philharmonic. First Prize Winner of the "American Conducting Prize" in the professional orchestra category.
and KEVIN SÜTTERLIN, Director of Orchestral Activities at Concordia College in Moorhead, MN, and Music Director of Sinfonietta Memphis. An awardee of the highly respected Hirschmann Foundation Switzerland scholarship and a finalist in the renowned American Prize Conducting Competition 2013 and 2014.
This was great fun and an action-packed success!
Enjoy the photo gallery below at the bottom of this page.
Conduct major repertoire with full orchestra in a supportive environment. Workshop is limited to 7 participants to ensure 95 minutes of podium time over three days. Excerpts to be conducted will be chosen in consultation with workshop faculty. 40 minutes with full orchestras and 55 minutes with string quartet and piano.
Seminar fee: $1,200 for conducting participants (arrangements can be made for spreading $200 payments over six months), and $150 for auditors.
More details and a full schedule are included on the Conductor Seminar Announcement.
Application deadline is October 12, 2018. Participants will be notified by October 19 - download the Application.
The Fargo-Moorhead Symphony Orchestra serves the community by enriching, educating and engaging audiences of all ages through inspired performances of orchestral and chamber music.


text:
Brain Trauma Affects One in Three NFL Players
The National Football League hired actuaries to analyze and prepare data related to brain damage in players as a result of the blows to the head they receive in the contact sport. The league has long disputed evidence that their players have high rates of brain damage. However, they are now changing that opinion to admit not only the levels of brain damage among their players but that they anticipate players will develop long-term cognitive problems at a much younger age than players of other sports and the general population.
The statement clears up any confusion the league has manufactured over the years. The study also supports what doctors and scientists have been saying for years: playing football increases the risks of traumatic brain injuries. Additionally, the younger an individual starts playing, the greater the risk and the earlier in life these injuries can occur. The studies used by actuaries pointed out that 28 percent of former players will develop Alzheimer's or dementia. Although the numbers cannot yet be quantified for younger players, the rates of these disease are anticipated to be higher.
Football and Your Brain
Purdue University did a study on what happens to players' brains during a season of football. In this study they used sensors during practice as well as brain scans before, during and after the season. Damage was found in parts of the players' brains that affect emotional responses, higher reasoning and ability to visualize. The study also found that over 50 percent of high school football players exhibited appreciable changes in their brain that persisted for up to two years after they stopped playing.
The University of Rochester researchers also conducted a similar study. However, they analyzed their data different, and came to the conclusion that a season of "routine" hits during practice and games can inflict enough damage that six months of non-contact activity is not enough time for the brain to heal.
One interesting fact about these studies is that none of the players studied actually suffered from a concussion; instead, the players simply suffered regular, routine hits. None of the players complained of soreness, dizziness or nausea, but their brain scans showed minor to significant changes in their brain. The clear implication is that there may be many football players with significant brain damage who do not even realize that they are injured.
Brain injuries are damaging not only to players involved but to families and loved ones as well. The attorneys at Barber and Associates, LLC in Anchorage take your injuries seriously and will fight for your right to compensation. Contact us today for a free consultation.

redpajama_set_name: RedPajamaCommonCrawl


WiFi and Bluetooth have their use cases, but both have certain demands on things like battery life and authentication that make them unsuitable for a lot of low-power use cases. They're also quite limited in range. There are other standards out there more suitable for low-power and wide area work, and thankfully, LoRa is one of them. Having created some LoRa pagers, [Moser] decided to head out and test their range.
It's a painful way to do a range test. [Moser]'s method is much simpler; set a cellphone to log GPS position, and have the pager attempt to send the same data back to the base station. Then, go out for a drive, and compare the two traces. This method doesn't just report straight range, either — it can be used to find good and bad spots for radio reception. It's great when you live in an area full of radio obstructions where simple distance isn't the only thing affecting your link.
Build details on the pagers are available, and you can learn more about LoRa here. While you're at it, check out the LoRa tag for more cool builds and hacks.


- Do you maintain your body through nutrition and lifestyle?
- Do you want to develop mental strength and resilience?
- Do you want to feel empowered to effectively manage everyday struggles?
- Review your core values and set achievable goals that will lead towards an enriched, full and meaningful life.
We value servicing our car to keep it functioning well and avoid breakdowns - don't we owe our minds the same care?
After experiencing a number of challenging situations in my life I felt that my feelings were getting out of control which has led me to feeling increasingly anxious. Constantly worrying about "What ifs" and "maybes", became an obsession and made me become so wrapped up in negative thoughts it was stopping me enjoying my life.
Lorraine was recommended to me, so I took the plunge and called her for an informal chat. Lorraine was non-judgemental and explained a couple of ways she may be able to help.
I now consider myself to be at a better point in my life, Lorraine has taught me some coping strategies which help me rationalise my thought process, recognise those thoughts for the "negative thoughts" that they are and learn to deal with them.
I think in life we all come across situations we can't change, Lorraine has taught me how to accept these, instead of beating myself up, and blaming myself.
I was experiencing panic attacks during exam situations and found the whole exam period very stressful which was unusual for me. Lorraine helped me understand my thoughts and develop relaxation techniques and strategies that I could use when I started to feel stressed. She also has enabled me to understand why I put myself under so much pressure and I now feel strong enough mentally to address this.
I am enjoying learning Mindfulness and relaxation techniques and am finding these new skills help me deal with stress. Reassessing my values has helped give me direction and increased a feeling of value and meaning in my life.


Having range of antique and modern furniture designs made of best materials and crafted with great care, we are one of the most recommended and reliable online furniture shops in India.
The expertise of F9 Furnichair lies in providing and manufacturing certified products under the particular label of F9 Certified. This certification is a symbol of style, assurance and also quality.
A quality assessment process is undergone by all our products right from scratch. We have set specific standards for allour processes from the wood procurement to décor pieces and polished finished furniture.
While considering ecosystem and human health impacts, energy usage, material selection, social actions, a company's processes, the F9 Certification generally assesses the environmental impact of a furniture product, and so, it is considered as a sustainability standard. We make sure that a tree is planted in place of every tree cut and thus, our wood is mainly procured from the commercially grown trees.
We believe in elegance, comfort, and quality. We are thus involved in incorporating our values and beliefs into our products. Our products are made by taking care of the human requirement, processes, and surroundings.


Fannie Mae's FNMA -0.37%national housing survey for September showed 37% of respondents expecting house prices to go up in the coming year – the highest level since the survey was launched in 2010. That number has more than doubled in the past year: just 18% believed the same thing in September 2011.
Few are expecting a bull market – the average expectation is for home prices to rise by 1.5% – but there increasing confidence that homes will retain their value or appreciate. Just 11% of people now think house prices will go down, compared to 25% a year ago.
Big builders have been claiming there is a housing recovery underway, although many in the markets are not buying it. But numbers like this are making it increasingly hard to deny there is a recovery in sentiment, if not action.


text:
Malware disables ambulance response systems
An unspecified malware variant recently disabled the automated response systems of a New Zealand-based ambulance service.
The service – which provides 90% of the emergency and non-emergency ambulances in the country – was forced to manually allocate the rescue vehicles over a 24-hour period.
"Anti-virus software protected the systems but as a result of the virus it impacted on some of the systems services, mainly those related to paging and radio," Alan Goudge, communications operations manager for the St John Ambulance, confirmed.
"Back-up systems immediately took over when it was detected and the workload was managed manually."
Although the malware did not seem to specifically target the ambulance service, the incident is obviously not the first time a medical entity has been affected by viruses or worms.
As Sophos security expert Graham Cluley notes, the Mytob worm hit a number of London hospitals in 2008, while the Northwest Hospital and Medical Center in north Seattle was affected by a 2005 attack which shut down computers in the facility's intensive care unit and prevented pagers from working properly.
A 21-year-old man was ultimately sentenced to three years in prison and fined a quarter of a million dollars in connection with the Seattle case.
"The fact is that malware often doesn't discriminate between who its victims might be. Whether you're running a computer in your spare bedroom, or operating critical systems in a medical environment, your PC may still be at risk," said Cluley.
"Anyone who still thinks that virus-writing is 'mostly harmless' and only really impacts the foolish who don't have backups, should consider what the possible consequences of taking down the systems of an ambulance emergency service might mean."
Will Fassbender be the new RoboCop?
TSA eyes Christmas present inspections


Smart Footer System is a WordPress plugin that lets you take control of your footers. No longer is your footer an afterthought. Easily create custom footers that will help you increase conversions with relevant promotions, advertisements, and calls-to-action.


text:
New York Labor and Employment Law Report
Arrest Record
Ban the Box Legislation
Call-In Pay
Class Action Waivers
Compensation History
Criminal Conviction Record
Discharge and Discipline
Domestic Violence Victim Status
Earned Sick Time Act
EEO-1 Form
Faithless Servant Doctrine
Family and Medical Leave Act
Federal Contractors
Long Term Leave of Absence
Marital Status Discrimination
New York City Commission on Human Rights
New York City Human Rights Law
New York Minimum Wage Rate
New York State Department of Labor
NYSDOL
OFCCP
Office of Federal Contract Compliance Programs
Paid Leave for Cancer Screenings
Presidential Proclamation
Reductions in Force
Safe Time
Title VII of the Civil Rights Act
Union Organizing
USDOL
White-Collar Exemptions
Search New York Labor and Employment Law Report
Employers Need to Develop an Action Plan to Deal With Workplace Violence
By: Katherine R. Schafer
If the recent and tragic shootings at an office holiday party in San Bernardino, California, and at a lawn care company in Kansas have taught us anything, it is that these unfortunate incidents of workplace violence are becoming more and more commonplace. In addition to the devastating human cost of these tragedies, workplace violence can also bring significant liability for employers. According to the Occupational Safety and Health Administration, workplace violence is responsible for $55 million in lost wages each year. When the cost of lost productivity, legal expenses, property damage, diminished public image, and increased security are factored in, workplace violence costs the American workforce approximately $36 billion dollars per year. Among other sources of potential liability, employers may be cited by OSHA for violating the "General Duty Clause" of the OSH Act, which requires employers to maintain workplaces free from "recognized hazards" that are likely to cause death or serious physical harm to employees. OSHA has previously published guidance citing certain types of workplace violence as recognized hazards for "heightened-risk industries," which include healthcare and social services, late-night retail establishments, and taxi and for-hire drivers. But an employer in any industry may be considered to have a recognized hazard of workplace violence based on factors like previous incidents, employee complaints, injury and illness data, prior corrective actions, and its own safety rules and policies. Last month, Bond attorneys presented a breakfast briefing on workplace violence at 12 locations across the state, providing guidance on developing an action plan to address workplace violence, identifying the potentially violent employee, and best practices for responding to an incident of violence in the workplace. To avoid liability and prevent the unthinkable, employers should start taking steps to develop a workplace violence prevention program.


text:
Q: OpenEBS cstore volume is very slow on delete and overwrite I have an OpenEBS setup with 3 data nodes and a default cstore storage class.
Creation of a file works pretty good:
time dd if=/dev/urandom of=a.log bs=1M count=100
real    0m 0.53s

I can delete the file and create it again with the quite the same times.
But when I rewrite onto the file it takes ages:
time dd if=/dev/urandom of=a.log bs=1M count=100
104857600 bytes (100.0MB) copied, 0.596577 seconds, 167.6MB/s
real    0m 0.59s

time dd if=/dev/urandom of=a.log bs=1M count=100
104857600 bytes (100.0MB) copied, 16.621222 seconds, 6.0MB/s
real    0m 16.62s

time dd if=/dev/urandom of=a.log bs=1M count=100
104857600 bytes (100.0MB) copied, 19.621924 seconds, 5.1MB/s
real    0m 19.62s

When I delete the a.log and write it again, then the ~167MS/s are back. Only writing onto an existing file takes so much time.
My problem is that i think this is why some of my applications (e.g. Databases) are too slow. Creation of a table in mysql took over 7sek.
Here is the spec of my testcluster:
# Create the OpenEBS namespace
apiVersion: v1
kind: Namespace
metadata:
  name: openebs
---
# Create Maya Service Account
apiVersion: v1
kind: ServiceAccount
metadata:
  name: openebs-maya-operator
  namespace: openebs
---
# Define Role that allows operations on K8s pods/deployments
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: openebs-maya-operator
rules:
- apiGroups: ["*"]
  resources: ["nodes", "nodes/proxy"]
  verbs: ["*"]
- apiGroups: ["*"]
  resources: ["namespaces", "services", "pods", "pods/exec", "deployments", "replicationcontrollers", "replicasets", "events", "endpoints", "configmaps", "secrets", "jobs", "cronjobs"]
  verbs: ["*"]
- apiGroups: ["*"]
  resources: ["statefulsets", "daemonsets"]
  verbs: ["*"]
- apiGroups: ["*"]
  resources: ["resourcequotas", "limitranges"]
  verbs: ["list", "watch"]
- apiGroups: ["*"]
  resources: ["ingresses", "horizontalpodautoscalers", "verticalpodautoscalers", "poddisruptionbudgets", "certificatesigningrequests"]
  verbs: ["list", "watch"]
- apiGroups: ["*"]
  resources: ["storageclasses", "persistentvolumeclaims", "persistentvolumes"]
  verbs: ["*"]
- apiGroups: ["volumesnapshot.external-storage.k8s.io"]
  resources: ["volumesnapshots", "volumesnapshotdatas"]
  verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
- apiGroups: ["apiextensions.k8s.io"]
  resources: ["customresourcedefinitions"]
  verbs: [ "get", "list", "create", "update", "delete", "patch"]
- apiGroups: ["*"]
  resources: [ "disks", "blockdevices", "blockdeviceclaims"]
  verbs: ["*" ]
- apiGroups: ["*"]
  resources: [ "cstorpoolclusters", "storagepoolclaims", "storagepoolclaims/finalizers", "cstorpoolclusters/finalizers", "storagepools"]
  verbs: ["*" ]
- apiGroups: ["*"]
  resources: [ "castemplates", "runtasks"]
  verbs: ["*" ]
- apiGroups: ["*"]
  resources: [ "cstorpools", "cstorpools/finalizers", "cstorvolumereplicas", "cstorvolumes", "cstorvolumeclaims"]
  verbs: ["*" ]
- apiGroups: ["*"]
  resources: [ "cstorpoolinstances", "cstorpoolinstances/finalizers"]
  verbs: ["*" ]
- apiGroups: ["*"]
  resources: [ "cstorbackups", "cstorrestores", "cstorcompletedbackups"]
  verbs: ["*" ]
- apiGroups: ["coordination.k8s.io"]
  resources: ["leases"]
  verbs: ["get", "watch", "list", "delete", "update", "create"]
- nonResourceURLs: ["/metrics"]
  verbs: ["get"]
- apiGroups: ["*"]
  resources: [ "upgradetasks"]
  verbs: ["*" ]
---
# Bind the Service Account with the Role Privileges.
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: openebs-maya-operator
subjects:
- kind: ServiceAccount
  name: openebs-maya-operator
  namespace: openebs
- kind: User
  name: system:serviceaccount:default:default
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: ClusterRole
  name: openebs-maya-operator
  apiGroup: rbac.authorization.k8s.io
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: maya-apiserver
  namespace: openebs
  labels:
    name: maya-apiserver
    openebs.io/component-name: maya-apiserver
    openebs.io/version: 1.3.0
spec:
  selector:
    matchLabels:
      name: maya-apiserver
      openebs.io/component-name: maya-apiserver
  replicas: 1
  strategy:
    type: Recreate
    rollingUpdate: null
  template:
    metadata:
      labels:
        name: maya-apiserver
        openebs.io/component-name: maya-apiserver
        openebs.io/version: 1.3.0
    spec:
      serviceAccountName: openebs-maya-operator
      containers:
      - name: maya-apiserver
        imagePullPolicy: IfNotPresent
        image: quay.io/openebs/m-apiserver:1.3.0
        ports:
        - containerPort: 5656
        env:
        - name: OPENEBS_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        - name: OPENEBS_SERVICE_ACCOUNT
          valueFrom:
            fieldRef:
              fieldPath: spec.serviceAccountName
        - name: OPENEBS_MAYA_POD_NAME
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        - name: OPENEBS_IO_CREATE_DEFAULT_STORAGE_CONFIG
          value: "true"
        - name: OPENEBS_IO_INSTALL_DEFAULT_CSTOR_SPARSE_POOL
          value: "true"
        - name: OPENEBS_IO_JIVA_CONTROLLER_IMAGE
          value: "quay.io/openebs/jiva:1.3.0"
        - name: OPENEBS_IO_JIVA_REPLICA_IMAGE
          value: "quay.io/openebs/jiva:1.3.0"
        - name: OPENEBS_IO_JIVA_REPLICA_COUNT
          value: "1"
        - name: OPENEBS_IO_CSTOR_TARGET_IMAGE
          value: "quay.io/openebs/cstor-istgt:1.3.0"
        - name: OPENEBS_IO_CSTOR_POOL_IMAGE
          value: "quay.io/openebs/cstor-pool:1.3.0"
        - name: OPENEBS_IO_CSTOR_POOL_MGMT_IMAGE
          value: "quay.io/openebs/cstor-pool-mgmt:1.3.0"
        - name: OPENEBS_IO_CSTOR_VOLUME_MGMT_IMAGE
          value: "quay.io/openebs/cstor-volume-mgmt:1.3.0"
        - name: OPENEBS_IO_VOLUME_MONITOR_IMAGE
          value: "quay.io/openebs/m-exporter:1.3.0"
        - name: OPENEBS_IO_CSTOR_POOL_EXPORTER_IMAGE
          value: "quay.io/openebs/m-exporter:1.3.0"
        - name: OPENEBS_IO_ENABLE_ANALYTICS
          value: "false"
        - name: OPENEBS_IO_INSTALLER_TYPE
          value: "openebs-operator"
        livenessProbe:
          exec:
            command:
            - /usr/local/bin/mayactl
            - version
          initialDelaySeconds: 30
          periodSeconds: 60
        readinessProbe:
          exec:
            command:
            - /usr/local/bin/mayactl
            - version
          initialDelaySeconds: 30
          periodSeconds: 60
---
apiVersion: v1
kind: Service
metadata:
  name: maya-apiserver-service
  namespace: openebs
  labels:
    openebs.io/component-name: maya-apiserver-svc
spec:
  ports:
  - name: api
    port: 5656
    protocol: TCP
    targetPort: 5656
  selector:
    name: maya-apiserver
  sessionAffinity: None
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: openebs-provisioner
  namespace: openebs
  labels:
    name: openebs-provisioner
    openebs.io/component-name: openebs-provisioner
    openebs.io/version: 1.3.0
spec:
  selector:
    matchLabels:
      name: openebs-provisioner
      openebs.io/component-name: openebs-provisioner
  replicas: 1
  strategy:
    type: Recreate
    rollingUpdate: null
  template:
    metadata:
      labels:
        name: openebs-provisioner
        openebs.io/component-name: openebs-provisioner
        openebs.io/version: 1.3.0
    spec:
      serviceAccountName: openebs-maya-operator
      containers:
      - name: openebs-provisioner
        imagePullPolicy: IfNotPresent
        image: quay.io/openebs/openebs-k8s-provisioner:1.3.0
        env:
        - name: NODE_NAME
          valueFrom:
            fieldRef:
              fieldPath: spec.nodeName
        - name: OPENEBS_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        livenessProbe:
          exec:
            command:
            - pgrep
            - ".*openebs"
          initialDelaySeconds: 30
          periodSeconds: 60
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: openebs-snapshot-operator
  namespace: openebs
  labels:
    name: openebs-snapshot-operator
    openebs.io/component-name: openebs-snapshot-operator
    openebs.io/version: 1.3.0
spec:
  selector:
    matchLabels:
      name: openebs-snapshot-operator
      openebs.io/component-name: openebs-snapshot-operator
  replicas: 1
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        name: openebs-snapshot-operator
        openebs.io/component-name: openebs-snapshot-operator
        openebs.io/version: 1.3.0
    spec:
      serviceAccountName: openebs-maya-operator
      containers:
        - name: snapshot-controller
          image: quay.io/openebs/snapshot-controller:1.3.0
          imagePullPolicy: IfNotPresent
          env:
          - name: OPENEBS_NAMESPACE
            valueFrom:
              fieldRef:
                fieldPath: metadata.namespace
          livenessProbe:
            exec:
              command:
              - pgrep
              - ".*controller"
            initialDelaySeconds: 30
            periodSeconds: 60
        - name: snapshot-provisioner
          image: quay.io/openebs/snapshot-provisioner:1.3.0
          imagePullPolicy: IfNotPresent
          env:
          - name: OPENEBS_NAMESPACE
            valueFrom:
              fieldRef:
                fieldPath: metadata.namespace
          livenessProbe:
            exec:
              command:
              - pgrep
              - ".*provisioner"
            initialDelaySeconds: 30
            periodSeconds: 60
---
# This is the node-disk-manager related config.
# It can be used to customize the disks probes and filters
apiVersion: v1
kind: ConfigMap
metadata:
  name: openebs-ndm-config
  namespace: openebs
  labels:
    openebs.io/component-name: ndm-config
data:
  node-disk-manager.config: |
    probeconfigs:
      - key: udev-probe
        name: udev probe
        state: true
      - key: seachest-probe
        name: seachest probe
        state: false
      - key: smart-probe
        name: smart probe
        state: true
    filterconfigs:
      - key: os-disk-exclude-filter
        name: os disk exclude filter
        state: true
        exclude: "/,/etc/hosts,/boot"
      - key: vendor-filter
        name: vendor filter
        state: true
        include: ""
        exclude: "CLOUDBYT,OpenEBS"
      - key: path-filter
        name: path filter
        state: true
        include: ""
        exclude: "loop,/dev/fd0,/dev/sr0,/dev/ram,/dev/dm-,/dev/md"
---
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: openebs-ndm
  namespace: openebs
  labels:
    name: openebs-ndm
    openebs.io/component-name: ndm
    openebs.io/version: 1.3.0
spec:
  selector:
    matchLabels:
      name: openebs-ndm
      openebs.io/component-name: ndm
  updateStrategy:
    type: RollingUpdate
  template:
    metadata:
      labels:
        name: openebs-ndm
        openebs.io/component-name: ndm
        openebs.io/version: 1.3.0
    spec:
      nodeSelector:
        "openebs.io/nodegroup": "storage-node"
      serviceAccountName: openebs-maya-operator
      hostNetwork: true
      containers:
      - name: node-disk-manager
        image: quay.io/openebs/node-disk-manager-amd64:v0.4.3
        imagePullPolicy: Always
        securityContext:
          privileged: true
        volumeMounts:
        - name: config
          mountPath: /host/node-disk-manager.config
          subPath: node-disk-manager.config
          readOnly: true
        - name: udev
          mountPath: /run/udev
        - name: procmount
          mountPath: /host/proc
          readOnly: true
        - name: sparsepath
          mountPath: /var/openebs/sparse
        env:
        - name: NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        - name: NODE_NAME
          valueFrom:
            fieldRef:
              fieldPath: spec.nodeName
        - name: SPARSE_FILE_DIR
          value: "/var/openebs/sparse"
        - name: SPARSE_FILE_SIZE
          value: "10737418240"
        - name: SPARSE_FILE_COUNT
          value: "3"
        livenessProbe:
          exec:
            command:
            - pgrep
            - ".*ndm"
          initialDelaySeconds: 30
          periodSeconds: 60
      volumes:
      - name: config
        configMap:
          name: openebs-ndm-config
      - name: udev
        hostPath:
          path: /run/udev
          type: Directory
      - name: procmount
        hostPath:
          path: /proc
          type: Directory
      - name: sparsepath
        hostPath:
          path: /var/openebs/sparse
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: openebs-ndm-operator
  namespace: openebs
  labels:
    name: openebs-ndm-operator
    openebs.io/component-name: ndm-operator
    openebs.io/version: 1.3.0
spec:
  selector:
    matchLabels:
      name: openebs-ndm-operator
      openebs.io/component-name: ndm-operator
  replicas: 1
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        name: openebs-ndm-operator
        openebs.io/component-name: ndm-operator
        openebs.io/version: 1.3.0
    spec:
      serviceAccountName: openebs-maya-operator
      containers:
        - name: node-disk-operator
          image: quay.io/openebs/node-disk-operator-amd64:v0.4.3
          imagePullPolicy: Always
          readinessProbe:
            exec:
              command:
                - stat
                - /tmp/operator-sdk-ready
            initialDelaySeconds: 4
            periodSeconds: 10
            failureThreshold: 1
          env:
            - name: WATCH_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
            # the service account of the ndm-operator pod
            - name: SERVICE_ACCOUNT
              valueFrom:
                fieldRef:
                  fieldPath: spec.serviceAccountName
            - name: OPERATOR_NAME
              value: "node-disk-operator"
            - name: CLEANUP_JOB_IMAGE
              value: "quay.io/openebs/linux-utils:3.9"
---
apiVersion: v1
kind: Secret
metadata:
  name: admission-server-certs
  namespace: openebs
  labels:
    app: admission-webhook
    openebs.io/component-name: admission-webhook
type: Opaque
data:
  cert.pem: <...pem...>
  key.pem: <...pem...>
---
apiVersion: v1
kind: Service
metadata:
  name: admission-server-svc
  namespace: openebs
  labels:
    app: admission-webhook
    openebs.io/component-name: admission-webhook-svc
spec:
  ports:
  - port: 443
    targetPort: 443
  selector:
    app: admission-webhook
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: openebs-admission-server
  namespace: openebs
  labels:
    app: admission-webhook
    openebs.io/component-name: admission-webhook
    openebs.io/version: 1.3.0
spec:
  replicas: 1
  strategy:
    type: Recreate
    rollingUpdate: null
  selector:
    matchLabels:
      app: admission-webhook
  template:
    metadata:
      labels:
        app: admission-webhook
        openebs.io/component-name: admission-webhook
        openebs.io/version: 1.3.0
    spec:
      serviceAccountName: openebs-maya-operator
      containers:
        - name: admission-webhook
          image: quay.io/openebs/admission-server:1.3.0
          imagePullPolicy: IfNotPresent
          args:
            - -tlsCertFile=/etc/webhook/certs/cert.pem
            - -tlsKeyFile=/etc/webhook/certs/key.pem
            - -alsologtostderr
            - -v=2
            - 2>&1
          volumeMounts:
            - name: webhook-certs
              mountPath: /etc/webhook/certs
              readOnly: true
      volumes:
        - name: webhook-certs
          secret:
            secretName: admission-server-certs
---
apiVersion: admissionregistration.k8s.io/v1beta1
kind: ValidatingWebhookConfiguration
metadata:
  name: validation-webhook-cfg
  labels:
    app: admission-webhook
    openebs.io/component-name: admission-webhook
webhooks:
  # failurePolicy Fail means that an error calling the webhook causes the admission to fail.
  - name: admission-webhook.openebs.io
    failurePolicy: Ignore
    clientConfig:
      service:
        name: admission-server-svc
        namespace: openebs
        path: "/validate"
      caBundle: <...ca..>
    rules:
      - operations: [ "CREATE", "DELETE" ]
        apiGroups: ["*"]
        apiVersions: ["*"]
        resources: ["persistentvolumeclaims"]
      - operations: [ "CREATE", "UPDATE" ]
        apiGroups: ["*"]
        apiVersions: ["*"]
        resources: ["cstorpoolclusters"]
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: openebs-localpv-provisioner
  namespace: openebs
  labels:
    name: openebs-localpv-provisioner
    openebs.io/component-name: openebs-localpv-provisioner
    openebs.io/version: 1.3.0
spec:
  selector:
    matchLabels:
      name: openebs-localpv-provisioner
      openebs.io/component-name: openebs-localpv-provisioner
  replicas: 1
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        name: openebs-localpv-provisioner
        openebs.io/component-name: openebs-localpv-provisioner
        openebs.io/version: 1.3.0
    spec:
      serviceAccountName: openebs-maya-operator
      containers:
      - name: openebs-provisioner-hostpath
        imagePullPolicy: Always
        image: quay.io/openebs/provisioner-localpv:1.3.0
        env:
        - name: NODE_NAME
          valueFrom:
            fieldRef:
              fieldPath: spec.nodeName
        - name: OPENEBS_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        - name: OPENEBS_IO_ENABLE_ANALYTICS
          value: "true"
        - name: OPENEBS_IO_INSTALLER_TYPE
          value: "openebs-operator"
        - name: OPENEBS_IO_HELPER_IMAGE
          value: "quay.io/openebs/openebs-tools:3.8"
        livenessProbe:
          exec:
            command:
            - pgrep
            - ".*localpv"
          initialDelaySeconds: 30
          periodSeconds: 60

I am on Kubernetes:
Client Version: version.Info{Major:"1", Minor:"16", GitVersion:"v1.16.2", GitCommit:"c97fe5036ef3df2967d086711e6c0c405941e14b", GitTreeState:"clean", BuildDate:"2019-10-15T19:18:23Z", GoVersion:"go1.12.10", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"16", GitVersion:"v1.16.2", GitCommit:"c97fe5036ef3df2967d086711e6c0c405941e14b", GitTreeState:"clean", BuildDate:"2019-10-15T19:09:08Z", GoVersion:"go1.12.10", Compiler:"gc", Platform:"linux/amd64"}

How can i investigate why it takes so long?
What can me bring some more insides?

A: Spoke to Peter over Slack. The following changes helped to improve the performance:

*

*Increasing the queue size of the iSCSI Initiator

*Increase the cpu from 2 to 6 for each of the three nodes

*Create my own sc to use a ext3 instead of the default ext4.

With the above changes, the subsequent writes numbers were around 140-150 MB/s.

redpajama_set_name: RedPajamaStackExchange


Luxury vehicles are being auctioned off at Jidd Motors in Des Plaines. Be the highest bidder and you can drive away in a Porsche or Lamborghini. Preview the event Friday, July 14th and ante up on Saturday, July 15th. It's an event you can't miss!


Since I've been laid up for over a week and this pain won't let me rest I decided to take a walk down memory lane as inspired by the photo of me at around age five or six.
Age 50+ My body sounds like Rice Krispies. "Snap, Krackle, Pop!" Every muscle, tendon and joint aches and if I tried to do that Pop and Lock dance from the 70s I'd stay that way!
Age 5: I could eat anything I wanted and as much of it as my little tummy could hold.
Age 50+ My various doctors tell me constantly what I can and cannot eat.
Age 5: Never knew what sickness was. I do remember having Chicken Pox and the Measles but a visit to the doctor, a little calamine lotion, plus extra love from Mommy and Daddy and I was up on my feet in a few days.
Age 50+ I estimate I've been in some doctor's office, sick at home or in the hospital every other month for the last two years.
Age 5: I was very inquisitive, creative and innovative. My parents encouraged and supported me in all my endeavors. Of course as a child I also had endless amounts of playtime. Even when my baby brother couldn't or wouldn't play with me I had imaginary friends plus I loved playing with ants. Probably from the ants point of view I was a sadistic child torturing them and they were glad when my Mom called me in for lunch or dinner!
Age 50+ The institution discourages creativity or thinking of any kind. I'm now paid to be a drone, a wage slave, a mindless robot. In fact I've been warned by a certain supervisor that any deviation from the expected will result in disciplinary action. So I hide my intelligence and let everyone think that I'm stupid. Makes life easier.
However during the little free time I have when I'm not ill or an inmate of the asylum, I enjoy being outdoors watching people and exploring. I especially enjoy the Botanic Gardens. Any of them, in any borough. I'm not particular. If I could get to the zoo I'd go there also.
One good difference between ages five and fifty-one is that now I hate television. Got rid of the TV back in January and don't miss it. My vision has diminished so it's very difficult for me to watch television. Gives me headaches. Plus there is nothing of value on TV anymore. Growing up in the 1960s was the golden age of television; mind you I was watching Felix the Cat and other old time cartoons, a few game shows, Romper Room, Batman, the Green Hornet, etc….
Also unlike today's parents my mother and father restricted my TV viewing. I did not have a TV in my bedroom. We had that one Black & White RCA Victor vacuum tube television in the living room which my father controlled. To this day I still hate Lawrence Welk and Mitch Miller! My parents were old-fashioned and expected me to spend most of my free time if not playing, then reading or drawing. As a result I could read by the time I was four and was a pretty good artist ages seven to seventeen.
Well enough walking down memory lane. Time for lunch with a side of Advils. Oh yes, the glories of getting older.


text:
traditional newswire
Open Newswire
Files Gallery
Hidden Articles List
about us | help us
International | Anti-Capitalism
no events match your query!
25 Dec 43rd successive Cabhair Swim, Christmas Day , 2019...
The leveraged buyout, exploitation and punishment beating of Greece as warning to others. 11:45 May 11 0 comments
Red Banner issue 60 out now 13:18 Jun 22 0 comments
Red Banner issue 59 out now 17:46 Mar 28 0 comments
The Fake Banking Enquiry. More Cynical Theatre For The Downtrodden Irish Masses? 11:29 Dec 28 0 comments
Red Banner issue 58 out now 15:58 Dec 10 0 comments
A Blog About Human Rights
Latest Updates Thu Nov 21, 2019 20:32 | Human Rights
US Holds China To Account For Human Rights Violations Sun Oct 13, 2019 19:12 | Human Rights
UN Human Rights Council Should Address Human Rights Crisis in Cambodia Sat Aug 31, 2019 13:41 | Human Rights
Fijian women still face Human Rights violations Mon Aug 26, 2019 18:49 | Human Rights
Saudi Human Rights Violation Fri Aug 09, 2019 20:41 | Human Rights
Human Rights in Ireland >>
Cedar Lounge
"A flaky website that purports to be ?leftist,? The Cedar Lounge Revolution, occasionally makes a relevant point or two."
Scottish political websites? 14:14 Sat Jan 18, 2020 | WorldbyStorm
Space television series 11:46 Sat Jan 18, 2020 | WorldbyStorm
Space film 11:14 Sat Jan 18, 2020 | WorldbyStorm
This Weekend I?ll Mostly Be Listening to? The Clientele 06:41 Sat Jan 18, 2020 | WorldbyStorm
Long life? and prosperity? 12:08 Fri Jan 17, 2020 | WorldbyStorm
Cedar Lounge >>
Life should be full of strangeness, like a rich painting
Some Thoughts on the Brexit Joint Report 11:50 Sat Dec 09, 2017
IRISH COMMONWEALTH: TRADE UNIONS AND CIVIL SOCIETY IN THE 21ST CENTURY 14:06 Sat Nov 18, 2017
Notes for a Book on Money and the Irish State - The Marshall Aid Program 15:10 Sat Apr 02, 2016
The Financial Crisis:What Have We Learnt? 19:58 Sat Aug 29, 2015
Money in 35,000 Words or Less 21:34 Sat Aug 22, 2015
Dublin Opinion >>
NAMA Wine Lake
Test ? 12 November 2018 Mon Nov 12, 2018 14:28 | namawinelake
Farewell from NWL Sun May 19, 2013 14:00 | namawinelake
Happy 70th Birthday, Michael Sun May 19, 2013 14:00 | namawinelake
Of the Week? Sat May 18, 2013 00:02 | namawinelake
Noonan denies IBRC legal fees loan approval to Paddy McKillen was in breach of E... Fri May 17, 2013 14:23 | namawinelake
NAMA Wine Lake >>
Resource Wars: a feature of the 21st century?
international | anti-capitalism | opinion/analysis Wednesday June 08, 2011 14:32 by James Blake
Food Security and Conflicts Due to Resource Wars
Conflict over resources has been a particularly strong feature of the last 500 years or a period regarded by some commentators as the five centuries of global capitalist expansion, begun by Christopher Columbus and followed up by the brutal conquistadors such as Hernan Cortes and Francisco Pizarro, who searched for 'Eldorado', the city of gold. Millions of indigenous Indians from the islands of the Caribbean down the spine of Central America to the mountains of Bolivia, toiled the fields and mines to serve European monarchies who, on the back of exploited labour, built ostentatious cathedrals and palaces as, according to Eduardo Galeano in 'Open Veins of Latin America', tens of millions of souls perished from this earth.
Nchanga Copper Mine in Zambia
Of all the enemies to public liberty war is, perhaps, the most to be dreaded because it comprises and develops the germ of every other. War is the parent of armies; from these proceed debts and taxes … known instruments for bringing the many under the domination of the few.… No nation could preserve its freedom in the midst of continual warfare ~ James Madison
The early Spanish and Portuguese seaborne empires spread from small Iberian ports to Central and South America, Africa and even as far away as Goa in India and were followed in violent succession by the Dutch (New York was originally a Dutch settlement called New Amsterdam, Indonesia, Malaysia) and later the French (Indochina, Africa, Middle East) and British (Africa, India, Middle East).
The so called 'Scramble for Africa' involved all the major European powers including a late arrival, Germany, under Kaiser Wilhelm II seeking colonies abroad or as he put it: "Germany's place in the sun". The fact that native people's already occupied these lands meant nothing to the Imperial mindset while historical precedent such as the campaigns of genocide, subjugation and 'reservatisation' of the Indians in the Western hemisphere clearly indicated the direction these later waves of colonisation would take. Given the eventual decimation of the Ameri and Carrib Indians, the 'entrepreneurs' and 'innovators' of their time came up with the so called 'slave trade'. Tens of millions of African men, women and children were transported across the Atlantic to North and South America in the most appalling conditions imaginable to toil in the cotton plantations and tobacco fields, enriching Western merchants and burgeoning financial institutions.
The enormous sums of money and trade involved turned coastal towns such as Liverpool and Portsmouth into bustling ports and all from the proceeds of the most despicable commerce in human history. Nor can the Belgians go without special mention, the Belgian King Leopold II turned the Congo (50 times the size of Belgium) into his personal fiefdom where some of the most grievous colonial crimes were carried out against the domestic population, documented by the Irishman Roger Casement in one of the first successful human rights reports of the era (Casement also later documented the brutal treatment of the Putumayo Indians of Peru).. Is it any wonder that this resource rich country in particular still does not know peace?
The exploitation of Africa's natural resources by European powers is well documented in Walter Rodney's masterful PhD, which later became a bestselling book: "How Europe Underdeveloped Africa". Rodney put forth a convincing thesis that European colonisation had a several fold impact in that not only did it lead to underdevelopment of the 'host' countries but also to the enrichment and rapid expansion of European economies who in effect piggy-backed on what are now termed 'developing countries'. In doing so, they left a lasting legacy of hate, favoured elites and disgruntled populations, scenarios which spawned civil wars and outbreaks of genocide, Rwanda being a case in point.
Today, it is said a new 'scramble for Africa' is underway (the Irish Times recently carried an editorial with that exact title). European powers despite their 'fall' remain influential on the African continent in what is regarded as a form of 'unofficial Empire' through organisations like the British 'Commonwealth' or France's 'Organisation de la Francophonie'. There have been several Western interventions in Africa in this new century including Liberia, the Ivory Coast and Libya. Meanwhile, India and China are investing heavily in Africa, building roads and infrastructure so that they can gain access to badly needed natural resources in the hope that they can in some way satisfy their insatiable hunger for natural resources which fuel their manic economic expansion. The long term environmental consequences of exponential growth in the BRIC economies (Brazil, Russia, India & China) are almost too horrifying to contemplate especially with China building coal burning stations at an alarming rate and its massively expanding middle class with their seemingly endless desire for the latest fashion accessories, motor car or technological gadget (it was recently reported in China that a teenager sold one of his kidney's so he could purchase an I-pad).
What is currently cited as a modern day 'scramble for Africa' should probably be called 'the scramble for African resources' which may lead to 'Resource Conflict'. As in the 19th century, leading powers are coming into conflict as they put in their respective territorial claims, however spurious, along with rights over the oil and gas reserves which rest beneath the ice caps, deserts, grasslands and seas of this increasingly resource drained planet. One potential recent flashpoint was the Arctic where several powers claimed dominion of the inhospitable region. It has been reported that two of the main protagonists, Norway and Russia, have negotiated some form of agreement; however, other issues remain outstanding.
Another area of growing conflict is the South China Seas. There are an estimated 213 billion barrels of oil and 2 quadtrillion cubic meters of natural gas in a region stretching from the south coast of China to the north coast of Malaysia. Six countries, Vietnam, the Philippines, Malaysia, Brunei, Taiwan, and China all have territorial claims. Something of an arms race is under way with one network reporting that Vietnam is going to spend $2 billion on six submarines, Thailand intends to spend $30 billion over ten years, and the Philippines is reportedly going to purchase a US submarine (its first) for over $100 million dollars. China has increased its overall military spend by 10%, bringing it up to €91.5 billion per annum, but this is dwarfed by the heavy weight in the region, the United States, which spent a record $707 billion on its military in 2010, which incidentally accounts for 40% of the global military spend. It is something of an irony that Southeast Asian countries in particular feel the need to spend vast quantities of much needed cash on military hardware given they are aid recipients who suffer enormous socio-economic problems, including endemic poverty. But then geopolitics is no stranger to irony or hypocrisy.
It seems certain in a world with governments who adhere slavishly to an economic philosophy that demands year on year growth coupled with dwindling natural resources and issues around fresh water supply and food security, that 'Resource Wars' will regrettably come to characterise the 21st century.
Food Security will be an issue
Add Your Comments >>
printable version with comments
RSS and atom feeds allow you to keep track of new comments on particular stories. You can input the URL's from these links into a rss reader and you will be informed whenever somebody posts a new comment.
Digg this del.icio.us Furl Reddit Technorati Facebook Twitter
Most Recent Comments First
View Comments Titles Only
Comments (8 of 8)
Jump To Comment: 1 2 3 4 5 6 7 8
by Serf Thu Jun 09, 2011 00:27
Interesting relevant and timely reminder and a decent well written article James. Thanks.
A warning to us here in Ireland of how ruthless the major european countries have been to other countries in the past when natural resources and their own personal gain were at stake. We'd do well to consider this when making deals with them for what remains of our resources, accepting their GM crops and selling ourselves and our children into indentured slavery to their financial institutions and banks for the foreseeable future
funny thing is serf
by opus diablos - the regressive hypocrite party Thu Jun 09, 2011 16:45
The ordinary Irish people are well aware from their not quite obliterated historic consciousness of just how the global cookie crumbles.
The famine, and agribusiness's part in the export of shiploads of food as the people starved still have a considerable hold.
Leopold's filthy regime in the Congo(I believe his flag was a single gold star on a blue background)are also documented by Joseph Conrad in Heart of Darkness, and he met Casement when he was out there(the book lists the atrocities and horrors suffered by the locals). If I recall right that was BEFORE he went to the Putamayo.
The Caribs proved unwilling slaves, and were soon virtually eliminated, so they started shipping Africans over..
Apocalypse Now was based on Conrads Congo book, transferred to Nam.
But slavery had been introduced centuries before by the Arabs who controlled the eastern coasts of Africa. Good article, I'm not going quibbling over details.
this seems the best place to post this
by opus diablos - the regressive hypocrite party Sat Nov 19, 2011 16:24
and again..
by opus diablos - the regressive hypocrite party Thu Dec 15, 2011 15:28
this seems the place
the filth continues
The Keystone Globocops..
by opus diablos - the regressive hypocrite party Tue Dec 20, 2011 15:31
..if only it WERE just some satirical take instead of the trailer to the Brave New Year http://www.globalresearch.ca/index.php?context=va&aid=28269
Resources Win.
by Ancient Celt. Thu Dec 22, 2011 21:05
All wars are about resources Opus. The Romans didn't bother invading Ireland because we weren't worth it.
Tautology, AC..
by opus diablos - the regressive hypocrite party Fri Dec 23, 2011 11:34
..dont teach. Besides, we were under the Roman psychological indoctrination since the Celtic version of the Christian interpretations were suppressed...and to a large extent still are. If the brits hadn't kept them occupied they would have been over to 'settle' us. As it was, they probably got all they wanted, as today the EU/US/Nato empire does, by ' free trade'. Its free, once they tie up the ownership of the base. And they lend us the means to trade them into ever deepening monetary/numerological riches. I'm not up to speed on detail, but I'd be surprised if the Paddy farmer pastoralists of the time were not feeding the legions. Given we traded North Africa and beyond before Rome took Britain, it seems likely Irish leather shod them.
meantime...
by opus diablos - the regressive hypocrite party Wed Dec 28, 2011 16:20
..back at the Rattlin' Good Sabre.... http://www.globalresearch.ca/index.php?context=va&aid=28018 the global politburo were polishing their jackboots for the dancefloor
Number of comments per page
<< Back To Newswire
© 2001-2020 Independent Media Centre Ireland. Unless otherwise stated by the author, all content is free for non-commercial reuse, reprint, and rebroadcast, on the net and elsewhere. Opinions are those of the contributors and are not necessarily endorsed by Independent Media Centre Ireland. Disclaimer | Privacy