LONDON. England... 3 Regent Street, S -W.
PARIS, France... 63 Rue Cambon :>?
for only one important office.
expression of republican party sentiment in California.
spirit and self-denial demonstrated by the returns.
mission disclosing the existence of foreign "shipping rings,"
or later going to result in the greatest injury to the nation." Mr.
to combines that grant rebates or special privileges.
trusts and combinations in restraint of trade.
was as follows: • V V .
to a man of his temperament.
kind of a man were nominated nor the ri^ht kind of a platform adoptecl.
were able for the chairmanship of the convention counts for nothing.
number y of important public functions.
"* and these missives are mostly directed at public officials.
in an arson case set for trial iir that county.
'Gallagher, was notified that he would soon be the victim of an attack.
as. evidence that the writers are. unbalanced or merely spiteful.
not at all likely to take shape in positive, action. \u25a0 .
hunting -and fishing, on the premises.
river $57.50, from Memphis $57.50.
people will attend the convention.
where he will spend his vacation.
the er»d of the week.
Indeed, I've felt the awful ruth.
She promised one day to be mine.
The dark ways of my life.
.The Germ that came with it!
Was Mistaken for a Lunatic.
back after so long a time searched out the foundations of the ruined house.
upon his knees. With a penknife he dug at the broken cement feverishly.
"Looking for a nickel," replied the man who wrote "In the Forest."
house because she thinks you may be crazy."
pere, who died in 1876, was a partner in the old time firm of Russell & Co".
Charlie Fair and his sisters, Tessie and Birdie, now Mrs. Oelrichs and Mrs.
Vanderbilt; Katherine Best, now Katherine Gray, the actress; Bruce Palmer.
nothing compared to the damage to the tempers of the victims.
novels, "In the Forest" and "Corrie Who?."
preparing to star in his new play, ."The Whirlpool."
dent Diaz met President Taft in Texas.
remove iodine stains from crochet lace?
CEMENT WALK— W. H. T.. CenteryHle.
or broken rock and sand should be nsed? * .
that will visit the orient.
Perm mutual insurance company. Is at the Pt.
J* J.VHAintAHAS of Loa.AnitelM. (feorg* T.
Barbara have apartments at the St. Francis.
cent arrivals at the Ar|on«ut.
railroad company. Coo« bay. is at the Dale.
, is among the recent arrivals at the Manx.
and rammed well Into place in the .
MINOR-F. C." R.. Fort Braze D~» .
rine corp* is a gnest at th* Stewart.
i» registered at the Fairmont.
w registered at the Palace.


text:
Armavir (cirill betűkkel Армави́р) város Oroszország  Krasznodari határterületén, a Kubán folyó bal partján. Fontos vasúti csomópont. Lakossága 188 832 fő (2010), 193 964 fő (2002), 160 983 fő (1989), 144 000 fő (1969) volt. Korábban a határterület második legjelentősebb ipari központja volt Krasznodar után.

Történelme
A mai város környékét eredetileg abazák lakták. Később a Krími Tatár Kánságból tatárok is települtek ide. A kaukázusi háborúk (1817–1864) következtében az abazák csoportjai a területről az Oszmán Birodalomba menekültek. Armavirt a cserkesz népek ősi földjének is tartják.

A jelenlegi települést 1839-ben ún. cserkesz-örmények alapították és Armjanszkij aulnak (Армянский аул, vagyis 'örmény aul') nevezték el. A jelenlegi nevén 1848 óta ismert, amikor az ősi örmény fővárosról, Armavirról kapta nevét. 1918 és 1920 között, az oroszországi polgárháború alatt több csatát is megvívtak a város környékén.
A Vörös Hadsereg 1943 januárjában szabadította fel a várost.

Gazdaság
Armavirban jelentős a gépipar, a fafeldolgozó- és az élelmiszerfeldolgozó ipar. Régi gépipari vállalata a Kubanzseldormas, amelyet 1933-ban alapítottak.

A várostól 12 kilométerre  délnyugatra van az Armaviri Radarállomás .

Jegyzetek

Források
Armavir hivatalos oldala

A Krasznodari határterület települései

redpajama_set_name: RedPajamaWikipedia


text:
Dollar General reaffirms bid for Family Dollar
G. Chambers Williams III
cwilliams1@tennessean.com
Dollar General Corp. said today that it will continue its pursuit of Family Dollar Stores Inc. despite rejection of its nearly $10 billion cash bid by the Family Dollar board of directors.
Family Dollar cited antitrust concerns in announcing this morning that it would continue on with a previously announced deal to be purchased by Dollar Tree Inc. for about $8.5 billion in cash and stock.
But in a statement this afternoon, Dollar General Chairman and CEO Rick Dreiling discounted those concerns and stressed that Dollar General believes it has the superior bid.
"We have done extensive antitrust analysis using experienced advisers, the results of which confirm that the transaction as proposed is capable of being completed," Dreiling said. "We remain willing to share this analysis with Family Dollar and its counsel and are confident that we will be able to quickly and efficiently resolve any potential antitrust issues."
As for Dollar General's next move, Dreiling said, "We are carefully reviewing and considering our options. Our existing all-cash proposal coupled with manageable antitrust issues continues to make our proposal superior to the current transaction agreement with Dollar Tree."
In its earlier statement, Family Dollar CEO Howard Levine took aim at Dollar General's offer.
"Our board of directors, with the assistance of outside advisers and consultants, has been carefully analyzing the antitrust issues in a potential combination with Dollar General since the beginning of this year," Levine said. "Our board reviewed, with our advisers, all aspects of Dollar General's proposal and unanimously concluded that it is not reasonably likely to be completed on the terms proposed."
On Monday, after making its offer public, Dollar General said that it believed it could quickly address any antitrust issues and was willing to sell off or close nearly 700 of its stores in order to get the necessary approvals.
MORE:Will Dollar General's offer spark all-out bidding war?
In a letter sent to Family Dollar on Wednesday, Dollar General said that it believed the number of stores it was offering to divest was "more than sufficient to take this (antitrust) issue off the table."
Last month, Dollar Tree announced plans to buy Family Dollar for $8.5 billion, significantly broadening its reach as it looks to fend off Wal-Mart, which has been stepping up its courtship of lower-income customers.
The deal would make Dollar Tree the biggest player in the dollar store segment, with its more than 13,000 combined locations eclipsing current leader Dollar General, which has about 11,300.
The bidding war comes as the chains look for ways to grow sales and market share at a time when their core low-income customers remain financially stretched during the economic recovery.
Dollar stores grew during the recession as people across income groups searched for cheaper options. To attract a broader array of customers, they also expanded their offerings to include more groceries and brand-name products, instead of the party favors and other knickknacks people often associated with them.
More recently, however, sales at dollar stores have been suffering because the lower-income customers who go to them are facing persistent job instability and slow wage growth in the aftermath of the recession. Wal-Mart Stores Inc. and Kroger Co. also have been opening smaller store formats to directly compete with dollar stores.

redpajama_set_name: RedPajamaCommonCrawl


text:
\section{Introduction}

Neural networks are able to learn from examples and to find
an unknown rule. Storage capacities and generalization abilities
have been calculated for a variety of network architectures within the
framework of statistical mechanics \cite{1}. Special interest has
been devoted to diluted networks, where only a fraction of the neurons
is connected \cite{2,3}. The most popular example of a diluted network
which appears in Nature is the human brain. Every neuron is connected
to roughly 10000 others, whereas their total number is about $10^{7}$
times larger. Theoretical studies indeed showed, that the effective
storage capacity per neuron in diluted systems can be substantially
larger than in undiluted networks \cite{4}.

So far, dilution of synapses has been considered in addition to
the usual dynamical modification of the bonds, which takes place
in the learning phase \cite{3}.
Motivated by biological observations
\cite{5}, which indicate that at early stages of development
of the brain, synapses are removed if their
strength is not appropriate, we address the question whether it is possible
to store patterns in a network with randomly chosen coupling strengths
only by removing a fraction of these bonds without changing their
strength. This is a nontrivial task,
since given a specific set of patterns it is a priori not clear which
of the bonds have to be removed. Previous studies \cite{8} have considered
a learning algorithm which removes weights that are frustrated in at least
one of the patterns. However, this simple method removes too many weights,
hence the storage capacity increases with $\log N$ only.
In this paper we show, that it is possible to learn of the order of $N$
patterns perfectly and to generalize by removing a fraction of the weights.
We focus on
the perceptron, as it is the simplest network for which a tractable
calculation is feasible. The same procedure however should be applicable
to general network classes such as multi-layer perceptrons or attractor
networks.

The paper is organized as follows: In section two we introduce the model
and calculate the critical storage capacity for a random input--output relation
following the standard statistical mechanics approach established by Gardner
\cite{6}. Section three examines the properties of a perceptron which learns
from an undiluted teacher perceptron. A simple Hebb--like dilution algorithm
that reaches the optimal generalization ability is presented in section four.
In the last section we close with a summary.

\section{The model for random input--output relations}

The diluted perceptron classifies an input pattern $\vec{\xi}^{\mu}  $ according
to:
\begin{equation}
\label{class}
\sigma^{\mu} = \mbox{sgn}\klm{\frac1{\sqrt{N}}\ \sum_{i=1}^N c_i J_i \xi_i^{\mu}} \ .
\end{equation}
Where  $J_i$  are the components of the weight vector drawn at random
from a distribution $P\klm{J_i}$. The $c_i$ are binary decision variables
which can take the values 0 or 1 and determine whether the $i$th coupling
$J_i$ has been removed or is kept, respectively.
For a given set of input--output pairs  $\{ \vec{\xi}^{\mu}  ,s^{\mu} \}_{\mu = 1}^p$
the classification is correct if:
\begin{equation}
\label{correct}
s^{\mu} \frac1{\sqrt{N}}\ \sum_{i=1}^N c_i J_i \xi_i^{\mu} \geq 0 \qquad,
\qquad \forall\ \mu = 1,\ldots ,p \ .
\end{equation}
We suppose that the inputs $\xi_i^{\mu}$ are drawn at random from a
distribution with zero mean and unit variance and we choose
$s_{\mu} = \pm 1$ with
equal probability and independently of the inputs.
The concentration of remaining bonds is defined to be
$c = N^{-1} \sum_{i=1}^N c_i$ and thus lies between 0 and 1.

We are interested whether the maximum number of patterns $p_{max}$,
which are correctly classified, can be of the order of their input
dimension $N$, resulting in a critical storage capacity of
$\alpha_c = p_{max}/N$, for
a fixed value of the concentration $c$ of remaining bonds in the thermodynamic
limit $(N \rightarrow \infty)$. Let us first consider the extreme cases.
For $c=0$ all bonds have been removed and no classification is possible, so
that $\alpha_c(c=0) = 0$. For $c=1$ all bonds are present and the
classification is at random, so that $\alpha_c(c=1) \rightarrow 0$ as
$N \rightarrow \infty$. For intermediate values of $c$ we shall calculate
$\alpha_c(c)$ using Gardner's phase space approach \cite{6,7}.

Form the technical point of view the problem is related to the
Ising perceptron \cite{7,16} and other discrete models \cite{9},
as well as to the knapsack problem \cite{10,11}, where
also binary dynamical variables appear. Note, that in contrast to the
common approach where the couplings $J_i$ are the dynamical variables,
here they represent in addition to the patterns a quenched disorder
which has to be averaged out.

For a fixed concentration $c$, the number of allowed configurations
according to \eref{correct} is given by
\begin{equation}
\label{number}
{\cal N}\klm{c} = \sum_{\{c_i\} }\ \prod_{\mu = 1}^p
              \Theta \klm{
              s^{\mu} \frac1{\sqrt{N}}\ \sum_{i=1}^N c_i J_i \xi_i^{\mu} - \kappa
                         }\
              \delta_{\mbox{\tiny  Kr}} \klm{
                     \sum_{i=1}^N c_i  -  c N
                      } \ .
\end{equation}
We introduced, as usual, the stability parameter $\kappa$ which should be
positive.
The corresponding entropy per bond of the microcanonical ensemble follows
from
\begin{equation}
S\klm{c} = \frac1N\ \skl{\skl{ \mbox{ln}\ {\cal N}\klm{c} }} =
       \lim_{n \rightarrow 0}\ \frac1{n N}
       \mbox{ln} \skl{\skl{ {\cal N}^n\klm{c} }}
       \ ,
\end{equation}
where the last equality results from the replica trick. The quenched
averages $\skl{\skl{\ldots}}$ have to be performed over the distributions
of the patterns, outputs and the couplings. Following the steps of the
calculation by Gardner and Derrida (1988) one can rewrite the replicated number of
configurations using the integral representation of the theta function
in \eref{number}.
The averages over the pattern and output distributions lead to the exponential
factor
\begin{equation}
\label{expon}
\skl{\prod_i \exp \klm{ - \frac{J_i^2}{2 N} \sum_{\mu}
                         \klm{\sum_{\alpha} c_i^{\alpha} x_{\mu}^{\alpha}
                         }^2
                       }
     }_{ \{ J \} } \ .
\end{equation}
Here $x_{\mu}^{\alpha}$ are the conjugate variables to the local fields
$\lambda_{\mu}^{\alpha}$ and $\alpha$ denotes the replica index running
from 1 to $n$. The average over the couplings has still to be done.
A straightforward evaluation however, leads to an expression which cannot
be rewritten in terms of an exponential, as it is convenient, for
the argument of the exponent in \eref{expon} is  not necessarily
infinitesimal due to the sum over all patterns. Instead, we introduce
at this point the order parameters
\begin{eqnarray}
 \label{qalpha}
q^{\alpha \beta} &=& \frac1N\ \sum_{i=1}^N J_i^2 c_i^{\alpha} c_i^{\beta}
 \qquad ( \alpha ,\beta = 1,\ldots , n \ ; \  \alpha < \beta) \\
 \label{Qalpha}
Q^{\alpha} &=& q^{\alpha \alpha} = \frac1N\ \sum_{i=1}^N J_i^2 c_i^{\alpha}
  \qquad (\alpha = 1,\ldots , n)
\end{eqnarray}
and leave the average over the distribution of the couplings for the
integral representations of the delta functions fixing $q^{\alpha \beta}$
and $Q^{\alpha}$. In addition one has a third order parameter $E^{\alpha}$
which fixes the concentration $c$.
We seek for a replica symmetric solution,
i.e. $q^{\alpha \beta} = q$ , $Q^{\alpha} = Q$  and $E^{\alpha} = E$.
Carrying out the sum over the $c_i^{\alpha}$ and using the saddle--point method
we obtain for the entropy in the thermodynamic limit:
\begin{eqnarray}
\label{entropy}
\fl  S\klm{c} =   \alpha \int Dt\ \ln \mbox{H} \klm{
             \frac{\kappa + \sqrt{q} t}{\sqrt{Q - q}}  }
                    + \frac12 F q - \frac12 f Q + \frac12 c E
                    \nonumber \\
 +        \int dJ P\klm{J} \int Dt\
          \ln \klm{ 1 + \exp \klm{ \sqrt{F} \btr{J} t +
                                    \frac12 \klm{f J^2 - F J^2 - E}
                                            }
           }  \ ,
\end{eqnarray}
where $F$ and $f$ are the conjugate order parameters to $q$ and $Q$
respectively and we have used the notations:
\begin{equation}
Dt \equiv dt\ \frac{\exp \klm{-\frac12 t^2} }{\sqrt{2 \pi}}
\qquad , \qquad
\mbox{H}(x) = \int_x^{\infty} Dt \ .
\end{equation}

For the distribution of the couplings $P\klm{J_i}$ we will focus on
two cases: $\btr{J_i}=1$ (note, that the entropy \eref{entropy} depends
only on the absolute value of $J_i$) and $J_i$ drawn from a normal
distribution i.e. $dJ P\klm{J} = DJ$. In the first case it follows that
$Q=c$ from \eref{Qalpha} and the definition of $c$. We solve the saddle
point equations $\partial S / \partial q = \partial S / \partial F
=  \partial S / \partial E = 0$. The critical storage capacity as determined
by Gardner and Derrida \cite{7} would be reached as $q$ approaches $c$.
This however, leads to a negative entropy of the system, as frequently
observed in discrete problems \cite{13}.
We therefore identify the critical storage capacity as the value of
$\alpha$ at which the entropy \eref{entropy} vanishes, as it has become
the standard way by now \cite{9,11,14}.
The resulting curve for the critical storage capacity $\alpha_c$ is
shown in \fref{jrand} as a function of the concentration $c$. As expected
$\alpha_c$ vanishes at both extremes of $c$. We observe a maximum  of
$\alpha_c \approx 0.59$ at $c \approx 0.32$. This means, that about
2/3 of the bonds have to be removed in order to reach the maximal value
of $\alpha_c$. It is somewhat surprising, since as a function of $c$, the
maximal number of configurations $\{c_i\}$ lies at $c=0.5$ and is exponentially
larger in $N$ than for any other value of the concentration. In
section~\ref{algorithm} we will come back to this point.

It is worth to note, that the case $\btr{J_i}=1$ can be mapped onto the
Ising perceptron with couplings 0 or 1, as in \eref{correct} one can define
the new patterns $\chi_i^{\mu} = J_i \xi_i^{\mu}$ and view the $c_i$ as the
couplings. The distribution of the $\chi_i^{\mu}$ has also zero mean and
unit variance. The critical storage capacity for the (0,1) Ising perceptron
has been calculated by Gutfreund and Stein \cite{9} with the zero-entropy (ZE)
ansatz and was found to be 0.59 in agreement with the value found here
at the maximum.

We performed an analysis of the local stability of the replica symmetric
(RS) solution according to de Almeida and Thouless \cite{15}
and obtained the curve
denoted as AT--line in \fref{jrand}. For values of $\alpha$ that lie
above this curve the RS solution is locally unstable.
Our result for $\alpha_c(c)$ lies below the AT--line for all $c$ and is
therefore locally stable. Nonetheless, global stability is not assured.
For this reason we also performed complete enumerations of all possible
dilution vectors for finite systems. The dots with their corresponding
error bars are results for systems with $9 \leq N \leq 24$. For $c=1$
finite size effects lead to $\alpha_c \sim N^{-1}$, since the probability of
classifying one pattern correctly by chance is 1/2. In contrast, for
values of $c$ around the maximum, the numerical results seem to underestimate
the theoretical values. A finite size scaling analysis for $c=1/3$ gives
the extrapolated value of $\alpha_c(1/3) = 0.586 \pm 0.004$ for
$N \rightarrow \infty$ in agreement with the RS--solution (0.58935)
at the same concentration. The general shape of the curve is well
confirmed by the numerical results.

In the case where the $J_i$ are drawn at random from a normal distribution
the picture changes quantitatively. Now, for a fixed $i$,
$\skl{J_i \xi_i^{\mu}} = 0$ as before, but
$\skl{\klm{J_i \xi_i^{\mu}}^2} = J_i^2 \skl{{\xi_i^{\mu}}^2} = J_i^2$ which is in general
different from unity as in the previous case. As a consequence $Q$ is
different from $c$ and we have to solve the additional saddle point equations
$\partial S / \partial Q = \partial S / \partial f = 0$. The ZE condition
yields for $\kappa = 0$ the critical storage capacity depicted also in
\fref{jrand}. Similarly, the curve has a maximum at $c \simeq 0.34$, the
critical capacity however is lowered over a wide range of the concentration
with respect to the binary case. Our interpretation for this effect is that
in the Gaussian case
the dilution variables are mainly used to remove the large couplings
$(\btr{J_i} > 1)$ and only few of them remain for learning. Therefore the
storage capacity $\alpha_c$ is lower than for the binary weights.
The order parameter $Q$ measures the
effective size of the remaining components $J_i$. For all $c$ the RS--solution
gives $Q < c$, supporting the above argument. In addition we measured
the probability
distribution of the size of remaining couplings in complete enumerations,
and found, that large couplings are likely to be removed. The values of
$\alpha_c$ for finite $N$ are displayed in \fref{jrand} as well.

\section{Learning with a teacher}
\label{teacher}

A teacher perceptron $\vec{B}$ classifies a pattern $\vec{\xi}^{\mu}  $ according
to:
\begin{equation}
\label{clteach}
s^{\mu} = \mbox{sgn}\klm{\frac1{\sqrt{N}}\ \vec{B} \cdot \vec{\xi}^{\mu}  }
\end{equation}
We choose a teacher vector which is not diluted and which has the
normalization $\vec{B}^2 = N$. A transition to perfect generalization through
dilution cannot be expected like in other discrete systems where the structure
of teacher and student coincides \cite{23}. The student
perceptron with components $c_i J_i$ can only remove part of
its weights in order to learn perfectly a set of examples given by
\eref{clteach}, resulting in a finite storage capacity.

A straightforward evaluation of the entropy under the RS assumption yields:
\begin{eqnarray}
\label{entroteach}
\fl  S\klm{c} = 2 \alpha \int Dt\
             \mbox{H} \klm{\frac{R\ \sqrt{Q}}{\sqrt{q - Q R^2}}\ t}
             \ln \mbox{H} \klm{
             \frac{\kappa + \sqrt{q} t}{\sqrt{Q - q}}  }
              + \frac12 F q\ - \frac12 f Q\ + \frac12 c E\ + \frac12 G R\ +
                    \nonumber \\
\fl       \int dB P_{\mbox{\tiny T}}\klm{B} \int dJ P\klm{J} \int Dt
          \ln \klm{ 1 + \exp \klm{ \sqrt{F} \btr{J} t +
                                    \frac12 \klm{f J^2 - F J^2 - E - G B J}
                                            }
           }  \ ,
\end{eqnarray}
where the overlap $R = \sum_i c_i J_i B_i / \klm{\sqrt{Q} N}$ between
diluted student and teacher, and its conjugate $G$ has been introduced.
$P_{\mbox{\tiny T}}\klm{B}$ is the distribution of the teacher components.

We focus again on the two cases where $\btr{J_i} = \btr{B_i} = 1$ or both
chosen independently at random from a normal distribution. The corresponding
critical storage capacity $\alpha_c$ determined  with the ZE condition
is shown in \fref{tuncorr} as a function of the concentration of remaining
bonds. Once more, we observe a maximum  of $\alpha_c$ at $c \approx 1/3$.
The critical storage capacity is higher than for random outputs indicating
that the problem, although unlearnable, is easier with examples from a teacher.

An important point to note is that the generalization ability defined as the
probability to classify a new unseen pattern correctly (as the teacher)
is poor compared to the value which could be achieved by {\it intelligent}
dilution. For $c=0.3$, $R \simeq 0.32$ in the binary case $\btr{J_i} =
\btr{B_i} = 1$, whereas an overlap of $R = \sqrt{0.3} \simeq 0.55$ could
be possible according to the following argument. The product $B_i J_i$
should be $+1$ for as many sites as possible in order to maximize $R$.
Since $B_i$ and $J_i$ are drawn at random they will coincide in $N/2$ of
the cases for $N \rightarrow \infty$. For $c \leq 0.5$ we choose all those
$c_i = 1$ for which $B_i J_i = +1$ up to a total number of $c N$, so that
$R_{max} = c N / \klm{ \sqrt{c} N} = \sqrt{c}$. If c is larger than $0.5$
then we also have to add up $\klm{ c N - N/2}$ times a value of $-1$ and
$R_{max} = \klm{N/2 - c N +N/2}/\klm{ \sqrt{c} N} = \klm{1-c}/\sqrt{c}$.
Perfect storage without errors lowers the overlap $R$, an effect known as
{\it overfitting}, which can be overcome by allowing a finite training error
(see section~\ref{algorithm}).

Up to now, we have assumed, that student $\vec{J}$ and teacher
$\vec{B}$ are uncorrelated before the dilution. In biological systems
however, we would rather expect to find synaptical structures, which
are already {\it prepared} for a specific task before the learning process
starts. In our model we can mimic it by allowing an initial positive overlap
$R_0$, and therefore certain similarity, between teacher and student.
Let us choose
\begin{equation}
P\klm{J} = \frac{1+R_0}2\ \delta\klm{J-B}\ +\ \frac{1-R_0}2\ \delta\klm{J+B}
\end{equation}
with $0 \leq R_0 \leq 1$. In $(1+R_0) N /2$ of the cases the components
of teacher and student will coincide and in $(1-R_0) N /2$ they will be
opposed. Since we are not allowed to change the values of $J_i$, but at best
to remove the bond, we will not reach perfect generalization even for
large $R_0$. The same would even hold if we chose a diluted teacher.

We have calculated the critical storage capacity for $\btr{B_i}=1$ as a
function of $R_0$ and $c$ and find the results plotted in \fref{tcorr}.
For $R_0=0$ we recover the uncorrelated case, whereas with increasing $R_0$
the storage capacity is enhanced for all $c$. At the same time the maximum
of the curve moves towards higher values of the concentration, as less bonds
have to be removed in order to mimic the teacher. For $R_0=1$ teacher and student
are identical before dilution. Although this situation is of less relevance
from the biological point of view, it offers an interesting physical
solution. If a fraction of the bonds is now removed, we obtain a finite storage
capacity. For increasing concentration $c$, we would expect the capacity
to increase at the same time. Above $c \simeq 0.82$ however, we find, that
it decreases and finally tends to zero for $c \rightarrow 1$. At $c=1$
we have $\alpha_c = \infty$ per definition, hence $c=1$ is a singular point.
This surprising behaviour may
be understood if we look at the annealed approximation \cite{22} for the
entropy
\begin{equation}
\label{sann}
S_{ann}\klm{c} = \frac1N\ \mbox{ln} \skl{\skl{ {\cal N}\klm{c} }}.
\end{equation}
The averaged number $\skl{\skl{ {\cal N}\klm{c} }}$ of allowed configurations
for fixed $c$ is simply given by the total number ${N \choose c N}$
times the probability that $p$ patterns are classified correctly:
\begin{equation}
\label{naver}
\skl{\skl{ {\cal N}\klm{c} }} =  {N \choose c N}
                                 \klm{1-\frac1{\pi} \arccos R}^p\ .
\end{equation}
For $R_0 =1$ we have $R=\sqrt{c}$ and using the ZE--condition one obtains
in the thermodynamic limit for the critical storage capacity in the annealed
approximation
\begin{equation}
\alpha_{ann}\klm{c} = \frac{c\ \mbox{ln} c + \klm{1-c}\ \mbox{ln} \klm{1-c}}
                           {\mbox{ln} \klm{1 - \frac1{\pi} \arccos \sqrt{c}}}\ ,
\end{equation}
which for $c \rightarrow 1$ results in
\begin{equation}
\alpha_{ann}\klm{c \rightarrow 1} \rightarrow \lim_{c \rightarrow 1} - 4 \pi
        \sqrt{1-c}\ \mbox{ln} \sqrt{1-c} \rightarrow 0.
\end{equation}
Since $\alpha_{ann}\klm{c}$ is an upper bound for $\alpha_c\klm{c}$, the
critical storage capacity has to decrease to zero as well when $c$ tends to 1.
From \eref{naver} we see, that although the probability of classifying one
pattern correctly tends to one for $c \rightarrow 1$, at the same time the
total number of dilution vectors decreases rapidly, such that the averaged
number of allowed configurations is no longer exponentially large in $N$.
Perfect storage of patterns is different from optimal generalization, which
in this case becomes better the closer $c$ is to 1.
In \fref{tcorr} we also included results from complete enumerations of
systems with $25 \leq N \leq 400$ for $R_0 =1$ and $c$ close to 1. They confirm
that $\alpha_c$ decreases in this region.

\section{A simple Hebb--like dilution algorithm}
\label{algorithm}

As we have seen in the previous sections, Gardner's method is very powerful
when asking if there exists, on average, a set of $c_i$ such that all perceptron
conditions \eref{correct} are satisfied, but it does not provide us with
the corresponding dilution vector for a specific set of patterns $\vec{\xi}^{\mu}  $, outputs
$s^{\mu}$ and couplings $J_i$.
The development of a learning , or in our case dilution algorithm,
is an independent task, which in the case of binary variables
$c_i=0,1$ becomes extremely difficult and compares with the binary
perceptron problem or the knapsack problem. In the worst case, the
number of computational steps towards the optimal solution scales
exponentially with the size $N$ of the system. The most successful
approaches try to find the global minimum of a
properly defined energy function, which penalizes the violation
of constraints, by using sequential descent \cite{17}
or simulated annealing \cite{18} strategies.
Although an algorithm based on mean field annealing has proven
to be very effective in finding solutions to the knapsack problem
\cite{10}, none of the known techniques yields the critical
values for the storage capacity predicted by Gardner calculations.
Typically, in large systems, the solutions still violate a finite
fraction of the imposed constraints.

In view of these general difficulties, we cannot expect to find the
optimal dilution vector $\vec{c}$, which allows us to store perfectly
$\alpha_c N$ many patterns for large $N$, within reasonable time.
We rather present here a simple dilution algorithm,
which gives us an insight into the basic properties of the solutions
and has optimal generalization ability for $\alpha \rightarrow \infty$.

Our aim is to fulfill all constraints \eref{correct} by removing
a fraction $\klm{1-c} N$ of the bonds $J_i$. If we think of the terms
$J_i \xi_i^{\mu}  s^{\mu}$ as matrix elements $a_{i \mu}$ of a $N \times p$
matrix, then we want all $p$ vertical sums $\sum_{i=1}^N a_{i \mu}\
(\mu = 1, \ldots, p)$ to be positive. For this purpose, we are allowed
to remove $\klm{1-c} N$ rows of the matrix. The idea is to remove those,
 which contain many negative elements $a_{i \mu}$, since these
contribute in many vertical sums negatively. Let us take away all
rows $i$ with horizontal sums $\sum_{\mu =1}^p a_{i \mu}$ smaller than
a threshold $h$, so that
\begin{equation}
\label{dilalg}
c_i = \theta \klm{ \frac1{\sqrt{N}} \sum_{\mu=1}^p J_i \xi_i^{\mu}  s^{\mu} - h}
\end{equation}
The larger $h$, the more $c_i$ will be zero, leading to a lower
concentration $c$. From a different perspective one can view \eref{dilalg}
as comparing the Hebb couplings
$H_i = \sum_{\mu =1}^p \xi_i^{\mu}  s^{\mu} /\sqrt{N}$
of the problem (Hopfield 1982) with $J_i$, the ones imposed at random.
If their product $H_i J_i$ is larger than the threshold $h$, then $J_i$
is accepted as coupling strength. As $h$ becomes more and more positive,
this is only the case if $H_i$ and $J_i$ agree in their sign.

Let us now give a simple derivation of the critical storage capacity
for random input--output pairs and $\btr{J_i} = 1$, which results from
\eref{dilalg} by allowing a certain percentage of errors. For simplicity,
suppose that $\xi_i^{\mu}  = \pm 1$ with equal probability. Then,
$a_{i \mu} = \pm 1$ with equal probability and the horizontal and vertical
sums have, in the limit $N \rightarrow \infty$, a Gaussian distribution
with zero mean and variance $p$ and $N$,
respectively. According to \eref{dilalg} we remove all horizontal sums
which are smaller than $h$. The resulting concentration is
$c = \int_{h/\sqrt{\alpha}}^{\infty} Dz = \mbox{H} \klm{h/\sqrt{\alpha}}$
and the new mean of the horizontal sums is
$\skl{\mbox{HS}} = \sqrt{p} \exp \klm{-\case12 h^2/\alpha}/\sqrt{2 \pi c^2}$.
The new vertical sums have still a Gaussian distribution, but now with
mean $\skl{\mbox{VS}} = c \skl{\mbox{HS}}/\alpha  $ and variance
$c N$ for $N \rightarrow \infty$. The fraction of errors is thus
equal to the integral over the Gaussian tail below zero:
\begin{equation}
\label{errrand}
\fl
\mbox{Learning Error} \equiv \epsilon_{\mbox{\tiny L}} =
                      \int_{- \infty}^0 \frac1{\sqrt{2 \pi}}
                      \frac1{\sqrt{c N}}
                      \exp \klm{-\frac12 \frac{\klm{z-\skl{\mbox{VS}}}^2}
                                             {c N}  }
                       = \mbox{H} \klm{\frac{\skl{\mbox{VS}}}{\sqrt{c N}}}
\end{equation}
For a fixed error $\epsilon_{\mbox{\tiny L}} = \mbox{H} \klm{A}$ and fixed
concentration $c$, we obtain for the storage capacity:
\begin{equation}
\label{alfalg}
\alpha \klm{c,A} = \frac{ \exp \klm{ - \klm{\mbox{H}^{-1}\klm{c}}^2 }}
                        { 2 \pi c A^2 }
\end{equation}
with $\mbox{H}^{-1}\klm{x}$ the inverse function of $\mbox{H}\klm{x}$.
\Fref{jbinrnalg} shows the resulting $\alpha$ for $A=1\
(\epsilon_{\mbox{\tiny L}} \simeq 15.9 \%)$ as a function of $c$. The maximum
of $\alpha$ is reached at $c \approx 0.27$, which is not too far from
$0.32$, the concentration at which the maximal $\alpha_c$ was obtained
according to the Gardner calculation with the ZE condition
(see \fref{jrand}). Also the shape
of the curve is similar, for different values of $\epsilon_{\mbox{\tiny L}}$
(or $A$), $\alpha\klm{c,A}$ is simply rescaled.
At first sight one would expect that as $h$ increases, also the mean of
the vertical sums $\skl{\mbox{VS}}$ increases, leading to a lower learning
error. This however is prevented by two effects. First, as $\skl{\mbox{HS}}$
increases, $\skl{\mbox{VS}}$ does not so necessarily, since
$\skl{\mbox{VS}} \sim c \skl{\mbox{HS}}$ and $c$ is lowered dramatically
with increasing $h$. As a result, the maximal storage capacity would be at
$c = 0.5$ for fixed $\epsilon_{\mbox{\tiny L}}$. The second effect is, that
the width of the distribution of vertical sums is proportional to $\sqrt{c}$,
lowering the learning error for smaller values of $c$. If $c$ is too small
however, the gain is compensated by the exponential factor in $\skl{\mbox{HS}}$,
which tends to zero. As a consequence, the maximum storage capacity for fixed
$\epsilon_{\mbox{\tiny L}}$ lies at a value of $c$ somewhat smaller than $0.5$.

In \fref{epsalg} the learning error $\epsilon_{\mbox{\tiny L}} \klm{c,\alpha}$
is plotted as a function of $\alpha$ for $h=0\ \klm{\Leftrightarrow c=0.5}$.
For small $\alpha$, $\epsilon_{\mbox{\tiny L}}$ is small, as typical for the
Hebb couplings and tends to 0.5 for $\alpha \rightarrow \infty$.

A more interesting quantity is the generalization error
$\epsilon_{\mbox{\tiny G}}$, defined as the probability to classify correctly
a new pattern ${\vec{\xi}}^{\ 0}$, which does not belong to the training set. For a
random input--output relation $\epsilon_{\mbox{\tiny G}} = 0.5$, since the
classification $s^0$ of ${\vec{\xi}}^{\ 0}$ is at random. In the presence of a teacher
$\vec{B}$ however, we can expect to reach a lower generalization error. A
straightforward evaluation (see e.g. \cite{20}) yields for
$\btr{J_i} = 1$:
\begin{eqnarray}
\epsilon_{\mbox{\tiny L}} \klm{c,\alpha} =
      2 \int_0^{\infty} Ds\ \mbox{H} \klm{\frac{s R \sqrt{c} +q}
                                              {\sqrt{c}\ \sqrt{1-R^2}}} \\
\epsilon_{\mbox{\tiny G}} \klm{c,\alpha} = \frac1{\pi} \arccos R \,
\end{eqnarray}
with
\begin{eqnarray}
c = \frac12\ \skl{\mbox{H} \klm{ h_- }
               + \mbox{H} \klm{ h_+ }}_{B_i}  \\
R = \frac1{2 \sqrt{c}}\ \skl{ B_i\ \klm{
                 \mbox{H} \klm{ h_- }
               - \mbox{H} \klm{ h_+ } } }_{B_i}
               \label{R}   \\
q = \frac1{2 \sqrt{2 \pi \alpha}}\ \skl{
                 \exp \klm{-\frac12 h_-^2 } +
                 \exp \klm{-\frac12 h_+^2 }  }_{B_i} \,
\end{eqnarray}
where
\begin{equation}
h_+ =   \frac{h}{\sqrt{\alpha}} + B_i\ \sqrt{ \frac{2\alpha}{\pi} }
           \qquad , \qquad
h_- =   \frac{h}{\sqrt{\alpha}} - B_i\ \sqrt{ \frac{2\alpha}{\pi} }  \ .
\end{equation}
The average is to be performed over the distribution $P_{\mbox{\tiny T}}\klm{B_i}$
of the teacher components $B_i$. As before, $c$ is the concentration of
remaining bonds and $R$ the normalized overlap between diluted student and
teacher. The parameter $q$ (not to be mismatched with $q$ defined by
\eref{qalpha}) does not seem to have a direct physical meaning, but in a
certain way it does take into account the randomness which is still inherent
for $R$ less than 1. In the extreme case where we ignore the teacher by
setting all $B_i =0$, we obtain $R=0\ ,\ q = \exp \klm{-\case12 h^2/\alpha}/
\sqrt{2 \pi \alpha}$ and recover for $\epsilon_{\mbox{\tiny L}}$ the expression
for random outputs \eref{errrand}. The above result for
$\epsilon_{\mbox{\tiny L}}$ and $\epsilon_{\mbox{\tiny G}}$ is similar to
the one obtained with the Clipped--Hebb algorithm for the perceptron
\cite{21}. This does not surprise, as our prescription
\eref{dilalg} is also in some sense a way of {\it clipping} the bonds.

For all $\alpha$ and $c$ we find
$\epsilon_{\mbox{\tiny L}} < \epsilon_{\mbox{\tiny G}}$
and for $\alpha \rightarrow \infty$ and $c$ fixed,
$\epsilon_{\mbox{\tiny L}} \rightarrow \epsilon_{\mbox{\tiny G}}$, as it
should.
The most important feature however is, that in this same limit the optimal
generalization error is reached. For $\alpha \rightarrow \infty$ and $c$
fixed by choosing $h$ appropriately, we obtain from \eref{R} for
$\btr{B_i} =1$:
\begin{equation}
R = \cases{\sqrt{c} &for $c \leq \frac12$  \\
            \frac{1-c}{\sqrt{c}} &for $c > \frac12$ \\ }
\end{equation}
which is exactly the maximal overlap that can be achieved by removing
$\klm{1-c} N$ bonds (see section~\ref{teacher}). For a teacher with
Gaussian $B_i$ we find in the same limit:
\begin{equation}
R = \frac1{\sqrt{2 \pi c}}\ \exp \klm{- \frac12\ \klm{\mbox{H}^{-1}\klm{c}}^2 }
\end{equation}
which is also the optimal overlap for this case, as can be shown easily.

\section{Conclusion}

We have shown, that it is possible to store information in a neural network
solely by dilution of synapses. Using Gardner's
phase space approach the critical storage capacity of a
perceptron with random coupling vector was calculated
as a function of the concentration of remaining
bonds for random input--output relations. We found a maximum
$\alpha_c \simeq 0.6$ of the
capacity at $c \simeq 1/3$, i.e.\ after 2/3 of the bonds have been removed.
Similar results are obtained if the desired outputs are generated by
an undiluted teacher perceptron, whose coupling vector is uncorrelated
to the initial student vector. In this case perfect learning is possible
up to a critical capacity $\alpha_c(c)$, only. If the initial network has
some preknowledge, i.e.\ if there is a nonzero overlap between teacher
and initial student vector we find, that the maximum of the capacity
moves towards higher concentrations $c$.

The problem of finding the subset of couplings which have to be removed
is extremely difficult and compares to problems which belong to the
NP--complete class. Nevertheless, properties of a Hebb--like learning
algorithm, which allows for a finite fraction of errors in the training
set, were calculated. The algorithm reaches for $\alpha \rightarrow \infty$
the maximal overlap between the diluted random student and the undiluted
teacher and thus the lowest possible generalization error. As the Hebb--rule,
it is a local algorithm that accumulates information about the training set
and decides at the end of this batch process which of the couplings are
removed. More desirable would be to find a prescription that {\it on--line}
removes disturbing couplings. In contrast to common on--line learning
algorithms \cite{12}, where infinitesimal changes of the coupling vector
are performed in every time step, here we would remove single couplings.
Whether this procedure can give satisfactory results, similar to those
obtained in the batch process, also for unlearnable rules, remains open
and should be studied in the future.

\ack
We would like to thank M Opper and A Mietzner for useful discussions.
This work has been supported by the Deutsche Forschungsgemeinschaft.

\Bibliography{10}
\bibitem{1} Watkin T L H, Rau A and Biehl M 1993 \RMP {\bf 65} 499 \nonum
            Opper M and Kinzel W 1995 {\it Models of Neural Networks III}
            ed E Domany \etal (Berlin: Springer-Verlag) p~151
\bibitem{2} Sompolinsky H 1987 {\it Heidelberg Colloquium on Glassy Dynamics}
            ed J L van Hemmen and I~Morgenstern (Berlin: Springer-Verlag) p~485
            \nonum
            Canning A and Gardner E 1988 \JPA {\bf 21} 3275
            \nonum
            Boll\'e D and van Mourik J 1994 \JPA {\bf 27} 1151
            \nonum
            Kree R and Zippelius A 1991 {\it Models of Neural Networks} ed
            E Domany \etal (Berlin: Springer-Verlag) p~193
\bibitem{3} Bouten M, Engel A, Komoda A and Serneels R 1990 \JPA {\bf 23} 4643
\bibitem{4} Kuhlmann P, Garc\'es R and Eissfeller H 1992 \JPA {\bf 25} L593
\bibitem{5} Scheich H \etal 1991  {\it Memory , Organization and Locus
            of Change} (Oxford:  Oxford University Press)
\bibitem{6} Gardner E 1988 \JPA {\bf 21} 257
\bibitem{7} Gardner E and Derrida B 1988 \JPA {\bf 21} 271
\bibitem{8} Kinzel W 1985 \ZP B {\bf 60} 205 \nonum
            van Hemmen J L and van Enter A C D 1986 \PR A {\bf 34} 2509
\bibitem{9} Gutfreund H and Stein Y 1990 \JPA {\bf 23} 2613
\bibitem{10} Ohlsson M, Peterson C and S\"oderberg B 1993 {\it Neural Comput.}
             {\bf 5} 808
\bibitem{11} Korutcheva E, Opper M and L\'opez B 1994 \JPA {\bf 27} L645
\bibitem{12} Hertz J, Krogh A and Palmer R G 1991 {\it Introduction to the
             Theory of Neural Computation} (Redwood City, CA:
             Addison--Wesley)
\bibitem{13} Iwanski J, Schietse J and Bouten M 1995 \PR E {\bf 52} 888
\bibitem{14} Urbanczik R 1994 {\it Europhys. Lett.} {\bf 26} 233
\bibitem{15} de Almeida J R L and Thouless D J 1978 \JPA  {\bf 11} 983
\bibitem{16} Krauth W and M\'ezard M 1989 \JP {\bf 50} 3057
\bibitem{17} K\"ohler H, Diederich S, Opper M and Kinzel W 1990 \ZP B
             {\bf 78} 333
\bibitem{18} Kirkpatrick S, Gelatt C D and Vecchi M P 1983 {\it Science}
             {\bf 220} 671 \nonum
             Horner H 1992 \ZP B {\bf 86} 291 \nonum
             Patel H K 1993 \ZP B {\bf 91} 257
\bibitem{19} Hopfield J J 1982 {\it Proc. Natl. Acad. Sci. USA} {\bf 79} 2554
\bibitem{20} Vallet F 1989 {\it Europhys. Lett.} {\bf 8} 747
             \nonum
             Vallet F and Cailton J G 1990 \PR A {\bf 41} 3059
\bibitem{21} Van den Broeck C and Bouten M 1993 {\it Europhys. Lett.}
             {\bf 22} 223
\bibitem{22} Seung H S , Sompolinsky H and Tishby N 1992 \PR A {\bf 45} 6056
\bibitem{23} Gy\"orgyi G 1990 \PR A {\bf 41} 7097
\endbib

\Figures
\begin{figure}
\hbox{
\epsfysize 12cm
\hbox to \hsize{\hss
\epsfbox{pl1p.eps} \hss}}
\caption{The critical storage capacity $\alpha_c$ as a function of the
         concentration $c$ for $\kappa = 0$ and random outputs. The solid
         line represents the zero entropy solution for $\btr{J_i}=1$.
         The dots with error bars are results from complete enumerations
         of systems with sizes $9 \leq N \leq 24$. The long dashed
         curve is the corresponding AT--line beyond which the RS--solution becomes
         locally unstable. The dashed curve is the critical storage
         capacity for $J_i$ drawn from a normal distribution.}
\label{jrand}
\end{figure}
\begin{figure}
\hbox{
\epsfysize 8cm
\hbox to \hsize{\hss
\epsfbox{pl2p.eps} \hss}}
\caption{The critical storage capacity $\alpha_c$ as a function of the
         concentration $c$ for $\kappa = 0$ and outputs from an undiluted
         teacher perceptron. The upper curve is for $\btr{J_i}=\btr{B_i}=1$
         and the lower for $J_i$ and $B_i$ both Gaussian. The dots are
         numerical results from complete enumerations of systems with
         sizes up to $N=20$.}
\label{tuncorr}
\end{figure}
\begin{figure}
\hbox{
\epsfysize 8cm
\hbox to \hsize{\hss
\epsfbox{pl3p.eps} \hss}}
\caption{The critical storage capacity $\alpha_c$ as a function of the
         concentration $c$ for $\kappa = 0$ and outputs from a teacher
         with $\btr{B_i}=1$ for different values of the initial overlap
         $R_0$ between $\vec{J}$ and $\vec{B}$. $R_0 = 1.0, 0.9, 0.7,
         0.3, 0.0$ (from top to bottom). The dots are numerical results
         from complete enumerations of systems with sizes $25 \leq N
         \leq 400$ and $c$ close to 1.}
\label{tcorr}
\end{figure}
\begin{figure}
\hbox{
\epsfysize 8cm
\hbox to \hsize{\hss
\epsfbox{pl4p.eps} \hss}}
\caption{The storage capacity $\alpha$ as a function of the concentration
         $c$ for $\btr{J_i}=1$, $\kappa = 0$ and $A=1\
         (\epsilon_{\mbox{\tiny L}} \simeq 15.9 \%)$. The lower curve is
         for randomly chosen outputs and the upper curve for outputs from
         a teacher with $\btr{B_i}=1$.}
\label{jbinrnalg}
\end{figure}
\begin{figure}
\hbox{
\epsfysize 8cm
\hbox to \hsize{\hss
\epsfbox{pl5p.eps} \hss}}
\caption{Learning error $\epsilon_{\mbox{\tiny L}}$ (solid lines) and
         generalization error $\epsilon_{\mbox{\tiny G}}$ (dashed lines)
         as a function of $\alpha$ for $h=0\ \klm{\Leftrightarrow c=0.5}$
         and $\btr{J_i}=1$ for random outputs, a binary teacher
         and a Gaussian teacher.}
\label{epsalg}
\end{figure}

\end{document}

redpajama_set_name: RedPajamaArXiv


text:
Touring Portugal's Lisboa wine country
Where wine is part of history and everyday life
A great pleasure in moving to Portugal has been our introduction to all manner of excellent wines: sparkling, red, wine and fortified. One of our favorite memories from the past year was a tour through Lisboa wine country, aka of Oeste (West Lisbon). The region was previously known as Extremadura. What a confusing tangle of labels for a countryside of gorgeous simplicity!
Last spring, traveling with Patrícia and Luis of Singulartrips, we explored a landscape that climbs from the Atlantic coast through the hilly turf to Portugal's mid-section. We visited historical vineyards, vineyards that celebrate the country's seafaring past, and a small-production winemaker plying his craft in a former village school house. We discovered that each of the wines of the region presents a unique character. Here's a recap of our trip.
AdegaMãe winery, Torres Vedras
AdegaMãe, launched in 2011 as a project of the Riberalves Group, Portugal's largest codfish distributor, is set among rolling hills near Torres Vedras, just ten kilometers inland from the Atlantic. The vineyards are planted with both indigenous and international grape varieties, and the aim is to be one of the most modern winemaking facilities in the country, producing wines on a par with other national and international regions.
Lisboa Region wines–minerality, freshness and even some salinity"
–Diogo Lourenço, winemaker
It's a strategy that is garnering results. In October, AdegaMãe introduced the Terroir brand, high-end wines that aim to convey the genuine traits of the Lisboa Region. These are limited-edition wines, to be issued only in years of outstanding quality. Earlier this week, one of them, AdegaMãe Terroir White 2013, received the Revista de Vinhos' Excellence Award—the highest award in Portuguese winemaking. Previously, Revista de Vinhos named AdegaMãe the 2015 Wine Company of the Year.
Ricardo and Bernardo Alves named the winery AdegaMãe to honor their mother.
Our host, winemaker Diogo Lourenço, pours a welcome drink at AdegaMãe.
Dory, the winery's signature label, is dedicated to Portugal's cod fishermen. In July, AdegaMãe launched its newest Dory wines at an event in Lisbon aboard the Creoula, a four-masted schooner, and former cod-fishing vessel.
The village of Cheleiros, just thirty minutes' drive from Lisbon, was historically important for wine production and known for the quality of its vineyards and orchards. Its slopes and terraces and unique microclimate mark the boundary between the influence of the sea and the Cheleiros riverside. Kings bought their wine here, en route to their palaces at Mafra. The ManzWine logo is a tribute to the village's Roman bridge, once used to carry stones to build those palaces.
André Manz and his family came to Cheleiros in 2004, and planted just two acres with three grape varieties, making his first wine exclusively with an almost extinct grape, Jampal. "I don't want to produce a lot of wine," he said at the time. "I want to produce a great wine." By many accounts, that is exactly what he has done, and the winery is known for its use of Jampal.
Today, ManzWine comprises nine brands produced from local grapes, and in partnerships in Douro and Setubal, two of the owner's favorite regions. Among the wines we tasted: Dona Fatima, a single varietal named after André Manz's mother—with a unique fruity flavor and buttery mouth feel—and Cheleiros, named after the original vineyard of the winemaker—concentrated and imposing, with notes of blackberries.
While restoring the village, the winemaker installed his production facility in a former primary school. The output is very limited, and even as the winery approaches capacity, the emphasis remains on the production of boutique wines, rather than expansion.
One hundred years ago, there were 43 wine presses in the village. There is only one now, in the winery's shop, showcased with other wine-growing artifacts in a small museum gallery. The buildings of Cheleiros village are open to the public for guided visits and wine tastings.
The Roman bridge at Cheleiros once used to carry stones for building royal castles
Display of ManzWine labels and in the museum
Display of winemaking implements in the ManzWine museum
Quinta do Grandil
Quinta do Gradil is one of the oldest estates in Cadaval Municipality and is significant both historically and architecturally. The property was owned in the 18th century by the Marquês de Pombal, known for his leadership following the 1755 Lisbon earthquake. A 'Pombaline' palace, chapel, mill, granary add to the quinta's charm. On our visit, dodging raindrops, we enjoyed a memorable luncheon at the restaurant and ended our day with a wine tasting at a windmill atop nearby Monte Montejunto
The vineyards of Quinta do Gradil have been cultivated for several hundred years. It's more than 200 hectares are planted with both Portuguese and international grape varieties.
The restaurant is a casual setting filled with art, its food inspired by the farm's location between land and sea. We enjoyed grilled octopus with punched potatoes, salad and a dessert of regional Rocha pears filled with goat cheese. Sauvignon Blanc and Arinto, a balanced, well-structured wine with intense fresh fruit and mineral notes, and vibrant acidity were an excellent accompaniment.
After touring the winery, it was time for a wine-tasting at nearby Aviz windmill. Our hosts were miller and master craftsman Miguel Nobre and his nephew, who served us freshly baked bread stuffed with choriço while Bruno Gomes of Quinta do Grandil poured the wine. The miller showed us how the milling process works, and how terracotta pots provide the 'song of the wind', indicating its direction and strength. Before we left, the sun came out to shine a golden light on a fairytale scene.
Quinta do Gradil plans to restore the palace of the Marquês de Pombal.
The vineyards of Quinta do Gradil have been cultivated for several hundred years.
Food at Quinta de Grandil's restaurant is inspired by the quinta's location between land and sea.
Quinta do Gradil is active in wine tourism, offering tastings, horse rides, harvest events and other attractions.
The Lisbon wine region is made up of nine sub-regions, and our wine tour was too short to do more than giving us an idea of the types, and quality of wines, produced. Clearly, more research is required! If you're thinking of adding West Lisbon to your travel itinerary, here are a few suggestions for you:
Enjoy a visit to these and other wineries in the region on one of the West Lisbon excursions offered by Singular Trips.
Let Wine Folly help with background on Portuguese wine regions and grape varieties (with maps).
Read tips for visiting the region in André Ribeirinho'ss introduction to wines of Lisbon and Alentejo for Decanter.
For more about winery visits around Lisbon (including the ones we visited in Oeste), read what Julie Fox has to say!
Previous articleFusterlandia
Next articleIt's February–Let's celebrate Portuguese wine!
Christmas memories for a pandemic year
Alpabzug or Désalpe? Swiss cows come home
Living like a local in Lisbon
Tom - October 29, 2014 4
Eating light in Porto, Portugal
Anita - July 13, 2015 16
Anita's Feast celebrates ten years
Anita - February 10, 2020 0
Sao Miguel and Terceira offer a taste of the Azores


text:
Security Patterns
=================

Security Patterns are solution structures, which prescribe a solution to
a security problem arising in a context. Many components and connectors
in secure system architectures are instances of security patterns but,
in many cases, the reasoning for a given pattern's inclusion is not
always clear. The requirements needed to realise these patterns are also
often omitted, making the job of reasoning about the consequences of
situating the pattern difficult. Moreover, security patterns may be
described in a context, but not all collaborating assets in a security
pattern may be evident in all possible contexts of a system's use. The
following sections describe how CAIRIS treats security patterns and
deals with these weaknesses.

Security Patterns in CAIRIS consist of the following elements:

-  A description of the context a pattern is relevant for.

-  A problem statement motivating the need for the pattern.

-  A solution statement describing the intrinsics of the pattern.

-  The pattern structure, modelled as associations between collaborating
   asset classes.

-  A set of requirements, which need to be fulfilled in order to realise
   the pattern.  These requirements are based on template requirements.

Before a security pattern can be defined in CAIRIS, template assets --
which represent the collaborating asset classes -- need to be first
defined.

Before a security pattern can be situated in CAIRIS environments, the
environments themselves need to be first created.

Create a template asset
-----------------------

.. figure:: TemplateAssetForm.jpg
   :alt: Template Asset form

Template assets can be best described as context-free assets. When they
are created, template assets do not form part of analysis unless they
are implicitly introduced. This 'implicit introduction' occurs when a
security pattern is situated.

The Template Patterns dialog can be opened by selecting the
Options/Template Assets menu option.

The process for creating, updating, and deleting a template asset is
almost identical to the processes uses for normal assets. The only
difference is the lack of environment-specific properties. Security
properties are only defined once for the asset.

To situate an asset in an environment, click on the situate button in the template assets list, and
specify the environments to situate the template asset in. After a
template asset is situated within an environment, these properties
should be revised in the assets generated on the basis of these. This is
because the values associated with the template asset properties may not
be inline with assumptions held about Low, Medium, and High assets in
the specification being developed.

Create a security pattern
-------------------------

.. figure:: SecurityPatternForm.jpg
   :alt: Security Pattern form

-  Select the Risks/Security Patterns menu option to open the Security
   Patterns table, and click on the Add button to open the Security
   Pattern dialog form.

-  Enter the security pattern name, and description of the context the security pattern is relevant for,
-  a problem description motivationg the security pattern, and the intrinsics of how the security pattern
   solves the pre-defined problem.

-  Click on the Structure page, and the Add button in the associations table to add associations between template assets; these associations form the collaborative structure for the pattern. The procedure for entering associations is based on that used for associating assets.

-  Click on the Requirements page, and the Add button in the requirements
   table to add names of template requirements needing to be satisfied to realise
   the pattern.

-  Click on the Create button to add the new security pattern.

-  Existing security patterns can be modified by double clicking on the
   security pattern in the security patterns table, making the
   necessary changes, and clicking on the Update button.

-  To delete a security pattern, click on the Delete button besides pattern to delete in the
   security patterns table.

Situate a security pattern
--------------------------

-  To introduce a security pattern into the working project, select the Risks/Security Patterns menu, click on the situate button for the pattern to be situated.

-  Select the environment to situate the environment in, and click on Ok to situate the pattern.

Template assets will be instantiated as assets, and situate in the
stipulated assets. Requirements associated with the pattern, will be
introduce and associated with the stipulated assets in the pattern
definition. These assets will be ordered based on the order of
definition in the pattern structure.

redpajama_set_name: RedPajamaGithub


text:
Q: Vim: Is there a way to replace all occurrences of a word without typing it? I have the following code:
  long_word: ISubscription['long_word'];

This is what I normally do:
shift v
:s/long_word/new_word/g

It's tedious to have to type the word I'm trying to replace. So sometimes I just do
ciw
new_word
esc
$
hhhh
.

which feels inefficient.
Is there a way to do something like ciw but on the whole line?

A: You could do:
:s/short-string/new_word

to set the replacement pattern, then put the cursor on long_word and type * or # to set the search pattern, and then do:
g&

to replace all occurrences of long_word with new_word.  It's not super elegant, but it's not terrible.
Make sure short-string is easy to type and not actually a string that occurs on the current line (you don't actually want to replace it!). I find short-string usually ends up being something like kjkjkjj.  Not elegant, but effective.

A: First things first: V is not necessary because the default range for :help :s is the current line. So that's one improvement.
Then there is the unavoidable fact that the string to substitute has to be defined, somehow. You can't just have your cursor anywhere on the line and expect Vim to figure it out on its own. The only way to start the process is to literally point Vim at the string in question… by placing your cursor on it.
So, assuming the string to substitute is under your cursor and matches Vim's definition of a word (see :help word), you can use :help c_ctrl-r_ctrl-w to insert it in the command-line:
:s/<C-r><C-w>/new-word/g<CR>

If you have to do that often, you could save yourself a lot of effort with a simple mapping:
nnoremap <key> :s/\<<C-r><C-w>\>//g<Left><Left>

Note the \< and \>, which makes sure that only whole words are matched.
Now, the string to substitute might not be a word (or a WORD, in which case you would use :help c_ctrl-r_ctrl-a). In that case, you can still visually select it, yank it, and insert it in the command-line:
v{motion}
y
:s/<C-r>"/new-word/g<CR>

which, of course, could also be turned into a mapping:
xnoremap <key> y:s/<C-r>"//g<Left><Left>

redpajama_set_name: RedPajamaStackExchange


text:
Is This the Year for Electric Cars?
2017 experienced a sharp decrease in the number of vehicles sold for US automakers, putting an end to seven years of sales growth. Despite this drop, PVMI director John Paul MacDuffie asserts that "it was a strong year," as electric cars and autonomous vehicle are on the rise.…Read More
January 19, 2018 | Knowledge@Wharton, Podcast, PVMI In the News	autonomous vehicles, electric vehicles, John Paul MacDuffie
Will Self-Driving Electric Vehicles Make Conventional Cars Worthless?
In the coming years, conventional cars will be as obsolete as rolling typewriters, according to RethinX, a Bay Area think tank. With the emergence of electric vehicles and Transportation-as-a-Service (TaaS), experts disagree about the likely adoption rate of shared autonomous transportation. …Read More
September 20, 2017 | PVMI In the News	autonomous vehicles, electric vehicles, John Paul MacDuffie, Transportation-as-a-Service
Connected Vehicles: How Soon Will They Hit the Road?
Advances in vehicle technologies promise to reinvent how we get around. The PVMI Connected Vehicle conference attempted to predict a timeline for the coming disruption.…Read More
August 1, 2017 | Knowledge@Wharton, PVMI In the News	autonomous vehicles, connected vehicles, electric vehicles, John Paul MacDuffie
How the Waymo-Uber Lawsuit Could Rewrite Intellectual Property Rules
Alphabet's recent lawsuit against Uber for patent infringement and stealing trade secrets may lead to a reevaluation of competition and intellectual property protection within the automobile industry. According to PVMI Director John Paul MacDuffie,"These are trade secrets that are incredibly hot and at the heart of a fierce and fast-paced…Read More
May 25, 2017 | Knowledge@Wharton, Podcast, PVMI In the News	autonomous vehicles, intellectual property, John Paul MacDuffie, Uber
Changes Ahead in Transportation Signal Opportunity for CPAs
Ride-hailing and autonomous vehicles are poised to rapidly transform global transportation culture. According to PVMI Director John Paul MacDuffie, these changes will also give rise to new opportunities for businesses beyond the auto industry, including insurers, healthcare providers, delivery companies, auto repair companies, and parking garages, to name a few. Alongside…Read More
April 17, 2017 | PVMI In the News	autonomous vehicles, John Paul MacDuffie
Does the Google-Uber Lawsuit Hold the Key to Self-driving Cars?
The recent legal battle between Uber and Waymo, a subsidiary of Google's parent company Alphabet, has created some roadblocks along the path to commercializing self-driving cars. Waymo has accused Uber of violation of trade secrets and patent infringements, and sought to bar Uber from using its proprietary technology. According to PVMI…Read More
March 2, 2017 | Knowledge@Wharton, PVMI In the News	autonomous vehicles, Google, John Paul MacDuffie, Uber
Can U.S. Highways Handle the New Mobility?
The advent of self-driving vehicles brings about many challenges, particularly with regulations and insurance companies. PVMI Director John Paul MacDuffie points out in the Philadelphia Inquirer that many of these challenges arise from the lack of unified tech standards across different automakers and between automakers and software companies.…Read More
January 9, 2017 | PVMI In the News	autonomous vehicles, insurance, John Paul MacDuffie, regulations
India isn't the Best Market for Autonomous Vehicles: WEF's John Moavenzadeh
Transportation and mobility are rapidly evolving, according to PVMI member John Moavenzadeh, head of Mobility Industries and member of the executive committee of the World Economic Forum.…Read More
October 4, 2016 | PVMI In the News	autonomous vehicles, India, John Moavenzadeh
How Federal Policy Is Paving the Way for Driverless Cars
Automakers like like Tesla, Google and Uber are preparing for a full-fledged launch of driverless cars by 2021, prompting federal agencies to put new safety requirements and regulations in place.…Read More
September 28, 2016 | Knowledge@Wharton, Podcast, PVMI In the News	autonomous vehicles, John Paul MacDuffie
How Does a Police Officer Pull Over a Driverless Car?
As autonomous vehicles take to the road, officials are increasingly dealing with questions of safety and regulation. In partnership with the World Economic Forum and the Boston Consulting Group, Boston officials plan to begin testing self-driving cars by the end of 2016.…Read More
September 22, 2016 | PVMI In the News	autonomous vehicles, John Moavenzadeh, regulations