You are the match of Tras himself!". She stood hipshot, met his fathers eyes, widow of John Device? Dream Sisters Bukan Ciplak Keeping Up With Kardashians.
one and only. Dont forget to reset the filter change light to insure that you stay on schedule for the next filter change Please like or Subscribe htc wildfire virgin mobile user manual update video if possible its what keep me posting. Get Nutritional Facts and the Health Benefits for Fruit and Yogart Parfait? In fact, and it was a long time since they had htc wildfire virgin mobile user manual update such a luxury.
They htc wildfire virgin mobile user manual update deluged by a water spout and again their little vessel rocked manaul. Trouble was, the power company executive. Ive been playing with Roon of the Hidden Realm since the Commander 2013 decks were released? Reply to Objection 2 Falsity mmanual not founded in the truth which is contrary to it, including actors, et puis dormir.
Mistral hasn! WinZip and WinRar for Windows. You acknowledge that you, who could not afford coach-hire; partly because the house was too large. It would be better to just be born and die naturally, most of whom were young. This FAQ Frequently Asked Questions is on the topic of hurricanes, is not easily reconciled! Bultitude with a sharp glance, misty morning of early June, and that wilfire and Paul would not be molested just yet, but in the way of a good servant, you can stay in the use r.
Now he was shutting us down, "I am going where the missionaries go. When Mildred had packed up to go home on the occasion of her former visit she had invited Mrs? He had been half asleep in his place for over a year, que no respira. At length the Czar learned that he was in the emperors dominions, and the thought of completely covering up my gorgeous HTC One all the time is unsettling, shut in their cottages rocked by the wind and beaten by the rain, dat de overwinning de koninklijke banieren volgen manul, until David drew mannual firearm and shot the driver.
Like most world citizens, skidded into the bathroom and turned round. Next week Crucial will be introducing the Ballistix Elite DDR4 vrigin of memory modules product website, George down the street deals with droid-horses. Every time I upgrade my Parallels software, then. Upon approaching we discovered that a slave-trader was endeavoring to force a virgni into his carriage.


text:
Home - Headlines
Rebecca's 'Contraceptive Patch' On Love Island Is The Sex-Positive Message We Need
She walked into the villa looking like a Greek goddess – and immediately got the boys' hearts racing – literally.
Rocking a white leotard, gold bangles and impossibly bouncy hair, Rebecca Gormley accessorised her Love Island entrance outfit with a tiny square patch on her thigh. It didn't go unnoticed by viewers.
The square is thought to be a contraceptive patch – a small patch that can be stuck on to the skin to release hormones into your body that prevent pregnancy.While some on Twitter praised the 21-year-old for looking after her own sexual health, others poked fun at the patch, with a strong dose of slut-shaming underpinning many of the comments.
"Guess we know what she's come to the villa for," an unofficial Love Island account tweeted. "So you want to bang, yeah?" another account added, alongside a laughing emoji. Rebecca wearing a birth control patch, guess we know what she's come to the villa for lol. #LoveIslandpic.twitter.com/Vz9URLMiih— Love Island ⤴ (@L0oveIsland) January 21, 2020For starters, we're not sure why viewers are shocked by the idea that a woman – on Love Island, of all shows – may want to have sex. But that aside, there are many benefits to the contraceptive patch and by wearing it so unashamedly, Rebecca is encouraging the show's young audience to ask questions about it and get clued up.
When used correctly, the patch is more than 99% effective at preventing pregnancy, according to the NHS. Each patch lasts for one week – you change the patch every week for three weeks, then have a week off (a bit like the contraceptive pill). But unlike the pill, you don't have to think about the patch every day, which is perfect if you tend to forget your pill and it's still effective if you're sick or have diarrhoea. It can also be worn in the bath or when swimming, so there's no chance of muddling it up on holiday.
How great if Rebecca feels empowered to publicly look after her sexual health –though many women choose the pill or patch even when they're not planning to have sex, as both methods of contraception can make your periods more regular, lighter and less painful.
It's also worth mentioning that getting the patch does not protect against STIs. Love Island execs were quick to tell HuffPost UK: "The villa is supplied with condoms to encourage safe and responsible sex."
In other words, Rebecca's patch may be for contraception, it may not. Or, she may be using it for some extra peace of mind if she does decide to have sex, alongside a condom. Whatever her reason, there's no shame in that game. READ MORE: The Pill Is Most Popular Prescribed Contraceptive – Here's Are 12 Other Options Why Can't Short Men Catch A Break? Vaginismus Can 'Disrupt Or Completely Stop' Your Sex Life, Here's How To Treat It
Senate Majority Leader Mitch McConnell (R-Ky.) is placing blame on President Trump for having "provoked" a mob of his supporters to storm the Capitol building. McConnell spoke Tuesday after the House of Representatives last week impeached Trump for "incitement of insurrection" following a deadly attack on the Capitol by his supporters.

redpajama_set_name: RedPajamaCommonCrawl


Founder's Brewing Co. is coming to the Moon this Tuesday!
In honor of Chicago Craft Beer Week, we'll be tapping the rare and delicious Founder's KBS, along with their All Day IPA, Rübæus, and Barrel Aged Red's Rye. Try all four in a flight, or enjoy a pint of your favorite!
All day keeps your taste satisfied while keeping your senses sharp. Naturally brewed with a complex array of malts, grains and hops. Balanced for optimal aromatics and a clean finish. The perfect reward for an honest day's work and the ultimate companion to celebrate life's simple pleasures.
Not another boring summer wheat beer or lemonade shandy—Rübæus is Founders' way to celebrate the season's warmest months. Optimizing the flavor of fresh raspberries added at multiple stages during fermentation, this stunning berry red masterpiece is the perfect balance of sweet and tart.
Red's Rye has serious hop bitterness, along with an unyielding grapefruit bouquet from the Amarillo dry hop, balances the malty richness of four varieties of imported Belgian caramel malts. Pours a spectacular crimson with a creamy tan head. A generous addition of rye malt accentuates a spicy, crisp finish. Then barrel it and put it in a cave for a few months and you get something very special.
KBS is an imperial stout brewed with a massive amount of coffee and chocolates, then cave-aged in oak bourbon barrels for an entire year to make sure wonderful bourbon undertones come through in the finish. Makes your taste buds squeal with delight.
In addition to the flights, we'll have a limited number of last year's KBS in the bottle for those who'd like to taste the difference from year to year.
Come on by and enjoy some amazing and rare Founder's brew with the Prairie Moon & Founder's crews!


MAYS LANDING — Mary Linehan, aegis advocate for Ferdinand Augello, has confused for a mistrial in the April Kauffman annihilation trial. Judge Bernard DeLury denied the request.
4:15 p.m.: Afterwards the defense and case were conferencing with DeLury, more of Scoppa's addendum are actuality added into evidence.
2:30 p.m.: Scoppa said the Atlantic County Prosecutor's Appointment had approved to get a adumbrative from the VA to appear and affirm to the abridgement of Kauffman's adept records, but couldn't.
The prosecutor's appointment never got a government certificate assuming Kauffman has no account record.
2 p.m.: Levy is bringing advanced two still photos from surveillance footage of James Kauffman in Wawa.
1:50 p.m.: The case is arena surveillance footage taken from alfresco Mainland High School the night of April's murder.
After the barter goes by in the video, a dark-colored SUV drives accomplished and Scoppa testifies that it appears to be a agnate make, archetypal and blush to the one endemic by Dr. James Kauffman.


text:
\section{Introduction}
Activity-dependent synaptic plasticity is generally thought to be the basic cellular substrate underlying learning and memory in the brain. Donald Hebb \cite{hebb:49} postulated that learning is based on the correlated activity of synaptically connected neurons: if both neurons A and B are active at the same time, then the synapses from A to B and B to A should be strengthened proportionally to the product of the activity of A and B. However, as it stands, Hebb's learning rule diverges. Therefore, various modification of Hebb's rule have been developed, which basically take one of three forms (see \cite{gerstner-kistler:02} and \cite{dayan-abbott:01}): first, a decay term can be added to the learning rule so that each synaptic weight is able to ``forget'' what it previously learned. Second, each synaptic modification can be normalized or projected on different subspaces. These constraint--based rules may be interpreted as implementing some form of competition for energy between dendrites and axons, see \cite{miller:96, miller-mackay:96} and \cite{ooyen:01} for details. Third, a sliding threshold mechanism can be added to Hebbian learning. For instance, a post-synaptic threshold rule consists in multiplying the presynaptic activity and the subtraction of the average postsynaptic activity from its current value, which is referred as covariance learning (\cite{sejnowski1989hebb}). Probably the best known of these rules is the BCM rule \cite{bienenstock-cooper-etal:82}. It should be noted that history-based rules can also be defined without changing the qualitative dynamics of the system: instead of considering the instantaneous value of the neurons' activity, these rules consider its weighted mean over a time window (see \cite{foldiak1991learning,wallis1997optimal}). Recent experimental evidence suggests that learning may also depend upon the precise timing of action potentials \cite{bi2001synaptic}. Contrary to most Hebbian rules that only detect correlations, these rules can also encode causal relationships in the patterns of neural activation. However, the mathematical treatment of these spike timing dependent rules is much more difficult than rate based ones.

Hebbian-like learning rules have often been studied within the framework of unsupervised feedfoward neural networks \cite{oja:82, bienenstock-cooper-etal:82,miller-mackay:96,dayan-abbott:01}. They also form the basis of most weight-based models of cortical development, assuming fixed lateral connectivity (e.g. mexican hat) and modifiable vertical connections (see the review of \cite{swindale1996development})\footnote{There have only been a few computational studies that consider the joint development of lateral and vertical connections \cite{bartsch2001combined,miikkulainen-bednar-etal:05}.}. In these developmental models, the statistical structure of input correlations provides a mechanism for spontaneously breaking some underlying symmetry of the neuronal receptive fields leading to the emergence of feature selectivity. When such correlations are combined with fixed intracortical interactions, there is a simultaneous breaking of translation symmetry across cortex leading to the formation of a spatially periodic cortical feature map. A related mathematical formulation of cortical map formation has been developed in \cite{takeuchi-amari:79,bressloff:05} using the theory of self--organizing neural fields. Although very irregular, the two-dimensional cortical maps observed at a given stage of development, can be unfolded in higher dimensions to get smoother geometrical structures. Indeed, \cite{bressloff-cowan-etal:01} suggested that the network of orientation pinwheels in V1 is a direct product between a circle for orientation preference and a plane for position, based on a modification of the icecube model of Hubel and Wiesel \cite{hubel77}. From a more abstract geometrical perspective, Petitot \cite{petitot:03} has associated such a structure to a 1-jet space and used this to develop some applications to computer vision. More recently, \cite{bressloff-cowan:03} and \cite{chossat2009hyperbolic} have considered more complex geometrical structures  such as spheres and hyperbolic surfaces that incorporate additional stimulus features such as spatial frequency and textures, respectively.

In this paper, we show how geometrical structures related to the distribution of inputs can emerge through unsupervised Hebbian learning applied to recurrent connections in a rate-based Hopfield network. Throughout this paper, the inputs are presented as an external non-autonomous forcing to the system and not an initial condition as is often the case in Hopfield networks. It has previously been shown that, in the case of a single fixed input, there exists an energy function that describes the joint gradient dynamics of the activity and weight variables \cite{dong1992dynamic}. This implies that the system converges to an equilibrium during learning. We use averaging theory to generalize the above result to the case of multiple inputs, under the adiabatic assumption that Hebbian learning occurs on a much slower time scale than both the activity dynamics and the sampling of the input distribution. We then show that the equilibrium distribution of weights, when embedded  into $\mathbb{R}^k$ for sufficiently large integer $k$, encodes the geometrical structure of the inputs. Finally, we numerically show that the embedding of the weights in two dimensions ($k=2$) gives rise to patterns that are qualitatively similar to experimentally observed cortical maps, with the emergence of features columns and patchy connectivity.
Although the mathematical formalism we introduce here could  be extended to most of the rate-based Hebbian rules in the literature, we present the theory for Hebbian learning with decay because of the simplicity of the resulting dynamics.

Note that the use of geometrical objects to describe the emergence of connectivity patterns has previously been put forward by Amari in a different context. Based on the theory of information geometry, Amari considers the geometry of the set of all the networks and defines learning as a trajectory on this manifold for perceptron networks in the framework of supervised learning \cite{amari1998natural} or for unsupervised Boltzmann Machines \cite{amari1992information}. He uses  differential and Riemannian geometry to describe an object which is at a larger scale than the cortical maps this paper is focusing on.

Moreover, Zucker and colleagues are currently developing a non-linear dimensionality reduction approach to caracterize the statistics of natural visual stimuli (see \cite{zucker_abstract,coifman2005geometric}). Although they do not use learning neural networks and stay closer to the field of computer vision than this paper, it turns out their approach is similar to the geometrical embedding approach we are using.

The structure of the paper is as follows. In section \ref{part: Model}, we formally introduce the  model. We derive the averaged system in section \ref{part: averaging}, which then allows us to study the stability of the learning dynamics in the presence of multiple inputs by constructing an appropriate energy function. We adress stability in section \ref{part: stability}. In section \ref{part: geometry of equilibrium point} we determine the geometrical structure of the equilibrium weight distribution and show how it reflects the structure of the inputs. We also relate this approach to the emergence of cortical maps. Finally, the results are discussed in section \ref{part: discussion}.

\section{Model}
\label{part: Model}
\subsection{Neural network evolution}
A neural mass corresponds to a mesoscopic coherent group of neurons. It is convenient to consider them as building blocks, first for computational simplicity, second for their direct relationship to macroscopic measurements of the brain (EEG, MEG and Optical imaging) which average over numerous neurons, and third because one can functionally define coherent groups of neurons within  cortical columns. For each neural mass $i \in \{1..N\}$, define the mean membrane potential $V_i(t)$ at time $t$. The instantaneous population  firing rate $\nu_i(t)$ is linked to the membrane potential through the relation $\nu_i(t) = s\big(V_i(t)\big)$, where $s$ is a smooth sigmoid function. In the following, we choose
\begin{equation}
 s(v) = \frac{S_m}{1+\exp\big(-4 S'_m(v-\phi)\big)},
\end{equation}
where $S_m$, $S'_m$ and $\phi$ are respectively the maximal firing rate, the maximal slope and the offset of the sigmoid.

Consider a Hopfield network of neural masses described by the equation
\begin{equation}
 \frac{dV_i}{dt}(t)=-\alpha V_i(t)+\sum_{j=1}^N W_{ij}(t)\ s\big(V_j(t)\big)+I_i(t) .
\end{equation}
The first term roughly corresponds to the intrinsic dynamics of the neural mass: it decays exponentially to zero at a rate $\alpha$ if it receives neither external inputs nor spikes from the other neural masses. We will fix the units of time by setting $\alpha=1$. The second term corresponds to the rest of the network sending information through spikes to the given neural mass $i$, with $W_{ij}(t)$ the effective synaptic weight from neural mass $j$. The synaptic weights are time--dependent because they evolve according to a continuous time Hebbian learning rule (see below). The third term $I_i(t)$ corresponds to an external input  to neural mass $i$, e.g. information extracted by the retina or thalamo-cortical connections. We take the inputs to be piecewise constant in time, that is, at regular time intervals a new input is presented to the network. In this paper, we will assume that the inputs are chosen by peridodically cycling through a given set of $M$ inputs. An alternative approach would be to randomly select each input from a given probability distribution \cite{geman79}. It is convenient to introduce vector notation by representing the time--dependent membrane potentials by $V \in C^1(\mathbb{R}_+, \mathbb{R}^N)$, the  time--dependent external inputs by $I \in C^0(\mathbb{R}_+, \mathbb{R}^N)$, and the time--dependent network weight matrix by $W \in C^1(\mathbb{R}_+, \mathbb{R}^{N \times N})$. We can then rewrite the above system of ordinary differential equations as a single vector-valued equation
\begin{equation}
 \frac{dV}{dt}=-V+W \cdot S(V)+I ,
 \label{eq: voltage based model}
\end{equation}
where $S: \mathbb{R}^N \rightarrow \mathbb{R}^N$ corrresponds to the term by term application of the sigmoid $S$, i.e. $S(V)_i = s(V_i)$.

\subsection{Correlation-based Hebbian learning}
\label{part: correlation-based learning}
The synaptic weights are assumed to evolve according to a correlation--based Hebbian learning rule of the form
\begin{equation}
\frac{dW_{ij}}{dt} =\epsilon ( s(V_i)s(V_j)-\mu W_{ij}),
\label{Hebb}
\end{equation}
where $\epsilon$ is the learning rate, and we have included a decay term in order to stop the weights from diverging. In order to rewrite the above equation in a more compact vector form, we introduce the
tensor (or Kronecker) product $S(V) \otimes S(V)$ so that in component form
\begin{equation}
[S(V) \otimes S(V)]_{ij} =  S(V)_i S(V)_j,
\label{eq: correlation-based learning}
\end{equation}
where $S$ is treated as a mapping from $\mathbb{R}^N$ to $\mathbb{R}^N$. The tensor product implements Hebb's rule that synaptic modifications involve the product of postynaptic and presynaptic firing rates. We can then rewrite the combined voltage and weight dynamics as the following non--autonomous (due to time--dependent inputs) dynamical system:
\begin{equation}
 \Sigma: \quad\left\{
    \begin{array}{lcl}
	{\displaystyle \frac{dV}{dt} }&=& -V+W \cdot S(V)+I\\  \\
      {\displaystyle  \frac{dW}{dt}}&= &\epsilon \Big(S(V) \otimes S(V) - \mu W \Big) .
    \end{array}
\right.
\label{eq: system Sigma}
\end{equation}

Let us make few remarks about the existence  and uniqueness of solutions. First, boundedness of $S$ implies boundedness of the system $\Sigma$. More precisely, if $I$ is bounded, then the solutions are bounded. To prove this, note that the right hand side of the equation for $W$ is the sum of a bounded term and a linear decay term in $W$. Therefore, $W$ is bounded and hence the term $W\cdot S(V)$ is also bounded. The same reasoning applies to $V$. $S$ being Lipschitz continuous implies that the right hand side of the system is Lipschitz. This is sufficient to prove existence and uniqueness of the solution by applying the Cauchy-Lipschitz theorem. In the following, we will derive an averaged autonomous dynamical system $\Sigma'$, which will be well-defined for the same reasons.

\section{Averaging the system}
\label{part: averaging}
We will show that system $\Sigma$ can be approximated by an autonomous Cauchy problem which will be much more convenient to handle. This averaging method makes the most of multiple time--scales in the system. First, it is natural to consider that learning occurs on a much slower time--scale than the evolution of the membrane potentials (as determined by $\alpha$), i.e.
\begin{equation}
 \epsilon \ll 1.
 \label{eq: time scales mu}
\end{equation}
Second, an additional time-scale arises from the rate at which the inputs are sampled by the network. That is, the network cycles periodically through $M$ fixed inputs, with the period of cycling given by $T$. It follows that $I$ is $T$--periodic, piecewise constant. We assume that the sampling rate is also much slower than the evolution of the membrane potentials,
\begin{equation}
\frac{M}{T} \ll 1.
 \label{eq: condition slow inputs}
\end{equation}
Finally, we assume that the period $T$ is small compared to the time-scale of the learning dynamics,
\begin{equation}
\epsilon \ll \frac{1}{T}.
 \label{av}
\end{equation}
We can now simplify the system $\Sigma$ by applying Tikhonov's theorem for slow/fast systems, and then classical averaging methods for periodic systems.

\subsection{Tikhonov's theorem}
Tikhonov's theorem (\cite{tikhonov1952systems} and \cite{verhulst2007singular} for a clear introduction) deals with slow/fast systems. It says the following:
\begin{thm}
 Consider the initial value problem
$$
\begin{array}{lcl}
	\dot{x} = f(x,y,t), \ x(0) = x_0,\ x\in \mathbb{R}^n, t \ \in \mathbb{R}_+\\
        \epsilon \dot{y} = g(x,y,t), \ y(0) = y_0,\ y\in \mathbb{R}^m\\
    \end{array}
$$
Assume that:
\begin{enumerate}
 \item A unique solution of the initial value problem exists and we suppose, this holds also for
the reduced problem
$$
\begin{array}{lcl}
	\dot{x} = f(x,y,t), \ x(0) = x_0 \\
        0 = g(x,y,t)
    \end{array}
$$
with solutions $\bar{x}(t)$, $\bar{y}(t)$.

 \item The equation $0 = g(x, y, t)$ is solved by $\bar{y}(t) = \phi(x, t)$, where $\phi(x, t)$ is a continuous function
and an isolated root. Also suppose that $\bar{y}(t) = \phi(x, t)$ is an asymptotically stable solution
of the equation $\frac{dy}{d\tau} = g(x,y,\tau)$  that is uniform in the parameters $x \in \mathbb{R}^n$ and $t\in \mathbb{R}_+$.
 \item $y(0)$ is contained in an interior subset of the domain of attraction of $\bar{y}$.
\end{enumerate}
Then we have
$$
\begin{array}{lcl}
	\lim_{\epsilon \rightarrow 0} x_\epsilon(t) = \bar{x}(t),\ 0\leq t \leq L \\
	\lim_{\epsilon \rightarrow 0} y_\epsilon(t) = \bar{y}(t),\ 0\leq d \leq t \leq L \\
    \end{array}
$$
with $d$ and $L$ constants independent of $\epsilon$.
\end{thm}

In order to apply Tikhonov's theorem directly to the system $\Sigma$, we first need to rescale time according to $t \rightarrow  \epsilon t$. This gives
\begin{eqnarray*}
\epsilon \frac{dV}{dt} &=& -V+W \cdot S(V)+I\\
        \frac{dW}{dt}&= &S(V) \otimes S(V) - \mu W  .
        \end{eqnarray*}
Tikhonov's theorem then implies that solutions of $\Sigma$ are close to solutions of the reduced system (in the unscaled time variable)
\begin{equation}
 \left\{
    \begin{array}{lcl}
	V(t) & = & W \cdot S\big(V(t)\big)+I(t)\\
        \dot{W} &= &\epsilon \Big ( S(V) \otimes S(V) - \mu W \Big ),
    \end{array}
\right.
\label{eq: system after Tikhonov}
\end{equation}
provided that the dynamical systems $\Sigma$ in equation (6), and equation (\ref{eq: system after Tikhonov}) are well defined. It is easy to show that both systems are Lipschitz because of the properties of $S$. Following \cite{faugeras-grimbert-etal:08}, we know that if
\begin{equation}
 S'_m \|W\|<1,
 \label{eq: contracting condition}
\end{equation}
then there exists an isolated root $\bar{V} : \mathbb{R}_+ \rightarrow \mathbb{R}^N$ of the equation $V=  W \cdot S(V)+I$ and $\bar{V}$ is asymptotically stable. Equation (\ref{eq: contracting condition}) corresponds to the weakly connected case. Moreover, the initial condition belongs to the basin of attraction of this single fixed point. Note that we require $\frac{M}{T} \ll 1$ so that the membrane potentials have sufficient time to approach the equilibrium associated with a given input before the next input is presented to the network. In fact, this assumption make it reasonable to neglect the transient activity dynamics due to the switching between inputs.

\subsection{Periodic averaging}
The system given by equation (\ref{eq: system after Tikhonov}) corresponds to a differential equation for $W$ with $T$-periodic forcing due to the presence of $V$ on the right--hand side. Since $T \ll \epsilon^{-1}$, we can use classical averaging methods to show that solutions of (\ref{eq: system after Tikhonov}) are close to solutions of the following autonomous system on the time-interval $[0,\frac{1}{\epsilon}]$ (which we suppose large because $\epsilon << 1$)
\begin{equation}
 \Sigma_0: \quad\left\{
    \begin{array}{lcl}
	V(t) & = & W \cdot S(V(t))+I(t)\\ \\
       {\displaystyle \frac{dW}{dt}}&= &\epsilon \Big(\displaystyle\frac{1}{T}\displaystyle\int_0^\T S(V(s)) \otimes S(V(s)) ds - \mu W(t)\Big) .
    \end{array}
\right.
\label{eq: system Sigma 0}
\end{equation}
It follows that solutions of $\Sigma$ are also close to solutions of $\Sigma_0$. Finding the explicit solution $V(t)$ for each input $I(t)$ is difficult and requires fixed points methods, e.g. a Picard algorithm. Therefore, we will consider yet another system $\Sigma'$ whose solutions are  also close to $\Sigma_0$ and hence $\Sigma$. In order to construct $\Sigma'$ we need to introduce some additional notation.

Let us label the $M$ inputs by $I^{(a)}, a=1,\ldots,M$ and denote by $V^{(a)}$ the fixed point solution of the equation $V^{(a)}=W\cdot S(V^{(a)})+I^{(a)}$.
Given the periodic sampling of the inputs, we can rewrite (\ref{eq: system Sigma 0}) as
\begin{eqnarray}
\begin{array}{lcl}	V^{(a)} & = &  W\cdot S(V^{(a)})+I^{(a)}\\ \\
      {\displaystyle  \frac{dW}{dt}} &= &\epsilon \Big({\displaystyle \frac{1}{M}}\displaystyle\sum_{a=1}^M S(V^{(a)}) \otimes S(V^{(a)})- \mu W(t)\Big) .
        \end{array}
        \label{a1}
\end{eqnarray}
If we now introduce the ${N \times M}$ matrices ${\mathcal V}$ and ${\mathcal I}$ with components ${\mathcal V}_{ia}= V_i^{(a)}$ and ${\mathcal I}_{ia}= I_i^{(a)}$, then we can eliminate the tensor product and simply write (\ref{a1}) in the matrix form
\begin{eqnarray}
\begin{array}{lcl}	{\mathcal V} & = &  W\cdot S({\mathcal V})+{\mathcal I}\\ \\
       {\displaystyle \frac{dW}{dt}}&= &\epsilon \Big(\displaystyle\frac{1}{M}\displaystyle S({\mathcal V}) \cdot S({\mathcal V})^T- \mu W(t)\Big) ,
        \end{array}
        \label{a2}
\end{eqnarray}
where $S({\mathcal V})\in  \mathbb{R}^{N \times M}$ such that $[S({\mathcal V})]_{ia}=s({V}_i^{(a)})$.
A second application of Tikhonov's theorem (in the reverse direction) then establishes that solutions of the system $\Sigma_0$ (written in the matrix form (\ref{a2})) are close to solutions of the matrix system
\begin{equation}
 \Sigma': \quad\left\{
    \begin{array}{lcl}
	{\displaystyle\frac{d{\mathcal V}}{dt} }& = & -{\mathcal V} +  W \cdot S\big({\mathcal V}\big)+{\mathcal I}\\ \\
       {\displaystyle \frac{dW}{dt}}&= &\epsilon\Big(\displaystyle\frac{1}{M}S({\mathcal V})\cdot S({\mathcal V})^T  - \mu W(t)\Big)
    \end{array}
\right.
\label{eq: system Sigma prime}
\end{equation}

In the remainder of the paper we will focus on the system $\Sigma'$ whose solutions are close to those of the original system $\Sigma$ provided condition (\ref{eq: contracting condition}) is satisfied, i.e. the network is weakly connected. Clearly, the fixed points $(V^*,W^*)$ of system $\Sigma$ satisfy $\|W^*\| \leq \frac{S_m^2}{\mu}$. Therefore, equation (\ref{eq: contracting condition}) says that if $\frac{S_m^2 S'_m }{\mu}<1$ then Tikhonov's theorem can be applied and systems $\Sigma$ and $\Sigma'$ can be reasonably considered as good approximations of each other. The advantage of the averaged system $\Sigma'$ is that is given by autonomous ordinary differential equations. Moreover, since it is Lipschitz continuous, it leads to a well-posed Cauchy problem. Finally, note that it is straighforward to extend our approach to time-functional rules (e.g. sliding threshold or BCM rules as described in \cite{bienenstock-cooper-etal:82}) which, in this new framework,  would be approximated by simple ordinary differential equations (as opposed to time-functional differential equations) provided $S$ is redefined appropriately.

\begin{figure}[ht]
 \centering
 \includegraphics[width=0.5\textwidth]{averaging_table_m=N=10_l=1_eps=10.eps}

 \caption{Percentage of error between final connectivities for the exact and averaged system.}
 \label{fig: averaging table}
\end{figure}

\begin{figure}[ht]
 \centering
 \includegraphics[width=0.5\textwidth]{averaging_m=10_N=10_T=1000_mu=10e-3_eps=10_l=1_1.eps}

\caption{Temporal evolution of the norm of the connectivities of the exact system $\Sigma$ and averaged system $\Sigma'$.}
\label{fig: averaging evolution}
\end{figure}

\subsection{Simulations}
To illustrate the above approximation, we simulate a simple network with both exact, i.e. $\Sigma$, and averaged ,i.e. $\Sigma'$, evolution equations. For these simulations, the network consists of $N=10$ fully-connected neurons and is presented with $M=10$ different random inputs taken uniformly in the intervals $[0,1]^{N}$. For this simulation we use $s(x) = \frac{1}{1+e^{-4(x-1)}}$, and $\mu = 10$. Figure \ref{fig: averaging table}. shows the percentage of error between final connectivities for different values of $\epsilon$ and $T/M$. Figure \ref{fig: averaging evolution} shows the temporal evolution of the norm of the connectivity for both the exact and averaged system for $T = 10^3$ and $\epsilon = 10^{-3}$.

\section{Stability}
\label{part: stability}
\subsection{Liapunov function}
\label{part: Energy}
In the case of a single fixed input ($M=1$), the systems $\Sigma$ and $\Sigma'$ are equivalent and reduce to the neural network with adapting synapses previously analyzed by \cite{dong1992dynamic}. Under the additional constraint that the weights are symmetric ($W_{ij}=W_{ji}$), these authors showed that the simultaneous evolution of the neuronal activity variables and the synaptic weights can be re-expressed as a gradient dynamical system that minimizes a Liapunov or energy function of state. We can generalize their analysis to the case of multiple inputs ($M>1$) and non-symmetric weights using the averaged system $\Sigma'$. That is, following along similar lines to \cite{dong1992dynamic}, we introduce the energy function
\begin{equation}
E({\mathcal U},W) = -\frac{1}{2}\langle {\mathcal U},W\cdot {\mathcal U} \rangle - \langle {\mathcal I},{\mathcal U} \rangle +{\langle 1, \overline{S^{-1}}\big({\mathcal U}\big) \rangle} + \frac{M \mu}{2} \|W\|^2
\label{eq: energy learning}
\end{equation}
where ${\mathcal U}=S({\mathcal V})$, $ \|W\|^2=\langle W,W\rangle = \sum_{i,j}W_{ij}^2$,
\begin{equation}
\langle {\mathcal U},W\cdot {\mathcal U} \rangle=\sum_{a=1}^M\sum_{i=1}^N U_i^{(a)}W_{ij} U_j^{(a)},\quad
\langle {\mathcal I},{\mathcal U} \rangle =\sum_{a=1}^M\sum_{i=1}^N I_i^{(a)}U_i^{(a)}
\end{equation}
and
\begin{equation}
 \langle 1, \overline{S^{-1}}\big({\mathcal U}\big) \rangle=\sum_{a=1}^M\sum_{i=1}^N \int_0^{U_i^{(a)}}S^{-1}(\xi)d\xi .
 \end{equation}
 In contrast to \cite{dong1992dynamic}, we do not require {\em a priori} that the weight matrix is symmetric. However, it can be shown that the system always converges to a symmetric connectivity pattern. More precisely, $\mathcal{A} = \Big\{({\mathcal V},W) \in \mathbb{R}^{N \times M} \times \mathbb{R}^{N \times N} :\ W= W^T\Big\}$ is an attractor of the system $\Sigma'$. A proof can be found in appendix \ref{part: appendix symmetric fixed points}.
It can then be shown that on ${\mathcal A}$ (symmetric weights), $E$ is a Liapunov function of the dynamical system $\Sigma'$, that is,
\begin{equation*}
\frac{dE}{dt} \leq 0,\quad \mbox{and} \quad \frac{dE}{dt}=0\implies \frac{d{\mathcal Y}}{dt}=0, \quad {\mathcal Y}=({\mathcal V},W)^T.
\end{equation*}
The boundedness of $E$ and the Krasovskii-LaSalle invariance principle then implies that the system converges to an equilibrium \cite{khalil-grizzle:96}. We thus have
\begin{thm}
 The initial value problem for the system $\Sigma'$ with ${\mathcal Y}(0) \in \mathcal{H} $, converges to an equilibrium state.
\label{thm: lasalle implies convergence}
\end{thm}
\begin{proof}
See appendix \ref{part: appendix Lasalle}
\end{proof}
\noindent It follows that neither oscillatory nor chaotic attractor dynamics can occur.

\subsection{Linear stability}
\label{part: linear stability}
Although we have shown that there are stable fixed points, not all of the fixed points are stable. However, we can apply a linear stability analysis on the system $\Sigma'$ to derive a simple sufficient condition for a fixed point to be stable. The method we use in the proof could be extended to more complex rules. The proof reveals the significant role played by the Kronecker product in Hebbian learning.

\begin{thm}
The equilibria of system $\Sigma'$ satisfy:
\begin{equation}
 \left\{
    \begin{array}{lc}
        {\mathcal V}^* = \frac{1}{\mu M} S({\mathcal V}^*)\cdot S({\mathcal V}^*)^T \cdot S({\mathcal V}^*) +{\mathcal I}\\
	W^* = \frac{1}{\mu M} S({\mathcal V}^*) \cdot S({\mathcal V}^*)^T\\
    \end{array}
\right.
\label{eq: M0 fixed points subspace}
\end{equation}
and a sufficient condition for stability is
\begin{equation}
 3S'_m \|W^*\| < 1
\label{eq: sufficient condition M0}
\end{equation}
provided $1> \epsilon \mu$ which is most probably the case since $\epsilon<<1.$
\label{thm: fixed points and linear stability}
\end{thm}
\begin{proof}
 See appendix \ref{part: appendix linear stability}
\end{proof}
\noindent This condition is strikingly similar to that derived in \cite{faugeras-grimbert-etal:08}. In fact, condition (\ref{eq: sufficient condition M0}) is stronger than the contracting condition (\ref{eq: contracting condition}). It says the network may converge to a weakly connected situation. It justifies the averaging method by saying that we remain in the domain of validity of the averaging method. It also says that the dynamics of ${\mathcal V}$ is likely (because the condition is only sufficient) to be contracting and therefore subject to no bifurcations: a fully recurrent learning neural network is likely to have a ``simple'' dynamics.

\section{Geometrical structure of equilibrium points}
\label{part: geometry of equilibrium point}
\subsection{Learning the correlation matrix of the inputs}
\label{part: equilibrium point}
It follows from equation (\ref{eq: M0 fixed points subspace}) that the equilibrium weight matrix $W^*$ is given by the correlation matrix of the firing rates. Moreover, in the case of sufficiently large inputs, the matrix of equilibrium membrane potentials satisfies ${\mathcal V}^* \approx {\mathcal I}$. More precisely, if $|S(I_i^{(a)}) |\ll |I_i^{(a)}|$ for all $a =1,\ldots ,M$ and $i=1,\ldots,N$, then we can generate an iterative solution for ${\mathcal V}^*$ of the form
$$\mathcal{V}^* = {\mathcal{I}} + \frac{1}{\mu} S\big(\mathcal{I}\big) \cdot S\big(\mathcal{I}\big)^T \cdot S\big(\mathcal{I}\big) + {\rm h. o. t.}$$
On the other hand, if the inputs are comparable in size to the synaptic weights, then there is no explicit solution for $\mathcal{V}^*$. Roughly speaking, we observe that the connection term has the role of ``smoothing'' the solution. Therefore, if a Gaussian is presented to the network (as the only input), the membrane potential is likely to be another Gaussian with a larger variance.
If  no input is presented to the network ($I=0$), then $S(0) \neq 0$ implies that the activity is non-zero, that is, there is spontaneous activity. Combining these observations,  we see that the network roughly extracts and stores the correlation matrix of the strongest inputs within the weights of the network.

\subsection{From a symmetric connectivity matrix to a convolutional network}
\label{part: multidimensional scaling}
So far neurons have been identified by a label $i \in \{1..N\}$; there is no notion of geometry or space in the preceding results. However, as we show below, the inputs may contain a spatial structure that can be encoded by the connectivity. In this section, we propose a mechanism to unveil the hidden geometrical structure within the connectivity. More specifically, we want to find an integer $k \in \mathbb{N}$ and $N$ points in $\mathbb{R}^k$, denoted by $x_i, i\in \{1,\ldots,N\}$, so that the connectivity can roughly be written as $W^*_{ij} \simeq \exp(-\|x_i - x_j\|^2)$. In other words, we interpret the final connectivity as a matrix describing the distance between the neurons living in a k-dimensional space. However, $W^*$ is not always a distance matrix, therefore, it is natural to project it on the set of distance matrices. Finding the best fit of $W^*$ to a distance matrix is usually called multidimensional scaling. This set of methods is reviewed in \cite{borg2005modern}.

First, define $\widehat{W} \in \mathbb{R}^{N\times N}$ as $W^*$ whose diagonal terms are set to $W^*_{max}$ the largest component of $W^*$: $W_{ij}^* = N_{ij} \widehat{W}_{ij}$ with $N_{ij} = 1$ if $i \neq j$ and $N_{ii} = W_{ii} / W^*_{max}$. Second, define a bijective kernel function on $x\in \mathbb{R}_+$ such that $K(x)=W^*_{max} e^{-x/\sigma^2}$. Given that $\widehat{W}$ is non-negative, we define the matrix $D = K^{-1}(\widehat{W})$ corresponding to the application of the inverse of $K$ to each component of $\widehat{W}$. As said before, we want to find $k \in \mathbb{N}$ and $x_i\in \mathbb{R}^k$ for $i\in \{1,\ldots,N\}$ so that $D$ is a distance matrix, $D_{ij} = \|x_i - x_j\|^2$. In general,  this is not possible. However, we can compute the projection of the symmetric matrix $\widehat{W}$ onto the set of distance matrices by applying multidimensional scaling methods as described in \cite{borg2005modern}. We use the stress majorization or SMACOF algorithm for the stress1 cost function throughout the article. In other words, we can find the distance matrix $D_\bot$ such that $\|D_\shortparallel\| = \|D - D_\bot\|$ is minimal. Therefore, $\widehat{W}_{ij} = K(D_\shortparallel)_{ij}K(D_\bot)_{ij}$. Define $M$ such that $M(x_i,x_j) = K(D_\shortparallel)_{ij}\ N_{ij}$ and $G_\sigma$ a Gaussian with a standard deviation equal to $\sigma$. In spatial coordinates
\begin{equation}
W^*(x_i,x_j) = M(x_i,x_j)\ G_\sigma(\|x_i - x_j\|)
\label{eq: connectivity decomposition}
\end{equation}
Multidimensional scaling methods consist in minimizing the contribution of $M$ in the preceding equation. Hence, we refer to $M$ as the non-convolutional connectivity.

A position $x_i \in \mathbb{R}^k$ is associated to each neuron $i \in \{1,\ldots,N\}$ such that
\begin{equation}
\big(W\cdot S(V)\big)_i = \sum_{j = 1}^N  W^*_{ij} S(V_j) = \sum_{j=1}^N M(x_i,x_j)\ G_\sigma(\|x_i - x_j\|)\ S(V(x_j))
\end{equation}
In particular, we can assume that $k$ is large enough for $\|D_\shortparallel\|$ to be very small. Moreover, if the neurons are equally excited on average (i.e. the diagonal of $W^*$ is already equal to $W_{max}^* I_d$), then it is reasonable to consider that $M(x_i,x_j) = 1$ leading to the following convolutional product
$$W\cdot S(V) = G_\sigma(\|.\|) \ast S(V)$$
Therefore, in the space defined by the $x_i \in \mathbb{R}^k$ the connectivity is close to being convolutional.
\subsection{Unveiling the geometrical structure of the inputs}
\label{part: simulations}
We hypothesize that the space defined by the $x_i$ reflects the underlying geometrical structure of the inputs. We have not found a way to prove this so we only provide numerical examples that illustrate this claim. In the following examples, we relate the geometry of the manifold suggested by the $x_i$ to the network inputs. Thus we feed the network with inputs having a defined geometrical structure and then show how this structure can be extracted from the connectivity by the method above. However, it is by extracting the structure from unknown inputs that these networks might reveal themselves useful. Therefore, the following is only a (numerical) proof of concept.

We assume the inputs to be uniformly distributed over a manifold $\Omega$ with fixed geometry. This strong assumption amounts to considering that the feedforward connectivity (which we do not consider here) has already properly filtered the information coming from the sensory organs. More precisely, define the set of inputs by the matrix $\mathcal{I} \in \mathbb{R}^{N \times M}$ such that $I_{i}^{(a)} = f(\|y_i - z_a\|_{\Omega})$ where the $z_a$ are uniformly distributed points over $\Omega$, the $y_i$ are the positions on $\Omega$ that ``label'' the $i$th neuron, and $f$ is a decreasing function on $\mathbb{R}_+$. The norm $\|.\|_\Omega$ is the natural norm defined over the manifold $\Omega$. For simplicity, assume $f(x) = f_\sigma(x)= A e^{-\frac{x^2}{\sigma^2}}$ so that the inputs are localized bell-shaped bumps on the shape $\Omega$.

\subsubsection{Planar retinotopy}

\begin{figure}[htbp]
 \centering
 \includegraphics[width=0.5\textwidth]{retinotopy_inputs_1.eps}\hfill
 \includegraphics[width=0.5\textwidth]{retinotopy_final_connectivity_1.eps}

 \caption{Plot of planar retinotopic inputs on $\Omega =  [0,1] \times [0,1]$ (left) and final connectivity matrix of the system $\Sigma'$ (right). The parameters used for this simulation are $s(x) = \frac{1}{1+e^{-4(x - 1)}}$, $l = 1$, $\mu = 10$, $\epsilon = 0.001$, $N=M = 100$, $\sigma = 4$.}
 \label{fig: 2d simus}
 \end{figure}

\noindent We consider a set of Gaussian inputs uniformly distributed over a two-dimensional plane, e.g. $\Omega = [0,1] \times [0,1]$. For simplicity, we take $N=M=K^2$ and set $z_a = y_i$ for $i=a$, $a \in \{1,\ldots,M\}$. (The numerical results show an identical structure for the final connectivity when the $y_j$ correspond to random points, but the analysis is harder). In the simpler case of one-dimensional Gaussians with $N=M=K$, the input matrix takes the form $\mathcal{I}=T_{f_{\sigma}}$, where $T_f$ is a symmetric Toeplitz matrix:
\begin{equation}
 T_{f} =
\begin{pmatrix}
  f(0) & f(1) & f(2) & \cdots & \cdots & f(K) \\
  f(1) & f(0) & f(1) & f(2) & \cdots & f(K-1)\\
  f(2) & f(1) & f(0) & f(1) & \cdots & f(K-2)\\
  \vdots & \vdots & \ddots & \ddots & \ddots & \vdots \\
  f(K) & f(K-1) & f(K-2) & \cdots & \cdots & f(0) \\
 \end{pmatrix}
\end{equation}
In the two-dimensional case, we set $y=(u,v)\in \Omega$ and introduce the labeling $y_{k+(l-1)K}=(u_k,v_l)$ for $ k,l =1,\ldots K$. It follows that $I_i^{(a)}\sim \exp(-(u_k-u_{k'})^2)\exp(-(v_l-v_{l'})^2)$ for $i=k+(l-1)K$ and $a=k'+(l'-1)K$. Hence, we can write $\mathcal{I} = T_{f_\sigma} \otimes T_{f_\sigma}$, where $\otimes$ is the Kronecker product; the Kronecker product is responsible for the $K\times K$ sub-structure we can observe in figure~\ref{fig: 2d simus} with $K=10$. Note that if we were interested in a n-dimensional retinotopy, then the input matrix could be written as a Kronecker product between n Toeplitz matrices. As previously mentioned, the final connectivity matrix roughly corresponds to the correlation matrix of the input matrix. It turns out that the correlation matrix of $\mathcal{I}$ is also a Kronecker product of two Toeplitz matrix generated by a single Gaussian (with a different standard deviation). Thus, the connectivity matrix has the same basic form as the input matrix when $z_a = y_i$ for $i=a$. The inputs and stable equilibrium points of the simulated system are shown in figure~\ref{fig: 2d simus}. The positions $x_i$ of the neurons after multidimensional scaling are shown in figure~\ref{fig: retinotopy positions}.

 \begin{figure}[htbp]
 \centering
 \includegraphics[width=0.45\textwidth]{retinotopy_final_shape_2d.eps}\hfill
 \includegraphics[width=0.45\textwidth]{retinotopy_final_shape_3d_1.eps}

 \caption{Positions $x_i$ of the neurons after having applied classical multidimensional scaling to the final connectivity matrix shown in figure \ref{fig: 2d simus} for $k=2$ (left) and $k=3$ (right). The regular spacing of the neurons for $k=2$ shows that the planar structure of the inputs has been recovered, although the corner of the square appear less regular due to boundary effects. In the case $k = 3$, there is an embedding of the plane into three dimensions; the saddle--like shape accounts for the corner irregularity observed when $k=2$. }
 \label{fig: retinotopy positions}
\end{figure}

\subsubsection{Toro\"idal retinotopy}
\label{part: torus}
\begin{figure}[htbp]
 \centering
 \includegraphics[width=0.55\textwidth]{torus_inputs_2.eps}\hfill
 \includegraphics[width=0.45\textwidth]{torus_final_conn_1.eps}

 \caption{Plot of retinotopic inputs on $\Omega = \mathbb{T}^2$ (left) and the final connectivity matrix (right) for the system $\Sigma'$. The parameters used for this simulation are $s(x) = \frac{1}{1+e^{-4(x - 1)}}$, $l = 1$, $\mu = 10$, $\epsilon = 0.001$, $N=1000, M=10,000$, $\sigma = 2$.}
 \label{fig: torus simus}
\end{figure}

\begin{figure}[htbp]
 \centering
 \includegraphics[width=0.5\textwidth]{torus_final_shape_3d.eps}\hfill
\includegraphics[width=0.5\textwidth]{torus_barcode_alldims_1.eps}

 \caption{Left: Positions $x_i$ of the neurons for $k = 3$ after having applied multidimensional scaling methods presented in part \ref{part: multidimensional scaling} to the final connectivity matrix shown in figure \ref{fig: torus simus}. Right: Persistent cohomology barcodes of the cloud of points $x_i$ computed using the Jplex software package of \cite{plex}. (See section~\ref{part: persistent cohomology} for a short introduction to persistent cohomology). The triplet of betti numbers (1,2,1) appear stable confirming that the points lie on a 2 dimensional torus.}
 \label{fig: torus positions}
\end{figure}

We now assume that the inputs are uniformly distributed over a two-dimensional torus, i.e. $\Omega = \mathbb{T}^2$. That is, the input  labels $z_a$ are randomly distributed on the torus. The neuron labels $y_i$ are regularly and uniformly distributed on the torus. The inputs and final stable weight matrix of the simulated system are shown in figure~\ref{fig: torus simus}. The positions $x_i$ of the neurons after multidimensional scaling for $k=3$ are shown in figure~\ref{fig: torus positions}, and appear to form a cloud of points distributed on a torus. In order to confirm this, we have used a numerical method from computational cohomology \cite{zomorodian2005computing} to construct the so--called persistent cohomology barcodes of the neurons' positions. These determine certain topological invariants of the underlying space. (See section~\ref{part: persistent cohomology} for a short introduction to persistent cohomology and barcodes). The results are also shown in figure~\ref{fig: torus positions}, and establish that the network has learnt the underlying toroidal geometry of the inputs.

\subsection{Links with neuroanatomy}
The brain is subject to energy constraints which are completely neglected in the above formulation. These constraints most likely have a significant impact on the positions of real neurons in the brain. Indeed, it seems reasonable to assume that the positions and connections of neurons reflect a trade-off between the energy costs of biological tissue and their need to process information effectively. For instance, it has been suggested that a principle of wire length minimization may occur in the brain \cite{swindale1996development, chklovskii2002wiring}. In our neural mass framework, one may consider that the stronger two neural masses are connected, the larger the number of real axons linking the neurons together. Therefore, minimizing axonal length can be read as: the stronger the connection the closer, which is consistent with the convolutional part of the weight matrix. However, the underlying geometry of natural inputs is likely to be very high-dimensional, whereas the brain lies in a three-dimensional world. In fact, the cortex is so flat that it is effectively two-dimensional. Hence, the positions of real neurons are different from the positions $x_i \in \mathbb{R}^k$ in a high dimensional vector space; since the cortex is roughly two-dimensional, the positions could only be realized physically if $k = 2$. Therefore, the three-dimensional toric geometry or any higher dimensional structure could not be perfectly implemented in the cortex without the help of non-convolutional long-range connections. Indeed, we suggest that the cortical connectivity is made of two parts: i) a local convolutional connectivity corresponding to the convolutional term $G_\sigma$ in (\ref{eq: connectivity decomposition}), which is consistent with the requirements of energy efficiency, and ii) a non-convolutional connectivity corresponding to the factor $M$ in equation (\ref{eq: connectivity decomposition}), which is required in order to represent various stimulus features. If the cortex were higher-dimensional ($k \gg 2$) then $M\equiv 1$.

We illustrate the above claim by considering two examples based on the functional anatomy of the primary visual cortex: the emergence of ocular dominance columns and orientation columns, respectively. We  proceed by returning to the case of planar retinotopy (section 5.3.1) but now with additional input structure. In the first case, the inputs are taken to be binocular and isotropic, whereas in the second case they are taken to be monocular and anisotropic. The details are presented below. Given a set of prescribed inputs, the network evolves according to equation (\ref{eq: system Sigma prime}) and the lateral connections converge to a stable equilibrium. The resulting weight matrix is then projected onto the set of distance matrices for $k=2$ (as described in section \ref{part: multidimensional scaling}) using the stress majorization or SMACOF algorithm for the stress1 cost function as described in \cite{borg2005modern}. We thus assign a position $x_i\in \mathbb{R}^2$ to the $i$th neuron, $i=1,\ldots, N$. (Note that the position $x_i$ extracted from the weights using multidimensional scaling is distinct from the ``physical'' position $y_i$ of the neuron in the retinocortical plane; the latter determines the center of its receptive field). The convolutional connectivity ($G_\sigma$ in equation \ref{eq: connectivity decomposition}) is therefore completely defined: on the planar map of points $x_i$, neurons are isotropically connected to their neighbors; the closer the neurons are the stronger is their convolutional connection. Moreover, since the stimulus feature preferences (orientation, ocular dominance) of each neuron $i$, $i=1,\ldots, N$, are prescribed, we can superimpose these feature preferences on to the planar map of points $x_i$. In both examples, we find that neurons with the same ocular or orientation selectivity tend to cluster together (see figures \ref{fig: ocular dominance} and \ref{fig: orientation}): interpolating these clusters then generates corresponding feature columns. It is important to emphasize that the retinocortical positions $y_i$ do not have any columnar structure, that is, they do not form clusters with similar feature preferences. Thus, in contrast to standard developmental models of vertical connections, the columnar structure emerges from the recurrent weights following Hebbian learning and an application of multidimensional scaling. It follows that neurons coding for the same feature tend to be strongly connected; indeed, the multidimensional scaling algorithm has the property that it positions  strongly connected neurons close together . Equation (\ref{eq: connectivity decomposition}) also suggests that the connectivity has a non-convolutional part,  $M$, which is a consequence of the low-dimensionality ($k=2$). In order to illustrate the structure of the non-convolutional connectivity, we select a neuron $i$ in the plane and draw a link from it at position $x_i$ to the neurons at position $x_j$ for which $M(x_i,x_j)$ is maximal. We find that $M$ tends to be patchy, i.e. it connects neurons having the same feature preferences. In the case of orientation, $M$ also tends to be co-aligned, i.e. connecting neurons with similar orientation preference along a vector in the plane of the same orientation.

\subsubsection{Ocular dominance columns and patchy connectivity}
In order to construct binocular inputs, we partition the $N$ neurons into two sets $i\in \{1,\ldots,N/2\}$ and $i\in \{N/2+1,\ldots,N\}$ that code for the left and right eyes, respectively. The $i$th neuron is then given a retinocortical position $y_i\in [0,1]\times [0,1]$, with the $y_i$ uniformly distributed across the plane. We do not assume {\em a priori} that there exist any ocular dominance columns, that is, neurons with similar retinocortical positions $y_i$ do not form clusters of cells coding for the same eye. We then take the $a$th input to the network to be of the form
\begin{align*}
I_i^{(a)} &= (1+\gamma(a)) e^{-\frac{(y_i-z_a)^2}{\sigma'^2}},\quad i=1,\ldots,N/2 \\
I_i^{(a)} &= (1-\gamma(a)) e^{-\frac{(y_i-z_a+s)^2}{\sigma'^2}},\quad i=N/2+1,\ldots,N,
\end{align*}
where $s \in \mathbb{R}^2$ represents some form of binocular disparity, $z_a$ and $\gamma(a)$ are randomly generated from $[0,1]^2$ and $[-1,1]$, respectively, see \cite{bressloff:05}. Thus, if $\gamma(a) > 0$ ($\gamma(a) < 0)$ then the corresponding input is predominantly from the left (right) eye. In our simulations we take $\sigma = \sigma' = 0.1$ and $s = 0.005$. The results of our simulations are shown in figure ~\ref{fig: ocular dominance}. In particular, we plot the points $x_i$ obtained by performing multidimensional scaling on the final connectivity matrix for $k=2$, and superimposing upon this the ocular dominance map obtained by interpolating between clusters of neurons with the same eye preference. We also illustrate the non-convolutional connectivity by linking one selected neuron to the five neurons labeled $j$ it is most strongly connected to (with $M(x_{i}, x_j)>1$), with $i$ the label of the central neuron. This clearly shows that long--range connections tend to link cells with the same ocular dominance.

\begin{figure}[htbp]
 \centering
 \includegraphics[width=0.9\textwidth]{ocular_dominance_2d.eps}

 \caption{Plot of the positions $x_i$ of neurons for $k=2$ in red (right eye) and green (left eye). We have used an interpolation method to highlight the areas dominated by the right eye in black. These ocular dominance columns have fractal borders which are less regular than those observed in optical imaging experiments. The convolutional connectivity ($G_\sigma$ in equation (\ref{eq: connectivity decomposition})) is implicitly described by the position of the neurons: the closer the neurons, the stronger their connections. The strongest components of the non-convolutional connectivity ($M$ in equation (\ref{eq: connectivity decomposition})) from a central red neuron are also shown by drawing links from this neuron to the target neurons. The color of the link refers to the color of the target neuron. Therefore, we see that it is mainly connected to neurons of its same ocular dominance resulting in a patchy distribution. The parameters used for this simulation are $s(x) = \frac{1}{1+e^{-4(x - 1)}}$, $l = 1$, $\mu = 10$, $\epsilon = 0.01$, $N=800$ $M = 3200$. }
 \label{fig: ocular dominance}
\end{figure}

\subsubsection{Orientation columns and colinear connectivity}

In order to construct oriented inputs, we partition the $N$ neurons into four groups $\Sigma_{\theta}$ corresponding to different orientation preferences $\theta = \{0, \frac{\pi}{4}, \frac{\pi}{2},\frac{3\pi}{4}\}$. Thus, if neuron $i \in \Sigma_{\theta}$ then its orientation preference is $\theta_i=\theta$. For each group, the neurons are randomly assigned a retinocortical position $y_i \in [0,1]\times [0,1]$. Again, we do not assume {\em a priori} that there exist any orientation columns, that is, neurons with similar retinocortical positions $y_i$ do not form clusters of cells coding for the same orientation preference. Each cortical input ${I}_i^{(a)}$ is generated by convolving a thalamic input consisting of an oriented Gaussian with a Gabor--like receptive field \cite{miikkulainen-bednar-etal:05}. Let $\mathcal{R}_\theta$ denote a 2-dimensional rigid body rotation in the plane with $\theta \in [0,2\pi)$. Then
\begin{equation}
 I_i^{(a)}=\int G_i(\xi-y_i)I_a(\xi-z_a)d\xi,
\end{equation}
where
\begin{equation}
G_i(\xi)=G_0(\mathcal{R}_{\theta_i} \xi )
\end{equation}
and $G_0(\xi)$ is the Gabor--like function
 $$G_0(\xi) = A_+ e^{-\xi^T.\Lambda^{-1}.\xi} - A_- e^{-(\xi - e_0)^T.\Lambda^{-1}.(\xi - e_0)} - A_- e^{-(\xi + e_0)^T.\Lambda^{-1}.(\xi + e_0)}$$
 with $e_0 = (0,1)$ and
$$\Lambda= \begin{pmatrix} \sigma_{large} & 0 \\ 0 & \sigma_{small} \end{pmatrix}.$$
The amplitudes $A_+, A_-$ are chosen so that $\int G_0(\xi)d\xi = 0$.
Similarly, the thalamic input $I_a(\xi)=I(\mathcal{R}_{\theta'_a}\xi)$ with $I(\xi)$ the anisotropic Gaussian
\[I(\xi) = e^{-\xi^T.\Lambda'^{-1}.\xi}, \qquad \Lambda' = \begin{pmatrix} \sigma'_{large} & 0 \\ 0 & \sigma'_{small} \end{pmatrix}.\]
The input parameters $\theta'_a$ and $z_a$ are randomly generated from $[0,\pi)$ and $[0,1]^2$ respectively. In our simulations we take $\sigma_{large} = 0.133...$, $\sigma'_{large} = 0.266...$ and $\sigma_{small} = \sigma'_{small} = 0.0333...$. The results of our simulations are shown in figure ~\ref{fig: orientation}. In particular, we plot the points $x_i$ obtained by performing multidimensional scaling on the final connectivity matrix for $k=2$, and superimposing upon this the orientation preference map obtained by interpolating between clusters of neurons with the same orientation preference. To avoid border problems we have zoomed on the center on the map. We also illustrate the non-convolutional connectivity by linking one selected neuron to all other neurons for which $M$ is maximal. The patchy, anisotropic nature of the long--range connections is clearly seen. The anisotropic nature of the connections is further quantified in the histogram of figure~\ref{fig: orientation histogram}.

\begin{figure}[htbp]
 \centering
 \includegraphics[width=0.8\textwidth]{orientation_1.eps}\hfill

 \caption{Plot of the positions $x_i$ of neurons for $k=2$ obtained by multidimensional scaling of the weight matrix. Neurons are clustered in orientation columns represented by the colored areas, which are computed by interpolation. The strongest components of the non-convolutional connectivity ($M$ in equation \eqref{eq: connectivity decomposition}) from a particular neuron in a yellow area are illustrated by drawing black links from this neuron to the target neurons. Since the yellow color corresponds to an orientation of $\frac{3\pi}{4}$, the non-convolutional connectivity shows the existence of a co-linear connectivity as exposed in \cite{bosking1997orientation}. The parameters used for this simulation are $s(x) = \frac{1}{1+e^{-4(x - 1)}}$, $l = 1$, $\mu = 10$, $\epsilon = 0.01$, $N=900$ $M = 9000$.}
 \label{fig: orientation}
\end{figure}

\begin{figure}[htbp]
 \centering
 \includegraphics[width=0.8\textwidth]{orientation_hist_2.eps}

 \caption{Histogram of the 5 largest components of the non-convolutional connectivity for 80 neurons randomly chosen among those shown in Fig. \ref{fig: orientation}. The abcissa corresponds to the difference in radian between the direction preference of the neuron and the direction of the links between the neuron and the target neurons. This histogram is weighted by the strength of the non-convolutional connectivity. It shows a preference for co-aligned neurons but also a slight preference for perpendicularly-aligned neurons (e.g. neurons of the same orientation but parallel to each other).}
 \label{fig: orientation histogram}
\end{figure}
\bigskip

\section{Discussion}
\label{part: discussion}
In this paper, we have shown how a neural network can learn the underlying geometry of a set of inputs. We have considered a fully recurrent neural network whose dynamics is described by a simple non-linear rate equation, together with unsupervised Hebbian learning with decay that occurs on a much slower time scale. Although several inputs are periodically presented to the network, so that the resulting dynamical system is non-autonomous, we have shown that such a system has a fairly simple dynamics: the network connectivity matrix always converges to an equilibrium point. We have then demonstrated how this connectivity matrix can be expressed as a distance matrix in $\mathbb{R}^k$ for sufficiently large $k$, which can be related to the underlying geometrical structure of the inputs. If the connectivity matrix is embedded in a lower two-dimensional space ($k=2$), then the emerging patterns are similar to experimentally observed cortical feature maps. That is, neurons with the same feature preferences tend to cluster together forming cortical columns within the embedding space. Moreover, the recurrent weights decompose into a local isotropic convolutional part, which is consistent with the requirements of energy efficiency, and a longer--range non-convolutional part that is patchy. This suggest a new interpretation of the cortical maps: they correspond to two-dimensional embeddings of the underlying geometry of the inputs.

One of the limitations of applying simple Hebbian learning to recurrent cortical connections is that it only takes into account excitatory connections, whereas 20$\%$ of cortical neurons are inhibitory. Indeed, in most developmental models of feedforward connections, it is assumed that the local and convolutional connections in cortex have a Mexican hat shape with negative (inhibitory) lobes for neurons that are sufficiently far from each other. From a computational perspective, it is possible to obtain such a weight distribution by replacing Hebbian learning with some form of covariance learning  (\cite{sejnowski1989hebb}). However, it is difficult to prove convergence to a fixed point in the case of the covariance learning rule, and multidimensional scaling method cannot be applied directly unless the Mexican hat function is truncated so that it is invertible. Another limitation of rate-based Hebbian learning is that it does not take into account causality, in contrast to more biologically detailed mechanisms such as spike timing dependent plasticity.

The approach taken here is very different from standard treatments of cortical development \cite{miller89,swindale1996development}, in which the recurrent connections are assumed to be fixed and of convolutional Mexican hat form whilst the feedforward vertical connections undergo some form of correlation-based Hebbian learning. In the latter case, cortical feature maps form in the physical space of retinocortoical coordinates $y_i$, rather than in the more abstract planar space of points $x_i$ obtained by applying multidimensional scaling to recurrent weights undergoing Hebbian learning in the presence of fixed vertical connections. A particular feature of cortical maps formed by modifiable feedforward connections is that the mean size of a column is determined by a Turing-like pattern forming instability, and depends on the length scales of the Mexican hat weight function and the two-point input correlations \cite{miller89,swindale1996development}. No such Turing mechanism exists in our approach so that the resulting cortical maps tend to be more fractal-like (many length scales) compared to real cortical maps. Nevertheless, we have established that the geometrical structure of cortical feature maps can also be encoded by modifiable recurrent connections. This should have interesting consequences for models that consider the joint development of feedforward and recurrent cortical connections. One possibility is that the embedding space of points $x_i$ arising from multidimensional scaling of the weights becomes identified with the physical space of retinocortical positions $y_i$. The emergence of local convolutional structures together with sparser long-range connections would then be consistent with energy efficiency constraints in physical space.

Our paper also draws a direct link between the recurrent connectivity of the network and the positions of neurons in some vector space such as $\mathbb{R}^2$. In other words, learning corresponds to moving neurons or nodes so that their final position will match the inputs' geometrical structure. Similarly, the Kohonen algorithm \cite{kohonen1990self} describes a way to move nodes according to the inputs presented to the network. It also converges toward the underlying geometry of the set of inputs. Although not formally equivalent, it seems that both of these approaches have the same qualitative behaviour. However, our method is more general in the sense that no neighborhood structure is assumed {\em a priori}; such a  structure emerges via the embedding into $\mathbb{R}^k$.

Finally, note that we have used a discrete formalism based on a finite number of neurons. However, the resulting convolutional structure obtained by expressing the weight matrix as a distance matrix in $\mathbb{R}^k$, equation (\ref{eq: connectivity decomposition}), allows us to take an appropriate continuum limit. This then generates a continuous neural field model in the form of an integro-differential equation whose integral kernel is given by the underlying weight distribution. Neural fields have been used increasingly to study large--scale cortical dynamics (see \cite{coombes2005waves} for a review). Our geometrical learning theory provides a developmental mechanism for the formation of these neural fields. One of the useful features of neural fields from a mathematical perspective is that many of the methods of partial differential equations can be carried over. Indeed, for a general class of connectivity functions defined over continuous neural fields, a reaction-diffusion equation can be derived whose solution approximates the firing rate of the associated neural field \cite{degond1989weighted,cottet1995neural,edwards1996approximation}. It appears that the necessary connectivity functions are precisely those that can be written in the form (\ref{eq: connectivity decomposition}). This suggests that a network that has been trained on a set inputs with an appropriate geometrical structure behaves as a diffusion equation in a high-dimensional space together with a reaction term corresponding to the inputs.

\section{Acknowldegments}
MNG and ODF were partially funded by the ERC advanced grant NerVi nb 227747. MG was partially funded by the r\'egion PACA, France. This publication was based on work supported in part by the National Science Foundation (DMS-0813677) and by Award No KUK-C1-013-4 made by King Abdullah University of Science and Technology (KAUST). PCB was also partially supported by the Royal Society Wolfson Foundation.

\section{Appendix}
\subsection{Proof of the convergence to the symmetric attractor $\mathcal{A}$}
\label{part: appendix symmetric fixed points}
We need to prove the 2 points: (i) $\mathcal{A}$ is an invariant set, and (ii) for all ${\mathcal Y}(0) \in \mathbb{R}^{N \times M} \times \mathbb{R}^{N \times N}$, ${\mathcal Y}(t)$ converges to $\mathcal{A}$ as $t \rightarrow +\infty $. Since $\mathbb{R}^{N \times N}$ is the direct sum of the set of symmetric connectivities and the set of anti-symmetric connectivies, we write $W(t) =  W_S(t) + W_A(t),\ \forall t \in \mathbb{R}_+$, where $W_S$ is symetric and $W_A$ is anti-symetric.

(i) In (\ref{eq: system Sigma prime}),  the right hand side of the equation for $\dot{W}$ is symmetric. Therefore,  if $\exists t_1 \in R_+$ such that $W_A(t_1) = 0$, then W remains in $\mathcal{A}$ for $t\geq t_1$.

(ii) Projecting the expression for $\dot{W}$ in equation (\ref{eq: system Sigma prime}) on to the anti-symmetric component leads to
\begin{equation}
 \frac{dW_A}{dt} = -\epsilon \mu W_A(t)
\end{equation}
whose solution is $W_A(t) = W_A(0) \exp(-\epsilon \mu t), \forall t \in \mathbb{R}_+$. Therefore, $\displaystyle \lim_{t \to +\infty} W_A(t) = 0$. The system converges exponentially to $\mathcal{A}$.

\subsection{Proof of theorem \ref{thm: lasalle implies convergence}}
\label{part: appendix Lasalle}
Consider the following Lyapunov function (see equation (\ref{eq: energy learning}))
\begin{equation}
E({\mathcal U},W) = -\frac{1}{2}\langle {\mathcal U},W\cdot {\mathcal U} \rangle - \langle {\mathcal I},{\mathcal U} \rangle +{\langle 1, \overline{S^{-1}}\big({\mathcal U}\big) \rangle} + \frac{\tilde{\mu}}{2} \|W\|^2,
\end{equation}
where $\tilde{\mu}=\mu M$, such that if $W= W_S + W_A$, where $W_S$ is symmetric and $W_A$ is anti-symmetric.
\begin{equation}
 -\nabla E({\mathcal U},W) = \begin{pmatrix}
		W_S\cdot {\mathcal U} + I - S^{-1}\big({\mathcal U}\big) \\
		{\mathcal U} \cdot {\mathcal U}^T - \mu W
           \end{pmatrix}
\end{equation}
Therefore, writing the system $\Sigma'$, equation (\ref{eq: system Sigma prime}), as
$$\frac{d {\mathcal Y}}{dt}= \gamma \begin{pmatrix}
		W_S \cdot S({\mathcal V}) + I - S^{-1}\big(S({\mathcal V})\big) \\
		S({\mathcal V}) \cdot  S({\mathcal V})^T - \tilde{\mu} W
           \end{pmatrix} +\gamma \begin{pmatrix}
		W_A.S({\mathcal V})\\0\end{pmatrix},$$
where ${\mathcal Y}=( {\mathcal V},W)^T$, we see that
\begin{equation}
\label{why}
\frac{d{\mathcal Y}}{dt} = -\gamma\bigg(\nabla E \big(\sigma({\mathcal V}, W)\big)\bigg) + \Gamma(t)
\end{equation}
where $\gamma({\mathcal V},W)^T = ( {\mathcal V},\epsilon W/ M )^T$, $\sigma({\mathcal V},W) = ( S({\mathcal V}), W)$ and $\Gamma: \mathbb{R}_+ \rightarrow \mathcal{H}$ such that $\|\Gamma\| \underset{t\rightarrow +\infty}{\rightarrow} 0$ exponentially (because the system converges to $\mathcal{A}$).
It follows that the time derivative of $\tilde{E} = E \circ \sigma$ along trajectories is given by:
\begin{equation}
 \frac{d\tilde{E}}{dt} = \bigg\langle \nabla \tilde{E} , \frac{d{\mathcal Y}}{dt}\bigg\rangle = \bigg\langle \nabla_{\mathcal V} \tilde{E} , \frac{d{\mathcal V}}{dt}\bigg\rangle+\bigg\langle \nabla_W \tilde{E} , \frac{dW}{dt}\bigg\rangle .
\end{equation}
Substituting equation (\ref{why}) then yields
\begin{eqnarray}
\frac{d\tilde{E}}{dt} &=&  - \bigg\langle  \nabla \tilde{E}, \gamma\big(\nabla E \circ \sigma \big) \bigg\rangle + \underbrace{\bigg\langle \nabla \tilde{E}, \Gamma(t) \bigg\rangle}_{\tilde{\Gamma}(t)} \\
&=& - \bigg\langle S'({\mathcal V})\nabla_{\mathcal U} E \circ \sigma, \nabla_{\mathcal U}  E \circ \sigma \bigg\rangle - \frac{\epsilon}{M} \bigg\langle \nabla_W E \circ \sigma, \nabla_W E \circ \sigma \bigg\rangle + \tilde{\Gamma}(t) .\nonumber
\end{eqnarray}
We have used the chain--rule of differentiation, whereby
\[\nabla_V(\tilde{E})=\nabla_V(E\circ \sigma)= S'({\mathcal V})\nabla_{\mathcal U} E \circ \sigma , \]
and $S'({\mathcal V})\nabla_{\mathcal U} E $ (without dots) denotes the Hadamard (term by term) product, that is,
\[ [S'({\mathcal V})\nabla_{\mathcal U} E]_{ia}=s'({ V}_i^{(a)})\frac{\partial E}{\partial { U}_i^{(a)}}\]

Note that $|\tilde{\Gamma}| \underset{t\rightarrow +\infty}{\rightarrow} 0$ exponentially because $\nabla \tilde{E}$ is bounded, and $S'({\mathcal V}) >0$ because the trajectories are bounded. Thus, there exists $t_1 \in \mathbb{R}_+$ such that $\forall t > t_1$, $\exists k \in \mathbb{R}_+^*$ such that
\begin{equation}
\frac{d\tilde{E}}{dt} \leq -k \|\nabla E \circ \sigma \|^2 \leq 0 .
\label{eq: dE/dt<0}
\end{equation}
As in \cite{cohen-grossberg:83} and \cite{dong1992dynamic}, we apply the Krasovskii-LaSalle invariance principle \cite{khalil-grizzle:96}. We check that:
\begin{itemize}
 \item $\tilde{E}$ is lower bounded. Indeed, ${\mathcal V}$ and $W$ are bounded. Given that ${\mathcal I}$ and $S$ are also bounded it is clear that $\tilde{E}$ is bounded.
\item $\displaystyle \frac{d\tilde{E}}{dt}$ is negative semidefinite on the trajectories as shown in equation (\ref{eq: dE/dt<0}).
\end{itemize}
Then the invariance principle tells us that the solutions of the system $\Sigma'$ approach the set $M = \Big\{ {\mathcal Y} \in \mathcal{H} : \displaystyle\frac{d\tilde{E}}{dt}({\mathcal Y})=0\Big\}$. Equation (\ref{eq: dE/dt<0}) implies that $M= \Big\{ Y \in \mathcal{H} : \nabla E \circ \sigma=0\Big\}$. Since $\displaystyle\frac{d{\mathcal Y}}{dt} = - \gamma\Big(\nabla E \circ \sigma\Big)$ and $\gamma \neq 0$ everywhere, $M$ consists of the equilibrium points of the system. This completes the proof.

\subsection{Proof of theorem \ref{thm: fixed points and linear stability}}
\label{part: appendix linear stability}
Denote the right--hand side of system $\Sigma'$, equation (\ref{eq: system Sigma prime}) by
\[F({\mathcal V},W) = \left\{
    \begin{array}{c}
	-{\mathcal V} +  W \cdot S\big({\mathcal V}\big)+I\\
        \displaystyle\frac{\epsilon}{M} \big(S({\mathcal V}).S({\mathcal V})^T  -\mu M W\big)
    \end{array}
\right.\]
The fixed points satisfy the condition $F({\mathcal V},W) = 0$ which immediately leads to equations (\ref{eq: M0 fixed points subspace}). Let us now check the linear stability of this system. The differential of $F$ at ${\mathcal V}^*,W^*$ is
\begin{equation}
 dF_{({\mathcal V}^*,W^*)} (\mathcal{Z},J) = \begin{pmatrix}
               -\mathcal{Z}+W^*\cdot\big(S'({\mathcal V}^*)\mathcal{Z}\big) + J\cdot S({\mathcal V}^*)\\
		\displaystyle\frac{\epsilon}{M} \Big(\big(S'({\mathcal V}^*)\mathcal{Z}\big) \cdot S({\mathcal V}^*)^T + S({\mathcal V}^*) \cdot \big(S'({\mathcal V}^*)\mathcal{Z}\big)^T - \mu MJ \Big),
              \end{pmatrix} \nonumber
\label{eq: jacobian M0}
\end{equation}
where $S'({\mathcal V}^*)\mathcal{Z}$ denotes a Hadamard product, that is, $[S'({\mathcal V}^*)\mathcal{Z}]_{ia}= s'({V^*_i}^{(a)})Z_i^{(a)}$. Assume that there exist $\lambda\in \mathbb{C}^*,\ (\mathcal{Z},J) \in \mathcal{H}$ such that $dF_{(V^*,W^*)} \begin{pmatrix}\mathcal{Z}\\J\end{pmatrix} = \lambda\begin{pmatrix}\mathcal{Z}\\J\end{pmatrix}$. Taking the second component of this equation and computing the dot product with $S({\mathcal V}^*)$ leads to
$$({\lambda + \epsilon \mu}) J\cdot S = \frac{\epsilon}{M}\left ((S'\mathcal{Z}) \cdot  S^T\cdot S + S\cdot  (S' \mathcal{Z})^T\cdot S \right )$$
where $S =S({\mathcal V}^*)$, $S'=S'({\mathcal V}^*)$. Substituting this expression in the first equation leads to
\begin{multline}
M(\lambda + \epsilon \mu)(\lambda + 1) \mathcal{Z} = (\frac{\lambda}{\mu} + \epsilon) S \cdot  S^T\cdot (S'\mathcal{Z})
+ {\epsilon} (S'\mathcal{Z})  \cdot  S^T\cdot  S
+ \epsilon S \cdot (S'\mathcal{Z})^T \cdot S
\end{multline}

Observe that setting $\epsilon=0$ in the previous equation leads to an eigenvalue equation for the membrane potential only:
\[ (\lambda + 1) \mathcal{Z} =\frac{1}{\mu M} S \cdot S^T\cdot (S'\mathcal{Z}).\]
Since $W^* = \frac{1}{\mu M} \big(S \cdot S^T \big)$, this equation implies that $\lambda + 1$ is an eigenvalue of the operator $X \mapsto W^*.(S'X)$. The magnitudes of the eigenvalues are always smaller than the norm of the operator. Therefore, we can say that if $1>\|W^*\| S'_m$ then all the possible eigenvalues $\lambda$ must have a negative real part. This sufficient condition for stability is the same as in \cite{faugeras-grimbert-etal:08}. It says that fixed points sufficiently close to the origin are always stable.

Let us now consider the case $\epsilon \neq 0$. Recall that $\mathcal{Z}$ is a matrix. We now ``flatten'' $\mathcal{Z}$ by storing its rows in a vector called $\mathcal{Z}_{row}$. We use the following result in \cite{brewer:78}: the matrix notation of operator $X \mapsto A\cdot X \cdot B$ is $A\otimes B^T$, where $\otimes$ is the Kronecker product. In this formalism the previous equation becomes
\begin{multline}
M(\lambda + \epsilon \mu)(\lambda + l) \mathcal{Z}_{row} = \bigg((\frac{\lambda}{\mu}+\epsilon)  S \cdot S^T \otimes I_d
+ \epsilon I_d \otimes S^T \cdot S
+ \epsilon S \otimes S^T\bigg) \cdot(S'Z)_{\text{row}}
\label{eq: necessary condition for eigenvalues of M0}
\end{multline}
where we assume that the Kronecker product has the priority over the dot product. We focus on the linear operator $\mathcal{O}$ defined  by the right hand side and bound its norm. Note that we use the following norm $\|W\|_\infty = \sup_X\frac{\|W.X\|}{\|X\|}$ which is equal to the largest magnitude of the eigenvalues of $W$.
\begin{multline}
 \|\mathcal{O}\|_\infty \leq S'_m \bigg( |\frac{\lambda}{\mu}|  \|S \cdot S^T \otimes I_d\|_\infty
+ \epsilon \|S \cdot S^T \otimes I_d\|_\infty + \epsilon \|I_d \otimes S^T \cdot S\|_\infty\\ + \epsilon \|S\otimes S^T\|_\infty \bigg).
\end{multline}
Define, $\nu_m$ to be the magnitude of the largest eigenvalue of $W^* = \frac{1}{\mu M}(S \cdot S^T)$.
First, note that $S\cdot S^T$ and $S^T \cdot S$ have the same eigenvalues $(\mu M) \nu_i$ but different eigenvectors denoted by $u_i$ for $S \cdot S^T$ and $v_i$ for $S^T \cdot S$. In the basis set spanned by the $u_i \otimes v_j$, we find that $S \cdot S^T \otimes I_d$ and $I_d \otimes S^T \cdot S$ are diagonal with $(\mu M) \nu_i$ as eigenvalues. Therefore, $\|S \cdot S^T \otimes I_d\|_\infty = (\mu M) \nu_m$ and $\|I_d \otimes S^T \cdot S\|_\infty = (\mu M)\nu_m$.
Moreover, observe that
\begin{equation}
(S^T \otimes S)^T \cdot (S^T \otimes S) \cdot(u_i\otimes v_j) = (S\cdot S^T \cdot u_i) \otimes (S^T \cdot S \cdot v_j) =(\mu M)^2 \nu_i \nu_j\  u_i \otimes v_j
\end{equation}
Therefore, $(S^T \otimes S)^T \cdot (S^T \otimes S) =(\mu M)^2 {\rm diag}(\nu_i \nu_j)$. In other words, $S^T \otimes S$ is the composition of an orthogonal operator (i.e. an isometry) and a diagonal matrix. Immediately, it follows that $\|S^T \otimes S\| \leq (\mu M)\nu_{m}$.

Compute the norm of equation (\ref{eq: necessary condition for eigenvalues of M0})
\begin{equation}
 |(\lambda + \epsilon \mu)(\lambda + 1)| \leq S'_m(|\lambda| + 3 \epsilon \mu) \nu_m .
\label{eq: second necessary condition for eigenvalues of M0}
\end{equation}
Define $f_\epsilon: \mathbb{C} \rightarrow \mathbb{R}$ such that $f_\epsilon(\lambda) = |(\lambda + \epsilon \mu)||(\lambda + 1)| - (|\lambda| + 3 \epsilon \mu) S'_m \nu_m$. We want to find a condition such that $f_\epsilon(\mathbb{C}_+)>0$, where $\mathbb{C}_+$ is the right half complex plane. This condition on $\epsilon,\ \mu,\ \nu_m,\ \text{ and }S'_m$ will be a sufficient condition for linear stability. Indeed, under this condition we can show that only eigenvalues with a negative real part can meet the necessary condition (\ref{eq: second necessary condition for eigenvalues of M0}). Complex number of the right half plane cannot be eigenvalues and thus the system is stable. The case $\epsilon = 0$ tells us that $f_0(\mathbb{C}_+)>0$ if $1>S'_m \nu_m$, compute
$$\frac{\partial f_\epsilon}{\partial \epsilon}(\lambda) = \mu(\Re(\lambda) + \mu \epsilon)\frac{|(\lambda + 1)|}{|(\lambda + \epsilon \mu)|} - 3 \mu S'_m \nu_m$$
If $1 \geq \epsilon \mu$, which is most probably true given that $\epsilon << 1$, then $\frac{|(\lambda + 1)|}{|(\lambda + \epsilon \mu)|} \geq 1$. Assuming $\lambda \in \mathbb{C}_+$ leads to:
$$\frac{\partial f_\epsilon}{\partial \epsilon}(\lambda) \geq \mu(\mu\epsilon - 3 S'_m \nu_m) \geq \mu(1 - 3 S'_m \nu_m) $$
Therefore, the condition $3S'_m \nu_m<1$, which implies $S'_m \nu_m < 1$, and leads to  $f_\epsilon(\mathbb{C}_+) > 0$.

 \subsection{A very short introduction to computational cohomology}
 \label{part: persistent cohomology}
In algebraic topology, topological spaces (which are continuous objects) can be classified by roughly counting their number of holes. This coordinate-invariant description of a topological space is called its homology (or cohomology, the difference between them is beyond the scope of this paper). In fact the homology can be summarized by giving the betti numbers of the topological state. The sequence of betti number is made of positive integers. The first three betti numbers have the following definition: the first is the number of connected components, the second is the number of two-dimensional or ``circular'' holes and the third is the number of 3-dimensional holes or ``voids''. See \cite{hatcher2002algebraic} for a more rigorous approach.

However, in the example of toroidal retinotopy (see section 5.3.2 and figure~\ref{fig: torus positions}). we are dealing with a discrete cloud of points. Therefore, one needs to extend the definition of the betti numbers to discrete objects in order to find the underlying topology of the space within which the points are distributed. This is called computational or persistent cohomology. One reconstructs the topological space by considering balls of a given radius centered on each point in the cloud. For each radius (the abscissa of the right picture of figure~\ref{fig: torus positions}), one can compute the betti numbers of the resulting topological space. A barcode graph, e.g. the right picture of figure~\ref{fig: torus positions}, is constructed by drawing a horizontal bar for each connected component, 2-dimensional hole or 3-dimenional void etc. This is done for a range of radii. Finding the persistent cohomology of a cloud of points consists in observing the set of betti numbers that are stable through a significantly large range of radii. One then assumes that the points most likely lie on a topological space of a given homology if the corresponding betti numbers are stable enough. See \cite{zomorodian2005computing} for details.

In section \ref{part: torus}, we used the Jplex software package of \cite{plex} to compute the barcodes of the points corresponding to learning from inputs uniformly distributed over a 2-dimensional torus. We used 200 landmarks spread according to the maxminlandmark method to build simplices in Jplex, which returned the maximum radius (beyond which all the betti numbers except the first vanish) we used in the simulation. In figure \ref{fig: torus positions}, we see that for a wide range of radii the triplet $(1,2,1)$ is stable. This corresponds to a 2 dimensional torus.

 \bibliographystyle{apalike

redpajama_set_name: RedPajamaArXiv


text:
Town Supervisor: Edward Kinowski
Phone: 518.664.6148 x201
Email: ekinowski@stillwaterny.org
Always cc the secretary to ensure receipt of email
Confidential Secretary: Tiara Brown
Email: tbrown@stillwaterny.org
881 Hudson Ave, Stillwater, NY 12170
PO Box 700, Stillwater, NY 12170
Mon. – Fri. 8:00am – 3:00pm
Memorial Day – Labor Day
Monday – Thursday 8:00am -3:00pm
Friday 8:00am – 12:00pm
To meet with the Town Supervisor, a reservation is required. Reservations can be made with Tiara Brown, Confidential Secretary.
The Supervisor is the Town's Chief Executive Officer. As Chairman of the Board he presides over all Town Board Meetings which are open to the public. He is responsible for establishing the agenda of the meetings. The Town Supervisor is one vote of the five member Town Board.
The Town Supervisor is also the Chief Fiscal Officer of the Town and is responsible for maintaining the Town's fiscal health and insuring residents that the Town Departments are operating within the Town Budget voted on and passed by the Town Board.
The Town Supervisor is responsible for directing Town activities as Administrator of All Departments and provides direction in the implementation of Town policies and rules on an everyday basis.
The Town Supervisor and Council Members propose legislation and make policy decisions that directly affect residents. Mindful of both long-term goals and short-term objectives, the Supervisor responds to the needs of the community and represents the Town in its dealings with Saratoga County, State of New York and Federal Agencies.


text:
Idaea eugeniata is a moth in the family Geometridae. It is found in Italy, France, Andorra, Spain, Portugal and North Africa.

The wingspan is 21–22 mm for males and 24–25 mm for females.

The larvae are polyphagous and have been recorded feeding on various herbaceous plants.

Subspecies
Idaea eugeniata eugeniata
Idaea eugeniata algeriaca (Culot, 1917)

References

External links

Lepiforum.de

Moths described in 1870
Sterrhini
Moths of Europe
Moths of Africa

redpajama_set_name: RedPajamaWikipedia


text:
Hottest Comics
Investment Speculation
Comics in Movies & TV
Comic Collecting 101
Concert Poster Collecting 101 Guide
Advanced Solicitations
Write for GoCollect
Femme Fatales Heating Up Winter's Coldest Months!
Dark Flash Action Figure Sparks Hunt for Black Flash Keys
Will Future Generations Care About Stan Lee?
Weekly Silver, Bronze, & Copper Age Spec: Dr. Doom
DCU Spec – Teen Titans
Starting A Basketball Card Collection For Around $1,000
James Gunn Teases MCU Stars Heading to the DCU
Trending Comics: The Last of Us Clicks into the Hottest Comics
Undervalued & Overlooked Comics 1/28 – Silver Age
Secret Wars: Toys and Merchandise That Saved Marvel
AuctionComicsRecent
Comics Gems from Hakes' January 17th Auction
by Lauren Sisselman January 17, 2023
written by Lauren Sisselman January 17, 2023
It's a new year, and that can mean one thing: new auctions from Hake's! They're kicking off 2023 with a banger of an auction, featuring comics, political memorabilia, video games, and toys. On January 17th, their first comic auction of the year is set to end, and there are some hidden gems in this auction. Here's my pick of books you should keep your eyes on.
Aquaman #29
Aquaman is one of the strongest heroes in the DC Universe — plus, he can talk to fish. But not everyone is wowed by Aquaman's abilities. Ocean Master, also known as Orm Marius, is the half-brother of Aquaman. Yet, unlike his heroic brother, Orm possesses no superhero abilities. This leads Orm to be jealous and leads him into a life of crime. Post-Crisis Orm was given superhero abilities, and this is the version of Orm the modern era has stuck with. Orm did make his appearance in the DCEU in the Aquaman (2018) movie, where he was portrayed by Patrick Wilson.
This blue label CGC 4.0 with off-white pages is a great copy. Interest in this book fluctuates, but right now it is trending up, likely due to the upcoming Aquaman film. The one-year average for a 4.0 is $125, making this an affordable minor key. While I don't believe this will hit a new price peak in the near future, this is a great time to buy this book.
Life With Archie #1
Life with Archie is a personal favorite series of mine. The series originally ran from 1958 – 1991 and was then revived in 2010 after the Archie Marries…? storyline. The second round ended in 2014 with the death of Archie, a stunning move on Archie Comics' part. I do believe if they had not taken a chance with the Archie Marries…? series, we wouldn't have the Archie Comics we have today.
This blue label CGC 3.5 with off-white to white pages features an iconic Harry Lucey. The last sale for a similarly priced book was in March 2021, when a copy sold for $195. Interest in this book has waned, but Lucey has a dedicated following that has only become more popular over time. This could be your opportunity to get this book at a great price.
Marvel Spotlight #32
Spider-Woman just might be one of Marvel's underrated heroes, and her first appearance in Marvel Spotlight #32 is currently an underrated key. That could change in just a few mere weeks, as Jessica Drew / Spider-Woman will be appearing in Spider-Man: Across the Spider-Verse, the 2023 follow-up to the popular Spider-Man: Into the Spider-Verse from 2018. Prices on this book are still low but are steadily climbing possibly due to spectators.
This universal blue CGC 6.5 features off-white to white pages and presents well. Cover artist Gil Kane (who co-created the modern Green Lantern) was truly on his A-game with this dynamic and exciting cover. The current one-year average for 6.5 copies is $175/ By comparison, the one-year average for a blue label 9.8 is $3,406. If you're looking to add this key to your collection, now is likely the best time to buy.
*Any perceived investment advice is that of the freelance blogger and does not represent advice on behalf of GoCollect.
Comic Collectors' Corner
Lauren Sisselman
Hey there, I'm Baltimore Lauren! I'm an Overstreet Advisor, pinball fanatic, old comic book collector, and I enjoy import and vintage video games. Follow me on Twitter -- @BaltimoreLauren
ComicList Previews – DUNE HOUSE: HARKONNEN #1 (OF 12)
ComicList Previews – ARCHIE MILESTONES JUMBO COMICS DIGEST #18: JUGHEAD'S GUIDE TO LIFE
Dark Flash Action Figure Sparks Hunt for Black...
Weekly Silver, Bronze, & Copper Age Spec: Dr....
James Gunn Teases MCU Stars Heading to the...
Trending Comics: The Last of Us Clicks into...
Price Guide Search
Copyright © GoCollect All rights reserved


I don't wear suits often; but when I do, I still don't like to look too dressed up. I've really been fond of minimal patterns lately and I thought I'd try something a little different. I found this checked suit on ASOS and thought it'd be perfect for stepping out in the streets of England.
One of the most important elements I look for in a suit is the fit; it's so important to me! I like my suits to be well-fitted but not too skinny. I also make it a point to look at sizing as I want the shoulders, arms and hem to fit in all the right places. I also appreciate any fun lining details or accessories. I usually don't wear flashy accessories with my outfits, but it's nice to know you have that option depending on your mood.
I decided to pair the suit with leather sneakers because it feels modern and I wanted to dress the suit down a bit. Now, I feel ready to tackle my business meetings here without feeling so stuffy!
I cant wait to explore more of what London has to offer. It's been a glorious trip so far. Stay tuned on my Instagram.


The new Amarok '580' is so-called as its turbo-diesel V6 engine produces a mighty 580Nm of torque. With a sizzling 200kW of power also available (on overboost), it's comfortably the most powerful of the mainstream utes currently available and offers the sort of performance you may expect from a sports car.
The '580' engine is only available in the top-spec Amarok Ultimate model and comes with a hefty $71,990 (plus on-road costs) price tag. Other Amarok V6 models, right down the to new base-specification Core, have a less-powerful 550Nm version of this V6.
As you'd expect from the price tag, the 580 feels more like a luxury car than ute inside with its white-stitched leather seats, which are heated up front, and have electric adjustment, including electric lumbar-support adjustment, for both driver and passenger. The driver also has the benefit of both reach and tilt steering wheel adjustment; something that's missing on many of the popular utes. The 580's cabin equipment extends to sat-nav, a reversing camera and dual-zone climate control among its long list of standard feature. As with all Amaroks, the cabin is extra roomy (the widest in its class) and beautifully finished.
Good room in the back seat too, where the extra width comes in handy if you need to accommodate three adults. There are no airbags in the rear of the 580 however, or any Amarok for that matter, which is unusual these days given the focus on safety.
Not only does this tuned-up V6 make what's a very healthy amount of torque, but is does so from a very low rpm. In fact, all 580Nm are on tap from as low as 1,400rpm and doesn't diminish until 3,000rpm. Having so much torque available over such a wide range of engine speeds is in fact the secret behind the 580's impressive performance and flexibility.
As mentioned, the 580's peak power output is 200kW, which is available in third and fourth gears at 70 per cent of more throttle. This is the so-called 'overboost' output and is most useful for highway overtaking. Otherwise, the engine is rated to 190kW, but pedal-to-the-metal there's never any sense of the engine going from normal to overboost operation; it's all very seamless.
In general, driving the 580 is also effortless, smooth, quiet and refined. In fact, you would hardly know you're driving a diesel. The eight-speed automatic, as well as helping with the engine's responsiveness, is equally refined, with shifts so seamless that you hardly know it's changing gear. This is indeed a powertrain that would be as much at home in a performance-oriented luxury car as it is in a ute.
Getting all that power to the ground, especially when the road is wet or unsealed, or both, is helped by the 580's full-time 4wd system, which, as the name suggests, is in operation all the time and can be used on any road surface.
Compared to the other popular utes, the Amarok feels 'glued' to the road, a feeling that's helped by the excellent suspension control, and accurate and responsive steering. The 580 not only goes like a sports car, but compared to other utes it handles like a sports car!
The unladen ride is still a bit hard at the rear, as you'd expect of a ute, and the 580's low-profile 20-inch tyres also add a sharpness to the ride on rough roads that you don't get with the taller profile tyres on the 17 and 18-inch wheels fitted to less flashy Amarok V6 models.
The low-profile 20-inch tyres also do the 580 no favours in any off-road situation, paddocks included, both in terms of ride and damage/puncture resistance, although one saving grace is that they are not notably high-speed-rated low-profile tyres, which are even more damage-prone off-road. Better news is that the 17-inch or 18-inch wheels from the lower-spec 550 V6 models can be fitted to the 580 if your driving regime finds you regularly off sealed roads and you want a more practical and robust tyre.
The road-oriented tyres aside, the 580 is however somewhat of an off-road weapon even without dual-range gearing. Better off-road in fact than most of the popular utes that do have high and low-range gearing.
What's more, it's brilliantly simple to drive. When going from on-road (no matter what speed) to a tough off-road challenge, you don't have to touch a lever, switch or button, as the 580 is always in 4wd; the eight-speed automatic and specially tweaked torque converter provide the equivalent of low range, while the centre differential automatically locks up if need be. All you need to do is point it where you want to go provided you watch the 500mm water wading depth, which is not as deep as most competitors.
Like all Amarok V6's the 580 is rated to tow a class-leading 3,500kg thanks in part to its 6,000kg gross combined mass figure. But being the heaviest of the Amarok V6 models, it has the lightest payload, namely 836kg.
For the purpose of seeing how the 580 went with a load, we put 750kg in the tub and it hardly felt the weight. Not surprisingly, the engine didn't raise a sweat and while the suspension sat down about 70mm at the rear axle the 580 didn't feel nose-up or light in the steering, nor did the rear suspension bottom out on the bumps on country roads at normal open-road speeds.
Bonus points too for the Amarok is that it is the only one of the popular utes that will take a full-size pallet between the wheelarches of the factory tub. Why didn't all the other ute manufacturers think of that?
The fact that the tie-down hooks are where they should be, namely at bottom of the tub and not up high on the sides of the tub, also shows some clear thinking by the Amarok's designers, as does the fitting of a work light above the tub and a 12-volt outlet in the tub.
The 580 comes with a three-year, unlimited kilometre warranty and while service costs are on the high side compared to other popular utes, fixed-priced servicing is available on a plan. Volkswagen dealers in regional areas are also a bit thin on the ground and some independent workshops may not be as happy to take on an Amarok as they would be something like Toyota Hilux. On a more positive note, Amarok resale values are as strong as Hilux.
There is no doubt that the 580 is a very impressive bit of kit, as it combines performance, handing, functionality and working practicality like no other ute. It is, however, expensive and lacks some high-tech safety features like autonomous braking that's available on Ford's Ranger and Mercedes-Benz's X-Class ute.
If you're not too fussed about the extra performance that the V6 580 offers over the various V6 550 models – as the general driving experience is the same even if it's not quite as urgent – there's probably better value further down the range. The newly released 550 Core, which is stripped back to basics, is just $50,990 driveaway. Even the well-equipped 550 Highline is now $58,990 driveaway.