text:
\section{Introduction}
Complex biological systems are thought to be shaped through evolution  to  perform optimally a set of desired functions within the constraints of their biological features. A bulk of previous influential work \cite{olshausen1996natural,lewicki2002efficient,deneve2016efficient} has proposed that the organization of the mammalian cortex, the most sophisticated biological organ, makes no exception and follows too the general principles of evolution towards optimality under biological constraints.

One prominent theory is that the function of the cerebral cortex is organized for efficient coding, in that it optimizes the encoding of information under a number of biological constraints such as the metabolic expenditure, the speed and the timing of information coding. Imposed constraints can be formalized with a loss function.

To understand biological neural networks that operate in animal's brains, the mathematical description of networks optimizing functionally relevant loss functions has to include as many features of biological realism as possible, so that the constraints satisfied by the model network will be informative of those obeyed by biological neural circuits. Thus, all elements of the model should have a counterpart in biological circuits \cite{gerstner2014neuronal} and be directly relatable to them. It is still unclear how to derive efficient coding theories of biologically plausible neuronal networks.

An influential proposal of efficient coding in cortical circuits has been that visual cortical circuits  extract statistical regularities from static natural images with a sparse set of basis functions, which yielded basis functions that resemble receptive fields in primary visual cortex (V1) \cite{olshausen1996natural,olshausen1997sparse}. A related proposal in the domain of auditory processing yielded  auditory circuits that extract statistical regularities of natural sounds with a set of basis functions that resemble the tuning properties of auditory nerve fibers \cite{lewicki2002efficient}.
In these studies, the goal of efficient coding is to learn sparse spatial or temporal filters that enables optimal information encoding about the stimulus by the activity of each single neuron. Analyses of cortical networks, however, show that neurons are recurrently connected and that information processing tends to be distributed across neurons \cite{kafashan2021scaling}. In recurrently connected circuits, neuron-to-neuron interactions typically strongly impact on the information encoded in the spiking activity \cite{pillow2008spatio} as well as on how this information is transmitted to and read out by downstream structures to produce relevant behavioral outputs \cite{runyan2017distinct,valente2021correlations,panzeri2022structures}.
Moreover, analyses of spike trains recorded in the cerebral cortex show that millisecond precision of spike timing of sensory neurons carries information about both static \cite{petersen2001population,panzeri2001role} and dynamic stimuli \cite{kayser2010millisecond,nemenman2008neural}, which suggests that the temporal information in spiking activity is highly relevant for the information processing in the cortex. Thus, analyses of empirical neural data strongly suggest that biologically relevant theories of coding in the cerebral cortex must rely on spiking neuron models and on population codes.

To address this issue, models of how to optimally perform a given coding function have been extended to spiking dynamics \cite{boerlin2013predictive}. Models of efficient coding with spikes are developed from a loss function that includes a quadratic error between the desired signal and its estimate (see the Supplementary material). Equations for the subthreshold dynamics of membrane potentials and firing thresholds are analytically derived by assuming that the error is minimized with every spike \cite{alemi2018learning,bourdoukan2012learning}. However, the efficient coding with spikes, and the related theory of predictive coding \cite{zhu2013visual}, have been so far developed as abstract mathematical frameworks that lack biological realism. Namely, these models are not consistent with the observation that a single neuron can  be either excitatory or inhibitory, but not both (Dale's principle, \cite{eccles1986chemical,kandel1968dale}).
Very recently, progress has been made in this direction by deriving efficient coding with spiking networks with a more biologically plausible neural circuit architecture with interacting excitatory (E) and inhibitory (I) neurons \cite{li2020minimax}, opening the possibility to model neural computations with biologically relevant models.

Here, we make major further progress by deriving a  mathematically well-defined extension of efficient coding for considerably more biologically plausible spiking neural network models of cortical circuits. We develop a recurrently connected neural circuit with E and I neurons that performs a generic linear transformation between its inputs and its outputs. We impose the E-I architecture and recover a network with low-rank and structured connectivity that maximizes the information transmission about the stimulus with every spike. Here, we derive a solution of the spiking network model from optimality principles that includes many desirable and biologically realistic properties. Our solution takes the form of a biologically plausible generalized leaky integrate-and-fire (LIF) neuron model and satisfies  the principle of functional specificity of excitatory and inhibitory neurons (Dale's principle). Our solution also prescribes faster membrane time constant of I compared to E neurons, compatible with the biophysical properties of pyramidal neurons and fast-spiking inhibitory interneurons. Our derivations prescribe subthreshold dynamics, firing thresholds and resets after spiking, thus giving a complete description of a spiking neural network along with network's computation. Importantly, all elements of the model are biologically plausible and directly relatable to empirically measurable currents in neurons.

The paper is structured as follows. We first introduce the loss functions and explaining how the spike timing is conditioned on a minimization of loss functions of E and I neurons in every time step. We then outline the analytical treatment of loss functions that leads to expressions of membrane potentials and firing thresholds of a generalized LIF network model with structured, low-rank connectivity. Further, we provide a description and biologically plausible interpretation of derived currents. Finally, we simulate the network with biologically plausible parameters and analyze the behavior of the network with respect to the complexity of E-E synaptic connections.

\section{Results}
\subsection{Minimization of the loss function with spiking neurons}
From a spiking network of $N^E$ excitatory and $N^I$ inhibitory neurons, we define a linear population readout of the spiking activity of E and I neurons:
\begin{equation}
    \vec{\dot{\hat{x}}}_{y}(t)=-\frac{1}{\tau_y} \vec{{\hat{x}}}_{y}(t) + W_{Y} \vec{f}_y(t), \qquad y \in \{E,I\},
    \label{estimate}
\end{equation}
with $\tau_y>0$ the time constant of the population readout. In the population readout, the decoding matrix $W_y$ weights the vector of spikes $\vec{f}_y(t)=[f_1^y(t),\dots,f_{N_y}^y(t)]^\intercal$. The $M$-by-$N_y$ matrix $W_{y}$ associates each of the $N_E$ excitatory ($N_I$ inhibitory) neurons with $M$ weights, $W_y=(w^y_{mi}); m=1\dots,M,\ i=1,\dots,N_y$, where $N_y$ is the number of neurons of the cell type $y$, and $M$ is the number of stimulus features estimated by the network. The population readout in eq.~\eqref{estimate} is an estimate of the desired signal $\vec{x}(t)$, that is not directly accessible to the network.

We assume that the desired signal $\vec{x}(t)$ is a linear function of stimulus features\\ ${\vec{s}(t)=[s_1(t), \dots,s_m(t),\dots, s_M(t)]^\intercal}$ and is defined as follows:
\begin{equation}
    \vec{\dot{x}}(t)=A \vec{x}(t) + \vec{s}(t),
    \label{signal}
\end{equation}
where $A=(a_{mn});\ m,n=1,...,M$ is a square matrix determining the linear transformation between the input features $\vec{s}(t)$ and the desired signal $\vec{x}(t)$. We will refer to the transformation between the features and the desired signal as network's computation. If $A$ is a diagonal matrix, network's computation operates independently across features, while a non-diagonal matrix $A$ implements linear mixing of features.

We hypothesize that the objective of the excitatory network is to minimize the distance between the desired signal and the population readout of E neurons, while the objective of the inhibitory population is to minimize the distance between the readout of the E and I populations \cite{boerlin2013predictive,chalk2016neural}. Besides the coding error, we also take into account the metabolic cost on spiking, since spiking  activity in the brain is energetically expensive  \cite{niven2016neuronal}. These objectives are formalized by two loss functions with a quadratic coding error and a quadratic regularizer \cite{gutierrez2019population}, relative to E and I cell type:

\begin{subequations}
\begin{align}
&\mathrm L_E(t)=  \sum_{m=1}^{M}(x_m(t)-\hat{x}^E_{m}(t))^2 + \mu_E \sum_{i=1}^{N_E} (r^E _{i}(t))^2
\label{loss_exc}\\
&\mathrm L_I (t)=  \sum_{m=1}^{M}(\hat{x}^E_{m}(t) - \hat{x}^I_{m}(t))^2 + \mu_I \sum_{i=1}^{N_I} (r^I_{i}(t))^2,
\label{loss_inh}
\end{align}
\end{subequations}

where constants $\mu_E,\mu_I>0$ are weighting the relative contribution of the metabolic cost over the coding error. The low-pass filtered spike train of the neuron $i$ is defined as
\begin{equation}
    \dot{r}_i^y = - \frac{1}{\tau_i^{r,y}} r_i^y(t) + f_i^y(t), \qquad y\in \{E,I\},
    \label{filter}
\end{equation}
with $\tau_i^{r,y}>0$ the time constant of the low-pass filter. Gathering the variables $r_i^y(t)$ across neurons, we define a vector of time-dependent neural activities $\vec{r}_y(t)= [r_1^y(t)_,\dots, r^y_{N_y}]^\intercal$. Note that an alternative to the loss function in eq.~\eqref{loss_inh} would be to minimize the distance between the readout of the I population and the signal. However, such a formulation, without further approximations, leads to a network where E and I neurons are unconnected.

We assume that neuron $i$ of cell type $y$ will fire a spike at time $t=t^+$ if and only if this decreases the loss function, i.e.
\begin{equation}
	\mathrm{L}_y \left(t^+ |
	\left[f_i^y(t^+)=1 \right] + \eta^y_i(t^+) \right) < \mathrm{L}_y\left(t^- | \left[f_i^y(t^-)=0\right] \right),
    \label{condition_spiking}
\end{equation}
with the noise term $\eta_i^y(t) = \sigma^y_i \xi^y_i(t)$. Gaussian random variable $\xi_i^y(t)$ has zero mean and covariance $\langle\xi_i(t)\xi_j(t')\rangle=\delta_{ij}\delta(t-t')$, and $\sigma^y_i$ is the noise intensity. Contrary to previous approaches where the noise is added in the membrane potential \cite{boerlin2013predictive,koren2017computational}, we here assume noise in the condition for spiking (eq.~\ref{condition_spiking}). While the noise in the membrane potential models synaptic inputs that are unrelated to the coding function of the network, the noise as in eq.~\ref{condition_spiking} captures the noise in spike generation for a given membrane potential, a source of noise that characterizes spike generation in biological neurons \cite{faisal2008noise}. In biological neurons, a spike is initiated when a sufficient fraction of sodium channels in the neural membrane is opened, and the fraction of open channels for a given membrane potential is probabilistic.

Assuming that a spike in neuron $i$ at time $t^+$ is only fired if this decreases the loss function in eq.~\eqref{condition_spiking}, we apply an analytical treatment of eqs.~\ref{loss_exc}-\ref{loss_inh} similar to \cite{boerlin2013predictive,koren2017computational}, and obtain the following expressions:
\begin{align}
\begin{split}
 & \vec{w}_E^\intercal \left(\vec{x}(t)-\xe\right) - \mu_E r^E_i(t) > \frac{1}{2} \left(\|\vec{w}_i^E\|^2_2  + \mu_E \right) + \sigma_i^E\xi^E_i(t)
 \\
 & \vec{w}_I^\intercal \left(\xe - \vec{{\hat{x}}}_{I}(t)\right) - \mu_I r^I_i(t)  > \frac{1}{2} \left(\|\vec{w}_i^I\|^2_2  + \mu_I \right) + \sigma_i^I\xi^I_i(t)
 \end{split}
 \label{firing_rule}
\end{align}
with $\|\vec{w}_i^y\|^2_2$ the squared length of decoding vector of the neuron $i$, $\|\vec{w}_i^y\|^2_2=\sum_{m=1}^M(w^y_{mi})^{2}$.
Since we assumed threshold crossing, we interpret the left-hand side of eq.~\eqref{firing_rule} as the membrane potential and the right-hand side as the firing threshold of the neuron $i$. Note that the firing threshold has a deterministic and a stochastic part. Collecting membrane potentials across neurons, $\vec{u}_y(t) = [u^y_1(t),\dots,u^y_{N_y}(t)]^{\intercal}$ with $y \in \{E,I\}$, we express them with vector notation as follows:
\begin{align}
\begin{split}
 & \vec{u}_E(t) \equiv W_E^\intercal \left(\vec{x}(t)-\xe\right) - \mu_E \vec{r}_E(t)
 \\
 & \vec{u}_I(t) \equiv W_I^\intercal \left(\xe - \vec{{\hat{x}}}_{I}(t)\right) - \mu_I \vec{r}_I(t).
 \end{split}
 \label{uE}
\end{align}

To express the temporal dynamics of the membrane potentials, $\vec{\dot{u}}_E(t)$ and $\vec{\dot{u}}_I(t)$, we take the time-derivative of all time-dependent terms in eq.~\eqref{firing_rule}. Contrary to previous approaches and without loss of generality, we express the mixing matrix $A$ as a difference of a square matrix $B$ and a diagonal matrix $\lambda_E \mathbf{I}^{MxM}$,
\begin{equation}
    A = B - \lambda_E \mathbf{I}^{MxM},
    \label{expressA}
\end{equation}
with $\mathbf{I}$ the identity matrix. Seen that loss functions minimize the distance between the signal and the E estimate (eq.~\ref{loss_exc}), and between the E and the I estimates (eq.~\ref{loss_inh}), we use the approximations $\vec{x}(t) \approx \xe$ and $\xe \approx \vec{{\hat{x}}}_{I}(t)$. Moreover, we remove synaptic connections that violate Dale's law (see the Supplementary material). After re-arranging the terms and without further approximations, interestingly, the network model can be expressed as a generalized LIF model with structured connectivity.

\subsection{Generalized integrate-and-fire network with low-rank structured connectivity}

Due to substitutions of the signal and the estimates, we write the membrane potential as $u_i^y(t)\approx V_i^y(t)$. The subthreshold dynamics of E and I neurons is then dependent on the following currents:
\begin{subequations}
\begin{align}
    &\tau_E\dot{V}_i^E(t) = - V_i^E(t) + I_i^{\text{ff}}(t) + I_i^{EI}(t) + I_i^{EE}(t)+ I^{\text{local }E}_i(t) + I^{\text{rebound}}_i(t),
    \label{dVe}\\
    &\tau_I \dot{V}_i^I(t)= - V_i^E(t) + I_i^{IE}(t) + I_i^{II}(t) + I^{\text{local }I}_i(t),
     \label{dVi}
\end{align}
\end{subequations}
and complemented with fire-and-reset rule: $\text{if } V^y_i(t^-) \geq \vartheta^y_i(t^-) \rightarrow V^y_i(t^+)=V^{\text{reset }y}_i$. For simplicity, we set the resting potential of all neurons to $V^{\text{rest}}=0$, but a biologically plausible resting potential can be introduced in eqs.~\eqref{dVe}-\eqref{dVi} without affecting network's dynamics. From eq.\eqref{firing_rule}, the
firing threshold that takes into account the noise in spike generation (eq.~\ref{condition_spiking}) is the following:
\begin{equation}
    \vartheta_i^y(t)=\frac{1}{2} (\mu_y + \|\vec{w}_i^y\|^2_2  ) + \sigma_i^y \xi_i^y(t), \qquad y \in \{E,I\},
    \label{Threshold}
\end{equation}
while the reset potentials for E and I cell types are

\begin{align}
\begin{split}
    & V_i^{\text{reset }E}= -\frac{1}{2} ( \mu_E - \|\vec{w}_i^E\|^2_2)\\
    & V_i^{\text{reset }I}= -\frac{1}{2} ( \mu_I + \|\vec{w}_i^I\|^2_2).
    \end{split}
    \label{Reset}
\end{align}

The firing thresholds in eq.~\ref{Threshold} and reset potentials in eq.~ \ref{Reset} are thus proportional to the length of decoding vector of the local neuron $\|\vec{w}_i^y\|_2$, and to the regularizer $\mu_y$ that affects equally all the neurons of the same cell type. Note that if decoding weights $w^y_{mi}$ are heterogeneous across neurons, firing thresholds and resets are heterogeneous as well.

The membrane equations in eqs. \ref{dVe}-\ref{dVi} depend on a number of currents that, interestingly, all have a straightforward counterpart in biological networks. The first currents on the right-hand site of eqs. \ref{dVe}-\ref{dVi} are leak currents,
that result from absorbing of several terms, among others the diagonal matrix $\lambda_E \mathbf{I}^{MxM}$ from eq.~\ref{expressA}. The feedforward current to E neurons is given by a weighted sum of external inputs $s_k(t)$,
\begin{equation}
     I_i^{\text{ff}}(t)= \tau_E \sum_{m=1}^{M} w^E_{mi}s_m(t).
     \label{ff}
\end{equation}
Inhibitory synaptic currents to the postsynaptic neuron $i$ are given by a weighted sum of spikes of presynaptic neurons,
    \begin{equation}
      I^{II}_i(t)= - \tau_I \sum_{\substack{j=1\\j\neq i }}^{N_I}C_{ij}^{II} f_j^I(t), \qquad
    I^{EI}_i(t)= - \tau_I \sum_{j=1}^{N_I}C_{ij}^{EI} f_j^I(t),
    \label{Fast_syn}
\end{equation}
with structured connectivity matrices $C^{yz}_{ij}$, where the strength of synaptic connection is proportional to the similarity of decoding vectors of the presynaptic neuron $j$ and the postsynaptic neuron $i$,
\begin{equation}
     C^{yz}_{ij}=
    \begin{cases}
     (\vec{w}^y_i)^\intercal \vec{w}_j^z, & \text{ if $(\vec{w}^y_i)^\intercal \vec{w}_j^z > 0$} \\
     0, & \text{ otherwise}.
    \end{cases} \label{C_fast}
\end{equation}
Note that synaptic connections between neurons with different selectivity (i.e., neuronal pairs with negative dot product of decoding vectors, $(\vec{w}^y_i)^\intercal \vec{w}_j^z < 0$), have been set to zero to make the network consistent with Dale's law (see the Supplementary material). In particular, we had to ensure that a particular neuron can only send excitatory or inhibitory currents to other neurons, but not both. Even though there is no direct fast synaptic connections between E-to-E neurons, effectively, fast synaptic connectivity implements lateral inhibition between E neurons with similar selectivity. Lateral inhibition (or competition) between E neurons with similar selectivity is a dynamical effect that is essential for an efficient neural code \cite{king2013inhibitory} and has been demonstrated in biological circuits \cite{chettih2019single}.

While local inhibitory synapses have fast kinetics of spike trains (eq.~\ref{Fast_syn}), recurrent excitatory synaptic currents are slower since they are convolved with the synaptic filter $z_j^E(t)$:
\begin{align}
    \begin{split}
        I^{EE}_i(t) & = \tau_E \sum_{\substack{j=1 \\ j\neq i}}^{N_E}D^{EE}_{ij} z_j^E(t)
        \\
        I^{IE}_i(t) & =\tau_E\sum_{j=1}^{N_E}C_{ij}^{IE} f_j^E(t) + \left(\frac{\tau_E}{\tau_I}-1 \right)  \sum_{j=1}^{N_E}D^{IE}_{ij} z_j^E(t), \qquad  \tau_E>\tau_I\\
        \dot{z}^E_j(t) & =-\frac{1}{\tau^{\text{syn}}_E} z^E_j(t) + f^E_j(t), \qquad  \tau^{\text{syn}}_E=\tau_E,
    \end{split}
    \label{E_Syn}
\end{align}
with matrices of synaptic interactions:
\begin{equation}
    D^{yE}_{ij}=
    \begin{cases}
     (\vec{w}^y_i)^\intercal B \vec{w}_j^E, & \text{ if $(\vec{w}^y_i)^\intercal \vec{w}_j^E > 0$}, \qquad B \text{ positive semi-def.},\\
     0, & \text{ otherwise}.
    \end{cases}
    \label{D}
\end{equation}
where $y=E$ for E-to-E synapses and $y=I$ for E-to-I synapses. Same as with fast synaptic currents in eq.~\eqref{Fast_syn}, the strength of the synapse between the presynaptic neuron $j$ and the postsynaptic neuron $i$ in eq.~\eqref{D} depends on the similarity of decoding vectors of the presynaptic and postsynaptic neurons, $(\vec{w}^y_i)$ and $\vec{w}_j^E$.
To make slower synaptic currents consistent with Dale's law, we removed connections between neurons with different selectivity, i.e., connections for which the following is true: $(\vec{w}^y_i)^\intercal \vec{w}_j^z < 0$ (see eq.~\ref{D}). In addition, slower synaptic currents depend on the matrix $B$ that was expressed from the mixing matrix $A$ (see eq.~\ref{expressA}). To ensure that presynaptic excitatory neurons always cause an excitatory current in the postsynaptic neuron, we constrained the matrix $B$ to positive semi-definite. Moreover, the excitatory effect of an E-to-I synapse constrains the following relation of time constants: $\tau_I<\tau_E$. Since $\tau_E$ and $\tau_I$ are membrane time constants of the E and I cell type, respectively, this relation constrains the membrane time constant in E neurons to be slower than in I neurons, consistent with E cell type modeling pyramidal cells while the I cell type models fast-spiking interneurons \cite{mccormick1985comparative}. Fast-spiking (somatostatin) interneurons are a prevalent class of inhibitory neurons in cortical circuits \cite{tremblay2016gabaergic}. As pyramidal neurons account for about 80 $\%$ of cortical neurons, and fast-spiking interneurons account for at least a half of inhibitory neurons in the cortex~\cite{wonders2006origin}, our model altogether provides a description for about 90 $\%$ of cortical neurons. Note that alternative solutions for slow currents can be formulated (see the Supplementary Material). These alternative solutions are, however, of lesser biological relevance since they either result in a network without E-to-E connections, or lead to a global imbalance of E and I currents in the network. Contrary to fast connections that are expected to implement lateral inhibition among E neurons with similar selectivity (see eq.~\ref{Fast_syn}), slower E-to-E synaptic connections are expected to implement cooperation among excitatory neurons with similar selectivity.

The efficient E-I network has low-rank connectivity. The rank of fast synaptic connectivity is, by definition of the connectivity matrices $C$ (eq.~\ref{C_fast}) and $D$ (eq.~\ref{D}), equivalent to the maximal number of features encoded by the network, $M$. We assume that the number of active features $s_m(t)$ is much smaller than the number of neurons in the network, $M << N^E$, which gives low-rank connectivity matrices \cite{mastrogiuseppe2018linking}. Low-rank connectivity constrains the neural activity to a low-dimensional manifold, and the relevance of the latter for the description of the dynamics of biological neural ensembles has been strongly suggested by empirical neural recordings \cite{gallego2020long,yu2008gaussian}. Note also that using heterogeneous and sparse decoding weights $w^y_i$ gives heterogeneous and sparse synaptic connectivity.

Next, we have currents that are triggered by spiking of the local neuron,
\begin{equation}
     I^{\text{local }y}_i(t) = -\mu_y \left(1 -\frac{\tau_y}{\tau_i^{r,y}} \right) r_i^y(t),
     \qquad y \in \{E,I\}, \label{Local}
 \end{equation}
with $r^y_i(t)$ the single neuron readout with time constant $\tau_i^{r,y}$ (eq.~\ref{filter}). While the kinetics of the local current is given by the single neurons readout, the sign and the strength of the current depend on the relation of time constants between the population readout $\tau_y$, and the single neuron readout $\tau_i^{r,y}$. To allow the population readout $\vec{{\hat{x}}}_{y}(t)$ to track fast changes of the signal $\vec{x}(t)$, we consider the case where that the population readout is fast, while the single neuron readout, related to a homeostatic readout of single neuron's firing frequency \cite{desai1999plasticity}, is a slower process. Such a relation of time constants, $\tau_y<\tau_i^{r,y}$ for $y \in \{E,I\}$, constrains the local current in eq.~\ref{Local} to spike-triggered adaptation \cite{liu2001spike}. If, on the contrary, we assume the following relation: $\tau_y<\tau_i^{r,y}$, we get spike-triggered facilitation, a current that might be relevant during learning and restructuring of synaptic connections \cite{payeur2021burst}. Besides the difference of time constants between the population and the single neuron readout, the strength of the local current depends on the regularizer $\mu_y$. In loss functions, the regularizer weights the importance of the metabolic cost over the coding error (eq.~\eqref{loss_exc} -\eqref{loss_inh}). A strong regularizer $\mu_y$ implies that the average firing rate contributes strongly to the loss evaluated by the loss function, a situation where the spiking frequency should be kept in check to keep the loss as small as possible. The weighting of the metabolic cost over the coding error in the loss functions is implemented mechanistically as the local current in eq.~\ref{Local}. Its connection with the loss function makes most sense when the local current is spike-triggered adaptation. An increase of the regularizer $\mu_y$ increases the importance of the metabolic cost in the loss functions, mechanistically increasing the amplitude of adaptation, which reduces the firing frequency of the spiking neuron.

Finally, we derived a depolarizing current triggered by the spike of the local excitatory neuron, that we termed $I^{\text{rebound}}_i(t)$. In the efficient spiking network, the diagonal of the E-to-E recurrent connectivity matrix $D^{EE}$ (eq.~\ref{E_Syn}) implements a positive ``self-connection" - as the neuron spikes, it generates a local depolarizing current with slower kinetics of the low-pass filtered spike train. We write this effect as follows:
\begin{equation}
    I^{\text{rebound}}_i(t) = \tau_E D^{EE}_{ii} z^E_i(t), \label{rebound}
\end{equation}
with $z_i^E(t)$ the synaptic filter as in eq.~\ref{E_Syn}.
Since the matrix $B$ is constrained to be positive semi-definite (see eq.~\ref{D}), the current in eq.~\ref{rebound} is always depolarizing. After a spike, the local neuron is reset to its reset potential, which, in E neurons, is necessarily below the resting potential (see eq.~\ref{Reset}; the resting potential is here at 0 mV). Right after the spike, the local neuron is therefore strongly hyperpolarized, and precisely at this moment, the rebound current activates. The current $I^{\text{rebound}}_i(t)$ thus creates a rebound of the membrane potential after a strong hyperpolarization. In biology, a current with these properties is the hyperpolarization-activated cation current (also called h-current \cite{pape1996queer}), a current important for the generation of network oscillations, spontaneous firing and in controlling the excitability of cortical neurons \citep{luthi1998h,pignatelli2013h}.

\subsection{Effect of complexity of E-E connectivity on network responses}

In the following, we analyze the effect of the complexity of E-E connections on network responses with analytical considerations and with simulations. We start by considering a simplified network model where we assume the matrix $A$, that determines the transformation between the external input $\vec{s}(t)$ and the internal signal $\vec{x}(t)$ (eq.~\ref{expressA}), to be a diagonal matrix. A simple way to do so is to set the elements of the matrix $B$ to 0, i.e., $B=(b_{mn});\ b_{mn}=0\ \forall m,n=1,\dots,M$, which constrains the desired signal to a leaky integration of input features that is independent across features,
\begin{equation}
    \vec{\dot{x}}(t)=-\lambda_E \vec{x}(t) + \vec{s}(t).
    \label{dx_leaky}
\end{equation}
The time constant of the desired signal $\vec{x}(t)$ is now equal to the time constant of the population readout of the E cell type in eq.~\eqref{estimate}, and such a network is devoid of E-E and slower component of E-I (eq.~\ref{E_Syn}) synaptic currents, as well as of the rebound current (eq.~\ref{rebound}). We further simplify the network by considering the special case where the time constants of the population and the single neuron readout are equal, $(\tau_i^{r,y}) =\tau_y \ \forall i; y \in \{E,I\}$, which sets the local currents in eq.~\eqref{Local} to zero.

To simulate the network (the simplified one or a more complex one), we have to set values for decoding weights $w^y_{mi}$, which are free parameters. We use unstructured decoding weights that we draw from the normal distribution with zero mean and standard deviation $\sigma^y_w$, $w^y_{mi} \sim \mathcal{N}(0,\sigma_w^y)$, with $\sigma^E_w=1$ and the ratio $\sigma^I_w:\sigma^E_w=3:1$. We chose decoding weights in the I cell type to be stronger than in the E cell type, but the number of I neurons is smaller then the number of E neurons, with a biologically plausible ratio $N^E:N^I=4:1$. Decoding weights then determine the strength of recurrent synaptic connections (eqs.~\ref{C_fast} and \ref{D}).

As we stimulate the simplified network with a step input in one of the input features, the network responds with asynchronous spiking of neurons that are aligned to the active feature (Figure~\ref{net_full}B). The feedforward current to neuron $i$ depends on the alignment of the input $\vec{s}(t)$ with neuron's decoding vector $\vec{w}^E_{i}$ (see eq.~\ref{ff}). When a single feature is activated, e.g. $s_n(t) \neq 0$, a positive input, $s_n(t)>0$, drives neurons with positive decoding weight for the $n-$th feature (i.e., neurons with $w^E_{ni}>0$), while a negative input, $s_n(t)<0$, drives neurons with negative decoding weights (neurons with $w^E_{ni}<0$). If several features are active simultaneously, the feedforward current to E neurons is given by a linear mixture of features. The simplified network is purely feedforward-driven and the network response never outlasts the stimulus (Figure~\ref{net_full}B). The recurrent connectivity of such a network consists of structured fast connections between neurons with similar decoding selectivity (eq.~\ref{C_fast}), and the stronger the similarity of decoding vectors of neurons $i$ and $j$, the stronger their synapse.

We then considered a network with E-E and slower E-I connections. In general, E-E and slower E-I connections emerge when the matrix $A$ in eq~\ref{expressA} is non-diagonal. The minimal requirement for the emergence of E-E and slower E-I connections is that at least one diagonal element of the matrix $A$ in eq.~\ref{expressA} is different than the inverse membrane time constant of E neurons, $\lambda_E$. Recurrent excitatory (E-E) connections and slower E-I connections, $D^{EE}$ and $D^{IE}$, are then determined by the similarity of decoding vectors $\vec{w}^E_i$ and $\vec{w}^E_j$ (same as fast connections), as well as on the matrix $B$ (in contrast to fast connections; see eqs.~\ref{C_fast} and \ref{D}). Square matrix $B$ is constrained to be positive semi-definite to ensure that the network obeys Dale's law (eq.\ref{D}), and positive semi-definiteness in turn constrains the matrix $B$ to be symmetric and to have non-negative eigenvalues. To ensure that these properties are satisfied, we write the matrix $B$ as follows:
\begin{equation}
B=a \Gamma \Gamma^\intercal, \qquad
    \Gamma = [\vec{b_1},\dots,\vec{b_{M'}}]
    \label{defineB}
\end{equation}
with $a>0$ a positive constant influencing the strength of E-E and slower E-I connections, and ${\vec{b_1},\dots,\vec{b_{M'}}}$, a set of linearly independent column vectors, with entries ${\vec{b_n}=[b_{n1},\dots,b_{nM}]^\intercal, n=1,\dots,{M'}}$, and where the number of vectors is constrained as follows: ${1\leq M' \leq M}$.
In it important to consider that matrix $\Gamma$ determines the rank of the matrix $B$, as $M'$ linearly independent column vectors set the rank of $B$ to $\text{rank}(B)=M'$. The rank of the matrix B in turn determines the rank of connectivity matrices $D^{EE}$ and $D^{IE}$ through eq.~\eqref{D}.

The response of the network now critically depends on the complexity of recurrent excitatory (E-E) connections.
In the simplest case, beyond the trivial case where all elements of $B$ are set to zero that we discussed above, $\Gamma$ is defined with a single column vector, $\Gamma=\vec{b_1}$. This sets the matrix $B$ to the following: $B=a \vec{b_1} \vec{b_1}^\intercal$, and determines the rank of the matrix $B$, as well as the rank of the E-E connectivity matrix $D^{EE}$, to $\text{rank}(B)=\text{rank}(D^{EE})=1$. We simulate such network, setting the membrane time constants of E and I cell type to biologically plausible values, with $\tau_E=10$ ms for pyramidal neurons, and $\tau_I=5$ ms for inhibitory interneurons, as measured empirically in the cerebral cortex \cite{tremblay2016gabaergic}. In addition, we assumed time constants of the single neuron readout to be longer than the time constants of the population readout, as single-neuron readout is assumed to be a slower process of homeostatic regulation of the firing rate in single neurons. In particular, we set the time constant of the single neuron read-out to be twice as long as the time constant of the population read-out. According to eq.~\eqref{Local}, such relation of time constants between the single neuron and population read-out gives spike-triggered adaptation in E and I neurons.

As we simulated such a network, we found a population-wide oscillation in the network activity (Figure~\ref{net_full}C). Such a population-wide oscillation is in contrast to the response of the simpler network without E-E connections and without spike-triggered adaptation, where the population firing rate with constant stimulus converges to a constant value after an initial transient (Figure~\ref{net_full}B). Similarly to the simpler network, however, we still have that only neurons aligned with the input feature are active (e.g., neurons with $w^y_{1i}>0$ when the active stimulus feature is $s_1(t)>0$).

\begin{figure}[htpb]

  \centering
  \includegraphics[width=0.99\linewidth]{fig/fig_full_net.eps}
  \caption{(A) Schema of the E-I network. E neurons (gray) receive a linear sum of external inputs $\vec{s}(t)$. E and I (magenta) neurons with similar selectivity are recurrently connected. Inhibitory synapses  $C^{EI}$ and $C^{II}$ have fast kinetics, E-to-I synapses have a fast and a slower component ($C^{IE}$ and $D^{IE}$), while E-to-E synapses only have the slower kinetics $C^{II}$ and , as well as through slower synapses $D^{EE}$ and  (green). Linear population readout of the spiking activity of E and I cell types define estimates $\xe$ and $\vec{{\hat{x}}}_{I}(t)$, respectively. (B) Activity of the simplified network without E-E connections and without the adaptation current. Top: Step input in the first feature $s_1(t)$. The remaining features are inactive. Middle: Spike trains of E and I neurons aligned with the positive input in the first feature (black and magenta, respectively), and aligned with the negative input in the first feature (blue and red, respectively). Bottom: Neuron-averaged firing rate, with the same color code as in the middle plot. Parameters: $M=3$,  $N_E=400$, $N_E:N_I = 4:1$, $\tau_E=\tau_I = \tau_i^{r,E} =\tau_i^{r,I}=10 \text{ ms}\ \forall i$, $b_{mn}=0\ \forall m,n=1,\dots,M$, $\mu_E = \mu_I=6$, $\sigma^E_i=\sigma^I_i = 0.167\ \forall i$, $\sigma^E_w=1$, $\sigma^I_w:\sigma^E_w=3:1$, $dt=0.02\ ms$. (C) Same as in B but showing the activity of the network with E-E connections and where E-I synapses have both fast and slow component. The network also has adaptation current in both E and I neurons. Parameters: $\tau_E=10$ ms, $\tau_I=5$ ms, $\tau_i^{r,E} =20$ ms $\forall i$, $\tau_i^{r,I}=10 \text{ ms}\ \forall i$, $B=a \vec{b_1}^\intercal \vec{b_1}$ with $a=0.035$ and $\vec{b_1}=[0.5,0,0.3]^\intercal$, $\sigma^E_i=\sigma^I_i = 0.25\ \forall i$. Other parameters as in B.}

  \label{net_full}
\end{figure}

With higher rank of E-E connectivity, the network response reflects mixing of input features.
We now define the matrix $B$ with $M' > 1$ linearly independent vectors, where ${a_1 \vec{b_1} + \dots +a_{M'} \vec{b_{M'}}=\vec{0}}$  if and only if $ {a_n=0\ \forall n}$. Linear independence of column vectors $\vec{b_n}$ implies that the matrix $B$ has rank $M'>1$. The rank of the matrix $B$ than determines the rank of E-E connectivity matrix due to eq.~\eqref{D}. We find that with rank of E-E connectivity bigger than 1, also neurons that are not aligned with the active input feature participate in the network response. As shown on Figure~\ref{net_rankB}A-B, with $s_1(t)>0$, excitatory neurons with a positive decoding weight for the first feature, $w^E_{1i}>0$ (black), have similar firing rate than neurons with negative decoding weight for the same feature, $w^E_{1i}<0$ (blue). Since the feedforward current only drives neurons with decoding weights aligned to the input feature (eq.~\ref{ff}; here neurons with $w^E_{1i}>0$  in black), this means that the response of misaligned neurons (i.e. neurons with $w^E_{1i}>0$), is driven by the E-E connectivity. In summary, E-E connectivity with rank 1 only drives neurons that are aligned with the active stimulus feature, while higher rank of E-E connectivity also drives misaligned neurons.

For a fixed strength of E-E connectivity, increasing the noise results in longer duration of network's response to the stimulus, and in spontaneous activation of the network long after the external input is turned off (Figure~\ref{net_rankB}B). In the absence of the external stimulus, such a network responds with rhythmic spontaneous activation (Up state), followed by a period of silence (Down state; Figure~\ref{net_rankB}C). Up states are triggered by the noise at threshold, that by itself occasionally leads to a spike of a single neuron (see eq.~\ref{Threshold}). A random spike is amplified by the recurrent excitatory synaptic activity, and spontaneous activity is suppressed by the adaptation current.

\begin{figure}[htpb]

  \centering
  \includegraphics[width=0.99\linewidth]{fig/fig_complex.eps}
  \caption{(A) Same as in Figure~\ref{net_full}C, but using the matrix $B$ with rank bigger than 1. To construct the matrix $B$, we impose the rank $M'=M=3$ and use the following column vectors: $\vec{b_1}=[0.4,0.1,0.1]^\intercal$, $\vec{b_2}=[0.1,0.4,0.1]^\intercal$, $\vec{b_3}=[0.1,0.1,-0.4]^\intercal$. Other parameters as in Figure~\ref{net_full}C. (B) As in A, but with increased strength of the noise. Parameters: $\sigma^E_i=\sigma^I_i = 0.3\ \forall i$. Other parameters as in A. (C) Same as in B, but without the external stimulus, $s_m(t)=0\ \forall m,t$. In the network with rank of E-E connectivity matrix larger than 1, all neurons participate in the population responses.}

  \label{net_rankB}
\end{figure}

\section{Discussion}

We developed an E-I spiking network that makes information coding efficient under biologically relevant constraints. Our model is made efficient by minimizing two loss functions comprising a quadratic estimation error and a quadratic regularizer, and assuming an arbitrary linear input/output transformation. The general solution of an online minimization of loss functions with spikes is a generalized LIF model with a specific set of structural and dynamical features. Structural features include low-rank structured connectivity \cite{song2005highly,ko2011functional,koren2020pairwise}, where fast synapses between E and I neurons implement competition between neurons with similar decoding selectivity and slower synapses implement cooperation. Moreover, structural features resulting from optimization with spikes are external inputs that are shared across neurons, spike-triggered adaptation currents in single neurons and hyperpolarization-triggered rebound current in E neurons. Assuming heterogeneous decoding weights, all currents with the exception of leak currents are heterogeneous across neurons, increasing the biological plausibility of the model.

Similarly to the seminal paper by Boerlin and colleagues \cite{boerlin2013predictive}, the spiking network proposed here assumes that spikes are fired if and only if they minimize the quadratic error between the signal and the population readout. As the network model is analytically developed from such optimization principles, efficient coding is a build-in property of the network. In earlier works on efficient coding \cite{bourdoukan2012learning}, however, the leak current is not derived exactly but only added to the membrane equation for biological plausibility. Our derivations define an exact expression for the leak current, similarly as in \cite{koren2017computational}. Contrary to most previous works on efficient coding with spikes that use a single loss function to derive the membrane equations, we here propose two separate loss functions that relate to the activity of E and I neurons. With this, we impose the E-I architecture and allow for functional diversity between E and I neurons. A previous proposal that already used separate loss functions for E and I neurons \cite{chalk2016neural} yielded a network similar to our simplified network, but the behavior or their network is markedly different than our simplified model, possibly due to very long and unrealistic synaptic time constants they use.

Most previous proposals on efficient coding \cite{brendel2020learning,chalk2016neural,koren2017computational,bourdoukan2012learning,barrett2016optimal} assumed a specific neural computation, namely a leaky integration of external inputs (but see \cite{boerlin2013predictive,alemi2018learning}). We here developed the case of the general linear transformation between the external inputs and the internal signals of the network with a biologically plausible spiking network. This generalization allowed us to formulate a biologically plausible expression for E-E connections and to gain insights on how the structure of the mixing matrix qualitatively changes network responses. Without mixing of features, only neurons that are aligned to the external stimulus, and consequently driven by the feedforward current, participate in network responses. With increased complexity of E-E connections, we also observe activation of neurons that are not driven by the feedforward current, but are instead engaged through recurrent E-E connections. Moreover, in the absence of the external stimulus and with sufficient background noise, such a complex network produces spontaneous Up and Down states that could describe spontaneous Up and Down states observed in biological networks \cite{sanchez2000cellular}.

Our results demonstrate that the minimization of the quadratic loss function with spikes results in a generalized LIF model, a spiking neuron model that has been shown to provide a good fit to the spiking dynamics of biological networks \cite{gerstner2009good,jolivet2008quantitative}.
An important advantage of the present computational framework over previously proposed generalized LIF models is the access to a functional interpretation of network parameters. In general, a change in a network parameter can have a strong influence on network's dynamics. Besides characterizing the effect on neural dynamics, it is insightful to also have the interpretation of the role of a specific parameter on the functional/computational level. In the present setting, parameters of the spiking network are directly related to the variables minimized in loss functions, which allows to connect their effect on dynamics with their role for network's computation.

In the present work, we emphasized the theoretical development of an optimization-based approach towards a biologically plausible spiking neural network with complex, non-linear responses. Future work could introduce performance measures, for example by measuring the distance between the signal and the population readout, or by using information-theoretic measures \cite{borst1999information,montemurro2004optimal,pica2017quantifying}. Moreover, future work could perform an analysis of network parameters and assess how are particular dynamical phenomena, such as rhythmic spontaneous Up and Down states \cite{sanchez2000cellular}, related to information processing in efficient spiking networks.

\section*{Acknowledgments}
This work was supported by the NIH Brain Initiative Grants U19 NS107464, R01 NS109961, and R01 NS108410.

{
\small

\section{From the loss function to the membrane potentials}
\label{supp_1}

In \cite{boerlin2013predictive}, a recurrently connected spiking network is developed from a single loss function of the form:
\begin{equation}
    \mathrm L(t)=  \sum_{m=1}^{M}(x_m(t)-\hat{x}_{m}(t))^2 + \nu \sum_{i=1}^{N} r _{i}(t) + \mu \sum_{i=1}^{N} r^2_{i}(t)
    \label{loss_1ct}
\end{equation}
with $\nu,\mu > 0$ and where $x_m(t)$, $\hat{x}_m(t)$ are the signal and the estimate of the $m-$th feature of the stimulus, respectively, and $r_i(t)$ is the low-pass filtered spike train of the neuron $i$.

In the present work, we introduced two loss functions, one for the excitatory (E) and one for the inhibitory (I) neurons, which allowed us to define biologically plausible membrane equations for E and I neurons.
We consider a sensory stimulus with $M$ independent features, encoded by $N_E$ excitatory (E) and $N_I$ inhibitory (I) neurons.
We define the loss functions related to the activity of E and I neurons as follows:
\begin{subequations}
\begin{align}
&\mathrm L_E(t)=  \sum_{m=1}^{M}(x_m(t)-\hat{x}^E_{m}(t))^2 + \mu_E \sum_{i=1}^{N_E} (r^E _{i}(t))^2
\label{loss_exc}\\
&\mathrm L_I (t)=  \sum_{m=1}^{M}(\hat{x}^E_{m}(t) - \hat{x}^I_{m}(t))^2 + \mu_I \sum_{i=1}^{N_I} (r^I_{i}(t))^2,
\label{loss_inh}
\end{align}
\end{subequations}
with $\mu_E,\mu_I>0$ and where $\hat{x}^E_m(t)$ ($\hat{x}^I_m(t)$) is the estimate of the desired signal $x_m(t)$, formed by the population read-out of the spiking activity of E (I) neurons. The variable $r_i^y(t)$ is the low-pass filtered spike train of neuron $i$,
\begin{equation}
    \dot{r}^y_i(t) = -\alpha^y_i r^y_i(t) + f^y_i(t), \qquad y \in \{E,I\}
    \label{dot_rY}
\end{equation}
with $\alpha^y_i>0 \ \forall i$ the inverse time constant of the single neuron read-out, $(\alpha^y_i)^{-1}=\tau^{r,y}_i$. Note that $r^y_i(t)$ is
proportional to the instantaneous firing rate of the neuron $i$.

We write definitions for the signal $\vec{x}(t) = [x_1(1),\dots,x_M(t)]^\intercal$ and the estimates $\vec{{\hat{x}}}_{y}(t)=[x^y_1(t),\dots,x_M^y(t)]^\intercal, y\in \{E,I\}$, as follows:
\begin{subequations}
\begin{align}
    &\vec{\dot{x}}(t)=A \vec{x}(t) + \vec{s}(t) \label{signal} \\
    &\dxe=-\lambda_E \xe + W_E \vec{f}_E(t) \label{est_E}\\
    &\vec{\dot{\hat{x}}}_{I}(t)=-\lambda_I \vec{{\hat{x}}}_{I}(t) + W_I \vec{f}_I(t) \label{est_I}
\end{align}
\end{subequations}
where $A \in \mathbb{R}^{M \text{x} M}$ is the mixing matrix of the features of sensory stimuli $\vec{s}(t) = [s_1(t),\dots,s_M(t)]^ \intercal$. Scalars $\lambda_E,\lambda_I > 0$ are the inverse time constants of the population read-out of E and I neurons, respectively, with $(\lambda_y)^{-1}=\tau_y$. Vector of spike trains,  $\vec{f}_y(t)=[f^y_1(t),\dots,f^y_{N_y}(t)]^\intercal$, assembles spike trains across excitatory ($y=E$) and inhibitory ($y=I$) neurons, where the spike train of the neuron $i$ is defined as the sum of Dirac delta distributions, $f_i^y(t)=\sum_k \delta(t-t^k_{i,y})$, with $t^k_{i,y}$ the $k-$th spike time of neuron $i$.

Every neuron is assigned a decoding vector, $\vec{w}^y_i=[w^y_{1i},\dots,w^y_{Mi}]^\intercal$, as $m-$th element of the decoding vector relates the spike train of the neuron $i$ to the $m-$th dimension of the estimate $\hat{x}^y_m(t)$. The weighting matrix ${W_y \in \mathbb{R}^{M \text{x} N_y}}$ assembles decoding vectors across neurons, ${W_y=[\vec{w}^y_1,\dots,\vec{w}^y_N]}$. We also gather the low-pass filtered spike trains across neurons, $\vec{r}_y(t) = [r^y_1(t),\dots,r^y_{N_y}]^\intercal$, and express their definition in vector notation:
\begin{align}
\begin{split}
    &\vec{\dot{r}}_E(t)=- \Lambda_E^r \vec{r}_E(t) + \vec{f}_E(t) \\
    &\vec{\dot{r}}_I(t) = - \Lambda_I^r \vec{r}_I(t) + \vec{f}_I(t)
\end{split}
\label{dr_vec}
\end{align}
with $\Lambda_y^r=\text{diag}(\vec{\alpha}_y)$ a square diagonal matrix with diagonal $\vec{\alpha}_y=[\alpha^{y}_1, \dots, \alpha^{y}_{N_y}] ^\intercal$. Note that the number of features encoded by the network, $M$, determines the dimensionality of the desired signal $\vec{x}(t)$ and of the estimates $\xe$ and $\vec{{\hat{x}}}_{I}(t)$. The number of neurons $N_y$ is typically larger than the number of features $M$.

Similar to refence \cite{boerlin2013predictive}, we assume that a spike of the neuron $i$ at time $t$ will be fired only if this minimizes the loss function. Additionally, we assume that in a biological network, the condition on spiking is subjected to noise. It is unlikely that biological circuits could implement spiking as an entirely noiseless process. The condition to have a spike in the neuron $i$ of cell type $y$ is formulated as:
\begin{equation}
	\mathrm{L}_y \left(t^+ | [f_i^y(t^+)=1] + \eta^y_i(t^+) \right) < \mathrm{L}_y\left(t^- | [f_i^y(t^-)=0] \right),
    \label{condition_spiking}
\end{equation}
where $\eta^y_i(t^+)=\sigma_i^y \xi_i^y(t)$ models the noise at threshold crossing. The noise at threshold crossing has intensity $\sigma_i^y$ while $\xi_i^y(t)$ is a Gaussian random process with zero mean and unit standard deviation, $\xi_i^y(t) \sim \mathcal{N}(0,1)$, with $\xi_i^y(t)$ independent and identically distributed over time, across neurons and across the two cell types.

Taking into account the effect of a spike on the estimates (eq.~\ref{est_E}-\ref{est_I}) and on the low-pass filtered spike trains (eq.~\ref{dot_rY}) and applying those in the condition on spiking (eq.~\ref{condition_spiking}), we arrive to the following condition for the spiking neuron $i$:
\begin{align}
\begin{split}
 & \vec{w}_E^\intercal \left(\vec{x}(t)-\xe\right) - \mu_E r^E_i(t) > \frac{1}{2} \left(\|\vec{w}_i^E\|^2_2  + \mu_E \right) + \sigma_i^E\xi^E_i(t)
 \\
 & \vec{w}_I^\intercal \left(\xe - \vec{{\hat{x}}}_{I}(t)\right) - \mu_I r^I_i(t)  > \frac{1}{2} \left(\|\vec{w}_i^I\|^2_2  + \mu_I \right) + \sigma_i^I\xi^I_i(t)
 \end{split}
 \label{firing_rule}
\end{align}
with $\|\vec{w}_i^y\|^2_2=\sum_{m=1}^M(w^y_{mi})^{2}$ the squared length of decoding vector of the neuron $i$.
As in \cite{bourdoukan2012learning,boerlin2013predictive}, we interpret the left-hand side of eq.~\ref{firing_rule} as the membrane potential of neuron $i$ and the right-hand side as the firing threshold,
\begin{align}
\label{defs}
\begin{split}
 u_i^E(t) & \equiv \vec{w}_E^\intercal \left(\vec{x}(t)-\xe\right) - \mu_E r_i^E(t)
 \\
 u_i^I(t) & \equiv \vec{w}_I^\intercal \left(\xe - \vec{{\hat{x}}}_{I}(t)\right) - \mu_I r_i^I(t)
 \\
 \vartheta_i^y  & \equiv \frac{1}{2} \left( \|\vec{w}_i^I\|^2_2+ \mu_y \right) + \sigma_1^y \xi_1^y(t), \qquad y \in \{E,I\}.
 \end{split}
\end{align}
Note that the firing threshold of the neuron $i$ is proportional to the squared length of the decoding vector, ${||\vec{w}^y_i||^2_2=\sum_m (w_{mi}^y)^2}$, and the constant of the regularizer $\mu_y$ (eq.~\ref{defs}).
The vector of the membrane potentials for $N_E$ excitatory and $N_I$ inhibitory neurons, can now be written in vector notation as follows:
\begin{align}
    \begin{split}
    \vec{u}_E(t) & = W_E^\intercal \left(\vec{x}(t)-\xe\right) - \mu_E \vec{r}_E(t) \\
    \vec{u}_I(t) & = W_I^\intercal \left(\xe - \vec{{\hat{x}}}_{I}(t)\right) - \mu_I \vec{r}_I(t).\\
    \end{split}
    \label{u_vec}
\end{align}
The membrane potentials $\vec{u}_y(t)$ are thus given by the projection of the coding error on the matrix of decoding weights, and in addition depend on the spiking frequency of the local neuron (eq.~\ref{u_vec}).

\section{Dynamics of the membrane potential}
 \label{supp_2}

The second part consists in calculating the difference equation for membrane potentials. To obtain a difference equation, we take derivatives with respect to time of $\vec{u}_E(t)$ and $\vec{u}_I(t)$,
\begin{align}
\begin{split}
    \vec{\dot{u}}_E(t)= W_E^\intercal \left(\vec{\dot{x}}(t)-\dxe\right) - \mu_E \vec{\dot{r}}_E(t)
     \\
    \vec{\dot{u}}_I(t) = W_I^\intercal \left(\dxe - \vec{\dot{\hat{x}}}_{I}(t) \right) - \mu_I \vec{\dot{r}}_I(t).
    \end{split}
    \label{du_0}
\end{align}
In eq.~\eqref{du_0} we use definitions of the temporal derivatives of the signal (eq.~\ref{signal}), the estimate by E neurons (eq.~\ref{est_E}), the estimate by I neurons (eq.~\ref{est_I}), and the definition of low-pass filtered spike trains (eq.~\ref{dot_rY}).
Without loss of generality, we also use the following substitutions:
\begin{subequations}
\begin{align}
& A = B - \lambda_E \mathbf{I}^{M \text{x} M} \label{substitute_A}\\
& \Delta_E^r = \lambda_E \mathbf{I}^{[N_E \text{x} N_E]} - \Lambda^r_E \label{substitute_deltare}\\
& \Delta_I^r = \lambda_I \mathbf{I}^{[N_I \text{x} N_I]} - \Lambda^r_I \label{substitute_deltari}
\end{align}
\end{subequations}
where $\mathbf{I}$ is an identity matrix, and $\Delta_y^r \in \mathbb{R}^{N_y \text{x} N_y}$ are square diagonal matrices with diagonal elements $\delta^{r,y}_i=\lambda_y-\alpha^y_i, \text{ for } i=1,\dots,N_y$. Diagonal elements of $\Delta_E^r$ and $\Delta_I^r$ therefore evaluate the difference of the time constants of the population read-out (eq.~\ref{est_E}-\ref{est_I}) and the single neuron read-out (eq.~\ref{dr_vec}) in E and I neurons, respectively. Using substitutions in eq.~\eqref{substitute_A}-\ref{substitute_deltari}, the exact solutions for the time-derivative of the membrane potentials are:
\begin{subequations}
\begin{multline}
    \vec{\dot{u}}_E(t) = -\lambda_E \vec{u}_E(t) + W_E^\intercal \vec{s}(t) - W_E ^\intercal W_E \vec{f}_E(t) +  W_E^\intercal B \vec{x}(t)
     - \mu_E \Delta_E^r \vec{r}_E(t)  - \mu_E \vec{f}_E(t)
	\label{due}
\end{multline}
\begin{multline}
    \vec{\dot{u}}_I(t) = -\lambda_I \vec{u}_I(t) + W_I ^\intercal W_E \vec{f}_E(t) - W_I ^\intercal W_I \vec{f}_I(t) + \delta_I W_I^\intercal B \xe
      - \mu_I \Delta_I^r \vec{r}_I(t)   - \mu_I \vec{f}_I(t)
    \label{dui}
\end{multline}
\end{subequations}
with $\delta_I=\lambda_I - \lambda_E$. Right-hand side of eq.~\ref{due}-\ref{dui} comprise a term proportional to the leak current, terms with synaptic interactions $W_y^\intercal W_z \vec{f}_z(t)$ for $y,z \in \{E,I\}$, terms involving the signal $\vec{x}(t)$ and the estimate $\xe$, and terms with local feedback with slower and faster dynamics, $\mu_y \Delta_y^r \vec{r}_y(t)$, and $\mu_y \vec{f}_y(t)$, respectively. Excitatory neurons in addition have a term proportional to the feedforward current, $W_E^\intercal \vec{s}(t)$. However, eqs.~\ref{due}-\ref{dui} do not yet express a biologically plausible membrane equation, and several of the terms have to be constrained in order to obtain a framework that is consistent with know properties of biological networks.

\section{Efficient spiking network as a generalized leaky integrate-and-fire neuron model.}
\label{text:describe_lif}
The following section considers biologically plausible and computationally efficient solutions derived from eqs.~\ref{due}-\ref{dui}. We examine the terms one by one, and express a biologically plausible solution in the form of an E-I network of generalized leaky integrate-and-fire neurons.

\paragraph{Leak current}

The terms $-\lambda_y \vec{u}_y(t)$ for $y \in \{E,I\}$ define the leak current in E and I cell type. In neuron $i$ of cell type $y$, the leak current is:
\begin{equation}
     I^{\text{leak }y}_i \propto -\lambda_y u^y_i(t) = -\frac{1}{\tau_y}u^y_i(t), \qquad y\in \{E,I\}
    \label{leak}
\end{equation}
with $\tau_y = (\lambda_y)^{-1}$ the membrane time constant of E ($y=E$) and I ($y=I$) neurons.
Leak currents in eq.~\eqref{leak} result from absorbing terms that define the membrane potential, as in eq.~\eqref{u_vec}, and are, contrary to \cite{boerlin2013predictive}, calculated without approximations. In the E cell type, in particular, inserting the eq.~\eqref{substitute_A} in eq.~\eqref{signal}, we get
\begin{equation}
    A\vec{x}(t) = B\vec{x}(t) - \lambda_E \mathbf{I}^{M \text{x} M} \vec{x}(t).
\end{equation}
The leak current in the E cell type (eq.~\ref{leak} with $y=E$) absorbs, among others, the term $- \lambda_E \mathbf{I}^{M \text{x} M} \vec{x}(t)$, while the remaining term, $B\vec{x}(t)$, is part of a synaptic current that is discussed further on.
Similar leak term has been obtained in a previous work on efficient spiking networks \cite{koren2017computational}, where the leak also emerged from analytical treatment of the loss function. However, in \cite{koren2017computational}, a simplified network with diagonal matrix $A=-\lambda_E \mathbf{I}^{M \text{x} M}$ has been considered, together with the assumption that the time constant of the signal $\vec{x}(t)$ is equivalent to the time constant of the neural membrane $\tau$. Here, writing the matrix $A$ as in eq.~\ref{substitute_A} allows us to drop this assumption and propose a more general solution for the membrane equation with the leak current.

\paragraph{Feedforward current}

The term $W_E^\intercal \vec{s}(t)$ in the E cell type (eq.~\ref{due}) defines a feedforward current and has been proposed before \cite{boerlin2013predictive,bourdoukan2012learning}. The feedforward current to the neuron $i$ is proportional to the sum of feedforward inputs $s_m(t)$, weighted by decoding vector of the neuron,
\begin{equation}
    I_i^{\text{ff}}(t) \propto (\vec{w}_i^E)^\intercal \vec{s}(t)=\sum_{m=1}^{M}w_{mi}^E s_m(t).
    \label{ff}
\end{equation}
In case we assume the variables $s_m(t) \text{, for } m=1\dots,M$, to correspond to $M$ features of an external stimulus that the network is receptive to (i.e., sensory features of an image such as the orientation, the spatial frequency, the color, etc.), the eq.~\eqref{ff} is a plausible expression of the feedforward current. This is also the interpretation that we follow in the main paper.

\paragraph{Fast synaptic currents}

In eq.~\eqref{due}-\eqref{dui}, the terms of the form $W_y^\intercal W_z \vec{f}_z(t)$ with $\{yz\}\in \{EE,IE,II\}$ define fast synaptic interactions between E-to-E, E-to-I and I-to-I neurons. We write the absolute value of fast synaptic currents at a postsynaptic neuron $i$ as the following sum of presynaptic inputs:
\begin{equation}
     |\tilde{I}^{yz}_i(t)| \propto \sum_{j=1}^{N_z} (\vec{w}^y_i)^\intercal \vec{w}^z_j f^z_j(t), \qquad \{yz\}\in \{IE,II,EE\},
    \label{Fast_syn_unconstrained}
\end{equation}
with $\vec{w}^y_i$ the decoding vector of the postsynaptic neuron, and $\vec{w}^z_j$, $f^z_j(t)$ the decoding vector and  the spike train of the presynaptic neuron, respectively. These currents are in general not biologically plausible and have to be constrained. The sign of the synaptic interaction from the presynaptic neuron $j$ of cell type $z$ to the postsynaptic neuron $i$ of cell type $y$ depends on the similarity of decoding vectors between the presynaptic and the postsynaptic neuron:
\begin{align}
\label{sign_fast_syn}
\begin{split}
    &(\vec{w}^y_i)^\intercal \vec{w}^z_j f^z_j(t) > 0 \text{ if } (\vec{w}^y_i)^\intercal \vec{w}^z_j >0   \\
    &(\vec{w}^y_i)^\intercal \vec{w}^z_j f^z_j(t) < 0 \text{ if } (\vec{w}^y_i)^\intercal \vec{w}^z_j <0.
    \end{split}
\end{align}
If the two neurons have similar decoding vectors, dot product of their decoding vectors is positive, while neuronal pairs with dissimilar decoding vectors have a negative dot product of their decoding vectors. Irrespectively of the sign in front of the synaptic current (see eq.~\ref{due}-\ref{dui}), therefore, the same presynaptic neuron $j$ sends positive (excitatory) and negative (inhibitory) synaptic currents to other neurons, depending on the similarity of decoding vectors of the presynaptic and the postsynaptic neuron. This is inconsistent with Dale's law that constrains a particular neuron to only send either excitatory or inhibitory currents to the postsynaptic neuron, but not both. A simple solution that enforces Dale's law consists in removing connections between neurons with dissimilar selectivity (second line in eq.~\ref{sign_fast_syn}). We get:
\begin{align}
\begin{split}
    I^{\text{fast }IE}_i(t) \propto \sum_{j=1}^{N_E}
        C^{IE}_{ij} f^E_j(t), \qquad
    & I^{\text{fast }II}_i(t) \propto - \sum_{\substack{j=1 \\ j\neq i}}^{N_I}
        C^{II}_{ij} f^I_j(t), \qquad
    I^{\text{fast }EE}_i(t) \propto - \sum_{j=1}^{N_E}
        C^{EE}_{ij} f^E_j(t),
   \end{split}
   \label{syn_fast_bad}
\end{align}
with $C^{yz}$ the connectivity matrix between the postsynaptic population $y\in \{E,I\}$ and the presynaptic population $z\in \{E,I\}$,
\begin{equation}
    C^{yz}_{ij} = \begin{cases}
     (\vec{w}^y_i)^\intercal \vec{w}_j^z, & \text{ if $(\vec{w}^y_i)^\intercal \vec{w}_j^z > 0$} \\
     0 & \text{ otherwise}.
     \end{cases}
     \label{C}
\end{equation}
In I neurons, fast synaptic currents as prescribed by eq.~\ref{syn_fast_bad} obey Dale's law, while in E neurons, they do not.
Elements of the matrix $C^{yz}$ in eq.~\ref{C} are always positive and the sign of the synaptic interaction is given by the sign in front of the synaptic term in eq.~\ref{syn_fast_bad}. In I neurons, we get a positive (excitatory) current originating from E neurons ($I^{IE}_i(t)$), and a negative (inhibitory) current originating from I neurons ($I^{II}_i(t)$), which is consistent with Dale's law. In E neurons, on the contrary, synaptic current originate from E neurons ($I^{EE}_i(t)$), but has a negative sign. Since in biological networks, a negative current cannot originate from excitatory neurons, we make the following replacement: $-W_E ^\intercal W_E \vec{f}_E(t) \approx -W_E ^\intercal W_I \vec{f}_I(t)$, and get the following fast synaptic currents:
\begin{equation}
\begin{gathered}
    I^{\text{fast }IE}_i(t) \propto \sum_{j=1}^{N_E}
     C^{IE}_{ij} f^E_j(t), \qquad
     I^{\text{fast }II}_i(t) \propto - \sum_{\substack{j=1 \\ j\neq i}}^{N_I}
     C^{II}_{ij} f^I_j(t), \qquad
     I^{\text{fast }EI}_i(t) \propto - \sum_{j=1}^{N_I}
     C^{EI}_{ij} f^I_j(t),\label{syn_fast_good}
\end{gathered}
\end{equation}
with the matrix of fast synaptic connections as in eq.~\ref{C}.
In E neurons, an inhibitory synaptic current now originates from I neurons (right-most term in eq.~\ref{syn_fast_good}), thus contributing fast inhibition to E neurons that is consistent with Dale's law.

\paragraph{Synaptic currents with kinetics of low-pass filtered spikes}

Next, we address synaptic terms $W_E^\intercal B \vec{x}(t)$ in the E cell type and $\delta_I W_I^\intercal B \xe$ in the I cell type (see eq.~\ref{due}-\ref{dui}). These terms will give synaptic currents with kinetics of low-pass filtered spikes (see eqs. \ref{signal} and \ref{est_E}), thus contributing slower channels to synaptic transmission.

In the E cell type (eq.~\ref{due}), the term $W_E^\intercal B \vec{x}(t)$ contains the signal $\vec{x}(t)$, a variable that does not by itself define a biologically plausible current to single neurons. By construction of the loss function of E neurons (eq. \ref{loss_exc}), the signal $\vec{x}(t)$ is approximated by the E estimate $\xe$, allowing us to make the substitution $\vec{x}(t) \approx \xe$. Using the definition of the E estimate (eq.~\ref{est_E}), we get the following E-to-E synaptic current to the postsynaptic neuron $i$:
\begin{equation}
\begin{split}
    & \dot{I}^{EE}_i(t) \propto -\frac{1}{\tau^{\text{syn}}_E} I^{EE}_i(t) + \sum_{\substack{j=1 \\ j\neq i}}^{N_E}
     D^{EE}_{ij} f^E_j(t), \qquad \tau^{\text{syn}}_E = \tau_E \\
    & D^{EE}_{ij} = \begin{cases}
     (\vec{w}^E_i)^\intercal B \vec{w}^E_j, & \text{ if $(\vec{w}^E_i)^\intercal \vec{w}^E_j > 0$,\qquad B positive semi-def.}\\
     0 & \text{ otherwise}.
   \end{cases}
\end{split}
\label{slow_EE}
\end{equation}
To ensure that synaptic interactions are consistently excitatory, we only allowed connections between neurons with similar selectivity and constrained the matrix ${B=(b_{mn});\  m,n=1,\dots,M}$, to be positive semi-definite.

In the I cell type in eq.~\eqref{dui}, we have the term $\delta_I W_I^\intercal B \xe$ with $\delta_I=\lambda_I - \lambda_E$ and $\lambda_y = (\tau_y)^{-1}$ for $y\in\{E,I\}$. We again use the definition of the E estimate (eq.~\ref{est_E}), and get the following E-to-I synaptic current in I neurons:
\begin{equation}
\begin{split}
    & \dot{I}^{\text{slow} IE}_i(t) \propto -\frac{1}{\tau^{\text{syn}}_E} I^{\text{slow }IE}_i(t) + (\frac{1}{\tau_I} - \frac{1}{\tau_E}) \sum_{j=1}^{N_E}
     D^{IE}_{ij} f^E_j(t),
     \qquad \tau_I < \tau_E,\\
    &  D^{IE}_{ij} = \begin{cases}
     (\vec{w}^I_i)^\intercal B \vec{w}^E_j, & \text{ if $(\vec{w}^I_i)^\intercal \vec{w}^E_j > 0$,\quad B positive semi-def.}\\
     0 & \text{ otherwise}
   \end{cases}
\end{split}
\label{slow_IE}
\end{equation}
Since E-to-I currents originate from E neurons, they have to be excitatory. To ensure that E-to-I synapses are consistently excitatory (and taking into account that the matrix $B$ has been constrained to be positive semi-definite in eq.~\ref{slow_EE}), we get the following constraint on time constants: $\tau_I < \tau_E$, constraining the membrane time constant in I neurons to be faster than in E neurons. In summary, the strength of slower synaptic currents is proportional to the similarity of decoding vectors of the presynaptic and the postsynaptic neuron, similarly as with fast synaptic currents (eq.~\ref{syn_fast_good}). Moreover, slower synaptic currents in addition depend on the matrix $B$ (eqs.~\ref{slow_EE}-\ref{slow_IE}).

We note that E-to-E synaptic currents in eq.~\ref{slow_EE} as well as E-to-I synaptic currents in eq.~\ref{slow_IE} have kinetics of low-pass filtered spike trains of excitatory presynaptic neurons. Defining a low-pass filtered spike train with synaptic time constant $\tau^{\text{syn}}_E$, we can simplify the notation and write these synptic currents as follows:
\begin{align}
\begin{split}
    & I^{EE}_i(t) = \sum_{\substack{j=1 \\ j\neq i}}^{N_E}
     D^{EE}_{ij} z^E_j(t), \qquad \tau^{\text{syn}}_E = \tau_E \\
    & I^{\text{slow }IE}_i(t) = (\frac{1}{\tau_I} - \frac{1}{\tau_E}) \sum_{j=1}^{N_E}
     D^{IE}_{ij} z^E_j(t),
     \qquad \tau_I < \tau_E,\\
    & \dot{z}^E_i(t)=-\frac{1}{\tau^{\text{syn}}_E} z^E_i(t) + f^E_i(t),
    \end{split}
    \label{slow_synaptic}
\end{align}
with the matrix $D^{EE}$ and $D^{IE}$ as in eqs.~\ref{slow_EE}-\ref{slow_IE}.

\paragraph{Local currents}
The terms $- \mu_y \Delta_y^r \vec{r}_y(t)$ in eq.~\ref{due}-\ref{dui} define local, spike-triggered currents with dynamics of the low-pass filtered spike train $r^y_i(t)$. We defined $\Delta_y^r$ as a diagonal matrix with $i-$th diagonal element $(\Delta_y^r)_{ii} = \lambda_y - \alpha^y_i$ (eq.~\ref{dr_vec}). Using that $\lambda_y=(\tau_y)^{-1}$ and $\alpha_i^y=(\tau^{r,y}_i)^{-1}$ are inverse time constants, we can write the local current in neuron $i$ as:
\begin{equation}
    I^{\text{local }y}_i(t) \propto - \mu_y \left( \frac{1}{\tau_y} -\frac{1}{\tau_i^{r,y}} \right) r_i^y(t), \qquad y \in \{E,I\}.
    \label{local_current}
\end{equation}
Using the definition of low-pass filtered spike train $r_i^y(t)$ in eq.~\ref{dot_rY}, we can rewrite eq.~\ref{local_current} with the spike train of the local neuron $f^y_i(t)$:
\begin{equation}
     \dot{I}^{\text{local }y}_i(t) \propto -\frac{1}{\tau_i^{r,y}} I^{\text{local }y}_i(t) - \mu_y \left( \frac{1}{\tau_y} -\frac{1}{\tau_i^{r,y}} \right) f_i^y(t), \qquad y \in \{E,I\}.
     \label{Local}
 \end{equation}
Local current as in eq.~\eqref{Local} is consistent with biological constraints, however, different solutions are obtained depending on the relation of time constants between the population read-out $\tau_y$ and the single neuron read-out $\tau^{r,y}_i$.
The regularizer $\mu_y$ is non-negative by definition (see eqs.\ref{loss_exc}-\ref{loss_inh}) and does not influence the sign of the local current in eq.~\eqref{Local}.
If we constrains the time constant of the single neuron read-out to be longer than the time constant of the population read-out: $\tau_i^{r,y} > \tau_y$, current in eq.~\ref{Local} is negative (hyperpolarizing), and we interpret is as spike-triggered adaptation.  If, on the contrary, we have the following relation of inverse time constants: $\tau_i^{r,y} < \tau_y$, the local current in eq.~\ref{Local} is positive (depolarizing), and we interpret it as spike-triggered facilitation. In the special case when the two time constants are equal, $\tau^{r,y}_i=\tau_y$, the local current vanishes. Note that this special case has been assumed in previous works \cite{bourdoukan2012learning,boerlin2013predictive,koren2017computational,chalk2016neural,brendel2020learning}, while we here recovered a more general solution. Note that the kinetics as well as the strength of local currents is heterogeneous across neurons due to the heterogeneity of the time constant $\tau_i^{r,y}$ across neurons (see eq.~\ref{Local}).

\paragraph{Reset current}
The last terms on the right-hand side of eq.~\ref{due}-\ref{dui} is of the following form: $-\mu_y \vec{f}_y(t)$, and defines resetting of the local neuron after a spike. In a single E and I neuron, respectively, the reset current is the following:

\begin{align}
\label{reset}
\begin{split}
    & I^{\text{reset }E}_i(t) = -\mu_E f_i^E(t)\\
    & I^{\text{reset }I}_i(t) = -(\mu_I + C^{II}_{ii}) f_i^E(t)
    \end{split}
\end{align}
where in I neurons, we also have the contribution of the negative self-connection $C^{II}_{ii}$ with $C^{II}$ the matrix of recurrent inhibitory connections as in eq.~\ref{C}. Since the regularizer $\mu_y$ is by definition positive, the reset current in eq.~\ref{reset} is always a negative (hyperpolarizing) current, which ensures its biological plausibility as a current that resets the membrane potential after the neuron has reached the firing threshold.

\paragraph{Spike-triggered rebound current}
In the definition of the recurrent E-to-E synaptic current, we omitted the positive self-connection in eq.~\eqref{slow_EE}, since a self-connection is not a synaptic current. This self connection is activated by the spike of the local neuron and has the dynamics of the low-pass filtered spike train:
\begin{align}
\begin{split}
    \dot{I}^{\text{rebound}}_i(t) & = -\frac{1}{\tau_h} I^{\text{rebound}}_i(t) + D^{EE}_{ii} f_i^E(t),\qquad \tau_h=\tau_E\\
    D^{EE}_{ii} &=(\vec{w}^E_i)^\intercal B \vec{w}^E_i, \qquad B \text{ positive semi-def.}
    \label{rebound}
    \end{split}
\end{align}
Rebound current is always a positive (depolarizing) current, since the coefficient $D^{EE}_{ii}$ is given by the product of the decoding vector of the spiking neuron, $\vec{w}^E_i$, with the positive semi-definite matrix $B$. Immediately after a spike of the neuron $i$, the neuron is strongly hyperpolarized by the reset current (eq.~\ref{reset}). Hence, the rebound current in eq.~\ref{rebound} activates when the neuron is strongly hyperpolarized and counteracts the strong hyperpolarization with a depolarizing rebound current.

\paragraph{Integrate-and-fire formulation}

Finally, we gather results and express the efficient spiking network as an E-I network of generalized LIF neurons. We use the fact that the activation of the reset current is instantaneous (eq.~\ref{reset}) and creates a jump in the membrane potential as the neuron reaches the threshold. The jump in the membrane potential corresponds to the amplitude of the reset current, and the dynamics of E and I neurons can be expressed as a generalized LIF neuron model:
\begin{subequations}

\begin{align}
\begin{split}
    \tau_E \dot{V}_i^E(t) & =-V^E_i(t) + I_i^{\text{ff}}(t) + I_i^{EI}(t) + I^{EE}_i(t) + I^{\text{local }E}_i(t) + I^{\text{rebound}}_i(t)
    \\
    \tau_I \dot{V}_i^I(t) & =-V^I_i(t) + I_i^{IE}(t) + I_i^{II}(t) + I^{\text{local }I}_i(t)
    \\
    \text{if } V^y_i(t^-) & \geq \vartheta^y_i(t^-) \rightarrow V^y_i(t^+)=V^{\text{reset }y}_i, \qquad y \in \{E,I\},
\end{split}
\label{lif}
\end{align}
The firing thresholds and resets are proportional to the regularizer $\mu_y$ and the squared length of the decoding vector $||\vec{w}^y_i||^2_2$:
\begin{align}
\begin{split}
    \vartheta_I^y(t)& =\frac{1}{2} (\mu_y + ||\vec{w}^y_i||^2_2 + \sigma_i^y \xi_i^y(t), \qquad y \in \{E,I\}
    \\
    V_i^{\text{reset }E} & = -\frac{1}{2} ( \mu_E - ||\vec{w}^E_i||^2_2)
    \\
    V_i^{\text{reset }I} & = -\frac{1}{2} ( \mu_I + ||\vec{w}^I_i||^2_2),
\end{split}
\end{align}
and the currents are:
\begin{align}
\label{currents}
\begin{split}
    & I_i^{\text{ff}}(t)= \tau_E\sum_{m=1}^{M} w^E_{mi}s_m(t)
    \\
    & I^{EE}_i(t) = \tau_E \sum_{\substack{j=1 \\ j\neq i}}^{N_E}D^{EE}_{ij} z_j^E(t)
    \\
    & I^{IE}_i(t)=\tau_E\sum_{j=1}^{N_E}C_{ij}^{IE} f_j^E(t) + \left(\frac{\tau_E}{\tau_I}-1 \right)  \sum_{j=1}^{N_E}D^{IE}_{ij} z_j^E(t), \qquad  \tau_E>\tau_I
    \\
    & I^{II}_i(t)= - \tau_I \sum_{\substack{j=1\\j\neq i }}^{N_I}C_{ij}^{II} f_j^I(t) \\
    & I^{EI}_i(t)= - \tau_I \sum_{j=1}^{N_I}C_{ij}^{EI} f_j^I(t)
    \\
    & I^{\text{local }y}_i(t) = -\mu_y \left(1 -\frac{\tau_y}{\tau_i^{r,y}} \right) r_i^y(t), \qquad y \in \{E,I\}
    \\
    & I^{\text{rebound}}_i(t) = \tau_E D^{EE}_{ii} z^E_i(t),
\end{split}
\end{align}
Matrices $C^{yz}_{ij}$ and $D^{yE}_{ij}$ give the strength of the fast and slow component in synaptic interactions, respectively:
\begin{align}
\label{connectivity}
\begin{split}
     & C^{yz}_{ij} = \begin{cases}
     (\vec{w}^y_i)^\intercal \vec{w}_j^z, & \text{ if $(\vec{w}^y_i)^\intercal \vec{w}_j^z > 0$} \qquad \{yz\}\in \{IE,II,EI\}\\
     0 & \text{ otherwise}
   \end{cases} \\
   & D^{yE}_{ij} = \begin{cases}
     (\vec{w}^y_i)^\intercal B \vec{w}^E_j, & \text{ if $(\vec{w}^y_i)^\intercal \vec{w}^E_j > 0$,\quad B positive semi-def.}, \qquad y \in \{E,I\} \\
     0 & \text{ otherwise}, \end{cases}
\end{split}
\end{align}
while $z^E_i(t)$ and $r_i^y(t)$ are low-pass filtered spike trains,
\begin{align}
    \begin{split}
        &\dot{z}^E_i(t)=-\frac{1}{\tau^{\text{syn}}_E} z^E_i(t) + f^E_i(t)\\
        &\dot{r}^y_i(t)=-\frac{1}{\tau^{r,y}_i} r^y_i(t) + f^E_i(t), \qquad y \in \{E,I\}.
    \end{split}
    \label{low-pass}
\end{align}
\end{subequations}
With eqs.\ref{lif}-\ref{low-pass}, we obtained a complete description of a biologically plausible spiking network model, where all elements obey constraints of biological neurons and networks and describe the set of  membrane currents that are highly relevant for the function and dynamics of networks in the cortex.

\iffalse
\section{Alternative solutions for feedforward currents}

\colorbox{yellow}{this discussion is not used in the main paper and could be cut out}\\
We so far assumed that the feedforward current is a linear sum of features of external stimuli. However, only sensory receptors receive direct inputs from features of sensory stimuli. The activity of sensory receptor neurons activates the feedforward sensory pathway in subcortical and cortical brain areas and the information about the stimulus reach the cortex only after several processing stages. Neurons in sensory cortical areas cannot directly "observe" the external world, but instead receive as the feedforward input the spiking activity from neurons in subcortical or downstream cortical areas.

\subsection{Biological plausibility of a particular feedforward synapse}

Let us therefore assume that $s_m(t)$ are presynaptic inputs from an upstream brain area (e.g., the thalamus), and $w^E_{mi}$ are postsynaptic weights. We have $m=1,\dots,M$ feedforward synaptic contacts to the postsynaptic neuron $i$ \colorbox{green}{schema}. While the feedforward synaptic current at the postsynaptic neuron $i$ can in principle be either excitatory or inhibitory, following Dale's law, a \textit{particular} synapse at the neuron $i$ has to be consistently either excitatory or inhibitory. With this, more biologically grounded interpretation, the general expression in
eq.~\ref{ff} is not anymore biologically plausible and has to be constrained. For the sake of biological plausibility, we have to ensure that the sign of the current at a \textit{particular} feedforward synapse is consistent over time, ${\text{sgn}(w^E_{mi}s_m(t)) = \text{sgn}(w^E_{mi}s_m(t+\Delta t))\  \forall\  \Delta t}$. There are several ways to achieve that discussed below.

\paragraph{Consistency of the presynaptic input over time}
A possible solution is to constrain the presynaptic input at the $m-$th synapse to keep the same sign over time,
\begin{subequations}
\begin{equation}
    \text{sgn}(s_m(t)) = \text{sgn}(s_m(t+\Delta t))\  \forall\  \Delta t.
    \label{ff1}
\end{equation}
The sign of the feedforward synaptic current at the $m-$th synapse, ${\text{sgn}(w^E_{mi}s_m(t))}$, now depends on the sign of the $m-$th presynaptic input $\text{sgn}(s_m(t))$ $m-$th and the $m-$th postsynaptic weight ${\text{sgn}(w^E_{mi}}$. In particular, if the sign of the postsynaptic weight and the presynaptic input is the same, the synaptic current at the $m-$th synapse is excitatory, while different signs result in an inhibitory synaptic current.

\paragraph{Positive postsynaptic weight}

An alternative solution is to constrain the postsynaptic weight $w^E_{mi}$ to be always positive,
\begin{equation}
    w^E_{mi}>0 \ \forall m=1,\dots,M; \forall i=1\dots,N_y.
    \label{ff2}
\end{equation}
The sign of the feedforward current at the $m-$th synapse is now determined by the sign of the presynaptic input $s_m(t)$.
Since the postsynaptic weight is also a decoding weight that determines the effect of neuron's spike on the population read-out in $m-th$ dimension $\hat{x}^E_m(t)$, constraining all decoding weights to be positive implies that spikes can pull the estimate in the positive, but not in the negative direction (see eq.~\eqref{est_E}). Potentially, this could lead to a less precise estimation of the signal.

\paragraph{Positive presynaptic input}

Another alternative solution to make the feedforward synapse biologically plausible is to impose that all $M$ presynaptic inputs are positive at all times, while the postsynaptic weight is allowed to be positive or negative,
\begin{equation}
    s_m(t) > 0 \ \forall m=1\dots,M;\ \forall t.
    \label{ff3}
\end{equation}
In this case, the sign of the feedforward current at the $m-$th synapse is determined by the sign of the postsynaptic weight.

\paragraph{Positive presynaptic input and positive postsynaptic weight}
As a last alternative, we consider the solution where both the postsynaptic weight and the presynaptic input are constrained to be positive at all times,
\begin{equation}
     w^E_{mi}>0 \ \forall m=1,\dots,M; \forall i=1\dots,N_y, \qquad s_m(t) > 0 \ \forall m=1\dots,M;\ \forall t.
     \label{ff4}
\end{equation}
\end{subequations}
While the constraints in eqs.~\eqref{ff1}-\eqref{ff3} allow for excitatory and inhibitory synaptic currents across $M$ synapses, the latter solution constrains the feedforward current to be always excitatory.

The total feedforward current is the sum of synaptic inputs across the feedforward synapses,
\begin{equation}
    I_i^{\text{ff}}(t) =\sum_{m=1}^{M}w_{mi}^E s_m(t).
    \label{ff_current}
\end{equation}
With constraints as in eqs. \eqref{ff1}-\eqref{ff3}, the total feedforward current is a sum of excitatory and inhibitory synaptic inputs. Drawing the variable that is allowed to be positive or negative from a symmetric distribution centered at 0, a small number of feedforward synapses $M$ can lead to an imbalance
of E and I synaptic currents in favor of excitation or inhibition. Increasing the number of feedforward synapses $M$, under the same conditions, in general increases the E-I balance in the feedforward current of a single neuron.

\subsection{Replacing independent feedforward inputs with a larger but low-rank spiking network}

In general, by having the feedforward current driven by $M$ independent presynaptic inputs as in eq.\ref{ff_current} we have to assume $M$ independent neurons at the presynaptic level. This assumption is not satisfying, since we would rather expect that the presynaptic current is send by a recurrently connected network with a large number of recurrently connected neurons. For example, we might expect that the feedforward current is generated by a network of the similar size as the network receiving the feedforward current. We therefore ask, how can the framework of efficient coding with spikes be extended to such cases that are closer to the way the feedforward signal flows between cortical areas in biological networks.

\fi

\section{Alternative solutions for slow synaptic currents}
\label{text:alternative}
While the recurrent excitatory synaptic currents suggested in eq.~\ref{slow_synaptic} seem the most biologically plausible solutions, we here, for completeness, present alternative solutions for excitatory synaptic currents with slower kinetics. These alternative solutions are mathematically well defined and obey Dale's law, but are less likely to describe biological neural networks because they lack global balance of excitation and inhibition, and/or because they do not provide a description of E-to-E connections. While the function of recurrent E-E connections in biological networks is still unclear, and in some instances, the probability of E-to-E connections in the local network can be very low \cite{seeman2018sparse}, recurrent E-to-E connections are presumably still relevant for the dynamics of the cortical circuitry, and the lack thereof only gives an incomplete description of cortical networks.

We so far defined recurrent excitatory synapses (eq.~\ref{slow_EE}) by substituting the signal $\vec{x}(t)$ with the excitatory estimate $\xe$ (eq.~\ref{slow_EE}), and justified the substitution by the fact that the loss function of E neurons minimizes the distance between these two variables (eq.~\ref{loss_exc}). Seen that the loss function of I neurons minimizes the distance between the E and the I estimates (eq.~\ref{loss_inh}), we can further assume the following: $\vec{x}(t) \approx \xe \approx \vec{{\hat{x}}}_{I}(t)$. With this assumption, several
alternative solutions emerge.

Let us first consider the solution that maintains slow recurrent excitation in E neurons and with that imposes positive semi-definiteness of the matrix $B$ (as in eq.~\ref{slow_EE}). In the membrane equation for the I cell type (eq.~\ref{dui}), where we have the term $(\lambda_I-\lambda_E) W_I^\intercal B \xe$, we now replace the E estimate with the I estimate,  $\xe \approx \vec{{\hat{x}}}_{I}(t)$. Using the definition of the I estimate (eq.~\ref{est_I}), we get the following solution for the slower component of synaptic currents:
\begin{equation}
\label{alt1}
\begin{split}
    & I^{EE}_i(t) = \sum_{\substack{j=1 \\ j\neq i}}^{N_E}
     D^{EE}_{ij} z^E_j(t) \\
    & I^{\text{slow} II}_i(t) = (\frac{1}{\tau_I} - \frac{1}{\tau_E}) \sum_{j=1}^{N_I}
     D^{II}_{ij} z^I_j(t),
     \quad \tau_I > \tau_E\\
    &  D^{yy}_{ij} = \begin{cases}
     (\vec{w}^y_i)^\intercal B \vec{w}^y_j, & \text{ if $(\vec{w}^y_i)^\intercal \vec{w}^y_j > 0$,\quad B positive semi-def.}, \qquad \{yy\} \in \{EE,II\}.\\
     0 & \text{ otherwise}
   \end{cases}
\end{split}
\end{equation}
The current $I^{\text{slow }II}_I(t)$ originates from I neurons and must therefore be inhibitory. To ensure the consistency of inhibitory connections, the membrane time constant of I neurons is slower than the membrane time constant of E neurons.

Moreover, slower E-to-E synaptic connections together with slower I-to-I synapses lead to a global imbalance of E-I currents. The network without slower synapses balances excitatory and inhibitory currents on its own. As we add slower E-to-E synapses, these bring additional excitation to the network that has to be counterbalanced by inhibition to maintain the global E-I balance. In the eq.~\ref{alt1}, we instead have a slower inhibitory current in I neurons. Since E-to-E and I-to-I synaptic currents both promote the excitation at the network level, such a network is imbalanced and risks runaway excitation.

Two other alternative solutions describe networks without E-to-E connections.
As we replace the signal $\vec{x}(t)$ in $W_E^\intercal B \vec{x}(t)$  with the estimate by I neurons, $\vec{x}(t) \approx \vec{{\hat{x}}}_{I}(t)$, this constrains the slow synaptic current in E neurons to originate from I neurons and the current in question is now constrained to be inhibitory. To ensure the synaptic current to E neurons to be inhibitory, the matrix $B$ to has to be negative semi-definite. Assuming that the slow synaptic current in the I cell type is excitatory, the negative semi-definite matrix $B$ now imposes the constant $(\lambda_I-\lambda_E)$ to be negative. This solutions reads as follows:
\begin{equation}
\label{alt2}
\begin{split}
    & RI^{\text{slow} EI}_i(t) = \sum_{j=1}^{N_I}
     D^{EI}_{ij} r^I_j(t) \\
    & RI^{\text{slow} IE}_i(t) = (\frac{1}{\tau_I} - \frac{1}{\tau_E}) \sum_{j=1}^{N_E}
     D^{IE}_{ij} z^E_j(t),
     \quad \tau_I > \tau_E\\
    &  D^{yz}_{ij} = \begin{cases}
     (\vec{w}^y_i)^\intercal B \vec{w}^z_j, & \text{ if $(\vec{w}^y_i)^\intercal \vec{w}^z_j > 0$,\quad B negative semi-def.}, \qquad \{yz\} \in \{EI,IE\}\\
     0 & \text{ otherwise}.
   \end{cases}
\end{split}
\end{equation}
Constraint $\tau_I > \tau_E$ imposes that the membrane time constant of I neuron is longer than in E neurons. Moreover, such a network again risks a global imbalance of E and I currents. Slow inhibition in E neurons is accompanied by slow excitation in I neurons, and both currents globally promote inhibition. The latter solution seems of lesser biological relevance also because the network does not have E-to-E connections that are known to exist among excitatory neurons.

To prevent the imbalance of E and I currents, we can replace the excitatory estimate in ${(\lambda_I - \lambda_E) W_I^\intercal B \xe}$ with the inhibitory estimate, $\xe \approx \vec{{\hat{x}}}_{I}(t)$. This gives slow recurrent inhibition in I neurons, and the following set of solutions:
\begin{equation}
\label{alt3}
\begin{split}
    & RI^{\text{slow} EI}_i(t) = \sum_{j=1}^{N_I}
     D^{EI}_{ij} r^I_j(t) \\
    & RI^{\text{slow} II}_i(t) = (\frac{1}{\tau_I} - \frac{1}{\tau_E}) \sum_{j=1}^{N_I}
     D^{II}_{ij} r^I_j(t),
     \quad \tau_I < \tau_E\\
    &  D^{yz}_{ij} = \begin{cases}
     (\vec{w}^y_i)^\intercal B \vec{w}^z_j, & \text{ if $(\vec{w}^y_i)^\intercal \vec{w}^z_j > 0$,\quad B negative semi-def.}, \qquad \{yz\} \in \{EI,II\}.\\
     0 & \text{ otherwise}
   \end{cases}
\end{split}
\end{equation}
In the solution as in eq.~\eqref{alt3}, we added an inhibitory current to E and to I neurons on top of a balanced network. Such a solution is expected to globally balance E and I currents in the network. However, the solution in eq.~\eqref{alt3} leads to an incomplete description of cortical networks because the network lacks E-to-E connections.

\section{Computational resources}
Spiking network has been implemented with own computer code in Matlab, Mathworks, version 2021b.
Integration of the membrane potential is done with Euler integration scheme. The network with 400 E and 100 I units is computed within seconds on a standard laptop.

{
\small

redpajama_set_name: RedPajamaArXiv


text:
Q: Objective C access to multidimensional array value for(id key in stats) {
    NSLog(@"key=%@ value=%@", key, [stats objectForKey:key]);
}

I have a dictionary of arrays, and each of these arrays has a number of arrays itself. Its a multidimensional array in a dictionary.
However the above code gives me the following
2012-11-30 21:36:07.203 key=main cat c value=(
"<Food: 0x6e5fb70>"
)
2012-11-30 21:36:07.205 key=main cat b value=(
"<Food: 0x6e5fa00>",
"<Food: 0x6e5faa0>"
)
2012-11-30 21:36:07.207 key=bakery_products value=(
"<Food: 0x6e5f510>",
"<Food: 0x6e5f660>",
"<Food: 0x6e5f700>",
"<Food: 0x6e5f810>",
"<Food: 0x6e5f900>",
"<Food: 0x6e5d5d0>"
)

How can I access the values in those arrays that are being displayed as Food 0x6e5fb70 ?
I've literally been at it for hours now and cannot find a solution.

A: Something like this perhaps:
for (id key in [stats allKeys]) {
    NSArray *foodArray = [stats objectForKey:key];
    for (Food *food in foodArray) {
        // do stuff with Food object
    }
}

Update (for Ploto's 2nd comment):
Do you mean this:
NSArray *array = [stats objectForKey:@"bakery_products"];

Update #2 for description:
- (NSString *)description {
    return [NSString stringWithFormat:@"Food: %@, %@", whatever1, whatever2];
}

A: On top of what @rmaddy mentioned, if you use Apple LLVM Compiler 4.0 + (which comes with xCode 4.4+, I think) there is a neat way to do what you require, with no boilerplate code at all, using literals.
Here's how you would do it:
for (id key in [stats allKeys]) {
    for (Food * food in stats[key]) {

        //this is the object you're looking for
        NSLog(@"Object: %@", food);

    }
}

Where stats is your dictionary object that has an array for each of its keys. Updating to LLVM 4.0 worths the effort for this only :)
If you choose to enumerate each array using an iterator, you can even access the object like this:
stats[key][i]

A: A slightly more efficient way of enumerating a dictionary
[stats enumerateKeysAndObjectsUsingBlock:^(NSString *category, NSArray *foodItems, BOOL *stop) {
  for (Food *food in foodItems) {
    // do something with food
  }
}];

Obviously if you needed the dictionary to be iterated in the same order each time you'll need to use something a little longer
NSArray *sortedKeys = [[stats allKeys] sortedArrayUsingSelector:@selector(caseInsensitiveCompare:)];

for (id key in sortedKeys) {
  NSArray *foodArray = [stats objectForKey:key];
  for (Food *food in foodArray) {
    // do stuff with Food object
  }
}

redpajama_set_name: RedPajamaStackExchange


The Revenue Service of the Ministry of Finance of Georgia has published a document (tender) on its official website informing about the interest to acquire a project providing a mechanism for controlling gambling business in the country.
In order to provide the control over gambling business, the Revenue Service needs information on revenues and deductions from each gaming device, the percentage of the winning, the history of transactions in case of electronic gambling operations, and history of filled in tickets in case of totes.
Thus, the Revenue Service has announced a tender procedure for the development of the relevant project.
The tender procedure will end on October 2. And the estimated start time for the winning project is December 1.
According to the latest data, there are about 6.4 thousand gambling devices in Georgia, seven online casino websites and 90 totalizators (betting houses).


text:
Mount Pleasant Daily Voice serves Mount Pleasant & Valhalla
COVID-19: 'This Time, Unlike Last Year, We're Not Defenseless,' Hochul Says Of New Variant
New York Gov. Kathy Hochul said that the state is not defenseless against a winter COVID-19 surge. Photo Credit: ny.gov
New York Gov. Kathy Hochul said that the state is closely monitoring the new Omicron COVID-19 variant. Photo Credit: ny.gov
Gov. Kathy Hochul cautioned that New York is bracing for the new Omicron COVID-19 variant, declaring that the state will be ready for it when it does.
Hochul said during a COVID-19 briefing this week that the new variant is being closely monitored by state health officials, including the state testing center at Wadsworth Labs in Albany, and while Omicron is a concern, New York now has more tools to combat it than at the outset of the pandemic in March 2020.
Earlier story - COVID-19: First Case Of New Omicron Variant Identified In United States
"We're fully monitoring it - but the difference this time, unlike last year, is we're not defenseless," Hochul said. "We know exactly what has to happen. Anyone over the age of 5 can get vaccinated, and anyone 18 and up can get their booster shots.
"We've always encouraged mask use, and we're encouraging businesses to do the same among patrons and employees," she continued. "It's just simple measures … Mask wearing, hand sanitation, limit activities, but the difference this year and last is the vaccine."
Hochul cautioned that some parts of the state are facing potential hospital bed shortages, healthcare staffing shortages, and that she signed an Executive Order that will restrict non-essential surgeries at between 32 and 36 hospitals, largely in upstate New York beginning next month.
"If this new variant takes the state by storm and vaccinations and boosters don't fight it as well as we hope, we're going to be seeing more hospitals getting into trouble," the governor said. "We want to make sure this doesn't spread like wildfire with the variant coming to our shores if it's not here already.
"This new variant, we fully expect it to arrive after we got the notification it was in Ontario, which is literally across the bridge from where I live. I can see Canada from my house and Ontario is right there," Hochul said. "Now it just has to cross a bridge. We want to welcome our neighbors, but we don't want them bringing unwanted guests with them."
Not in Mount Pleasant?

redpajama_set_name: RedPajamaCommonCrawl


text:
Posted 14 January, 2014
Ogilvy appoints new director of scientific and editorial services
Ogilvy CommonHealth Asia Pacific have appointed Dr Robert Liversage as its director of scientific and editorial in its Australian office, strengthening their Australian team.
Full release below.
Ogilvy CommonHealth Asia Pacific strengthen Australia team in healthcare communications with key scientific hire
The creation of this new key role bolsters the scientific service offer and credentials of this integrated health focused communications agency
Sydney Australia January 2014 – Ogilvy CommonHealth Asia Pacific (OCHAP) announced today that they have appointed Dr Robert Liversage as their Director of Scientific & Editorial Services in their Australia office. The newly created role will see Dr Liversage take responsibility for content development, scientific council and medical advice across all areas of the business from prescription medications to OTC and pharmacy brands, and from medical education to disease awareness programmes.
Robert has a Neuroscience and Neuropharmacology PhD from the University of Leeds (UK) and spent several years in high-science, medical writing roles in the UK before relocating to Australia. Since 2008 he has been working within the editorial departments of Australian medical communications businesses and healthcare advertising agencies writing within the pharmaceutical space, leading the editorial and medical elements across both promotional and education programmes. He has a plethora of experience across disease areas spanning Oncology, Mental Health, Diabetes, Respiratory, CV, Dermatology, medical devices and Infectious Diseases.
Gary Pattison, Ogilvy CommonHealth Group Managing Director in Australia said "I am thrilled that Rob will be joining our growing, expert team at Ogilvy. He has a solid track record and a reputation as one of the best in the business."
Pattison went on to say, "At OCHAP we have spent the last 6-8 months looking carefully at our business, our structure and how we add value to our clients brands in this ever-changing world of health and wellness. We live and breath change here – we never stand still. We must offer excellence in client service and creative ideation of course – but none of this matters if you get the science wrong. Rob is a key hire in this respect as we plan for the future of the business and the future of our clients business."
Rohit Sahgal, Director for Ogilvy CommonHealth Asia Pacific commented "Staying one step ahead is a tradition at Ogilvy, and doing that in the area of scientific innovation has been a key part of a multi-faceted specialist strategy. Rob's appointment reinforces our commitment to providing cutting edge solutions and building enduring client partnerships."
Ogilvy & Mather is one of the largest marketing communications companies in the world. It was named both the Cannes Lions Network of the Year and the EFFIEs World's Most Effective Agency Network for two consecutive years, 2012 and 2013. The company is comprised of industry leading units in the following disciplines: advertising; public relations and public affairs; branding and identity; shopper and retail marketing; health care communications; direct, digital, promotion and relationship marketing; consulting, research and analytics; branded content and entertainment; and specialist communications. O&M services Fortune Global 500 companies as well as local businesses through its network of more than 500 offices in 126 countries. It is a WPP company (NASDAQ: WPPGY). For more information, visit http://www.ogilvy.com/, or follow Ogilvy on Twitter at @Ogilvy and on Facebook.com/Ogilvy.