text:
\section{Introduction} \label{sec:introduction}

The goal of meta-learning is to be able to generalise to new tasks from the same task distribution as the training tasks. In supervised meta-learning, a task can be described as making predictions on a set of unlabelled data points (\emph{target}) by effectively learning from a set of data points with labels (\emph{context}).
Various ideas have been proposed to tackle supervised meta-learning from different perspectives \citep{andrychowicz2016learning,ravi2016optimization,finn2017model,koch2015siamese,snell2017prototypical,vinyals2016matching,santoro2016meta,rusu2018meta}. In this work, we are particularly interested in a family of meta-learning models that use an encoder-decoder pipeline, such as Neural Processes \citep{garnelo2018conditional,garnelo2018neural}. The encoder is a permutation-invariant function on the context that summarises the context into a task representation, while the decoder produces a predictive model for the targets, conditioned on the task representation. The objective of meta-learning is then to learn the encoder and the decoder such that the produced predictive model generalises well to the targets of new tasks.

Previous works in this category such as the \gls{CNP} and the \gls{NP} \citep{garnelo2018conditional,garnelo2018neural} use sum-pooling operations to produce finite-dimensional, vectorial, task representations. In this work, we investigate the idea of summarising tasks with infinite-dimensional functional representations.
Although there is a theoretical guarantee that sum-pooling of instance-wise representations can express any set function (\textit{universality}) \citep{zaheer2017deep,bloem2019probabilistic}, in practice \gls{CNP} and \gls{NP} tend to underfit the context \citep{kim2019attentive}.
This observation is in line with the theoretical finding that for universality, the dimension of the task representation should be at least as large as the cardinality of the context set, if the encoder is a smooth function \citet{wagstaff2019limitations}.
We develop a method that explicitly uses functional representations. Here the effective dimensionality of the task representation grows with the number of context points, which addresses the aforementioned issues of fixed-dimensional representations.
Moreover, in practice it is difficult to model interactions between data points with only sum-pooling operations. This issue can be partially addressed by inserting modules such as relation networks \citep{sung2018learning,rusu2018meta} or set transformers \citep{lee2019set} before sum-pooling. However, only within-context but not context-target interactions can be modelled by these modules.
The construction of our functional representation involves measuring similarities between all data points, which naturally contains information regarding interactions between elements in either the context or the target.

Furthermore, rather than producing the functional representation in a single pass, we develop an approach that \textit{learns} iterative updates to encode the context into the task representation.
In general, learning via iterative updates is often easier than directly learning the final representation, because of the error-correcting opportunity at each iteration.
For example, an iterative parameterisation of the encoder in Variational Autoencoders (VAEs) has been demonstrated to be effective in reducing the amortisation gap \citep{marino2018iterative},
while in meta-learning, both learning to learn methods \citep{andrychowicz2016learning,ravi2016optimization} and \gls{MAML} \citep{finn2017model} use iterative updating procedures to adapt to new tasks, although these update rules operate in parameter space rather than function space.
Therefore, it is reasonable to conjecture that iterative structures are favourable inductive biases for the task encoding process.

In summary,
the primary contribution of this work is a meta-learning approach that learns to summarise a task using a functional representation constructed via iterative updates.
We apply our approach to solve  meta-learning problems on both regression and classification tasks, and achieve state-of-the-art performance on heavily benchmarked datasets such as miniImageNet \citep{vinyals2016matching} and tieredImageNet \citep{ren2018meta}, which has never been demonstrated with encoder-decoder meta-learning methods without \gls{MAML}-style gradient updates.
We also conducted an ablation study to understand the effects of the different model components.

\section{MetaFun} \label{sec:metafun}

\begin{figure}
  \centering
    \includegraphics[width=0.99\linewidth]{metafun-cartoon.png}
\caption{To illustrate the iterative procedure in MetaFun, we consider a simpler case where our functional representation is just a predictor for the task. (A) The figure depicts a 1D regression task with the current predictor. (B) Local updates are computed by evaluating the functional representation (the current predictor) on the context inputs, and comparing it to the corresponding context outputs. Here we simply measure differences between evaluations (predictions) and outputs. (C) We apply functional pooling to aggregate local updates into a global functional update, which generalises the local updates to the whole input domain. (D) The functional update is applied to the current functional representation with a learning rate $\alpha$.}
\label{fig:cartoon}
\end{figure}

Meta-learning, or learning to learn, leverages past experiences to quickly adapt to tasks $\mathcal{T}\sim p(\mathcal{T})$ drawn iid from some task distribution.
In supervised meta-learning, a task $\mathcal{T}$ takes the form of $\mathcal{T}=\{ \ell, \{(\textbf{x}_i,\textbf{y}_i)\}_{i\in \mathbf{C}}, \{(\textbf{x}'_j,\textbf{y}'_j)\}_{j\in \mathbf{T}}\}$, where $\textbf{x}_i,\textbf{x}'_j\in \mathcal{X}$ are inputs, $\textbf{y}_i,\textbf{y}'_j\in \mathcal{Y}$ outputs, $\ell$ is the loss function to be minimised, $\{(\textbf{x}_i,\textbf{y}_i)\}_{i\in \mathbf{C}}$ is the context, and $\{(\textbf{x}'_j,\textbf{y}'_j)\}_{j\in \mathbf{T}}$ is the target. We consider the process of learning as constructing a predictive model using the task context and refer to the mapping from context $\{(\textbf{x}_i,\textbf{y}_i)\}_{i\in \mathbf{C}}$ to a predictive model $f = \Phi(\{(\textbf{x}_i,\textbf{y}_i)\}_{i\in \mathbf{C}}; \phi)$ as the \emph{learning model} parameterised by $\phi$. In our formulation, the objective of meta-learning is to optimise the learning model such that the expected loss on the target under $f$ is minimised, formally written as:
\begin{align} \label{eq:meta-objective}
    f &= \Phi(\{(\textbf{x}_i,\textbf{y}_i)\}_{i\in \mathbf{C}}; \phi) \nonumber \\
    \phi^* &= \underset{\phi}{\arg\min}
    \; \mathbb{E}_{\mathcal{T} \sim p(\mathcal{T})} \left[ \frac{1}{|\mathbf{T}|} \sum_{j\in \mathbf{T}} \ell(f(\textbf{x}'_j),\textbf{y}'_j) \right]\,,
\end{align}
where both $\{(\textbf{x}_i,\textbf{y}_i)\}_{i\in \mathbf{C}}$, $\{(\textbf{x}'_j,\textbf{y}'_j)\}_{i\in \mathbf{T}}$ come from task $\mathcal{T}$.

\begin{figure*}
  \centering
    \includegraphics[width=0.99\linewidth]{metafun-main.png}
\caption{This figure illustrates the iterative computation of functional representation in MetaFun. At each iteration, we first evaluate the current functional representation at both context and target points. Then the shared local update function $u$ takes in each context point and the corresponding evaluation as inputs, and produces local update $u_i$. Next, we apply (kernel-based or attention-based) functional pooling  to aggregate local updates $u_i$ into a functional update $\Delta r(\cdot)$, which for each query is a linear combination of local updates $u_i$ weighted by similarities between this query and all keys. Finally, the functional updates are evaluated for both the context and the target, and are applied to the corresponding evaluations of functional representation with a learning rate $\alpha$.}
\label{fig:metafun}
\end{figure*}

\subsection{Learning Functional Task Representation} \label{sub:learning task representation}

Like previous works such as \gls{CNP} and \gls{NP}, we construct the learning model using an encoder-decoder pipeline, where the encoder $\Phi_e(\{(\textbf{x}_i,\textbf{y}_i)\}_{i\in \mathbf{C}}; \phi_e)$ is a permutation-invariant function of the context producing a task representation.
In past works, pooling operations are usually used to enforce permutation-invariance. \gls{CNP} and \gls{NP} use sum-pooling: $\textbf{r}=\sum_{i\in\mathbf{C}} \textbf{r}_i$, where $\textbf{r}_i=h(\textbf{x}_i,\textbf{y}_i;\phi_e)$ is a representation for context pair $\textbf{x}_i,\textbf{y}_i$, and $\textbf{r}$ is a fixed-dimensional task representation. Instead, we introduce functional-pooling operations, which also enforce permutation-invariance but output a function that can loosely be interpreted as an infinite-dimensional representation.

\theoremstyle{definition}
\begin{definition}[Functional pooling] Let $k(\cdot,\cdot)$ be a real-valued similarity measure, and $\{(\textbf{x}_i,\textbf{r}_i)\}_{i\in\mathbf{C}}$ be a set of key-value pairs with $\textbf{x}_i\in\mathcal{X}$, $\textbf{r}_i\in\mathcal{R}$. \emph{Functional pooling} is a mapping $\operatorname{\textsc{FunPooling}}:(\mathcal{X}\times\mathcal{R})^{|\mathbf{C}|}\rightarrow \mathcal{H}$ defined as
\begin{align}
\hspace*{-.8em}
    r(\cdot)=\operatorname{\textsc{FunPooling}}(\{(\textbf{x}_i,\textbf{r}_i)\}_{i\in \mathbf{C}}) = \sum_{i\in C} k(\cdot,\textbf{x}_i) \textbf{r}_i\,,
\end{align}
where the output is a function $r:\mathcal{X}\rightarrow \mathcal{R}$ and $\mathcal{H}$ is a space of such functions.
\end{definition}

In practice, we only need to evaluate this function on a finite query set $\{(\textbf{x}'_j,\textbf{y}'_j)\}_{j\in\mathcal{Q}}$ (consisting of both contexts and targets; see below). That is, we only need to compute $R=[r(\textbf{x}'_1),\ldots,r(\textbf{x}'_{|\mathcal{Q}|})]^{\top}$, which can be easily implemented using matrix operations.
We consider two types of \textsc{FunPooling} here, though others are possible. The kernel-based \textsc{FunPooling} reads as,
\begin{align}
    R=\KFP{Q,K,V} := k_{\text{rbf}}(Q,K) V\,,
\label{eq:kernel_gradient}
\end{align}
where $k_{\text{rbf}}$ is the RBF kernel, $Q=[a(\textbf{x}'_1),\ldots,a(\textbf{x}'_{|\mathcal{Q}|})]^\top$ is a matrix whose rows are queries,  $a(\cdot)$ is a  transformation mapping inputs into features,  $K=[a(\textbf{x}_1),\ldots,a(\textbf{x}_{|C|})]^\top$ a matrix whose rows are keys, and $V=[\textbf{r}_1,\ldots,\textbf{r}_{|C|}]^\top$ a matrix whose rows are values (using terminology from the attention literature). Parameterising input transformation $a$ with a deep neural network can be seen as using deep kernels \citep{wilson2016deep} as the similarity measure.
The second type of \textsc{FunPooling} is given by dot-product attention,
\begin{align}
    R=\DFP{Q,K,V} := \text{softmax}(QK^\top/\sqrt{d_k}) V\,,
\label{eq:attention}
\end{align}
where $d_k$ is the dimension of the query/key vectors.

Our second core idea is that rather than producing the task representation in a single pass (like  previous encoder-decoder meta-learning approaches), we start from an initial representation $r^{(0)}(\cdot)$, and iteratively produce improved representations $r^{(1)}(\cdot),\ldots,r^{(T)}(\cdot)$.
At each step, a parameterised local update rule $u$ compares $r^{(t)}(\cdot)$ to the context input/output pairs, producing local update values $\textbf{u}_i=u(\textbf{x}_i,\textbf{y}_i,r^{(t)}(\textbf{x}_i))$ for each $i\in\mathbf{C}$. These can then be aggregated into a global update to the task representation using functional pooling,
\begin{align}
    \textbf{u}_i &= u(\textbf{x}_i,\textbf{y}_i,r^{(t)}(\textbf{x}_i))\,, \nonumber \\
    \Delta r^{(t)}(\cdot) &= \textsc{FunPooling}(\{(\textbf{x}_i,\textbf{u}_i)\}_{i\in\mathbf{C}})\,, \nonumber \\
    r^{(t+1)}(\cdot) &= r^{(t)}(\cdot) - \alpha \Delta r^{(t)}(\cdot)\,,  \label{eq:r-iter}
\end{align}
where $\alpha$ is the step size.
Once the local update function $u$ and the functional pooling operations are parameterised by neural networks, \Cref{eq:r-iter} defines a  neural update rule operating directly in function space. The functional update $\Delta r^{(t)}(\cdot)$ depends on the current representation $r^{(t)}(\cdot)$ and the context $\{(\textbf{x}_i,\textbf{y}_i)\}_{i\in \mathbf{C}}$. \Cref{fig:cartoon} illustrates our iterative procedure in a simplified setting.

The final task representation can then be decoded into a predictor   $f(\cdot)=\Phi_d(r^{(T)}(\cdot); \phi_d)$.
The specific parametric forms of the decoder take different forms for regression and classification, and are described in \Cref{sub:regression-and-classification}. The decoder requires the evaluation of functional representation $r^{(T)}(\textbf{x})$ at $\textbf{x}$ only for predicting $f(\textbf{x})$. Therefore, it is unnecessary to compute the functional representations $r(\cdot)$ (including their functional updates) on all input points. Instead, we compute them only on the context $\{\textbf{x}_i\}_{i\in \mathbf{C}}$ and target inputs $\{\textbf{x}'_j\}_{j\in \mathbf{T}}$. We use $\textbf{r}^{(t)}\!=\![r^{(t)}(\textbf{x}_1)\ldots r^{(t)}(\textbf{x}_{|\mathbf{C}|}),r^{(t)}(\textbf{x}'_1) \ldots r^{(t)}(\textbf{x}'_{|\mathbf{T}|})]^\top$ to denote a matrix where each row is $r^{(t)}(\textbf{x})$ evaluated on either context or target inputs, and let $Q=[a(\textbf{x}_1) \ldots a(\textbf{x}_{|\mathbf{C}|}),a(\textbf{x}'_1) \ldots a(\textbf{x}'_{|\mathbf{T}|})]^\top$.
\Cref{eq:r-iter} can be implemented using matrix computations as follows,
\begin{align}
\textbf{u}_i^{(t)} &= u(\textbf{x}_i,\textbf{y}_i,\textbf{r}^{(t)}_i)\,,\label{eq:value_i_t} \\
U^{(t)} &= [\textbf{u}_1^{(t)}, \ldots, \textbf{u}_{|\mathbf{C}|}^{(t)}]^\top\,,\label{eq:values_t}\\
\Delta \textbf{r}^{(t)} &= \operatorname{\textsc{kfp}} \text{ or } \DFP{Q, K, U^{(t)}}\,, \label{eq:delta_r}\\
\textbf{r}^{(t+1)} &= \textbf{r}^{(t)} - \alpha \Delta \textbf{r}^{(t)}\, \label{eq:update_r}
\end{align}
where $\textbf{r}^{(t)}_i$ denotes the $i$-th row of $\textbf{r}^{(t)}$.

To obtain a prediction $\textbf{f}_j$ for the target $(\textbf{x}'_j,\textbf{y}'_j)$, we decode the final representation for this target point: $\textbf{f}_j=\Phi_d(\textbf{r}^{(T)}_{|\mathbf{C}|+j}; \phi_d)$, and the overall training loss can be written as:
\begin{equation}
    L(u,a,\phi_d) = \frac{1}{|\mathbf{T}|} \sum_{j\in \mathbf{T}} \ell(\textbf{f}_j,\textbf{y}'_j)\,,
\end{equation}
where the predictions $\textbf{f}_j$ depend on $u,a$ and $\phi_d$.

Assuming the width and depth of all our neural network components are bounded by $W$ and $D$ respectively, and the output dimension of $u$ is also less than $W$, the time complexity of our approach is $\mathcal{O}\big(W|\mathbf{C}|(|\mathbf{C}|+|\mathbf{T}|)+W^2 D (|\mathbf{C}|T+|\mathbf{T}|)\big)$, and the space complexity is $\mathcal{O}\big((|\mathbf{C}|+ WT)(|\mathbf{C}|+|\mathbf{T}|)+ W^2 D \big)$. For few-shot problems, $|\mathbf{C}|$ and $|\mathbf{T}|$ are typically small, and $T \leq 6$ in all our experiments.

\subsection{MetaFun for Regression and Classification} \label{sub:regression-and-classification}

While the proposed framework can be applied to any supervised learning task, the specific parameterisation of learnable components can affect the model performance.
In this section, we specify the parametric forms of our model that work well on regression and classification tasks.
\paragraph{Regression} For regression tasks, we parameterise the local update function $u(\cdot)$ using a multi-layer perceptron\glsunset{MLP} as $u([\textbf{x}_i,\textbf{y}_i,r(\textbf{x}_i)]) = \MLP{[\textbf{x}_i,\textbf{y}_i,r(\textbf{x}_i)]}$, $i \in C$, where $[\cdot]$ is concatenation.
We also use an \gls{MLP} to parametrise the input transformation $a(\cdot)$ in the functional pooling.
The decoder in this case is given by $\mathbf{w}=\MLP{r(\textbf{x})}$, another \gls{MLP}\footnote{
It might be desirable to use other parameterisations of the input transformation $a(\cdot)$, and the decoder $f(\cdot)$, e.g., $f(\textbf{x})~=~\MLP{[\textbf{x},r(\textbf{x})])}$, or feeding $r(\textbf{x})$ to each layer of the \gls{MLP}.
} that outputs $\mathbf{w}$, which then parameterises the predictive model $f=\MLP{\textbf{x};\mathbf{w}}$.

Note that our model can easily be modified to incorporate Gaussian uncertainty by adding an extra output vector for the predictive standard deviation: $P(\textbf{y}|\textbf{x})=\mathcal{N}(\mu_{\mathbf{w}}(\textbf{x}), \sigma_{\mathbf{w}}(\textbf{x})), \mathbf{w}=\MLP{r(\textbf{x})}$. For further architecture details, see Appendix.

\paragraph{Classification} For $K$-way classification, we divide the latent functional representation $r(\textbf{x})$ into $K$ parts $[r^1(\textbf{x}), \dots, r^K(\textbf{x})]$, where $r^k(\textbf{x})$ corresponds to the class $k$.
Consequently, the local update function $u(\cdot)$ also has $K$ parts, that is, $u([\textbf{x}_i,\textbf{y}_i,r(\textbf{x}_i)]) = [u^1(\cdot), \dots, u^K(\cdot)]$. In this case, $\textbf{y}_i=[y_i^1,\dots,y_i^K]$ is the class label expressed as a one-hot vector; the $u^k$ is defined as follows,
\begin{equation} \label{eq:value-function-classification}
    \begin{aligned}
    u^k([\textbf{x}_i,\textbf{y}_i,r(\textbf{x}_i)]) &= y_i^k u_{+}(m(r^k(\textbf{x}_i)), \bm{m}_i)\\
    &+ (1-y_i^k) u_{-}(m(r^k(\textbf{x}_i)), \bm{m}_i)\,,
    \end{aligned}
\end{equation}
where $\bm{m}_i = \sum_{k=1}^K m(r^k(\textbf{x}_i))$ summarises representations of all classes, and $m$, $u_{+}$, $u_{-}$ are parameterised by separate \gls{MLP}s.
With this formulation, we update the class representations using either $u_{+}$ (when the label matches $k$) or $u_{-}$ (when the label is different to $k$), so that labels are not concatenated to the inputs, but directly used to activate different model components, which is crucial for model performance. Furthermore, interactions between data points in classification problems include both within-class and between-class interactions. Our approach is able to integrate two types of interactions by having separate functional representation for each class and computing local updates for each class differently based on class membership of each data point.
In fact, this formulation resembles the structure of the local update rule in functional gradient descent for classification tasks, which is a special case of our approach (see \Cref{sec:related_work}).
Same as in regression tasks, the input transformation $a(\cdot)$ in the functional pooling is still an \gls{MLP}.
The parametric form of the decoder is the same as in \gls{LEO} \citep{rusu2018meta}.
The class representation $r^k(\textbf{x})$ generates weights $\mathbf{w^k}\sim \mathcal{N}(\mu(r^k(\textbf{x})),\sigma(r^k(\textbf{x})))$ where $\mu$ and $\sigma$ are \gls{MLP}s or just linear functions,
and the final prediction is given by
\begin{align}
    P(\textbf{y}=k | \textbf{x}) = \text{softmax} ( \textbf{x}^T \mathbf{w} )_k\,,
\end{align}
where $\mathbf{w} = [\mathbf{w^1}, \dots, \mathbf{w^K}]$, $k=1,\dots,K$.
Hyperparameters of all components are described in Appendix.

\section{Related Work} \label{sec:related_work}

\paragraph{Functional Gradient Descent} Functional gradient descent \citep{mason1999functional,guo2001norm} is an optimisation algorithm used to minimise the objective function by moving in the direction of the negative gradient in function space.
To ensure smoothness, we may work with functions in a \gls{RKHS} \citep{aronszajn1950theory,berlinet2011reproducing} defined by a kernel $k(\textbf{x},\textbf{x}')$.
Given a function $f$ in the \gls{RKHS}, we are interested in minimising the supervised loss $L(f)= \sum_{i\in C} \ell(f(\textbf{x}_i),\textbf{y}_i)$ with respect to $f$. We can do so by computing the functional derivative and use it to iteratively update $f$ (see Appendix for more details),
\begin{align}
    \hspace{-0.5em}f^{(t+1)}(\textbf{x}) = f^{(t)}(\textbf{x})
    - \alpha \sum_{i\in C} k(\textbf{x},\textbf{x}_i) \nabla \ell(f^{(t)}(\textbf{x}_i),\textbf{y}_i)
    \label{eq:functionupdate}
\end{align}
with step size $\alpha$, and $\nabla \ell(f^{(t)}(\textbf{x}_i),\textbf{y}_i)$ denotes gradient w.r.t. to predictions in the loss function $\ell$.

The update rule in \Cref{eq:r-iter} becomes that of functional gradient descent in \Cref{eq:functionupdate} when
\begin{enumerate}[label={\upshape(\roman*)}]
    \item A trivial decoder $f(\textbf{x})=\Phi_d(r(\textbf{x});\phi_d)(\textbf{x})=r(\textbf{x})$ is used, so the functional representation $r(\textbf{x})$ is the same as the predictive model $f(\textbf{x})$.
    \item Kernel functional pooling $\operatorname{\textsc{KFP}}$ is used and the kernel function is fixed.
    \item Using gradient-based local update function $u(\textbf{x},\textbf{y},f(\textbf{x}))=\nabla \ell(f(\textbf{x}),\textbf{y})$.
\end{enumerate}

Furthermore, for a $K$-way classification problem, we predict $K$-dimensional logits $f(\textbf{x})=[f^{1}(\textbf{x}),\dots,f^{K}(\textbf{x})]^\top$, and use cross entropy loss as follows:
\begin{equation}
    \ell(f(\textbf{x}),\textbf{y}) = - \sum_{k=1}^K y^k \log \frac{e^{f^{k}(\textbf{x})}}{\sum_{k'=1}^K e^{f^{k'}(\textbf{x})}}\,,
\end{equation}
where $\textbf{y}=[y^1,\dots,y^K]^\top$ is the one-hot label for $\textbf{x}$.

The gradient-based local update function is now $\nabla \ell(f(\textbf{x}),\textbf{y})=[\partial_1 \ell(f(\textbf{x}),\textbf{y}),\ldots,\partial_K \ell(f(\textbf{x}),\textbf{y})]^\top$ where $\partial_k \ell(f(\textbf{x}),\textbf{y})$ is partial derivative w.r.t. each predictive logit:
\begin{align} \label{eq:fgd-classification}
    \partial_k \ell(f(\textbf{x}),\textbf{y}) &= \frac{e^{f^{k}(\textbf{x})}}{\sum_{k'=1}^K e^{f^{k'}(\textbf{x})}} - y^k\,.
\end{align}
Here $\partial_k \ell(f(\textbf{x}),\textbf{y})$ is analogous to $u^k(\cdot)$ in \Cref{eq:value-function-classification}, which is the local update function for class $k$.

If $m,u_{+},u_{-}$ in \Cref{eq:value-function-classification} are specified rather than being learned, more specifically:
\begin{align}
    m(f^{k'}(\textbf{x})) &= e^{f^{k'}(\textbf{x})}  \nonumber \\
    u_{+}(m(f^k(\textbf{x})), \bm{m}) &= \frac{m(f^k(\textbf{x}))}{\bm{m}} - 1 \nonumber \\
    u_{-}(m(f^k(\textbf{x})), \bm{m}) &= \frac{m(f^k(\textbf{x}))}{\bm{m}} \nonumber \\
    \bm{m} &= \sum_{k=1}^K m(f^{k'}(\textbf{x}))\,,
\end{align}
\Cref{eq:fgd-classification} can be rewritten as:
\begin{align} \label{eq:fgd-classification-rewritten}
\partial_k \ell(f(\textbf{x}),\textbf{y}) &= y^k u_{+}(m(f^k(\textbf{x})), \bm{m})\nonumber \\
    &+ (1-y^k) u_{-}(m(f^k(\textbf{x})), \bm{m})\,,
\end{align}
which has a similar form as \Cref{eq:value-function-classification}.

Therefore, our approach can be seen as an extension of functional gradient descent, with an additional learning capacity due learnable neural modules which afford more flexibility. From this perspective, our approach tackles supervised meta-learning problems by learning an optimiser in function space.

\paragraph{Supervised Meta-Learning} Various ideas have been proposed to solve the problem of supervised meta-learning. \citet{andrychowicz2016learning,ravi2016optimization} learn the neural optimisers from previous tasks which can be used to optimise models for new tasks. However, these learned optimisers operate in parameter space rather than function space as we do.
\gls{MAML} \citep{finn2017model} learns the initialisation from which models are further adapted for a new task by a few gradient descent steps.
\citet{koch2015siamese,snell2017prototypical,vinyals2016matching} explore the idea of learning a metric space from previous tasks in which data points are compared to each other to make predictions at test time.
\citet{santoro2016meta} demonstrate that Memory-Augmented Neural Networks (MANN) can rapidly integrate the data for a new task into memory, and utilise this stored information to make predictions.

Our approach, in line with previous works such as \gls{CNP} and \gls{NP}, adopt an encoder-decoder pipeline to tackle supervised meta-learning.
The encoder in \gls{CNP} corresponds to a summation of instance-level representations produced by a shared instance encoder.
\Gls{NP}s, on the other hand, use a probabilistic encoder with the same parametric form as \gls{CNP}, but producing a distribution of stochastic representation.
The \gls{ANP} \citep{kim2019attentive} adds a deterministic path in addition to the stochastic path in \gls{NP}. The deterministic path produces a target-specific representation, which can be interpreted as
applying functional pooling (implemented with multihead attention \citep{vaswani2017attention}) to instance-wise representation. However, the representation is directly produced in a single pass rather than iteratively improved as we do, and only regression applications are explored as opposed to few-shot image classification.
In fact, to achieve high performance for classification tasks, it is crucial for \gls{CNP} to only apply sum-pooling within each class \citep{garnelo2018conditional}, and it is unclear how to follow similar practices in \gls{ANP} with both within-class and between-class interactions still being modelled.
Recently, \citet{gordon2019convolutional} have also extended \gls{CNP} to use functional representations, but for the purpose of incorporating translation equivariance in the inputs as an inductive bias rather than increasing representational capacity as we do. Their approach uses convnets to impose translation equivariance and does not learn a flexible iterative encoder.

Pooling operations are usually used in encoder-decoder meta-learning to enforce permutation invariance in the encoder.
As an example, encoders in both \gls{CNP} and \gls{NP} use simple \emph{sum-pooling} operations.
More expressive pooling operations have been proposed to model interactions between data points.
\citet{murphy2019janossy} introduces \emph{Janossy pooling} which applies permutation-sensitive functions to all reorderings and averages the outputs, while \citet{lee2019set} use \emph{pooling by multihead attention} (\textsc{pma}), which uses a finite query set to attend to the processed key-value pairs. Loosely speaking, attention-based functional pooling can be seen as having the whole input domain $\mathcal{X}$ as the query set in \textsc{pma}.

\paragraph{Gradient-Based Meta-Learning}
Interestingly, many gradient-based meta-learning methods such as \gls{MAML} can also be cast into an encoder-decoder formulation, because a gradient descent step is a valid permutation-invariant function. For a model $f(\cdot, \theta)$ parameterised by $\theta$, one gradient descent step on the context loss has the following form,
\begin{equation}
    \theta_{t+1} = \theta_t - \alpha \sum_{i\in C} \nabla_{\theta} \ell(f(\textbf{x}_i;\theta_t),\textbf{y}_i)\,,
\end{equation}
where $\ell$ is the loss function, $\alpha$ is the learning rate, and $\theta_t$ are the model parameters after $t$ gradient steps.
This corresponds to a special case of permutation-invariant functions where we take the instance-wise encoder to be $h_t(\textbf{x}_i,\textbf{y}_i;\theta_t) = \theta_t/|\mathbf{C}| - \alpha \nabla_{\theta} \ell(f(\textbf{x}_i;\theta_t),\textbf{y}_i)$ and apply sum-pooling $\theta_{t+1}=\sum_i h_t(\textbf{x}_i,\textbf{y}_i;\theta_t)$.
Multiple gradient-descent steps also result in a permutation-invariant function, which can be proved by induction. We refer to this as a gradient-based encoder.
What follows is that popular meta-learning methods such as \gls{MAML} can be seen as part of the encoder-decoder formulation.
More specifically, in \gls{MAML}, we learn an initialisation of the model parameters $\theta_0$ from training tasks, and adapt to new tasks by running $T$ gradient steps from the learned initialisation. Therefore, $\theta_T$ can be seen as the task representation (albeit very high-dimensional) produced by a gradient-based encoder.
The success of \gls{MAML} on a variety of tasks can be partially explained by the high-dimensional representation and the iterative adaptation by gradient descent, supporting our usage of a functional ('infinite-dimensional') representation and iterative updating procedure. Note, however, that the update rule in \gls{MAML} operates in parameter space rather than function space as in our case.

Under the same encoder-decoder formulation, a comparison regarding \gls{MAML} and MetaFun can be made, which partially explains why MetaFun can be desirable: Firstly, the updates in \gls{MAML} must lie in its parametric space, while there is no parametric constraint in MetaFun, which is better illustrated in \Cref{fig:sinusoid}. Secondly, \gls{MAML} uses gradient-based updates, while MetaFun uses learned local updates, which potentially contains more information than gradient. Finally, \gls{MAML} does not explicitly consider interactions between data points, while both within-context and context-target interactions are modelled in MetaFun.

\section{Experiments} \label{sec:experiments}

We evaluate our proposed model on both few-shot regression and classification tasks.
In all experiments that follow, we partition the data into training, validation and test meta-sets, each containing data from disjoint tasks.
For quantitative results, we train each model with $5$ different random seeds and report the mean and the standard deviation of the test accuracy. For further details on hyperparameter tuning, see the Appendix. All experiments are performed using TensorFlow \citep{tensorflow2015-whitepaper}, and the code is available online \footnote{A tensorflow implementation of our model is available at \url{github.com/jinxu06/metafun-tensorflow}}.

\subsection{1-D Function Regression}

\begin{figure*}
\begin{minipage}{0.49\textwidth}
    \includegraphics[width=1.0\linewidth]{sinusoid.pdf}
    \caption{MetaFun is able to learn smooth updates, and recover the ground truth function almost perfectly. While the updates given by \gls{MAML}s are relatively not smooth, especially for \gls{MAML} with less parameters.}
    \label{fig:sinusoid}
\end{minipage}
\hspace{0.5cm}
\begin{minipage}{0.49\textwidth}
    \includegraphics[width=1.0\linewidth]{gp-uncertainty.pdf}
\caption{Predictive uncertainties for MetaFun matches those for the oracle \gls{GP} very closely in both 5-shot and 15-shot cases. The model is trained on random context size ranging from $1$ to $20$.}
\label{fig:gp}
\end{minipage}
\end{figure*}

We first explore a 1D sinusoid regression task where we visualise the updating procedure in function space, providing intuition for the learned functional updates. Then we incorporate Gaussian uncertainty into the model, and compare our predictive uncertainty against that of a \gls{GP} which generates the data.

\begin{table}[!ht]
  \centering
  \footnotesize
  \caption{Few-shot regression on sinusoid. \gls{MAML} can beneift from more parameters, but MetaFun still outperforms all \gls{MAML}s despite less parameters being used compared to large \gls{MAML}. We report mean and standard deviation of $5$ independent runs.}
  \begin{tabular}{l|cc}
    \toprule
    Model &  $5$-shot MSE & $10$-shot MSE \\
    \midrule
    Original \gls{MAML} & $0.390\pm0.156$ & $0.114\pm0.010$ \\
    Large \gls{MAML} & $0.208\pm0.009$ & $0.061\pm0.004$ \\
    Very Wide \gls{MAML}  & $0.205\pm0.013$ & $0.059\pm0.010$ \\
    MetaFun & $\bm{0.040\pm0.008}$ & $\bm{0.017\pm0.005}$ \\
    \bottomrule
  \end{tabular}
  \label{tb:sinusoid}
\end{table}

\paragraph{Visualisation of functional updates} We train a $T$-step MetaFun with dot-product functional pooling, on a simple sinusoid regression task from \citet{finn2017model}, where each task uses data points of a sine wave. The amplitude $A$ and phase $b$ of the sinusoid varies across tasks and are randomly sampled during training and test time, with $A \in \mathcal{U}(0.1, 5.0)$ and $b \in \mathcal{U}(0, \pi)$. The x-coordinates are uniformly sampled from $\mathcal{U}(-5.0, 5.0)$.
\Cref{fig:sinusoid} shows that our proposed algorithm learns a smooth transition from the initial state to the final prediction at $t=T=5$. Note that although only $5$ context points on a single phase of the sinusoid are given at test time, the final iteration makes predictions close to the ground truth across the whole period.
As a comparison, we use \gls{MAML} as an example of updating in parameter space. The original \gls{MAML} ($40$ units $\times$ $2$ hidden layers) can fit the sinusoid quite well after several iterations from the learned initialisation. However the prediction is not as good, particularly on the left side where there are no context points (see \Cref{fig:sinusoid} B). As we increase the model size to large \gls{MAML} ($256$ units $\times$ $3$ hidden layers), updates become much smoother (\Cref{fig:sinusoid} C) and the predictions are closer to the ground truth.
We further conduct experiments with a very wide \gls{MAML} ($1024$ units $\times$ $3$ hidden layers), but the performance cannot be further improved (\Cref{fig:sinusoid} D). In \Cref{tb:sinusoid}, we compare the mean squared error averaged across tasks. MetaFun performs much better than all \gls{MAML}s, even though less parameters ($116611$ parameters) are used compared to large \gls{MAML} ($132353$ parameters).

\paragraph{Predictive uncertainties} As another simple regression example, we demonstrate that MetaFun, like \gls{CNP}, can produce good predictive uncertainties. We use synthetic data generated using a \gls{GP} with an RBF kernel and Gaussian observation noise ($\mu=0,\sigma=0.1$), and our decoder produces both predictive means and variances.
As in \citet{kim2019attentive}, we found that MetaFun-DFP can produce somewhat piece-wise constant mean predictions which is less appealing in this situation. On the other hand, MetaFun-KFP (with deep kernels) performed much better, as can be seen in \Cref{fig:gp}. We consider the cases of $5$ or $15$ context points, and compare our predictions to those for the oracle \gls{GP}. In both cases, our model gave very good predictions.

\begin{table*}[!htb]
  \centering
  \footnotesize
  \begin{threeparttable}[]
  \caption{Few-shot Classification Test Accuracy}
  \begin{tabular}{l|cc}
    \toprule
         & \textbf{miniImageNet 5-way}     & \textbf{miniImageNet 5-way}  \\
    Models    & \textbf{1-shot}     & \textbf{5-shot} \\
    \midrule
    \emph{(Without deep residual networks feature extraction):} & & \\
    Matching networks \citep{vinyals2016matching} & $43.56\pm0.84\%$ & $55.31\pm0.73\%$ \\
    Meta-learner LSTM \citep{ravi2016optimization}  & $43.44\pm0.77\%$ & $60.60\pm0.71\%$ \\
    MAML \citep{finn2017model} & $48.70\pm1.84\%$ & $63.11\pm0.92\%$ \\
    LLAMA \citep{grant2018recasting}  & $49.40\pm1.83\%$ & - \\
    REPTILE \citep{nichol2018first}  & $49.97\pm0.32\%$ & $65.99\pm0.58\%$ \\
    PLATIPUS \citep{finn2018probabilistic}  & $50.13\pm1.86\%$ & - \\
    \midrule
    \emph{(Without data augmentation):} & & \\
    Meta-SGD \citep{li2017meta} & $54.24\pm0.03\%$ & $70.86\pm0.04\%$  \\
    SNAIL \citep{mishra2017simple} & $55.71\pm0.99\%$ & $68.88\pm0.92\%$ \\
    \citet{bauer2017discriminative} & $56.30\pm0.40\%$ & $73.90\pm0.30\%$ \\
    \citet{munkhdalai2018rapid} & $57.10\pm0.70\%$ & $70.04\pm0.63\%$ \\
    TADAM \citep{oreshkin2018tadam} & $58.50\pm0.30\%$ & $76.70\pm0.30\%$  \\
    \citet{qiao2018few} & $59.60\pm0.41\%$ & $73.74\pm0.19\%$  \\
    LEO & $\bm{61.76\pm0.08\%}$ & $ 77.59\pm0.12\%$  \\

    MetaFun-DFP & $\bm{62.12\pm0.30\%}$ & $77.78\pm0.12\%$  \\
    MetaFun-KFP & $61.16\pm0.15\%$ & $\bm{78.20\pm0.16\%}$  \\

    \midrule
    \emph{(With data augmentation):} & & \\
    \citet{qiao2018few} & $\bm{63.62\pm0.58\%}$ & $78.83\pm0.36\%$  \\
    LEO & $63.97\pm0.20\%$ & $79.49\pm0.70\%$  \\
    MetaOptNet-SVM \citep{lee2019meta}\tnote{1} & $\bm{64.09\pm0.62\%}$ & $80.00\pm0.45\%$  \\
    MetaFun-DFP & $\bm{64.13\pm0.13\%}$  &
    $\bm{80.82\pm0.17\%}$  \\
    MetaFun-KFP & $63.39\pm0.15\%$  &
    $\bm{80.81\pm0.10\%}$  \\

    \bottomrule
  \end{tabular}

  \begin{tabular}{l|cc}
    \toprule
     & \textbf{tieredImageNet 5-way}     & \textbf{tieredImageNet 5-way}  \\
    Models    & \textbf{1-shot}     & \textbf{5-shot} \\
    \midrule
    \emph{(Without deep residual networks feature extraction):} & & \\
    MAML \citep{finn2017model} & $51.67\pm1.81\%$ & $70.30\pm0.08\%$ \\
    Prototypical Nets \citep{snell2017prototypical} & $53.31\pm0.89\%$ & $72.69\pm0.74\%$ \\
    Relation Net [in \citet{liu2019learning}] & $54.48\pm0.93\%$ & $71.32\pm0.78\%$ \\
    Transductive Prop. Nets \citep{liu2019learning}\hspace{1cm} & $57.41\pm0.94\%$ & $71.55\pm0.74\%$ \\
    \midrule
    \emph{(With deep residual networks feature extraction):} & & \\
    Meta-SGD & $62.95\pm0.03\%$ & $79.34\pm0.06\%$  \\
    LEO & $66.33\pm0.05\%$ & $81.44\pm0.09\%$  \\
    MetaOptNet-SVM & $65.81\pm0.74\%$ & $81.75\pm0.58\%$  \\
    MetaFun-DFP & $\bm{67.72\pm0.14\%}$ & $82.81\pm0.15\%$  \\
    MetaFun-KFP & $67.27\pm0.20\%$ & $\bm{83.28\pm0.12\%}$  \\
    \bottomrule
  \end{tabular}
  \label{tb:classification}
  \end{threeparttable}
\end{table*}

\subsection{Classification: miniImageNet and tieredImageNet}\label{sub:imagenet}

The \emph{miniImageNet} dataset \citep{vinyals2016matching} consists of 100 classes selected randomly from
the ILSVRC-12 dataset \citep{russakovsky2015imagenet}, and each class contains 600 randomly sampled images. We follow the split in \citet{ravi2016optimization}, where the dataset is divided into training ($64$ classes), validation ($16$ classes), and test ($20$ classes) meta-sets.
The \emph{tieredImageNet} dataset \citep{ren2018meta} contains a larger subset of the ILSVRC-12 dataset. These classes are further grouped into $34$ higher-level nodes. These nodes are then divided into training ($20$ nodes), validation ($6$ nodes), and test ($8$ nodes) meta-sets. This dataset is considered more challenging because the split is near the root of the ImageNet hierarchy \citep{ren2018meta}. For both datasets, we use the pre-trained features provided by \citet{rusu2018meta}.

Following the commonly used experimental setting, each few-shot classification task consists of $5$ randomly sampled classes from a meta-set. Within each class, we have either $1$ example ($1$-shot) or $5$ examples ($5$-shot) as context, and $15$ examples as target. For all experiments, hyperparameters are chosen by training on the training meta-set, and comparing target accuracy on the validation meta-set. We conduct randomised hyperparameters search \citep{bergstra2012random}, and the search space is given in Appendix. Then with the model configured by the chosen hyperparameters, we train on the union of the training and validation meta-sets, and report final target accuracy on the test meta-set.

In \Cref{tb:classification} we compare our approach to other meta-learning methods. The numbers presented are the mean and standard deviation of $5$ independent runs. The table demonstrates that our model outperforms previous state-of-the-art on 1-shot and 5-shot classification tasks for the more challenging tieredImageNet.
As for miniImageNet, we note that previous work, such as MetaOptNet-SVM \citep{lee2019meta}, used significant data augmentation to regularise their model and hence achieved superior results. For a fair comparison, we also equipped each model with data augmentation and reported accuracy with/without data augmentation.
However, MetaOptNet-SVM \citep{lee2019meta} uses a different data augmentation scheme involving horizontal flip, random crop, and color (brightness, contrast, and saturation) jitter. On the other hand, MetaFun, \citet{qiao2018few} and LEO \citep{rusu2018meta}, only use image features averaging representation of different crops and their horizontal mirrored versions.
In 1-shot cases, MetaFun matches previous state-of-the-art performance, while in 5-shot cases, we get significantly better results. In \Cref{tb:classification}, results for both MetaFun-DFP (using dot-product attention) and MetaFun-KFP (using deep kernels) are reported. Although both of them demonstrate state-of-the-art performance, MetaFun-KFP generally outperforms MetaFun-DFP for 5-shot problems, but performs slightly worse for 1-shot problems.

\subsection{Ablation Study}

\begin{table*}[!htb]
  \centering
  \footnotesize
  \begin{threeparttable}[]
  \caption{Ablation Study. We conduct independent randomised hyperparameter search for each number presented, and reported means and standard deviations over 5 independent runs for each.}
  \label{table:few_shot}
  \begin{tabular}{ccc|cc|cc}
    \toprule
    Functional &  Local update &  \multirow{2}{*}{Decoder} &  \multicolumn{2}{c|}{\textbf{MiniImageNet}} & \multicolumn{2}{c}{\textbf{tieredImageNet}} \\
     pooling &  function &  & 1-shot   & 5-shot \\
    \midrule
    Attention & NN & \cmark & \bm{$62.12\pm0.30\%$} & $77.78\pm0.12\%$ & \bm{$67.72\pm0.14\%$} & $82.81\pm0.15\%$  \\
    Deep Kernel & NN& \cmark & $61.16\pm0.15\%$ & \bm{$78.20\pm0.16\%$} & $67.27\pm0.20\%$ & \bm{$83.28\pm0.12\%$} \\
    Attention & Gradient & \cmark & $59.63\pm0.19\%$ & $75.84\pm0.04\%$  & $62.55\pm0.10\%$ & $78.18\pm0.09\%$ \\
    Deep Kernel & Gradient & \cmark & $59.73\pm0.21\%$ & $76.41\pm0.14\%$ & $65.24\pm0.11\%$ & $80.31\pm0.16\%$ \\
    \midrule
    SE Kernel & NN & \cmark & $60.04\pm0.19\%$ & $75.25\pm0.12\%$  & $60.81\pm0.30\%$ & $79.70\pm0.20\%$ \\
    Deep Kernel & Gradient & \xmark & $57.67\pm0.16\%$ & $73.55\pm0.04\%$ & $62.53\pm0.17\%$ & $76.86\pm0.07\%$ \\
    \bottomrule
  \end{tabular}
  \label{tb:ablation}
  \end{threeparttable}
\end{table*}

\begin{figure*}
    \includegraphics[width=0.99\linewidth]{iterations.pdf}
    \caption{This figure illustrates the accuracy of our approach for varying number of iterations $T=1, \dots,6$, over different few-shot learning problems. For each problem, we use the same configuration of hyperparameters except for the number of iterations and the choice between attention and deep kernels. Error bars (standard deviations) are given by training the same model $5$ times with different random seeds.}
\label{fig:iteration}
\end{figure*}

As stated in \Cref{sub:regression-and-classification}, our model has three learnable components: the local update function, the functional pooling, and the decoder. In this section we explore the effects of using different versions of these components. We also investigate how the model performance would change with different numbers of iterations.

\Cref{tb:ablation} demonstrates that neural network parameterised local update functions, described in \Cref{sub:learning task representation}, consistently outperforms gradient-based local update function, despite the latter having build-in inductive biases.
Interestingly, the choice between dot-product attention and deep kernel in functional pooling is problem dependent. We found that MetaFun with deep kernels usually perform better than MetaFun with dot product attention on $5$-shot classification tasks, but worse on $1$-shot tasks. We conjecture that the deep kernel is better able to fuse the information across the 5 images per class compared to attention. In the comparative experiments in \Cref{sub:imagenet} we reported results on both.

In addition, we investigate how a simple Squared Exponential (SE) kernel would perform on these few-shot classification tasks. This corresponds to using an identity input transformation function $a$ in deep kernels. \Cref{tb:ablation} shows that using SE kernel is consistently worse than using deep kernels, showing that the heavily parameterised deep kernel is necessary for these problems.

Next, we looked into directly applying functional gradient descent with parameterised deep kernel to these tasks. This corresponds to removing the decoder and using deep kernels and gradient-based local update function (see \Cref{sec:related_work}). Unsurprisingly, this did not fare as well, given as it only has one trainable component (the deep kernel) and the updates are directly applied to the predictions rather than a latent functional representation.

Finally, \Cref{fig:iteration} illustrates the effects of using different numbers of iterations $T$. On all few-shot classification tasks, we can see that using multiple iterations (two is often good enough) always significantly outperform one iteration. We also note that this performance gain diminishes as we add more iterations. In \Cref{sub:imagenet} we treated the number of iterations as one of the hyperparameters.

\section{Conclusions and Future Work}

In this paper, we propose a novel functional approach for meta-learning called MetaFun. The proposed approach learns to generate a functional task representation and an associated functional update rule, which allows to iteratively update the task representation directly in the function space.
We evaluate MetaFun on both few-shot regression and classification tasks, and
demonstrate that it matches or exceeds previous state-of-the-art results on miniImageNet and tieredImageNet few-shot classification tasks.

Interesting future research directions include
a) exploring a stochastic encoder and hence working with stochastic functional representations, akin to the \glsreset{NP}\gls{NP},
and b) using local update functions and the functional pooling components whose parameters change with iterations instead of sharing them across iterations, where the added flexibility could lead to further performance gains.

\section*{Acknowledgements}

We would like to thank Jonathan Schwarz for valuable discussion, and the anonymous reviewers for their feedback. Jin Xu and Yee Whye Teh acknowledge funding from Tencent AI Lab through the Oxford-Tencent Collaboration on Large Scale Machine Learning project. Jean-Francois Ton is supported by the EPSRC and MRC through the OxWaSP CDT programme (EP/L016710/1).

\clearpage
\newpage

redpajama_set_name: RedPajamaArXiv


text:
@support
Speare Support posted on Feb 13 2017 3:08AM
Speare Background
https://speare.com
Kent Sisco's vision for Speare
Speare is an easy-to-use, web-based publishing platform where you can write and publish your writing online.
It's an "idea processor" that can be used as "a second brain" or a personal knowledge base. It makes your ideas tangible.
"Tools should enhance, not limit, your thinking." – Kent Sisco
The limitations of existing tools
Most linear writing programs don't help people to structure their thoughts. They may make writer's block worse because their complexity and learning curve creates a large amount of "friction," which gets in the way of capturing their ideas at the speed of thought.
Most word processing and writing applications were never designed to capture and organize ideas. Their primary purpose is to record your writing. That means content creators must think in complete sentences, paragraphs and narratives, and record their thoughts in a fairly rigid sequence. That's hard for many business people, entrepreneurs and writers of all stripes.
Existing tools make it hard to see the structure of your information. It's hidden in sentences and paragraphs and an endlessly scrolling text experience that's intimidating to many people.
How Speare enables creative thinking and problem solving
Speare enables emergent structure - the ability to capture ideas in a free-form manner, and then let patterns and structure gradually emerge. Why is that important? Because forcing ideas into a structure too quickly tends to restrict one's thinking. As this structure emerges, users should be able to discern and fill gaps in their information.
Speare gives its users the ability to think through and solve problems, and to discover new insights and connections along the way.
Speare is ideal for multi-threaded thinkers – people who tend to have multiple streams of thinking going on in their brains at the same time.
By forcing users to capture their ideas in concise, self-contained building blocks, Speare enables succinct thinking.
Characteristics of the ideal idea capture tool: Many text-based tools require multiple steps to capture ideas, which diminishes the odds that you'll do so. The ideal idea capture tool needs to have as little friction as possible. Idea capture should be easy and fast. Once you've captured a constellation of ideas, it should be just as easy to modify them, add to them and arrange them in ways that make the most sense to you. This definition describes Speare very accurately!
How Speare enables writing
Speare streamlines the process of writing and assembling content, drafts, and documents, using a building block approach to writing
Building blocks make the structure of your ideas visible.
Side-by-side boards enable users to build narratives out of multiple collections of information, ideas and knowledge.
Everything you write in Speare becomes part of your searchable thought library.
How Speare supports remote work
Telecommuters have the freedom to think differently, more creatively and in new ways, outside of the office.
Speare enables that by giving remote workers more ways to interact with concepts, data and ideas.
Kent Sisco
Speare, LLC
kent@speare.com

redpajama_set_name: RedPajamaCommonCrawl


Thinking about a Custom Print?
We offer full custom posters, design specifically for you from your brief, and ONLY available to you.
How much you say? Well, it depends on complexity, but as a ballpark they are around the $550-1,900 mark for the design/illustration component (includes 1 x A3 print) and additional prints are available at wholesale rates at the time of initial order.
Any questions, just get in touch, happy to talk through the process.
We are also offering a "Semi Custom" print.
Based on our bidon series or Team Print, you can have a custom design to "fill the gaps".
Love a certain team, your cycling club, bike, or just a favourite bit of kit, you can now have some personalised wall love.
Rest of the World add $40 postage.
This price includes the design and 1 x A3 print.
Here are some samples of what you can do!


text:
Q: Jasmine testing setting component variable I'm using spyOn, in a jasmine test, to "listen" to a function call from a service,
that function returns an Observable.
I'm getting the error unexpected token U JSON;
The error is generated from the component line:
this.config = JSON.parse(localStorage.getItem('configuration'));

The localStorage item was JSON.stringified;
I understand that this error is usually thrown when JSON.parse = undefined,
So I tried to set the variable within my test i.e.
component.config =  mockConfig;

So..
 // Storage Mock
  function storageMock() {
    var storage = {};
      return {
      setItem: function(key, value) {
        storage[key] = value || '';
      },
      getItem: function(key) {
        return key in storage ? storage[key] : null;
      },
      removeItem: function(key) {
        delete storage[key];
      },
      get length() {
        return Object.keys(storage).length;
      },
      key: function(i) {
        var keys = Object.keys(storage);
        return keys[i] || null;
      }
    };
  }
  let mockConfig = JSON.stringify({
    base_url:"http://image_url/",
    poster_sizes:['w9', 'w100']
  })
  //Set storage
 let m =  storageMock()
 m.setItem('configuration', mockConfig)

  it('Should set items array with values from MoviesService', () => {
    component.config = JSON.parse(m.getItem('configuration'));
       let spy = spyOn(moviesService, 'getPreview').and.callFake(()=>{
        return Observable.from([[{id1: 1, title: 'a'}, {id1: 2, title: 'b'}]])
  })
  component.ngAfterViewInit();
  expect(component.items.length).toBeGreaterThan(0);
});

A: For anyone who come across this problem I got this working by  by placing this code in the beforeEach wrapper
Object.defineProperty(window, 'localStorage', { value: m });

Basically it uses my mock localStorage variable instead of the one from the window object.

redpajama_set_name: RedPajamaStackExchange


text:
using System;
using System.Collections.Generic;

namespace GameFramework
{
    public static class MathExtensions
    {
        public static float Square(this float val)
        {
            return (float)Math.Pow(val, 2);
        }

        public static double Square(this double val)
        {
            return Math.Pow(val, 2);
        }

        public static float SquareRoot(this float val)
        {
            return (float)Math.Pow(val, 0.5);
        }

        public static double SquareRoot(this double val)
        {
            return Math.Pow(val, 0.5);
        }

        public static Vector RoundTo(this Vector vector, int digits)
        {
            return new Vector((float)Math.Round(vector.X, digits), (float)Math.Round(vector.Y, digits));
        }

        public static Vector TranslatePolar(this Vector vector, float radAngle, float distance)
        {
            return vector + new Vector((float)Math.Cos(radAngle) * distance, (float)Math.Sin(radAngle) * distance);
        }

        public static Vector TranslatePolar(this Vector vector, float radAngle, Vector translationVector)
        {
            return TranslatePolar(vector, translationVector.GetAngle() + radAngle, translationVector.Length);
        }

        public static Rectangle TranslatePolar(this Rectangle rectangle, float radAngle, float distance)
        {
            return rectangle.Translate(new Vector((float)Math.Cos(radAngle) * distance, (float)Math.Sin(radAngle) * distance));
        }

        public static Rectangle TranslatePolar(this Rectangle rectangle, float radAngle, Vector translationVector)
        {
            return TranslatePolar(rectangle, translationVector.GetAngle() + radAngle, translationVector.Length);
        }

        public static Vector Translate(this Vector vector, Vector delta)
        {
            return new Vector(vector.X + delta.X, vector.Y + delta.Y);
        }

        public static Vector Translate(this Vector vector, float offsetX, float offsetY)
        {
            return new Vector(vector.X + offsetX, vector.Y + offsetY);
        }

        public static Vector Scale(this Vector vector, float zoomFactor)
        {
            return new Vector(vector.X * zoomFactor, vector.Y * zoomFactor);
        }

        public static Vector Scale(this Vector vector, Vector scalingVector)
        {
            return new Vector(vector.X * scalingVector.X, vector.Y * scalingVector.Y);
        }

        public static Vector Clamp(this Vector vector, Rectangle rectangle)
        {
            return MathUtil.Clamp(vector, rectangle);
        }

        public static Point Translate(this Point point, Point delta)
        {
            return new Point(point.X + delta.X, point.Y + delta.Y);
        }

        public static Vector ToVector(this Point point)
        {
            return new Vector(point.X, point.Y);
        }

        public static Rectangle Scale(this Rectangle rectangle, float factor)
        {
            return new Rectangle(
                rectangle.X * factor,
                rectangle.Y * factor,
                rectangle.Width * factor,
                rectangle.Height * factor);
        }

        public static Rectangle Translate(this Rectangle rectangle, Vector vector)
        {
            return new Rectangle(rectangle.X + vector.X, rectangle.Y + vector.Y, rectangle.Width, rectangle.Height);
        }

        public static Rectangle Round(this Rectangle rectangle)
        {
            return new Rectangle((float)Math.Round(rectangle.X), (float)Math.Round(rectangle.Y),
                (float)Math.Round(rectangle.Width), (float)Math.Round(rectangle.Height));
        }

        public static bool Intercept(this Rectangle rectangle, Vector vector)
        {
            return vector.X >= rectangle.X
                && vector.Y >= rectangle.Y
                && vector.X < rectangle.X + rectangle.Width
                && vector.Y < rectangle.Y + rectangle.Height;
        }

        public static Color ChangeAlpha(this Color color, int alpha)
        {
            return new Color(color.R, color.G, color.B, alpha);
        }

        public static void ForEachPair<T>(this IEnumerable<T> enumerable, Action<T, T> action)
        {
            var first = default(T);
            var last = default(T);
            foreach (var item in enumerable)
            {
                if (object.Equals(first, default(T)))
                    first = item;
                else
                    action(last, item);

                last = item;
            }

            action(last, first);
        }
    }
}

redpajama_set_name: RedPajamaGithub


text:
from nose.tools import *
from dateutil.parser import parse as time_parse
import yawhois

class TestWhoisRegisterComStatusRegistered(object):

    def setUp(self):
        fixture_path = "spec/fixtures/responses/whois.register.com/status_registered.txt"
        host         = "whois.register.com"
        part         = yawhois.record.Part(open(fixture_path, "r").read(), host)
        self.record  = yawhois.record.Record(None, [part])

    def test_status(self):
        eq_(self.record.status, 'registered')

    def test_available(self):
        eq_(self.record.available, False)

    def test_domain(self):
        eq_(self.record.domain, "register.com")

    def test_nameservers(self):
        eq_(self.record.nameservers.__class__.__name__, 'list')
        eq_(len(self.record.nameservers), 6)
        eq_(self.record.nameservers[0].__class__.__name__, 'Nameserver')
        eq_(self.record.nameservers[0].name, "ns2.register.com")
        eq_(self.record.nameservers[1].__class__.__name__, 'Nameserver')
        eq_(self.record.nameservers[1].name, "ns3.register.com")
        eq_(self.record.nameservers[2].__class__.__name__, 'Nameserver')
        eq_(self.record.nameservers[2].name, "ns4.register.com")
        eq_(self.record.nameservers[3].__class__.__name__, 'Nameserver')
        eq_(self.record.nameservers[3].name, "ns5.register.com")
        eq_(self.record.nameservers[4].__class__.__name__, 'Nameserver')
        eq_(self.record.nameservers[4].name, "ns6.register.com")
        eq_(self.record.nameservers[5].__class__.__name__, 'Nameserver')
        eq_(self.record.nameservers[5].name, "ns1.register.com")

    def test_admin_contacts(self):
        eq_(self.record.admin_contacts.__class__.__name__, 'list')
        eq_(len(self.record.admin_contacts), 1)
        eq_(self.record.admin_contacts[0].__class__.__name__, 'Contact')
        eq_(self.record.admin_contacts[0].type, yawhois.record.Contact.TYPE_ADMINISTRATIVE)
        eq_(self.record.admin_contacts[0].name, "Domain Registrar")
        eq_(self.record.admin_contacts[0].organization, "Register.Com, Inc.")
        eq_(self.record.admin_contacts[0].address, "575 8th Avenue")
        eq_(self.record.admin_contacts[0].city, "New York")
        eq_(self.record.admin_contacts[0].zip, "10018")
        eq_(self.record.admin_contacts[0].state, "NY")
        eq_(self.record.admin_contacts[0].country_code, "US")
        eq_(self.record.admin_contacts[0].phone, "+1.9027492701")
        eq_(self.record.admin_contacts[0].fax, None)
        eq_(self.record.admin_contacts[0].email, "domainregistrar@register.com")

    def test_registered(self):
        eq_(self.record.registered, True)

    def test_created_on(self):
        eq_(self.record.created_on.__class__.__name__, 'datetime')
        eq_(self.record.created_on, time_parse('1994-11-01T00:00:00-0500'))

    def test_registrar(self):
        eq_(self.record.registrar.__class__.__name__, 'Registrar')
        eq_(self.record.registrar.id, "9")
        eq_(self.record.registrar.name, "Register.com")
        eq_(self.record.registrar.url, "http://www.register.com")

    def test_registrant_contacts(self):
        eq_(self.record.registrant_contacts.__class__.__name__, 'list')
        eq_(len(self.record.registrant_contacts), 1)
        eq_(self.record.registrant_contacts[0].__class__.__name__, 'Contact')
        eq_(self.record.registrant_contacts[0].type, yawhois.record.Contact.TYPE_REGISTRANT)
        eq_(self.record.registrant_contacts[0].name, "Domain Registrar")
        eq_(self.record.registrant_contacts[0].organization, "Register.Com, Inc.")
        eq_(self.record.registrant_contacts[0].address, "575 8th Avenue")
        eq_(self.record.registrant_contacts[0].city, "New York")
        eq_(self.record.registrant_contacts[0].zip, "10018")
        eq_(self.record.registrant_contacts[0].state, "NY")
        eq_(self.record.registrant_contacts[0].country_code, "US")
        eq_(self.record.registrant_contacts[0].phone, "+1.9027492701")
        eq_(self.record.registrant_contacts[0].fax, None)
        eq_(self.record.registrant_contacts[0].email, "domainregistrar@register.com")

    def test_technical_contacts(self):
        eq_(self.record.technical_contacts.__class__.__name__, 'list')
        eq_(len(self.record.technical_contacts), 1)
        eq_(self.record.technical_contacts[0].__class__.__name__, 'Contact')
        eq_(self.record.technical_contacts[0].type, yawhois.record.Contact.TYPE_TECHNICAL)
        eq_(self.record.technical_contacts[0].name, "Domain Registrar")
        eq_(self.record.technical_contacts[0].organization, "Register.Com, Inc.")
        eq_(self.record.technical_contacts[0].address, "575 8th Avenue")
        eq_(self.record.technical_contacts[0].city, "New York")
        eq_(self.record.technical_contacts[0].zip, "10018")
        eq_(self.record.technical_contacts[0].state, "NY")
        eq_(self.record.technical_contacts[0].country_code, "US")
        eq_(self.record.technical_contacts[0].phone, "+1.9027492701")
        eq_(self.record.technical_contacts[0].fax, None)
        eq_(self.record.technical_contacts[0].email, "domainregistrar@register.com")

    def test_updated_on(self):
        eq_(self.record.updated_on.__class__.__name__, 'datetime')
        eq_(self.record.updated_on, time_parse('2009-08-26T00:00:00-0400'))

    def test_domain_id(self):
        eq_(self.record.domain_id, None)

    def test_expires_on(self):
        eq_(self.record.expires_on.__class__.__name__, 'datetime')
        eq_(self.record.expires_on, time_parse('2019-08-04T00:00:00-0400'))


text:
package org.tzi.context.client;

import org.tzi.context.client.ContextClient.CommunicationState;

public interface ContextClientListener {

	public void processCommunicationState(CommunicationState state);
	public void processContextInformation(String message);
	public void processCommandResult(String command, String result);
}


text:
DBSS flat at Natura Loft in Bishan sold for $1.2m
In August this year, a similar-sized unit at Natura Loft was sold for about $1.08 million. TNP FILE PHOTO
120 sq m five-roomer is believed to have fetched highest resale price for a Design, Build and Sell Scheme unit
Adeline Tan
Flats at Natura Loft in Bishan continue to fetch a handsome price, with the recent sale of a five-room unit there again crossing the $1 million mark.
According to data from the Housing Board, the flat, with just under 90 years of its lease left, was sold for $1,208,000 this month.
It is believed to be the highest resale price for a Design, Build and Sell Scheme (DBSS) unit, Chinese newspaper Shin Min Daily News reported yesterday.
The 120 sq m unit is believed to be between the 28th and 30th storeys.
The previous record price for a DBSS unit was the $1,205,000 paid for a five-room flat on the 39th-storey at City View in Boon Keng.
In August this year, a similar-sized unit at Natura Loft, also on a high floor, sold for about $1.08 million.
HDB launches portal for flat pricing and loan information
Resale condo prices climb for fifth straight month in December
House in Yang Yin saga up for sale again at lower asking price of $25m
Record 82 HDB resale flats sold for $1 million last year
In 2017, a unit there made headlines after it fetched a record price of $1.18 million.
DBSS flats are Housing Board flats designed, built and sold by private developers.
Experts who spoke to The New Paper said the new record is not a surprise, given that the housing market has been on the rebound after the circuit breaker.
Ms Christine Sun, head of research and consultancy at OrangeTee & Tie, said the uptick in price is in tandem with what has been happening across all segments of the housing market.
"The sales volume has been going up and so are the prices at certain locations. We see this in the private housing market as well. Generally, that is where the trend is going," Ms Sun told The New Paper.
Flash estimates from real estate portal SRX revealed last Thursday that prices of Housing Board resale flats increased for the third consecutive month last month.
Housing Board resale prices rose by 0.8 per cent last month compared with August, a faster rate than the previous two months.
In all, 2,489 HDB flats were sold last month, a 2.2 per cent increase from August.
Eight resale flats also sold for above $1 million, one of them a five-room flat at The Pinnacle@Duxton in Tanjong Pagar, which sold for $1.258 million.
It is believed to be a record for a resale HDB flat.
Referring to the recent sale at Natura Loft, Mr Ismail Gafoor, chief executive officer of PropNex, said the location in Bishan is an attractive proposition.
He said: "That property is in a very good location, in terms of very good schools, and is also very centrally situated."
Both Ms Sun and Mr Ismail said the previous owners would have bought the flat at a premium price from the developers, which Mr Ismail estimated to be $600,000 and upwards.
Mr Ismail said he will not be surprised if the current record is broken in the coming months, although he is not expecting a huge increase.
"Today, the HDB market has taken a rebound and the private property prices are also positively increasing. Therefore, DBSS (flats), based on design, are seen as a value proposition."
tanpya@sph.com.sg
Read articles by Adeline Tan


Hi, I'm Dynamo Dog The Superhero and because of my super soft plush fur, people love to cuddle me (which I love). I am 9 inches (22cm) high when sitting and 15 inches (39cm) from head to toe.
My soft polyester fibre stuffing makes me super soft and cuddly. I comply to EN71 Safety Standards, so am safe for your little ones to play with. My eyes are embroidered so am suitable from birth and my cape and mask can't be removed.
We can ONLY embroider a first name on the BACK of the cape. If you would like the name to match the star then please select ORANGE. The longer the name, the smaller the text size will be.
Your chosen name will be expertly embroidered. We pride ourselves on providing very high quality gifts.
*Would you like to personalise the CAPE?
It is beautiful. Really soft and it's great quality.
The personalised cape is a really lovely touch.
Would definitely order from here again.


This auction is for a Brand New set of Original, Tequipment, OEM, Factory, Geninue Porsche 19" CAYENNE DESIGN wheels, Pirelli PZero Tires and Porsche OEM Black Crest Center Cap set for Cayenne, Cayenne S, Cayenne GTS and Cayenne Turbo. Porsche Cayenne Design Wheel features five dual-arm spokes and a 10-inch width at both front and rear wheel. The auction is for Wheels, Tires and Porsche Black Crest Center Caps. These set of wheels and Tires are in New condition with no chips or ware. There is no structural damage such as bends, cracks, curb rust or scratches. The wheels are straight, round and true. The wheels has the Porsche Color Logo Center Caps. The wheels will fit all Porsche Cayenne from 2003 to 2011 . This Wheel is very nice when mounted on a Porsche Cayenne and are very rear to find especially one that comes with Pirelli PZero Rosso Tires. Wheels and Tires are in New shape and condition.
Front Wheel: Part #: 955.044.600.21 Size: 9Jx 19 Offset: 50 Front Tire Size: 275/45/19 Pirelli PZero Rosso Speed Rating 108Y N0. Tire Tread Remaining: 100%.
The Front Tires has 100% of Tread Remaining and the Rear Tires has 100% Tread Remaining.
The total cost of these wheels, Tires and the Center Caps is $6495.50 plus tax. This is a nice opportunity to dress up your Porsche at a reasonable cost.


Adventure Time is a comedy fantasy cartoon show that first aired on April, 5th 2013 on Cartoon Network. The show has amassed an almost cult-like fanbase that only grew after the impeccable grand finale. The show finally came to the grand conclusion on September, 17th 2017. The lasting appeal of this timeless show was cemented with the release of character-themed box sets sold on Amazon.
Adventure Time did so well as a show for many reasons, the reason that stands out the most to me was way the writers developed the characters. All the character started off as juvenile happy go lucky kids and ended as conscious and self-aware young adults.
The character development also speaks to the progression of the fan base age. If you were a diehard fan like myself, you started watching the show as a 12-year-old and you finished watching as a 16-year-old. The writers wanted to keep the same fan base and appeal to the worldview of the age group appropriately, and they succeeded.
Another thing aspect of this show that stands out is the character designs. The protagonist, Finn, is the classic hero and sticks out like a sore thumb considering he's one of the only humans in the whole show. The second protagonist, Jake, is a dog with special stretchy powers who is always good for getting out of a sticky situation.
Jake balances out Finn's hero complex well by being the lazy, gruff, realist who knows way more about the complexities of life than he lets on. Jake is played by cartoon legend John DiMaggio. DiMaggio has been in Futurama, two Batman animated series, and Zootopia.
Adventure Time is a great light-hearted show that features great environments, characters, and character development. The appeal is universal if you are a younger viewer the newer seasons are perfect and as an older view, the later seasons are great.


Does the corpus iuris civilis show characteristics of an abstract system of transfer of property?
El estudio versa sobre la estructura o sistema de las exposiciones acerca de la stipulatio convencional en diversas obras de la jurisprudencia romana. Se propone que la base estuvo en el tratado sobre la materia que aparece en los iuris civilis libre tres de Sabino y que de él dependen en la materia tanto las obras del tipo digesta, quaestiones, responsa, sententiae y similares que desde Celso empezaron a escribir los juristas clásicos, como las institutiones de Gayo (y después las de Justiniano). Se examina también la estructura que ofrece el tema en el codex Iustinianus y en los Iustiniani digesta. This study is about the structure or system of the expositions about conventional stipulatio in several works of the Roman jurisprudence. It is suggested that the treatise on the topic appearing in Sabin's iuris civilis libre tres was the base, and that in the matter the works of the type digesta, quaestiones, responsa, sententiae and similar depend on it that, since Cels, the classic jurists started writing, such as Gayo's institutiones (and after that, Justinian's). The structure offered by the topic in the codex Iustinianus and in the Iustiniani digesta is also examined.


In a blender or food processor, pulse bananas until smooth. Divide sliced strawberries evenly among ice pop molds. Pour the pureed bananas into the molds. Freeze for 4-6 hours or overnight.
< Did OPI said Hello Kitty collection??


holistic and a constructivist view of children, childhood and teaching.
A holistic view of the development of children is seen as a dynamic and closely interwoven interaction between their physical and mental circumstances and the environment in which they grow up. Children are social players who themselves contribute to their own and other children's learning. Interaction with other people is crucial to children's development and learning.
Childhood is a phase of life with an intrinsic value, in the same way as other phases of human life.
One of the characteristics of childhood is interaction through play, which provides scope for initiative, imagination and enthusiasm. Children wonder about things and ask questions, seek adventures and gain experience in their own learning arenas. In close interaction with their surroundings, they are active in all areas of life. The care, upbringing, play and education that children experience at a young age shape their attitudes, values and confidence in themselves and in other people, and their motivation for learning later in life.
The UN Convention on the Rights of the Child emphasizes the fact that children are entitled to express their own views on everything that effects them. That their views should always be taken into consideration, and to influence on all aspects of their lives at kindergartens. The degree of participation will depend on the age and level of function of the child.
Children express their feelings through body language and through words. They express their views through body position, mimic and other forms of emotional expression.Preschool Staff will listen to and attemped to interpret their body language, and are observant in relation to their action, aesthetic expression and evaluate their verbal and non-verbal communication shown through play and in daily activities. Preschool Staff take children´s participation seriously, and the Staff will see each child individually and as a group. This requires good communication between children and staff. Children right to participate requires time and space for listening and talking. That means the Pedagogical activities must be planed and organized in a way that there is time and place for the children to participate.


Happy Monday! In case you all missed them, here are some posts from this weekend!
So yesterday definitely contained the good, the bad, and the ugly. I feel like sometimes when I'm blogging that I try and gloss over days when I'm doing so well. I mean, who wants to write about/read about unhappy things, when I have other great things to share?
But then I thought about it, and realized that I always appreciate when other bloggers are honest about how they're feeling, because I realize that everyone has bad days and that NO ONE is perfect. Even me!
I grew up in a bit of a "glossy" world, where you always pretended everything was great, even if it wasn't. I tend to do this because I don't want to dump any of my unhappiness on other people, but what I need to realize is that other people are there for support and won't just run away because I'm not perfect all the time.
I had a bad day and I'm still not feeling great about it. I couldn't do my 7 mile run yesterday because my knee was still bothering me and I think that sent me into a funk! I hate not being able to run! I decided to try out swimming laps at the pool, since I figured that wouldn't put pressure on my knee. I actually really enjoyed it. However, I couldn't find my goggles before I left (I think they got tossed when I moved from Charleston), so by the time I left the pool my eyes were completely swollen and red and my vision was blurry for hours! It hurt!
When I got back I made myself some pumpkin oats, and then was surprisingly productive for not being able to see well!
Like my mini pumpkin? I bought it at the farmers market yesterday to bring some Fall cheer to my apartment!
Later in the day I decided to make some low-fat pumpkin cookies that I found online. They were super easy to make and had very few ingredients.
Unfortunately, they didn't taste very great. They were fine right in the middle, but the outsides were dry and they were a bit tasteless. I decided to toss them after eating one (I was going to take them to my brother) but of course while throwing them out I ended up picking at them and probably ate a total of 3 or 4 in the course of things… I was feeling anxious and as I've said before, when I'm anxious, I eat!
Does anyone else do this? In reality I didn't eat much, but my stomach had been really bothering me yesterday and every time I ate anything I felt full really quickly and my stomach started to hurt. I think this added to my frustration.
I was also anxious because I've gained about a pound or two. I know this sounds like nothing (and it ISN'T) but when I can't run for stress relief and I'm trying to add healthy fats back in and not calorie count, it makes me feel out of control and anxious! I'm trying so hard to not let it bother me, but clearly deep down it does!
After visiting Molly, which ALWAYS makes me feel better, I headed home to attempt dinner. I'd been seeing people make savory oats a lot on blogs, so I thought I would give it a whirl. It was DELICIOUS! (another "good" of the day).
I made each element separately and then combined them together. The beans I microwaved, the veggies I cooked in a skillet and then added the apple cider vinegar and Braggs when it was almost done, and the oats I cooked on the stove. I used Unsweetened Vanilla So Delicious Coconut milk for the oats (1/3 cup milk, 1/3 cup water) and I really think that added a great flavor!
So you'd think this dinner, plus some yogurt after for dessert would be enough, right? Nope. I discovered how to make my own Cinnamon Toast Crunch Cereal (added Stevia and cinnamon to plain bran flakes) and literally ate cereal until I felt sick.
We call this binge eating. And that's why I feel so awful today. Part of me is horribly embarrassed to share this, especially since I know a lot of the people who read this blog, but I don't want to be "glossy," I want to be honest.
In reality I probably didn't eat that much, maybe two servings of cereal, but I ate even though my stomach hurt! I know the advice, take a walk around the block, call someone, distract yourself, but I didn't. We are all human, we all make mistakes, and so do I . I need to remind myself that I'm NOT PERFECT! I had a slip up! I used to do this all the time and haven't in a while. Time to pull myself up, brush myself off, and move forward.
Does anyone else feel like they need to seem perfect all the time? Or have a hard time being honest about when they have a bad day?
Also, any suggestions to improve my mood today? What do you do to make yourself feel better?


text:
Q: The strip command works "too much" it even erases what is not supposed to be erased I'm trying to use the.strip() function to remove a part of the text in several variables.
Here is my code :
image_list = [] # on créer un tableau dans lequel on va mettre toutes les images
for filename in glob.glob(chemin+"\*.jpg" ): # I browse all.jpg files that are in the directory indicated by the variable "chemin".
   im=Image.open(filename)
   image_list.append(im) # I add the image to a table
   print("\n",filename, "\n", chemin,"\n", filename.strip(chemin),"\n\n") # Here is the test I do to see the different texts and the text where I use the strip function

And this is what I get :
D:\Prog&Job\Ebay\Produit\Beyblade\toupie + boitier_bleu\images secondaire\boitier_8.JPG
D:\Prog&Job\Ebay\Produit\Beyblade\toupie + boitier_bleu\images secondaire
8.JPG
First there is the file name and path, then the directory name and then the path, and when I try to strip the file to remove all the path, it removes the beginning
(here: D:\Prog&Job\Ebay\Produit\Beyblade\toupie + boitier_bleu\images secondaire ), but it also removes "boitier_"
Do you know why?
Thank you in advance
EDIT : Here is the solution I wanted :
>>> import os
>>> full_path = '/book/html/wa/foo/bar/'
>>> print os.path.relpath(full_path, '/book/html')
'wa/foo/bar'

it's an example from this : How to remove a path prefix in python?

A: strip does not remove a certain substring found as prefix/suffix. It removes any characters found in the string passed to it that are at the start or end of the string. Note:
"TESTED SETS".strip("TES") # => "D " (T, E and S removed from front and back)

You can remove a prefix substring in any number of ways. This is one:
if string.startswith(prefix):
    string = string[len(prefix):]


text:
Q: How to save values with PHP in dynamic memory? I need to create a simple form which saves it's values in a dynamic memory. So far the code goes like this:

                    Code:
                    <input type="text" name="code" />

                    Name:
                    <input type="text" name="name" />

                    Inventory number
                    <input type="text" name="number" />

                    Value
                    <input type="text" name="value" />

                    Buy value
                    <input type="text" name="buy_value" />

                    <input type="submit" name="add" value="Add" />

So how can I add it not to a database, but to something like a multidimensional array or something like that, and I must be able to delete a selected record afterwards.

A: You can save it in session data. This, however, will be lost upon session end (say if user closes the browser).
session_start();
$_SESSION = array_merge($_SESSION, $_POST);

I used array_merge so that previous data you might had there won't be lost.

A: you can submit your form data into a file that displays them in tables, arrays and etc and yet you can delete the selected row or index or etc using jquery(javascript) functions and then what? where is the form data going to go? no sustainable storage? loosing data on page change?

A: Im not sure exactly what you are going for but if the code that you posted is inside of a form and the forms action is to post data to your php script then
  extract($_POST);

will store the values of each input field into a php scalar of the same name as the input field's name. From there you can easily go about storing the data into some type of array.


This is a simple guide on how to record videos from your fights and upload them on the forum.
A big "Thank you!" to @Mortal_Kombatnt for the iOS part of the tutorial.
Since many Android devices do not include a native screen-recording app, i'm going for a universal method that will work on all of them.
I'm pointing out that I don't have any connections with the app's developer.
I just found it the most useful and i'm using it for the tutorial.
Feel free to use any screen recording app.
Once you've installed the app, when you open it, the options will pop out as seen in the picture below.
2. Press the record button to start your video. A countdown will begin and your recording will start.
3. Press the stop button to finish.
A new native feature with iOS 11 is Screen Recording. I've tested different a couple of different resolution & fps settings, but not all. One setting that should be maintained is High Efficiency. High Efficency is a new format container for video and photos on iOS, iOS also gives you Most Compatible. I've tried Most Compatible when recording videos and when transferred to a Windows machine the audio does not sync, only in High Efficency mode will it playback videos correctly on Windows.
To reach this setting go to: Settings > Camera > Formats and choose High Efficency. It might already be set to HE by default.
1.) First, go to Settings > Control Center > Customize Controls.
3.) Now lift up the Control Center by swiping up from the bottom of the display and tap the newly added Screen Recording button. You'll first see a 3 second timer after which the recording will begin. Press the button again to stop the recording and the movie will be saved to the Camera Roll.


The Naot Black Leather Mark Console is a lace up shoe with a padded tongue and padded heel cup for extra comfort & support. This style features padded technical lining for comfort and moisture absorption. Naot's removable, anatomic cork & latex footbed is wrapped in pampering suede and molds to the shape of the foot with wear. The Men's Console Collection features a 1.5" polyurethane sole which is extremely lightweight, durable and slip resistant. This collection also features a stiffener in the toe box to avoid losing shape.


text:
Іван Лаврентійович Бороздна (близько 1680 — близько 1731, с. Медведове, нині Клинцівського району, Брянської області, Російська Федерація) — значковий товариш Стародубського полку (1709—1715), наказний полковник Стародубського полку (1722, 1724—1725), бунчуковий товариш (1723; 1727—1728). Син Лаврентія Осиповича Бороздни, сподвижника гетьмана Богдана Хмельницького та брат сотника Стародубського полку М. Л. Борздни.

Біографія

За батькової підтримки зарахований до військового товариства Стародубського полку.

У 1709—1715 роках — знатний військовий товариш. У 1709 році отримав гетьманський універсал, за яким став власником сіл Медведове, Куршовичі, Опалевичі, Бутовське, частини сіл Ущерп'я та Смолевичів, половини містечка Горська.

26 травня 1715 року отримав царську грамоту на «село Медведове з дворами і з усіма угіддями та млинами, село Куршановичі з млином та гутою, половину м. Горська з двома млинами, село Смолевичі придбане (від Смолевицьких), село Бутовське, в селі Ущерп'ю двох мужиків, у селі Халеєвичах придбаний двір та дворик у м. Стародубі».

За наказом правителя Малоросії, генерал-аншефа Олександра Рум'янцева у 1723 році призначений сотником Стародубської полкової сотні, на якій перебував до 1725 року. Крім того, у 1722 та 1724—1725 роках виконував обов'язки наказного полковника Стародубського полку. Бунчуковий товариш (1723; 1727—1728).

Через часті доноси колишнього топальського сотника Клима Янджула на прізвисько «Шпечак», Іван Борозна у 1725 році був заарештований та утримувався у Глухові під караулом. Там у 1726 році підданий тортурам, а пізніше засланий до Сибіру, звідки повернувся вже за рік.

Від 1728 року мешкав у селі Медведовому як бунчуковий товариш, де й помер близько 1731 року.

Одружився у 1700 році на Ганні Рашевській (Зеленківній), з якою мав шестеро дітей. Поріднився з Андрієм Миклашевським, Григорієм Андрійовичем Стаховичем, графом Гаврилом Рогузинським, Хомою Городиським.

Примітки

Джерела

Народились 1680
Померли 1731
Сотники Гетьманщини та Слобідської України
Стародубські полковники

redpajama_set_name: RedPajamaWikipedia


Finally! You can have the freedom to follow your passion and serve your community.
You have learned what you needed to learn from those past experiences.
Be seen as a leader through your creativity an anthology of work.
Take your art to a more radical state.
Be free to travel and enjoy a lifestyle away from menial jobs.
Learn how to utilize the sacred arts to heal others and yourself.
I am so humbled and grateful to have you as part of my life. The beats that you share here are a constant source of inspiration, and each time I flow to them, I am reminded of your awesome person, and it helps me feel connected to my larger community. I think you are doing an excellent job! Keep up the great work!
Get the tips and solutions to stepping out of the "people pleasing" pattern and into your LEADERSHIP and BOSS mentality.
I will never share your name or email to anyone. Promise.
My clients call me UNDAKOVA an Urban Shaman. I am the owner of the Hip Hop Yoga company Undakova Enterprises.
So that they are no longer dependent on menial jobs that deter them from more traveling, control over their life, and doing what they love.


text:
The Pasture-Fed Livestock Association urges the Brexit Government to take control of food, farming and fisheries
The Pasture-Fed Livestock Association is one of 84 organisations to urge Brexit Government to take control of food, farming and fisheries for the public good.
84 organisations have signed a letter to David Davies MP and Theresa May PM to stress the important implications of Brexit on food and farming.
With many of the UK's food and farming policies and subsidies being defined at EU level, the UK government now has an opportunity to reshape these to ensure that taxpayers money is spent for public good.
Organisations representing the health and long-term interests of millions of British citizens have called on government to adopt common-sense food, farming and fishing policies that are good for jobs, health and the environment, when they plan for the UK's exit from the European Union.
Concerns are expressed in a letter to Prime Minister Theresa May, and David Davies MP the Minister currently overseeing a new Unit advising the Government and PM on the post EU Referendum strategy.
The letter, co-signed by over 80 food, farming, fair trade, poverty, animal welfare, wildlife and environmental organisations, argues that good food, farming and fishing policies must be central to any post EU Referendum strategy for the UK.
The organisations point out that better food, farming and trade policies can help to cut greenhouse gas emissions from farming and food industries by 80% by 2050, and promote healthier diets to combat heart disease, cancers, diabetes and obesity, saving the NHS, and ultimately taxpayers millions.
Such policies can also support a vibrant and diverse economy, supporting good jobs and working conditions, in the UK and overseas. Further, the UK could prioritise ethical and sustainable production methods, improved animal welfare, more farmland and marine wildlife, a healthy future for bees and other pollinators, as well as enhancing the beauty of the countryside and protecting the environment, whilst also providing a safe and traceable food supply.
Kath Dalmeny, head of Sustain, an alliance of food and farming organisations, who coordinated the letter, said:
"The British public has given no mandate for a reduction in food and farming standards, a weakening of protection for nature, nor a reversal of the UK's commitment to lifting millions of the poorest people in the world out of poverty through trade. We are seriously concerned that such vital considerations may be over-run by a drive for new trade deals at any cost."
Dr John Meadley, Chair of the Pasture-Fed Livestock Association said:
"With two thirds of the UK's farmland being pasture, the way in which we manage it will have a critical effect on our ability to feed ourselves, to sequester carbon, to increase employment in the rural areas, as well as strongly affecting public health, our landscape and the ability of our watersheds to prevent flooding.
"Brexit offers the opportunity to shift farming policy away from one driven largely by the interests of the agricultural supply industry and the food retail sector, with its focus on controlling nature, to a policy that seeks to nurture nature. With both permanent pasture and rotational grass leys playing a vital role in building up carbon/moisture-retentive organic matter in the soil, we encourage the new government to support pasture-focused, mixed farming systems, that minimise the use of carbon-intensive (and frequently foreign-exchange dependent) inputs."
Professor Tim Lang from the Centre for Food Policy, City University London, said:
"Brexit was largely won on the idea that the UK can 'take back control' but what does this mean in a country that imports nearly a third of its food? How will we manage for fruit and veg pickers if we can no longer rely on the 65% of our farm workers that come from other EU countries?
"If we want a home-grown supply of fresh, healthy and sustainable food, then farm incomes must improve, including fair terms of trade for farmers, and better pay and conditions for farm workers, as well as some level of continued allowance for migrant and seasonal workers. Will Oliver Letwin advise the government to negotiate all that?"
Malcolm Clark, coordinator of the Children's Food Campaign, an alliance of health and children's organisations, said:
"Brexit must not mean an end to policies that can reduce the diet-related conditions, such as cancers, diabetes, obesity and heart disease, that threaten to overwhelm the NHS.
"We have already waited nearly a year for the government's Childhood Obesity Strategy, yet decisive action is still urgently needed to decrease consumption of junk food. Since the result of the EU Referendum, we have already seen the food and soft drinks industry calling for the sugary drinks tax to be shelved. But the UK's future outside Europe must not be allowed to result in a rise in diet-related disease."
Stephen Trotter, the Wildlife Trusts' Director for England, a federation of 47 charities, protecting, championing and taking action for wildlife and wild places at land and at sea, said:
"Public money should be spent on public goods. We have the chance now to increase the wildlife in our farmed landscapes, prevent flooding, stop pollution of our water supplies and reduce climate change. We will be looking to make sure that the proposals for what comes after the Common Agricultural Policy mean that wildlife and the environment don't lose out."
Tim Aldred, head of policy and research at the Fairtrade Foundation, a not-for-profit organisation that promotes the Fairtrade Mark on products in the UK, said:
"Millions of people in developing countries work hard and rely on their food exports to the UK for their livelihoods. It is essential that the UK takes their needs into consideration when negotiating new trade deals, to ensure there is no loss of livelihoods amongst the world's poorest people. With the right political commitment this could be an opportunity to strengthen truly fair trade that works for the poor, that delivers on the sustainable development goals of reducing poverty worldwide and towards a sustainable future."
Ruth Westcott, who coordinates the national Sustainable Fish Cities campaign, a collaboration of the UK's leading marine conservation and sustainable fish organisations, said:
"We can take back control of our seas to some extent. But it is vitally important that life outside the EU does not lead to a fishing free-for-all. Fish swim across national boundaries, and precious stocks that we share across the EU are still under threat from over-fishing. So stocks must still be managed sustainably and in negotiation with other countries. Oliver Letwin must put the science of marine conservation at the heart of government's decisions about leaving the European Union."
The signatory organisations also ask Oliver Letwin MP to ensure that the advice the new unit provides to government is drawn up in consultation with people with science, health and sustainability expertise in relation to food, farming and fishing, alongside economic concerns. Further, the signatory organisations urge that food, farming and fishing make up one of the Options Papers being developed by the unit, to advise the PM and government.
For a list of EU policies that influence our food and farming system, please see the paper: Food, the UK, and the EU: Brexit or Bremain. March 2016. Tim Lang and Victoria Schoen. http://foodresearch.org.uk/wp-content/uploads/2016/03/Food-and-Brexit-briefing-paper-2.pdf
Food employs 3.6 million people in the UK. Source Food Research Collaboration paper. Food, Brexit and the consequences. http://foodresearch.org.uk/2016/07/food-brexit-and-the-consequences-what-can-academics-and-the-uk-food-movement-do/
Sustain is the UK alliance for better food and farming. It advocates food and agriculture policies and practices that enhance the health and welfare of people and animals, improve the working and living environment, enrich society and culture and promote equity. www.sustainweb.org
The letter and full list of signatories can be downloaded below:
Download the letter (PDF): Brexit Letter to Government
PFLA, OF&G and BDA welcome Soil Association Certification as fellow auditor of the 'Pasture for Life' certification standard
PFLA launches its Regional Groups Strategy
First Russ Carrington Award announced
Mary Berry enjoys meeting 100% pasture-fed cattle on BBC Countryfile
to support our work
Becoming a member is easy, click here to get involved.