text:
\section{Introduction}

In recent years, with the advances in computational power and data accumulation, deep learning methods thrived, achieving the state of art in many benchmarks. However, these methods usually demand a large number of human-annotated data. While in many cases unlabeled data is relatively easy to achieve, human annotation remains costly.

\begin{figure}[thb]

\begin{subfigure}{.235\textwidth}
  \centering
 \includegraphics[width=\linewidth,height=2.7cm]{graphs/visulizations/phase_transition_visualization_v2.png}
\caption{Theoretical results}
\label{fig:example}
\end{subfigure}
\begin{subfigure}{.235\textwidth}
  \centering
 \includegraphics[width=\linewidth,height=2.7cm]{graphs/visulizations/all_budget_margin_scan_avihu_comparison_v2.png}
\caption{Empirical results}
\label{fig:phase_transition_real}
\end{subfigure}

\vspace{-.2cm}
\caption{
Visualization of phase transition in deep active learning, as revealed by plotting the difference in accuracy between different AL strategies to a random baseline as a function of budget size (the number of queried examples). We see a similar behavior both theoretically and empirically: when the budget is low, oversampling typical examples is more beneficial, whereas when the budget is high, oversampling atypical examples is more beneficial. (a) The behavior of an idealized model (see Section~\ref{sec:theoretical_analysis}). (b) The behavior of \textit{TypiClust}, contrasted with a basic uncertainty-based strategy, as seen in deep neural models trained on CIFAR-10 (see Section~\ref{sec:emp}).}
\vspace{-.5cm}
\end{figure}

Active Learning (AL) aims to alleviate this problem \citep[see surveys in][] {settles.tr09,schroder2020survey}. Given a large pool of unlabeled data, and possibly a small set of labeled examples, learning is done iteratively: the learner employs the labeled examples (if any), then queries an oracle by submitting examples to be annotated. This may be done repeatedly, often until a fixed budget is exhausted.

When submitting a query for a new label, two main principles guide the traditional active learning approach: (1) uncertainty sampling \citep[e.g.,][] {lewis1994sequential,scheffer2001active,culotta2005reducing,joshi2009multi,li2013adaptive,ranganathan2017deep,gal2017deep,beluch2018power,gissin2019discriminative,yoo2019learning,sinha2019variational}, and (2) diversity sampling \citep[e.g.,][] {elhamifar2013convex,yang2015multi,wang2016batch,yin2017deep,zhdanov2019diverse,he2019towards,kirsch2019batchbald,DBLP:conf/iclr/AshZK0A20,shui2020deep}. In uncertainty sampling, the learner queries examples about which it is least certain, presumably because such labels contain the most information about the problem at hand. In diversity sampling, queries are chosen to minimize the correlation between samples, to avoid redundancy in the annotations.
Notably, as deep active learning is practical only in batch settings, the importance of diversity is amplified \citep{geifman2017deep, sener2018active}.

\begin{figure}[thb!]
\begin{center}

\includegraphics[width=.47\textwidth]{avihu_graphs/method vis/try9.png}

\caption{Visualizing the selection of 30 examples from CIFAR-10 by \emph{TypiClust}. Left: the data is first clustered into 30 clusters, and the densest example within every cluster is queried. We show t-SNE dimensionality reduction of the feature space, colored by cluster assignment, where selected examples are marked by $\times$. Right: the selected images, organized column-wise by class. Note that the ensuing labeled set is approximately class-balanced, even though the queries are chosen without access to class labels.}
\vspace{-0.65cm}
\label{fig:cifar10_examples}
\end{center}
\end{figure}

At present, effective deep active learning strategies are known to require a large initial set of labeled examples to work properly \citep{DBLP:conf/emnlp/YuanLB20,pourahmadi2021simple}. We call this the \emph{high budget regime}, while the regime characterized by failure (or "cold start") is called the \emph{low budget regime}. In the low budget regime, it has been shown that random selection outperforms most querying strategies \citep[see][]{attenberg2010label,mittal2019parting,DBLP:journals/tkde/ZhuLHWGLC20,simeoni2021rethinking,chandra2021initial}. Different accounts have been offered to explain this observation: \begin{inparaenum}[(i)] \item failure to model uncertainty, which is more severe with a small budget of labels \citep{nguyen2015deep,gal2016dropout}; \item difficulty to accomplish diversity in high dimensional input spaces, where traditional metrics (such as the Euclidean distance) are not meaningful.\end{inparaenum}~In Section~\ref{sec:ablation_study}, we show that these accounts cannot fully explain the failure of uncertainty sampling in the low budget regime.

As a result, most deep active learning research is focused on the high-budget regime. Strategies designed to address the low-budget regime usually employ self-supervised or semi-supervised methods using the unlabeled pool \citep{gao2020consistency,hong2020deep,mahmood2021low}. In the ensuing embedding,
the estimation of uncertainty is often more reliable \citep{zhang2018unreasonable}.

In this paper, we start by showing a formal argument, which suggests that the uncertainty principle may be unsuited for the low-budget regime. We find that in the low-budget regime the most typical points should be queried, while points of high uncertainty should be deferred to higher budgets. More specifically, we analyze a mixture of two independent learners model.
We then prove, under some mild assumptions, that such models yield better performance if the distribution of the training data is allowed to be biased. The preferred distribution depends on the number of labeled examples -- when limited to only a few labels, it is beneficial to sample additional \emph{typical} examples (to be defined more precisely below), while with a large budget of labels, it is beneficial to sample more \emph{atypical} examples. This is illustrated in Fig.~\ref{fig:example} for an ideal model.

Guided by this theoretical analysis, we propose \emph{TypiClust} -- a novel strategy for active learning in the low-budget regime. \emph{TypiClust} utilizes the unlabeled data to estimate the typicality of unlabeled examples. To this end, \emph{TypiClust} employs self-supervised representation learning and then estimates each point's density in this representation. Diversity is obtained by clustering the dataset and picking the densest examples from each cluster (see Fig.~\ref{fig:cifar10_examples}).

We evaluate \emph{TypiClust} in three AL frameworks: \begin{inparaenum}[(i)]
\item A pool-based supervised framework, which repetitively queries labels and uses them to train a deep network model.
\item A pool-based supervised framework, which leverages self-supervised pre-trained embeddings.
\item An initial pool semi-supervised framework, which queries labels once and also uses the remaining unlabeled examples to train a deep network model.  \end{inparaenum}
This body of work is summarized in Fig.~\ref{fig:semi_supervised_vs_fully_acc_improvement}, plotting the accuracy gain over the random baseline in the three AL frameworks.

\begin{figure}[htb!]
\begin{center}
    \begin{subfigure}{.46\textwidth}
      \centering
      \includegraphics[width=\linewidth]{graphs/semi_supervised/ssl_vs_fully_combined.png}

      \vspace{-0.5cm}
    \end{subfigure}
\vspace{-0.25cm}
\caption{
\textit{TypiClust} achieves major accuracy gains as compared to the random selection baseline in the fully-supervised ($3$ repetitions on ImageNet, $10$ otherwise), semi-supervised ($3$ reps) and self-supervised embedding ($5$ reps) frameworks. We use $10$, $300$, $1000$, $50$, $100$, $200$ examples in CIFAR-10, CIFAR-100, TinyImageNet, ImageNet-50, ImageNet-100 and ImageNet-200 respectively.
\vspace{-.2cm}
\label{fig:semi_supervised_vs_fully_acc_improvement}
\end{center}
\end{figure}

More specifically, our empirical study focuses on the low-budget regime. We first show that the aforementioned theoretical predictions are observed in practice when training competitive deep learning models on various image recognition benchmarks (see Fig.~\ref{fig:phase_transition_real}).
In accordance with \citet{DBLP:journals/tkde/ZhuLHWGLC20}, we also observe that many well-known querying strategies are not effective in this domain, and are even detrimental. In contrast,  \emph{TypiClust} consistently improves generalization by a large margin, across different datasets and architectures (see Fig.~\ref{fig:semi_supervised_vs_fully_acc_improvement}).
We note that while the choice of a specific self-supervised task and a clustering algorithm may have an impact on performance, \emph{TypiClust} remains beneficial (with significant gains) in all settings.

Although in both supervised and semi-supervised AL frameworks the learner has access to a big pool of unlabeled data, many AL strategies do not exploit the promise of semi-supervised learning. Recent studies report that the benefit of AL is marginal when semi-supervised learning methods are employed \citep{chan2021marginal,bengar2021reducing}, with little added value to the active querying of labels. Re-examining this observation, we note that semi-supervised learning is most beneficial in the low-budget regime, wherein the explored AL strategies are inherently not suitable. By using \emph{TypiClust}, which is designed for the low-budget regime, we show that AL outperforms random sampling in the semi-supervised framework by a large margin, surpassing the state-of-the-art (SOTA) performance on several image classification tasks.

Specifically, we used \emph{TypiClust} to generate the labeled set for FlexMatch \cite{DBLP:journals/corr/abs-2110-08263}, a competitive semi-supervised algorithm. With $10$ queries from CIFAR-10, \emph{TypiClust} improves the test accuracy of FlexMatch by $39.4\%$ over a random selection. With $300$ queries from CIFAR-100, \emph{TypiClust} improves the accuracy of FlexMatch by $20.3\%$. This improvement is obtained even though \emph{TypiClust} may not deliver a balanced training set, unlike the strategy against which it is compared.

\vspace{-.2cm}
\subsection{Summary of Contribution}
\setlist{nolistsep}
\setlength\itemsep{30em}
\begin{itemize

\setlength\itemsep{.3em}
    \item We show theoretical and empirical results suggesting that typical examples are more important in low budgets, as opposed to atypical examples in high budgets.
    \item We propose a mixture learner model to analyze active learning; the model predicts a phase transition phenomenon that depends on the budget size.

    \item We empirically demonstrate this behavior in actual deep learning models trained on image benchmarks.
    \item We propose \emph{TypiClust}: an AL strategy that significantly improves performance in the low budget regime.
    \item We empirically show that \emph{TypiClust} can be used to boost SOTA semi-supervised learning methods.

\end{itemize}

\vspace{-.4cm}
\section{Theoretical Analysis
\label{sec:theoretical_analysis}

We now introduce a simple mixture model of two learners, which reveals a phase-transition phenomenon in AL. Its relevance to actual classifiers is discussed in Section~\ref{sec:common}.

\subsection{Mixture Model Definition}
\label{sec:model_def}

Active learning abandons random sampling in favor of biased sampling strategies. This can be formally investigated in the context of a mixture model. To this end, we start in Section~\ref{sec:model} by defining such a mixture model. We then formalize biased data sampling strategies in Section~\ref{sec:biased}.

\subsubsection{Mixture Learner Model}
\label{sec:model}

We adopt a mixture of two learners model, where each learner is independently trained on a distinct part of the domain of the data distribution. Formally, let ${\mathbf X}=[\bm{x},y]$ denote a single data point, where $\bm{x}\in\mathbb{R}^d$ denotes an input point and $y\in\mathbb{Y}$ its corresponding label or target value. For example, $\mathbb{Y}$ is $\mathbb{R}$ in regression problems, and $[k]$ in classification problems. Each point ${\mathbf X}$ is drawn from a distribution $\mathcal{D}$ with density $f_\mathcal{D}({\mathbf X})$. We denote an i.i.d sample of $m$ points from $\mathcal{D}$ as $\mathbb{X}^m=\left\{{\mathbf X}_1,...,{\mathbf X}_m\right\}\sim\mathcal{D}^m$.

Let $R_1}%f,R_2}%s\subseteq\mathbb{R}^{d}\times\mathbb{Y}$ denote a partition of the domain of $f_\mathcal{D}$, where $R_1}%f\cup R_2}%s=\mathbb{R}^{d}\times\mathbb{Y}$ and $R_1}%f\cap R_2}%s=\emptyset$.
Let $\mathcal{D}_1}%f, \mathcal{D}_2}%s$ denote the conditional distributions obtained when restricting $\mathcal{D}$ to regions $R_1}%f, R_2}%s$ respectively. Note that $\mathcal{D}$ can now be viewed as a mixture distribution, where points are sampled from $\mathcal{D}_1}%f$ with probability  $p=\int_{{\mathbf X}\in R_1}%f}f_\mathcal{D}({\mathbf X})d{\mathbf X}$, and $\mathcal{D}_2}%s$ with probability $(1-p)$.
Let $m_i,~i\in\{1}%f,2}%s\}$ denote the number of points in $R_i$ when sampling $m$ points from $\mathcal{D}$, and $\mathbb{X}^{m_i}$ denote  the restriction of sample $\mathbb{X}^m$ to $R_i$. The hypothesis obtained when training a learner independently on $\mathbb{X}^{m_i}$ is denoted $\hdD(\mathbb{X}^{m_i})$.

Next, we define the error score of a learner, which is a function of $m$ - the training set size. It measures the expected generalization error of the learner over all such training sets
\begin{defn}[Error score]
Assume training sample $\mathbb{X}^m\sim\mathcal{D}^m$, with the corresponding learned hypothesis $\hdD(\mathbb{X}^{m})$. Let $Er({\mathbf X},\hdD(\mathbb{X}^{m}))$ denote the error of this hypothesis on some point ${\mathbf X}\in\mathbb{R}^{d}\times\mathbb{Y}$. The expected error of the learner, over random variables ${\mathbf X}$, $\mathbb{X}^m$ and $\hdD(\mathbb{X}^{m})$, is given by
\begin{equation}
\label{eq:error-score}
E_{\mathcal{D}}(m) = \mathbb{E}_{{\mathbf X}}\left[\mathbb{E}_{\mathbb{X}^{m}}\left[\mathbb{E}_{\hD(\mathbb{X}^{m})}\left[Er({\mathbf X},\hD(\mathbb{X}^{m}))\right]\right]\right]
\end{equation}
\label{def:expected-error}
\vspace{-.5cm}
\end{defn}
We make two commonly adopted assumptions on $E_{\mathcal{D}}(m)$: \begin{inparaenum}[(i)] \item  \textbf{Efficiency.} $E_{\mathcal{D}}(m)$ is strictly monotonically decreasing, namely, on average the learner benefits from additional examples. \item \textbf{Realizability.} $\lim\limits_{m \rightarrow\infty}E_{\mathcal{D}}(m) =0$. \end{inparaenum}

During training, we assume a mixture of independent learners in $R_1}%f,R_2}%s$, and a training set composed of $m_1}%f,m_2}%s$ examples from each region respectively. The error score of the mixture learner on $\mathcal{D}$ for $m=m_2}%s+m_1}%f$ is:
\begin{equation}
\begin{split}
\label{eq:optimization}
    E_{\mathcal{D}}(m) = p\cdot E_{\mathcal{D}_1}%f}(m_1}%f) + (1-p)\cdot E_{\mathcal{D}_2}%s}(m_2}%s)
\end{split}
\end{equation}
As an important ingredient of the mixture model, we assume that one region requires fewer examples to be adequately learned, and call this region $R_1}%f$. Essentially, we expect the error score to decrease faster at $R_1}%f$, with $E_{\mathcal{D}_1}%f}(m)<E_{\mathcal{D}_2}%s}(m)~\forall m$. Other than this difference, we expect the error score to be similar in $R_1}%f$ and $R_2}%s$.
Making all this more precise, we formally assume that the error scores   in $R_1}%f,R_2}%s$ can be written as $E_{\mathcal{D}_1}%f}(m) =E(m)$ and $E_{\mathcal{D}_2}%s}(m)= E(\alpha m)$ for a single function $E(m)$ and learning speed parameter $\alpha\leq1$. We can now rewrite (\ref{eq:optimization}) as follows:
\begin{equation}
\begin{split}
\label{eq:Etotal}
E_\mathcal{D}(m)&= p \cdot E(m_1}%f) + (1-p) \cdot E(\alpha(m-m_1}%f))
\end{split}
\end{equation}

Finally, we extend $E(m)$ to domain $\mathbb{R}^+$ with the continuation of $E(m)$ denoted $E(x):\R^{+}\rightarrow\R^{+}$, which is in $C^\infty$
and positive. \textbf{Efficiency} is extended to imply $E'(x)<0$.

\subsubsection{Sampling Strategies}
\label{sec:biased}

Considering the extended error score $E(x):\R^{+}\rightarrow\R^{+}$, we define a biased sampling strategy as follows:
\begin{equation*}
\label{eq:m}
m_1}%f =p\cdot m+\Delta, \qquad m_2}%s =\left(1-p\right)\cdot m-\Delta
\end{equation*}
$\Delta=0$ is essentially equivalent to random sampling from $\mathcal{D}$.
$\Delta>0$ implies that more training points are sampled from $R_1}%f$ than $R_2}%s$, and vice versa.

In Thm.~\ref{thm:threshold_test} we show that when aiming to minimize the expected generalization error while considering a mixture of $2$ learners model as defined above, choosing between the different sampling strategies can be done using a simple threshold test. The proof for Thm.~\ref{thm:threshold_test} can be found in App~\ref{app:proof_of_thm_1}.

\begin{theorem}
\label{thm:threshold_test}
Let $0<p<1$, $0<\alpha<\frac{p}{1-p}$ denote some constants. Given error score $E(x)$, the following threshold test decreases the expected error for sample size $m$:
\begin{equation*}
\frac{E'\left(pm\right)}{E'\left(\alpha\left(1-p\right)m\right)}
\begin{cases} > \frac{\alpha\left(1-p\right)}{p}&\implies \begin{array}{cc}&\texq{over~sample}\vspace{-.15cm}\\  &\texq{region~R_1}%f} \end{array} \\
< \frac{\alpha\left(1-p\right)}{p}&\implies \begin{array}{cc}&\texq{over~sample}\vspace{-.15cm}\\  &\texq{region~R_2}%s} \end{array}
\end{cases}
\end{equation*}
\end{theorem}

\myparagraph{Example: exponentially decreasing function.}
Assume $E\left(m\right)=e^{-m}$, and a mixture model with $p=0.8,\alpha=0.1$. We simulate $E\left(m\right)$ when biasing the train sample with $\Delta=\pm 0.01$.
Fig.~\ref{fig:example} shows the differences between the error score of biased sampling (in favor of either $R_1}%f$ in blue or $R_2}%s$ in orange) and random sampling, as a function of the number of examples $m$. For small $m$ it is beneficial to favorably bias region $R_1}%f$, while for large $m$ it is beneficial to favorably bias $R_2}%s$. This is the behavior often seen in our empirical investigation, see Fig.~\ref{fig:phase_transition_real} and Section~\ref{sec:emp}.

\subsection{Error Scores Analyzed}
\label{sec:error-fs}

We now report sufficient conditions on $E(m)$, which guarantee the phase-transition-like behavior illustrated in Fig.~\ref{fig:example}. We start with some formal definitions, and then obtain sufficient conditions for these definitions to take hold.

To capture the sought behavior, we say that an error score is undulating if it displays at least two phases. In the beginning, with only a few training examples, such learners benefit from over-sampling region $R_1}%f$. After seeing sufficiently many training examples, such learners may benefit from over-sampling region $R_2}%s$. Formally:
\begin{defn}[Undulating] An error score $E(m)$ is undulating if there exist $z_1,z_2\in\mathbb{R}$ such that  $\frac{E'\left(pm\right)}{E'\left(\alpha\left(1-p\right)m\right)}>\frac{\alpha\left(1-p\right)}{p}~~\forall m< z_1$, and $\frac{E'\left(pm\right)}{E'\left(\alpha\left(1-p\right)m\right)}<\frac{\alpha\left(1-p\right)}{p}~~\forall m> z_2$.
\label{def:1}
\end{defn}
For undulating error scores, there could potentially be any number of transitions between the two conditions, switching the preference of $R_1}%f$ to $R_2}%s$, and vice versa. We extend the above definition to capture a case of particular interest, where this transition happens only once, as follows:
\begin{defn}[SP-undulating] An error score $E(m)$ is Single-Phase undulating if it is undulating, and $z_1=z_2$.
\label{def:2}
\end{defn}

\vspace{-1cm}
\subsubsection{Undulating Error Scores}
As motivated in Section~\ref{sec:model}, we define a proper error score as follows:
\begin{defn}[Proper error score] $E(x):\R^{+}\rightarrow\R^{+}$ is a proper error score if it is a positive twice differentiable function, which is strictly monotonically decreasing ($E>0$, $E'<0$), $E(0)=c_0\in\R^{+}_*$, and where $\lim\limits_{x\rightarrow\infty}E(x)=0$.
\label{def:proper}
\end{defn}

Not all proper error scores will exhibit the phase undulating behavior. In Thm.~\ref{thm:suffiecnt_coniditions_undulating} we state sufficient conditions that ensure this behavior (see proof in App~\ref{app:proof_for_thm_2}).
\begin{theorem}[Undulating error score: sufficient conditions]
\label{thm:suffiecnt_coniditions_undulating}
 Let $0<p<1$, $0<\alpha<\frac{p}{1-p}$ denote some constants. \emph{Error score} $E(x)$ is undulating if the following assumptions hold:
\begin{enumerate}
\item $E(x)$ is a proper \emph{error score} (see Def.~\ref{def:proper})
\item
$\lim\limits_{x\to\infty}\frac{E'(x)}{E(x)},\lim\limits_{x\rightarrow\infty}\frac{E(x)}{E(a x)},\lim\limits_{x\rightarrow\infty}\frac{E'(x)}{E'(a x)}$ exist $\foralla\!\in\! (0,1)$
\item
$-\log(E(x))=\omega\left(\log(x)\right)$
\end{enumerate}
\end{theorem}

\begin{corollary}[Exponential error as a bound]
\label{cor:exp_undulating}
\emph{Error score} $E(x)$ is undulating if it satisfies assumptions (1) and (2) of Thm.~\ref{thm:suffiecnt_coniditions_undulating}, and is bounded from above as follows
\begin{equation}
\label{eq:bound}
E(x) \leq k e^{-\nu x} \qquad\forall x\in\R^{+}
\end{equation}
for some constants $\nu,k\in\mathbb{R}^+_*$.
\end{corollary}
\begin{proof}
It can be readily verified that assumption (3) of Thm.~\ref{thm:suffiecnt_coniditions_undulating} follows from (\ref{eq:bound}).
\end{proof}

\subsubsection{SP-Undulating Error Scores}
\label{sec:SP_undulating}
Thm.~\ref{thm:suffiecnt_coniditions_sp_undulating} extends the results of the previous section, by stating a set of sufficient conditions that ensure an SP-undulating error score.
The proof can be found in App~\ref{app:proof_thm_3}.

\begin{theorem}[SP-undulating: sufficient conditions]
\label{thm:suffiecnt_coniditions_sp_undulating}
 Let $0<p<1$, $0<\alpha<\frac{p}{1-p}$ denote some constants. \emph{Error score} $E(x)$ is SP-undulating if the following assumptions hold:
\begin{enumerate}
\item $E(x)$ is an undulating proper \emph{error score}.
\item At least one of the following conditions holds:
\begin{enumerate}
\item
$\frac{-E''\left(x\right)\cdot x}{E'\left(x\right)}$ is
monotonically increasing with $x$.
\item
$-E'(x)$ is strictly monotonically decreasing and log-concave.
\end{enumerate}

\end{enumerate}
\end{theorem}

\begin{corollary}[Exponential error is SP-undulating]
\label{cor:exp}
Consider \emph{error scores} of the form $E(x) = k e^{-\nu x}$
for constants $\nu,k\in\mathbb{R}^+_*$. Such functions are SP-undulating.
\end{corollary}
Cor.~\ref{cor:exp} shows that classifiers with an exponentially decreasing \emph{error score} are SP-undulating, with a single transition from favoring $R_1}%f$ to favoring $R_2}%s$

In practice, we cannot assume that the \emph{error score} of commonly used learners is exponentially decreasing. However,
frequently we can bound the \emph{error score} from above by an exponentially decreasing function, as we demonstrate theoretically (see Section~\ref{sec:piecewise_linear_separator}) and empirically (see Fig.~\ref{fig:error_scores_neural_networks} in the App). In such cases, it follows from Cor.~\ref{cor:exp_undulating} that these functions are undulating.

\subsection{Simple Classification Models}
\label{sec:common}

To make the analysis above more concrete, we analyze a mixture of two linear classifiers in Section~\ref{sec:piecewise_linear_separator}. We then analyze the nearest neighbors classification model in Section~\ref{sec:high_density}, to shed light on the circumstances that make it possible to learn from fewer examples in certain regions.

\subsubsection{Mixture of Two Linear Classifiers}
\label{sec:piecewise_linear_separator}

Consider a binary classification problem and assume a learner that delivers a mixture of two linear classifiers in $\mathbb{R}^d$. The two classifiers are obtained by independently minimizing the $L_2$ loss on points in $R_1}%f$ and $R_2}%s$ respectively.

\myparagraph{I. Bounding the error of each mixture component.}
We first derive a bound on the error of the $j^{th}$ component, as it depends on the sample size $m_j$. Let $X\in\mathbb{R}^{d\times m_j}$ denote the matrix whose columns are the training vectors that lie in region $R_j$. Let $\bm{y}\in\{-1,1\}^{m_j}$ denote a row labels vector, where 1 marks positive examples and $-1$ negative examples. The learner seeks a separating row vector $\hat\bm{w}\in\mathbb{R}^d$, where
\begin{equation*}
\hat {\bm{w}} = \mathrm{arg}\min_{{\bm{w}}\in\mathbb{R}^d} \left\|{\bm{w}}X-{\bm{y}}\right\|^2 ~~ \implies \hat {\bm{w}} = {\bm{y}}X^\top (XX^\top)^{-1}
\end{equation*}

In Thm.~\ref{thm:error_bound_pw_linear}, we bound the error of a linear model by some exponential function of the number of training examples $m_j$. The proof and further details can be found in App~\ref{app:proof_thm_4}.

\begin{theorem}[Error bound on a linear classifier]
\label{thm:error_bound_pw_linear}
Assume: \begin{inparaenum}[(i)] \item a bounded sample $\|\bm{x}_i\|\le B$, where $X$ is sufficiently far from singular so that its smallest positive singular value is bounded from below by $\frac{1}{\Lambda}$; \item a realizable binary problem where the classes are separable by margin $\delta$; \item full rank data covariance, where $\frac{1}{\lambda}$ denotes its smallest singular value. \end{inparaenum} Then there exist some positive constants $k, \nu > 0$, such that $\forall m_j\in\mathbb{N}$ and every sample $\mathbb{X}^{m_j}$, the expected error of $\hat {\bm{w}}$ obtained using $\mathbb{X}^{m_j}$ is bounded by:
\begin{equation}
\label{eq:tata}
\mathbb{E}_{{\mathbf X}\sim\mathcal{D}_j}[\mathrm{0-1~loss~of~}\hat {\bm{w}}] \leq ke^{-\nu {m_j}}
\end{equation}
\end{theorem}

\myparagraph{II. A mixture classifier.}
Assume a mixture of two linear classifiers, and let $E(m)=p\cdot E_{\mathcal{D}_1}%f}(m_1}%f) + (1-p)\cdot E_{\mathcal{D}_2}%s}(m_2}%s)$ denote its error score. The following theorem characterizes this function (the proof can be found in App~\ref{app:proof_of_thm_5}):
\begin{theorem}[Undulating error]
\label{thm:undulating_pw_linear}
Retain the assumptions stated in Thm.~\ref{thm:error_bound_pw_linear}, and assume in addition that $\foralla\!\in\! (0,1)$ $\exists\lim\limits_{m\to\infty}\frac{E'(m)}{E(m)},\lim\limits_{m\rightarrow\infty}\frac{E(m)}{E(a m)} \lim\limits_{m\rightarrow\infty}\frac{E'(m)}{E'(a m)}$.
Then the error score of a mixture of two linear classifiers is undulating.
\end{theorem}

In practice, the \emph{error score} in this case is also SP-undulating. The empirical validation of this point can be seen in Fig.~\ref{fig:error_linear_classifier_mixture}.

\subsubsection{KNN Classifier and High-Density Regions}
\label{sec:high_density}

Our analysis in Section~\ref{sec:error-fs} showed that given some partition of the data into $R_1}%f$ and $R_2}%s$, where $R_1}%f$ is easier to learn in the sense that $E_{\mathcal{D}_{R_2}%s}}(m)<E_{\mathcal{D}_{R_1}%f}}(m)$ for every $m$, then oversampling from $R_1}%f$ is preferable in the low budget regime, while oversampling from $R_2}%s$ is preferable in the high budget regime. We now employ the discrete one-Nearest-Neighbor (1-NN) classification framework to shed light on the nature of the assumed partition. Specifically, we show that \textbf{selecting $R_1}%f$ as the set of the densest points in the dataset} has the desired property. In fact, we show in App~\ref{app:sec:1_nn_classifier_full_sec} that in this framework, the selection of an initial pool of size $m$ will benefit from the following heuristic:
\setlist{nolistsep}
\begin{itemize}[noitemsep]
    \item \textbf{Max density:} when selecting a point ${\mathbf X}_i$, maximize its density $f_\mathcal{D}({\mathbf X}_i)$.
    \item \textbf{Diversity:} select points that are far apart, so that their corresponding sets of nearest neighbors do not overlap.
\end{itemize}

While 1-NN is a rather simplistic model, we propose to use the derived heuristics to guide deep active learning. In the rest of this paper, we show how these guiding principles benefit deep active learning in the low-budget regime.

\section{Method: Low Budget Active Learning}
\label{sec:active_learning_low_budget}

In the low-budget regime, our theoretical analysis showed that it may be beneficial to bias the training sample in favor of certain regions in the data domain. It also established a connection between such regions and the principles of \emph{max density} (or \emph{typicality}) and \emph{diversity}. In this section, we incorporate  these principles into a new active learning strategy called \emph{TypiClust}, designed for the low-budget regime

\subsection{Framework and Definitions}

Let $L_0$ denote an initial labeled set of examples, and $U_0$ denote an initial unlabeled pool. Active learning is done iteratively: at each iteration $i$, a set of size $B$ of unlabeled examples is picked according to some strategy. These examples are annotated by an oracle, added to $L_i$, and removed from $U_i$. This process is repeated until the label budget is exhausted, or some predefined termination conditions are satisfied. In the low-budget regime, the total number of labeled examples, represented by $L_0$ and $B$, is small. The case where $L_0=\emptyset$ is called "initial pool selection".

\subsection{Proposed Strategy: Typical Clustering (\emph{TypiClust})}

We define the \emph{Typicality} of an example by its density in some semantically meaningful feature space. Formally, we measure an example's \emph{Typicality} by the inverse of the average Euclidean distance to its $K$ nearest neighbors\footnote{We use $K=20$, but other choices yield similar results.}, namely:
\begingroup\abovedisplayskip=4pt\belowdisplayskip=4pt
\begin{equation}
\label{eq:density}
    Typicality(x)=\bigg(\frac{1}{K}\sum_{x_i\in \text{K-NN}(x)}||x-x_i||_2\bigg)^{-1}
\end{equation}
\endgroup

In the low budget regime, an active learning strategy based on typical examples needs to overcome several obstacles:
\begin{inparaenum}[(a)]
    \item Networks trained on only a few examples are prone to overfit, making measures of typicality noisy and unreliable.
    \item Typical examples tend to be very similar, amplifying the need for diversity (see App ~\ref{app:clustring_importance} for further details).
\end{inparaenum}

To overcome these obstacles we propose a novel method, called \emph{TypiClust}, which attempts to select typical examples while probing different regions of the data distribution. In our method, self-supervised representation learning is used to overcome (a), while clustering is used to overcome (b). \emph{TypiClust} is therefore composed of 3 steps:

\myparagraph{Step 1: Representation learning.}
Utilize the large unlabeled pool $U_0$ to learn a semantically meaningful feature space: first train a deep self-supervised task on $U_0 \cup L_0$, then use the penultimate layer of the resulting model as feature space. Such methods are commonly used for semantic feature extraction \citep{chen2020simple, DBLP:conf/nips/GrillSATRBDPGAP20}.

\myparagraph{Step 2: Clustering for diversity.}
As \emph{typicality} in (\ref{eq:density}) is evaluated by measuring distances to neighboring points, the most typical examples are usually close to each other, often resembling the same image (see Fig.~\ref{subfig:clustering_matters_nocluster}). To enforce diversity and thus better represent the underlying data distribution, we employ clustering. Specifically, at each AL iteration $i$, we partition the data into $|L_{i-1}|+B$ clusters. This choice guarantees that there are at least $B$ clusters that do not intersect with the existing labeled examples. We refer to such clusters as \emph{uncovered clusters}.
All the implementation details can be found in App~\ref{app:method_implementation_details}.

\myparagraph{Step 3: Querying typical examples.}
We select the most typical examples from the $B$ largest \emph{uncovered clusters}. Selecting from \emph{uncovered clusters} enforces diversity (also w.r.t $L_{i-1}$), while selecting the most typical example in every cluster favors the selection of representative examples.

As the general steps above do not depend on the specific representation or clustering algorithm, different variants of the \emph{TypiClust} strategy can be constructed for different tasks. In our work we consider two variants, both of which outperform by a large margin the uncertainty based strategies in the low-budget regime:
\begin{enumerate}
    \item \textit{$TPC_{DC}$}: Using a deep clustering algorithm both for the self-supervised and clustering tasks. In our experiments, we used SCAN \citep{van2020scan}.
    \item \textit{$TPC_{RP}$}: Using representation learning followed by a clustering algorithm. We used DINO \citep{caron2021emerging} for ImageNet, and SimCLR \citep{chen2020simple} for all other datasets, followed by K-means.
\end{enumerate}
The pseudo-code of \emph{TypiClust} can be found in Alg.~\ref{alg:initial_pool_selection} in App~\ref{app:code_for_AL}. Note that, unlike traditional active learning strategies, \emph{TypiClust} relies on self-supervised representation learning, and therefore can be used for initial pool selection.

\section{Empirical Study}
\label{sec:emp}
In this section we report a set of empirical results, demonstrating the benefits of \emph{TypiClust} in the low-budget regime.
Section~\ref{sec:methodology} describes the evaluation protocol, specific datasets, and alternative methods. Section~\ref{sec:emp_results} describes the actual experiments and their results.

\subsection{Methodology}
\label{sec:methodology}
We evaluate active learning separately in the following three frameworks.  \begin{inparaenum}[(i)]
\item \textbf{Fully supervised}: training a deep network solely on the labeled set, obtained by active queries.
\item \textbf{Fully supervised using self-supervised embeddings}: training a linear classifier on the embedding obtained from a pre-trained self-supervised model.
\item \textbf{Semi-supervised}: training a deep network on the labeled and unlabeled sets, using FlexMatch -- a competitive semi-supervised algorithm.
\end{inparaenum}

In (i) and (ii), we adopted the AL evaluation framework created by \citet{Munjal2020TowardsRA}, which implements several AL methods, including all baselines used here. In (iii) we adopted the code and hyper-parameters provided by FlexMatch. As FlexMatch is computationally intensive, we did not evaluate it on ImageNet, confining the study to datasets it has been reported to handle; in all of them, \emph{TypiClust} is beneficial.
For all implementation details, see App~\ref{app:eval_impl_details}.

We compared \emph{TypiClust} to the following baseline strategies for the selection of $B$ points from $U$: \begin{inparaenum}[(1)] \item \emph{Random} -- uniformly. \item \emph{Uncertainty} -- lowest max softmax output. \item \emph{Margin} -- lowest margin between the two highest softmax outputs. \item \emph{Entropy} --  highest entropy of softmax outputs. \item \emph{DBAL} \citep{gal2017deep}.
\item \emph{CoreSet} \citep{sener2018active}. \item \emph{BALD} \citep{kirsch2019batchbald}. \end{inparaenum} All strategies are evaluated on several image classification tasks, including CIFAR-10/100 \citep{krizhevsky2009learning}, TinyImageNet \citep{le2015tiny} and ImageNet-50/100/200, which are subsets of ImageNet \citep{deng2009imagenet} containing 50/100/200 classes respectively \citep[following][]{van2020scan}. All code will be published upon acceptance.

\begin{figure}[thb]
\begin{center}
\begin{subfigure}{.45\textwidth}
  \centering
 \includegraphics[width=\linewidth]{avihu_graphs/legend_newnames.png}
\end{subfigure}
\\
    \begin{subfigure}{.154\textwidth}
      \centering
      \includegraphics[width=\linewidth]{avihu_graphs/main_al_graphs/cifar10_yname.png}

    \caption{CIFAR-10}
    \label{fig:main_al_graph_cifar10}
    \end{subfigure}
    \begin{subfigure}{.154\textwidth}
      \centering
      \includegraphics[width=\linewidth]{avihu_graphs/main_al_graphs/cifar100_yname.png}

    \caption{CIFAR-100}
    \label{fig:main_al_graph_cifar100}
    \end{subfigure}
    \begin{subfigure}{.154\textwidth}
      \centering
      \includegraphics[width=\linewidth]{avihu_graphs/main_al_graphs/imagenet100.png}

    \caption{ImageNet-100}
    \label{fig:main_al_graph_tinyimagenet}
    \end{subfigure}
\caption{Framework (i): comparing \emph{TypiClust} with baseline AL strategies on CIFAR10, CIFAR100, and ImageNet-100 for 5 active learning iterations in the low budget regime. The budget $B$ is equal to (top) the number of classes, or (bottom) $5$ times the number of classes. The final averaged test accuracy in each iteration is reported, using 10 (CIFAR) and 3 (ImageNet) repetitions. The shaded area reflects the standard error across repetitions.}
\label{fig:main_al_graph}
\end{center}
\vspace{-.55cm}
\end{figure}

\subsection{Results: Low Budget Regime}
\label{sec:emp_results}
The amount of labeled data that should be considered as low-budget is likely to vary between tasks. In the following experiments, unlike many previous works, we focus on scenarios where $0.02\%\sim12\%$ of the examples are labeled.

\myparagraph{(i) Fully supervised framework.}
Fig.~\ref{fig:main_al_graph} shows the accuracy of networks trained on CIFAR-10, CIFAR-100, and ImageNet-100, using the labeled examples queried by the different AL strategies. Denoting the number of classes by $M$, we use a budget of either $B=M$ or $B=5M$ labeled examples, with $L_0=\emptyset$. See App~\ref{app:more_sup_empirical} for other budget sizes.

We see that in the low budget regime, both \emph{TypiClust} variants outperform the baselines by a large margin. Specifically, all other baseline AL methods perform on par with random selection or worse, in accordance with \citet{pourahmadi2021simple}. In contrast, the typicality-based strategy achieves a large gain in accuracy.
Noting that most of the baselines are possibly hampered by their use of random initial pool selection when $L_0=\emptyset$, our ablation study in Section~\ref{sec:starting_from_random} demonstrates that this is not the decisive factor.

\myparagraph{(ii) Fully supervised with self-supervised embedding.}
As self-supervised embeddings can be semantically meaningful, they are often used as features for a linear classifier. Accordingly, in this framework, we use the extracted features from the representation learning step and train a linear classifier on the queried labeled set $L_i$. Unlike the fully supervised framework, here we use the unlabeled data while training the classifier, albeit in a basic manner. This framework outperforms the fully supervised framework, but still lags behind the semi-supervised framework. Once again, \textit{TypiClust} outperforms all baselines by a large margin, as shown in Fig.~\ref{fig:weak_semi} (see App~\ref{app:more_lin_empirical} for additional datasets).

\begin{figure}[htb!]
\begin{center}
\begin{subfigure}{.45\textwidth}
  \centering
 \includegraphics[width=\linewidth]{avihu_graphs/legend_newnames.png}
\end{subfigure}
\\
\begin{subfigure}{.154\textwidth}
      \centering
      \includegraphics[width=\linewidth]{avihu_graphs/linear_layer_onegraph/cifar10_yname.png}

    \caption{CIFAR-10}
    \label{fig:weak_semi_cifar10}
    \end{subfigure}
    \begin{subfigure}{.154\textwidth}
      \centering
      \includegraphics[width=\linewidth]{avihu_graphs/linear_layer_onegraph/cifar100_yname.png}

    \caption{CIFAR-100}
    \label{fig:weak_semi_cifar100}
    \end{subfigure}
    \begin{subfigure}{.154\textwidth}
      \centering
      \includegraphics[width=\linewidth]{avihu_graphs/linear_layer_onegraph/imagenet100_yname.png}

    \caption{ImageNet-100}
    \label{fig:weak_semi_imagenet200}
    \end{subfigure}
\caption
Similar to Fig.~\ref{fig:main_al_graph} in framework (ii), where a linear classifier is trained on features from a pre-trained self-supervised model.}
\label{fig:weak_semi}
\end{center}
\vspace{-.6cm}
\end{figure}

\myparagraph{(iii) Semi-supervised framework.}
In this framework, we evaluate \emph{TypiClust} and different AL strategies by examining the performance of FlexMatch when trained on their respective queried examples.
As semi-supervised methods often achieve competitive performance with only a few labeled examples, we focus on the extreme low-budget regime, where only $0.02\%\sim1\%$ of the data is labeled. Note that semi-supervised algorithms typically assume a class-balanced labeled set, which is not feasible in active learning. To compare with this scenario which dominates the literature, we add a balanced random baseline for reference.

In Fig.~\ref{fig:semi_supervised}, we compare the final performance of FlexMatch using the labeled sets provided by different AL strategies. We show results for a budget of $10$ examples in CIFAR-10 (Fig.~\ref{fig:ssl_cifar_10_with_10_examples}), $300$ examples in CIFAR-100 (Fig.~\ref{fig:ssl_cifar_100_with_300_examples}), and $1000$ examples in TinyImageNet (Fig.~\ref{fig:ssl_tiny_imagenet_with_1000_examples}). We see that both \emph{TypiClust} variants outperform random sampling, whether balanced or not, by a large margin.
In contrast, other AL baselines do not improve the results of random sampling. Similar results using additional budgets, baselines, datasets, and semi-supervised algorithms, can be found in App~\ref{app:more_ssl_empirical}.

\begin{figure}[htb!]
\vspace{-.3cm}
\begin{center}
    \begin{subfigure}{.154\textwidth}
      \centering
      \includegraphics[width=\linewidth]{graphs/semi_supervised/cifar10_10samples_ssl_comparison.png}
    \vspace{-0.75cm}
    \caption{CIFAR-10}
    \label{fig:ssl_cifar_10_with_10_examples}
    \end{subfigure}
    \begin{subfigure}{.154\textwidth}
      \centering
      \includegraphics[width=\linewidth]{graphs/semi_supervised/cifar100_ssl_comparison.png}
    \vspace{-0.75cm}
    \caption{CIFAR-100}
    \label{fig:ssl_cifar_100_with_300_examples}
    \end{subfigure}
    \begin{subfigure}{.154\textwidth}
      \centering
      \includegraphics[width=\linewidth]{graphs/semi_supervised/tiny_imagenet_ssl_comparison.png}
    \vspace{-0.75cm}
    \caption{TinyImageNet}
    \label{fig:ssl_tiny_imagenet_with_1000_examples}
    \end{subfigure}

\caption{Comparison of AL strategies in a semi-supervised task. Each bar shows the mean test accuracy after $3$ repetitions of FlexMatch trained on: (a) $10$ examples from CIFAR-10, (b) $300$ examples from CIFAR-100, (c) $1000$ examples from TinyImageNet. Error bars denote the standard error.}
\label{fig:semi_supervised}
\vspace{-0.3cm}
\end{center}
\end{figure}

\subsection{Ablation Study}
\label{sec:ablation_study}
We now report the results of a set of ablation studies, checking the added value of each step in our suggested strategy.

\begin{figure}[htb]

\begin{subfigure}{.48\textwidth}
  \centering
 \includegraphics[width=\linewidth]{avihu_graphs/ablations/pretty_abl_legend.png}
\end{subfigure}
\\
\begin{subfigure}{.154\textwidth}
  \centering
 \includegraphics[width=\linewidth]{avihu_graphs/ablations/from_random_yname.png}
\caption{}

\label{fig:cifar10_random_init}
\end{subfigure}
\begin{subfigure}{.154\textwidth}
  \centering
 \includegraphics[width=\linewidth]{avihu_graphs/ablations/tv_dist_half.png}
\caption{}
\label{fig:balancing_metric}
\end{subfigure}
\begin{subfigure}{.154\textwidth}
  \centering
   \includegraphics[width=\linewidth]{avihu_graphs/ablations/added_rand_inverse.png}
\caption{}
\label{fig:scores_ablation}
\end{subfigure}

\vspace{-.3cm}
\caption{(a-b) Same experiment as in Fig.~\ref{fig:main_al_graph_cifar10} top, but where: (a) \emph{TypiClust} uses random initial set selection; (b) the Total Variation (TV) distance between the labeled set distribution and the ground truth class distribution is shown.
(c) To isolate the added value of clustering for diversity and typical sample selection, we evaluate 3 additional selection heuristics on CIFAR-10 (see Section~\ref{sec:abb-denisity}).}
  \label{fig:agreement_testics}
  \vspace{-0.6cm}
\end{figure}

\subsubsection{Random Initial Pool Selection}
\label{sec:starting_from_random}

As our AL strategies are based on self-supervised learning, they are well suited for the case $L_0=\emptyset$, and can actively query the initial selection of labeled examples. By contrast, the other AL baselines use random initial pool selection when $L_0=\emptyset$. To isolate the effect of this difference, we conducted  the same experiment as reported in Fig.~\ref{fig:main_al_graph_cifar10}, giving \emph{TypiClust} a random initial pool selection just like the other baselines. Results are reported in  Fig.~\ref{fig:cifar10_random_init}, showing that \emph{TypiClust} still outperforms all baselines. Importantly, this comparison reveals that non-random initial pool selection yields further generalization gains when combined with active learning, which is useful in real-life problems.

\subsubsection{Comparing Class Distribution}
\label{sec:class_balancing}
With an extremely low budget, covering the support of the distribution comprehensively is challenging. To compare the success of the different AL strategies in this task, we measure the Total Variation (TV) distance between the labeled set class distribution and the ground truth class distribution on each strategy. Fig.~\ref{fig:balancing_metric} shows that \emph{TypiClust} variants achieve a significantly better (lower) score than the alternatives, resulting in queries with a better class balance.

\subsubsection{The Importance of Density and Diversity}
\label{sec:abb-denisity}
\emph{TypiClust} clusters the dataset and selects the most typical examples from every cluster. To assess the added value of clustering and typicality selection, we consider the following alternative selection criteria:
\begin{inparaenum}[(a)]
    \item Select a random example from each cluster (TPC$_{Rand}$).
    \item Select the most atypical example in every cluster (TPC$_{Inv}$).
    \item Select typical samples greedily, without clustering (TPC$_{NoClust}$).
\end{inparaenum}

The results in Fig.~\ref{fig:scores_ablation} show that both clustering and high-density sampling are crucial for the success of \emph{TypiClust}. The low performance of TPC$_{Rand}$ shows that representation learning and clustering alone cannot account for all the performance gain, while the low performance of TPC$_{NoClust}$ shows that typicality without diversity is not sufficient

\subsubsection{Uncertainty Delivered by an Oracle}
\label{sec:ablation_oracle_uncertainty}

When trained on only a few labeled examples, neural networks tend to overfit, which may result in the unreliable estimation of uncertainty. This offers an alternative explanation to our results -- uncertain examples may be a good choice in the low-budget regime as well, if only we could compute uncertainty accurately.

To test this hypothesis we first train an "oracle" network (see App.~\ref{app:sec:ablation_details}) on the entire CIFAR-10 dataset and use its softmax margin to estimate uncertainty. This "oracle margin" is a more reliable uncertainty measure, which is then used to choose the query examples. Subsequently, another network is trained similarly to the  setup of Fig.~\ref{fig:main_al_graph_cifar10}, adding in each iteration the examples with either the highest or lowest softmax response margin according to the oracle.

\begin{wrapfigure}{R}{0.24\textwidth}
\centering
\vspace{-0.35cm}
\includegraphics[width=0.24\textwidth]{graphs/ablations/ablation_calculate_hardness.png}
\vspace{-0.75cm}
\caption
Certainty, as estimated by the margin of an oracle that knows the labels, is used for AL. We plot mean test accuracy of $100$ models trained on CIFAR-10,
 $|L_0|=10$, $B=10$.}
\vspace{-0.3cm}
\label{fig:oracle}
\end{wrapfigure}

The results are shown in
Fig.~\ref{fig:oracle
. We see that even a reliable measure of uncertainty leads to poor performance in the low-budget regime, even worse than the baseline uncertainty-based methods. This may be because these methods compute the uncertainty in an unreliable way, and thus behave more like the random selection strategy.

\section{Summary and Discussion}

We show, theoretically and empirically, that strategies for active learning in the high and low-budget regimes should be based on opposite principles. Initially, in the low budget regime, the most typical examples, which the learner can learn most easily, are the most helpful to the learner. When reaching the high-budget regime, the best examples to query are those that the learner finds most confusing. This is the case both in the fully supervised and semi-supervised settings: we show that semi-supervised algorithms get a significant boost from seeing the labels of typical examples.

Our results are closely related to curriculum learning \citep{bengio2009curriculum,hacohen2019power}, hard data mining, and self-paced learning \citep{kumar2010self}, all of which reflect the added value of typical ('easy') examples when there is little information about the task, as against atypical ('hard') examples  which are more beneficial later on. The point of transition - what makes the budget 'small' or 'large', depends on the task and corresponding data distribution. In complex real-life problems, the low budget regime may still contain a large number of examples, increasing the usefulness of our method. Determining the range of training sizes with 'low budget' characteristics is a challenging problem, which we leave for future work.

\smallskip
\paragraph{\large{Acknowledgements.}}
This work was supported in part by a grant from the Israeli Ministry of Science and Technology, and by the Gatsby Charitable Foundations.

\bibliographystyle{plainnat}

redpajama_set_name: RedPajamaArXiv


International Journal of Law & Education is the journal of the Australia and New Zealand Education Law Association Ltd. ANZELA was established in 1991 in order to promote the interests of educators, academics and lawyers in Australia and New Zealand on issues related to law and education. Two objectives of ANZELA are to promote: (i) the research, study, discussion, writing and/or dissemination of information about issues on laws relating to education; and (ii) publication of articles, newsletters, journals and other literature.
The journal would appeal to persons with an interest in education and the law including: school principals, teaching and administrative staff; personnel in colleges of technical and further education; university staff in the faculties of law and education; education policy makers; and legal practitioners with an interest in education law.
Opinions expressed in the journal are those of the individual contributors and no responsibility is accepted for them by the Editorial Board or ANZELA. The contributors shall indemnify the editors and the publisher from all liability including costs which may arise as a result of the publication of any article.
Copyright © 2013 ANZELA, except where otherwise stated. The author(s) retain copyright in the article. The author(s) will be asked to grant a non-exclusive licence to ANZELA to authorise the reproduction of the article for further research, teaching, publication and other academic purposes.


text:
Q: $\int {3\over 2t+100}\;dt$ has two solutions The integral
$$\int {3\over 2t+100}\;dt$$
has two solutions, why is that?
Some websites show that the answer is ${3\over 2} ln(t+50)$ while other websites show that the answer is ${3\over 2} ln(2t+100)$. If I plot it, the graphs aren't the same.
They all use u-sub with $u=2t+100$ or with $u=t+50$ which give different answers, and it confuses me.

A: Note that $\log(2t+100)=\log(t+50)+\log2$ -- so the difference between the two antiderivatives is just a constant of integration.
Remember that antiderivatives are only unique up to an arbitrary constant term.

redpajama_set_name: RedPajamaStackExchange


This development of new 4 luxurious villas on private plots, is located to the eastern side of the "Mijas Triangle", in a quiet urbanization, easily accessible from the highway and with stunning sea and mountain views. Only 10 minutes to Malaga International Airport and 5 down to the amenities of Fuengirola city, Phase 2 of this new development, offers luxurious villas of 150 to 250 m2 constructed living area and 70 to 100 m2 of basement, that can be used as a gymnasium, games room, kids play area.
Turnkey villas, with modern and integral design and an efficient construction. Through the most innovative construction system in the market, a solution of ecological, economic and rapid construction is offered, complying with all aspects of the new energy consumption legislation of homes in Spain.
Building time: 8 months approx. from start of construction.
The Villas have modern architecure, will be designed with 3 and 4 ensuite bedrooms and each owner will have the final say in the layout and distribution of their project. The plots start at over 680 m2, up to 1,300 m2 +, so there is ample room for an extensive decking area around a private swimming pool and gardening area.
There is a great opportunity to design and develop 3 or 4 bedrooms luxuries contemporary villas that will enjoy the panoramic southern sea view!
Please, send me more information about ref LUX0277 in Buena Vista, Mijas Costa or similar properties.
Need Advice buying this property for sale in Buena Vista, Mijas Costa?
Disclaimer This is a property advertisement (REF LUX0277) provided and maintained by Cleox Inversiones, Urb. Aloha Gardens, Bloque 11, Oficina 8 29660 Marbella - Nueva Andalucía Málaga, and does not constitute property particulars. Whilst we require advertisers to act with best practice and provide accurate information, we can only publish advertisements in good faith and have not verified any claims or statements or inspected any of the properties. PropertyTop.com does not own or control and is not responsible for the properties, website content, products or services provided or promoted by third parties and makes no warranties or representations as to the accuracy, completeness, legality, performance or suitability of any of the foregoing. We therefore accept no liability arising from any reliance made by any reader or person to whom this information is made available to.


text:
70,000 refugees stranded on Jordan border will finally receive aid under new deal
Published October 10th, 2016 - 09:00 GMT
Syrian refugees at the Rukban camp, near the Jordanian border. (AFP/File)
Based on recent developments and upon further discussions with international agencies, Jordan will allow the delivery of aid to the berm through cranes, to be collected by the community leaders of the displaced Syrians near the border for distribution.
In remarks to The Jordan Times on Sunday, Minister of State for Media Affairs and Government Spokesperson Mohammad Momani said the distribution process will be monitored "in different ways".
Asked if the new measures entail any change to Jordan's policy regarding the berm, the no-man's land between the Jordanian and Syrian borders where the displaced Syrians are stranded, Momani said Jordan will maintain its sealed border policy, and this delivery of aid will not affect that.
The injured and humanitarian cases will be allowed entry based on the assessment of agencies on the ground.
According to the UN, more than 85,000 Syrians are stranded at Rakban settlement in the no-man's-land between Jordan and Syria.
Momani explained aid would be transported by cranes directly from the northeastern border to the other side.
Community leaders in Rukban will receive the aid once lowered and distribute it to the refugees there.
Members of the Tribal Council of Palmyra and Badia, a Syrian rebel group in charge of running the Rukban camp's daily affairs, told The Jordan Times that the preliminary agreement entails the resumption of aid delivery as of next week.
Jordan declared the northern and northeastern border areas a closed military zone in June in the aftermath of a terrorist attack that targeted a military post serving refugees near the border, killing seven troops and injuring 13 others.
Since then, Jordan has agreed with international relief agencies to allow a one-off aid delivery to the area in August, while global stakeholders were expected to seek alternative solutions to continue the mission.
Tags:Jordan TimesTribal Council of Palmyra and BadiaUnited NationsMohammad Momani
© Copyright The Jordan Times. All rights reserved.

redpajama_set_name: RedPajamaCommonCrawl


text:
Star Trek: The Mirror War #6 – Comic Review
by Lee Thacker@FubsyShabaroon 13 May 2022 12 May 2022
Leave a Comment on Star Trek: The Mirror War #6 – Comic Review
With the second season of Star Trek: Picard having drawn to a close recently, viewers were given an insight into Jean-Luc Picard's backstory, taking us back to his childhood, showing a traumatic event which had remained long-since buried, and in the process helping the Admiral to heal himself and come to terms with his past.
IDW's Star Trek: The Mirror War also gives us the chance to look back into Picard's history, albeit the bewhiskered, evil version of the familiar character, from the 'Mirror Universe'. In his latest issue, we get to see Picard having something of a reckoning with a figure with whom he has a long – and not always amiable – association, and has already crossed paths with earlier in the run, with Picard set on making sure things will come to a conclusion once and for all.
Writers Scott and David Tipton let us glimpse something of this alternative Picard's background, in the hope of making him into more than just a ranting, impulsive maniac, which is how he has often come across, shooting first and thinking later. We know how our Picard is a man of intellect as well as introspection, and while we might not expect this variant of Picard to possess much of the latter, his caprices and drives have somewhat hampered his use of the former.
READ MORE: Smallville 4×06 – 'Transference' – TV Rewind
Finally, however, Picard seems to be using more of his guile, and rather than acting like some blunt instrument hell bent on using physical over intellectual muscle, he now appears to be successful in keeping more of his rasher instincts in check. While still undeniably brutal, 'Mirror' Picard is using more of his tactical thinking, and acting as though he has a strategy, rather than just aiming to crush any opposition through use of sheer brute force alone. At last, we actually have the first glimmerings of some hidden depths to the man.
Our Captain is not the only one to shine in this issue, as his enforcer in the form of Deanna Troi gets some moments of her own in which to excel. Like Counsellor Troi, this Deanna helps people to get in touch with their pain; unlike our Troi, however, she is actually the cause of it in the first place. This is a very BDSM-inspired take on Deanna, with our Inquisitor having a sadistic streak, and sporting a very fetish-type look (good to see this Troi is not one for following uniform codes either).
READ MORE: Star Trek: Discovery – Adventures In The 32nd Century #3 – Comic Review
While Picard is slowly but surely drawing his plans, making a power play by assembling a very impressive and formidable armada of ships, the Klingon-Cardassian Alliance is growing increasingly suspicious of what he is doing tactically, having seen signs that he is changing his approach. All of this looks to be culminating in what should promise to be a conflict of major proportions, suggesting that a devastating campaign awaits them in the next chapter of the story.
The Mirror War artist Gavin Smith has been delivering some impressive spreads so far, but if the scale of the forthcoming battle – if not war – lives up to what appears to be promised, the Tiptons will certainly be calling upon Smith to pull out all the stops.
Star Trek: The Mirror War #6 is out now from IDW Publishing.
Tags: David Tipton IDW Publishing Scott Tipton Star Trek Star Trek: The Mirror War Star Trek: The Next Generation
Previous Entry Star Trek: Discovery – Adventures In The 32nd Century #3 – Comic Review
Next Entry Revolver (1973) – Blu-ray Review


Thinks this looks like FUN!
Hell yes that looks FUN! Josh Brooks was doing it the best - he did it big every lap through there (although I think in one of the races, he binned it).
BSB is an amazing series, probably the most entertaining series I watch. They're still playing it on Velocity.
OWWWW 2:35 that`s a NASTY looking hi-side!
There's no TC in BSB - it's against the rules.
Edit: And GPS is useless in BSB - they're not allowed to have different electronic settings corner-to-corner. They're allowed to adjust ignition timing and fuel mapping PER CYLINDER - and that's it.
It's old school - right wrist and left fingers for traction control.
Last edited by Conrice; 02-05-2014 at 10:50 PM.


text:
@{

# Script module or binary module file associated with this manifest.
ModuleToProcess = 'Exfiltration.psm1'

# Version number of this module.
ModuleVersion = '3.0.0.0'

# ID used to uniquely identify this module
GUID = '75dafa99-1402-4e29-b5d4-6c87da2b323a'

# Author of this module
Author = 'Matthew Graeber'

# Copyright statement for this module
Copyright = 'BSD 3-Clause'

# Description of the functionality provided by this module
Description = 'PowerSploit Exfiltration Module'

# Minimum version of the Windows PowerShell engine required by this module
PowerShellVersion = '2.0'

# Format files (.ps1xml) to be loaded when importing this module
FormatsToProcess = 'Get-VaultCredential.ps1xml'

# Functions to export from this module
FunctionsToExport = '*'

# List of all files packaged with this module
FileList = 'Exfiltration.psm1', 'Exfiltration.psd1', 'Get-TimedScreenshot.ps1', 'Out-Minidump.ps1',
           'Get-Keystrokes.ps1', 'Get-GPPPassword.ps1', 'Usage.md', 'Invoke-Mimikatz.ps1',
           'Invoke-NinjaCopy.ps1', 'Invoke-TokenManipulation.ps1', 'Invoke-CredentialInjection.ps1',
           'VolumeShadowCopyTools.ps1', 'Get-VaultCredential.ps1', 'Get-VaultCredential.ps1xml'

}

redpajama_set_name: RedPajamaGithub


text:
Home•Projects•THE LOLLYWOOD ART PROJECT
THE LOLLYWOOD ART PROJECT
"Lollywood once competed with its Indian counterpart, but has over the past many years seen a constant decline and is today being replaced by its foreign competitors. Lollywood through the ages aims to tap on those people who were responsible for the rise of the industry during the 60s, and the 70s and then those who witnessed the downward spiral in the later years." – Haroon Khalid, Director, Lollywood Through the Ages, Lahore
Lollywood, based in Lahore started in 1929 with the opening of the United Players' Studios on Ravi Road. The Pakistani film industry is known to have produced some of the most remarkable and talented writers, actors, directors and for introducing pop music to South Asia. The Citizens Archive of Pakistan has preserved a rare collection of film posters and film magazines from Lollywood to honor the efforts of all those that were and are involved in the industry. Films, images, articles and advertisements of films collected from libraries and private collectors have digitized. Our Archive has 60 rare posters in the print form and digitized. As our collection grows, we are hope to add rare records of Lollywood hits and original film reels too.
January 2013: 'The Lollywood Art Project' was launched at Port Grand, Karachi. The event featured workshops by Master Painter Faiz Rahi, live painting, and an art exhibition as a celebration and revival of the Lollywood art culture in Pakistan.
November 2012: Iqbal Day exhibit opened at Port Grand, Karachi. The exhibition included a timeline on Iqbal's life, his handwritten work and a virtual tour of the Iqbal Academy in Lahore.
November 2012: The Exchange for Change: Pakistan India 2012 2013 program press launch was held at Marriott Hotel, Karachi. At the press conference, singer-actor Ali Zafar presented the ideology behind the program and helped create widespread awareness of promoting a peaceful dialogue between the youth of Pakistan and India. Routes2Roots Board Members Ms. Tina Vachani and Mr. Rakesh Gupta were present along with CAP Board Members.


Create and manifest the life of your dreams.
It is about 'Owning Your Life' and not simply getting through the days. Nicky and I are both firm believers of manifesting your thoughts and we have both manifested some pretty amazing things, which I have told you a bit about in My Manifesting Journey.
define your vision or have any idea what your soul really desires.
Have you actually spent any time at all sitting down and thinking about what really matters in your life, what makes you happy, what you would want to spend your life doing?
Most of us don't. We are just too busy, being busy most of the time.
I was recently saying to Nicky that I wanted to create a new vision board for myself. We had both created one probably about a year ago and many of the things that were on my board I have already manifested in my life (same for Nicky too). Hence us being BELIEVERS!!
But before you can start on a vision board it is a good idea to try and get a clear picture of what you want to manifest into your life.
Do you want a new car? Do you want to meet the love of your life? A new kitchen? What are the things you want to attract into your life? Health, happiness, money … all of the above?
Since doing my first vision board, so many of the things I wanted to manifest I now have. So I need to do another one.
What do I now want in my life? I need to DEFINE my vision.
You can join me by completing the Message to Me. This is a letter you will write to yourself … a message to me, and can be purchased in our shop.
light a candle and spend some time just on YOU.
Sit and think about these things ….
I am ….. here you need to write all the things you wish to be. I am … confident. I am … brave. I am … healthy. Anything and everything that you wish to be. Not what you think you are now, what your soul feels will empower you to be who you really are.
I have … here you need to write about all the things you want. I have … a new car. I have … time with my family. I have … lots of good friends. I have … a happy and healthy family. Really think about this one. It is so easy to get caught up with wanting 'things' and that is fine, but in that peaceful place whilst you are doing this, search deep for the things that will really light up your life, into all you have ever dreamed of.
I will … here you need to write about all the things you will do. I will … travel to Japan. I will … have a family holiday to Australia. I will … build my dream home. I will … have a baby.
creating a Life Map in our Create a Vision Board post.
I was the same Maria when I first started doing it! But I found once I started everything became clearer… the "Message to Me" printable is a really great tool to help with this.


CD34 Rabbit Recombinant mAb detects endogenous level of total CD34.
CD34 has been widely used as a marker to assist in the identification and isolation of hematopoietic stem cells (HSCs) and progenitors in preparation for bone-marrow transplantation; more recently it has been employed as a marker to help identify other tissue-specific stem cells, including muscle satellite cells and epidermal precursors.
 Nielsen JS, et al. J Cell Sci. 2008, 121(Pt 22):3683-92.
 Laura E Sidney, et al. Stem Cells. 2014, 32(6): 1380–1389.


text:
Ricardo Bonilla González es un economista colombiano. Ha sido profesor de la Universidad Nacional y director del Centro de Investigaciones para el Desarrollo de dicha universidad. Ha participado como asesor e investigador de diversas organizaciones nacionales e internacionales. Fue secretario de Hacienda de Bogotá durante la alcaldía de Gustavo Petro y, junto a Jorge Iván González y Luis Jorge Garay, responsable del programa económico que llevó a la presidencia a Petro en 2022. Es seguidor de la línea keynesiana en economía.

En octubre de 2022 fue designado por el presidente Petro para ser el director de la Banca Financiera de Desarrollo Territorial (Findeter).

Estudios
Realizó estudios de Economía en la Universidad Nacional y se graduó de economista en la Universidad Jorge Tadeo Lozano en agosto de 1975. Entre los años 1988 a 1990 cursó una especialización de D.E.A Diploma de Estudios a Profundidad en «Economía Industrial y Cambio Mundial» en la Universidad de Rennes 1 en Francia y fue candidato a «Doctor en Ciencias Económicas» de la misma universidad.

Trayectoria
En el área educativa, de 1982 a 2011, Bonilla se desempeñó como profesor en diferente universidades. Fue Profesor de la Facultad de Ciencias Económicas de la Universidad Nacional con sede en Bogotá. Catedrático de las universidad Externado, Rosario, la ESAP, UPTC y Cooperativa.

Como investigador trabajó en el Centro de Investigaciones para el Desarrollo desde 1995 a 2011. Fue director del mismo centro de 1998 a 2001. Además participó en diferentes centros de investigación del país y ha sido consultor de la OIT, el BID, Fenalco, DANE y otras instituciones. Ha escrito diversos artículos o capítulos de libros de la Universidad Nacional y de entidades del Estado Colombiano entre otros.

Secretario de Hacienda de Bogotá
En el año 2011 fue designado por el recién electo Gustavo Petro como secretario de Hacienda de Bogotá, cargo que desempeñó desde el primero de enero de 2012 hasta abril de 2015 cuando renunció debido a que cumplió 65 años de edad que es la edad límite permitida por la ley para algunos funcionarios públicos.

Tras finalizar su trabajo en la alcaldía, Bonilla fue llamado por el gobierno de Juan Manuel Santos para integrar una comisión de nueve expertos que lo asesorarían en la formulación de una reforma tributaria.

Equipo económico de Petro
En 2018 acompañó a Gustavo Petro en la formulación de su propuesta económica para su candidatura presidencial . En 2022 continuó haciendo parte, junto con Jorge Iván González y Luis Jorge Garay, del equipo económico de la candidatura que llevó a la presidencia a Petro en el año 2022, e incluso llegó a ser mencionado por el presidente electo como posible ministro de Hacienda.

Referencias

Nacidos en Bogotá
Economistas de Colombia
Profesores de Colombia
Funcionarios durante la alcaldía de Gustavo Petro

redpajama_set_name: RedPajamaWikipedia


text:
Mobile Apps /
Declutter The Mind is an app that offers free guided meditations to help users deal with stress, anxiety, sleep, and more. In the app, users can browse through and find meditations on various subjects, such as eating, walking, breathing, and visualization for anxiety and sleep. The app is available for free from the app store, while monthly and annual subscriptions are available for full access to all courses and meditations for $7.99 and $79.99, respectively.
Mindfulness and meditation can help people navigate the stresses of everyday life and deal with depression and anxiety. The Declutter The Mind app allows users to learn how to meditate properly with guided audio meditations and articles with helpful tips. There are meditations available for beginners and for people who are very experienced.
"The Declutter Your Mind app allows users to learn how to meditate properly with guided audio meditations and articles with helpful tips."
Guided Meditation Library
The app features a library of guided meditations where users can find the meditation they need by category. Categories include Mindfulness, Anxiety and Stress, Sleep and Relaxation, Managing Emotions, and many more. Users can either select the meditation to start right away or add it to their favorites to try at a later time.
It can be difficult to find time to meditate, so Declutter The Mind allows users to set reminders through the app. You can set a reminder for a specific time and the app will send you a push notification to let you know when it's time to meditate.
Meditation Courses
All of the guided meditations are free. However, the app also features meditation courses that teach users how to meditate and learn how to practice. The first 5 days of these courses are available for free in the app and then if you like them, you can subscribe to the paid version of the app to continue enjoying the full course. The app is available at $7.99 for a month and at $79.99 for the full year.
Find Groups
Find Trials


In the 1930's, Hawaiian Music traveled to 750 radio stations around the world bringing "Aloha" and a visual image of Hawaii created solely by the music itself. People were lured to the Islands on steam ships after hearing the weekly radio program, "HAWAII CALLS." Legendary singers from that era and beyond include: Alfred Apaka, Hilo Hattie, Chick Daniels, Auntie Alice Namakelua, Gabby Pahinui and Don Ho.
A memorial walk has been constructed in Waikiki at a location where music was once written by the beloved Queen Lili`uokalani. There will be bronze plaques on the sidewalk with a description of each Legend and on the adjacent wall there will be another bronze plaque showing each of their portraits and names.
We need your help to make this memorial a reality and to preserve the history and tradition of Hawaiian Music for generations to come. Your assistance will enable millions of tourists and thousands of Hawai`i residents who visit Waikiki each year to learn about the Legends whose music helped to make Waikiki world famous. It will also serve as a living inspiration to young Hawai`i musicians. As a Founding Contributor, you will know that you have made this famous walk possible.


Word Lists are compilations of words from a wide variety of sources including some dictionaries. The word lists are commonly in the public domain.
What sets the lists apart are what they are used for. For example, the official Scrabble™ player's dictionary, known as OSPD, includes words that are playable under the "official rules" of Scrabble. The words are 8 letters or less in length, no proper names, no abbreviations.
Each list has a history, a source, and some bias. We've tried to credit all the sources, as well as any biases we've observed. On our page of word lists you will also find compilations of all the lists into a single list.
What do we use word lists for?
If you are new to Regular Expressions, they can be quite daunting. If you are familiar with the basics, they can still be quite daunting! They are an extremely powerful tool and a remarkably concise way to interpret text. They would certainly not be considered "light reading," but they can be read.
Caveat: When you get into more sophisticated features, regular expression systems vary. You'll want to confirm the syntax for the system you are using.
A regular expression is a symbolic representation of the most general search query one can make to that word finder. Any letter just means itself. A . (a period) means any single letter (or other character). A * after any expression means 0 or more consecutive occurrences of that expression (or more accurately, of anything that MATCHES that expression). Regular expressions generally match on lines of text in a file; since word lists have one word per line, your regular expressions will match on words.
Usually, regular expressions are case-SENSITIVE, so if you ask for a j you won't match J. Most regular expressions as well as the NPL word finder provide an option to override case sensistivity.
Regular expressions can be built up of simpler regular expressions; some examples follow.
Since we didn't user use ^ or $ here, this will find the letters anywhere in a line.
To search for "regex1 OR regex2" just use (regex1)|(regex2) — the parentheses ( ) serve to group the two halves of the regular expression so they are not combined in a way you did not intend. The vertical line character | as the character for OR).
There are a whole bunch of other things one can ask about. In these examples, regex can be any complete regular expression (excepting ^ and $, in most cases, as that would produce something that will find no matches).
Subexpressions are a very powerful concept. \( and \) indicate subexpressions. \n re-uses the nth subexpression.
Explanation: Using the pair \( as a left bracket and \) as a right one (note backslash), the subexpression \(something\) defines a pattern that the part of the regular expression to its right will understand as the pair \n, where n is the number of aforesaid subexpression counting from the left until it's first encountered. (All but the last example involved only one subexpression, so only \1 was used in those.) The notation is very dense and unforgiving, but it's really not complicated. Note: Some regular expression systems work with ( and ) for subexpressions, not \( and \).
If you want to match a character which is a regular expression character, surround it with , as in [(] to match a begin parentheses.
For futher reading, you want to check out an online tutorial like this one or this one. You also might find this summary sheet handy.
Once you're familiar with regular expressions, you'll probably want to use them to find some bases for puzzles. The following examples should help you get started.
Suppose you want to transpose "aeginrst". Put [^ganister] into the first search string, and specify "don't match"; then put \(.\).*\1 into the second search string, and specify "don't match" there, too. Then choose a word length of 8. If you use NI2 as your dictionary, this query will retrieve you three words: astringe, ganister and gantries, all of which are transposals of aeginrst.
This works because the first string, along with the "don't match", says: "find all words that don't contain any of the letters other than aeginrst". So only words that contain some or all of those letters will be returned. The second query says "don't match anything that has a repeated letter anywhere. Since aeginrst has no repeated letters, that's a requirement. Finally, the 8-letter length requirement forces every one of the 8 letters to be used, so that every returned word must be a transposal.
This won't work for transposals containing spaces, or if any letters are repeated. You can get an approximation to the list by omitting the second search string, though, and this is sometimes good enough.
What banks down to "lens"? Put [^lens] into the first search string and choose "don't match"; the result (using NI2) will be a list of 47 words, all of which contain only l's, e's, n's and/or s's. However, the list includes words like "eel" and "sense" which don't contain all four of the letters. The best that can be done here is to put …. into the second search string. This will force the resulting words to be at least four letters long, which eliminates "eel" but not "sense".
As with letter banks, this isn't perfect: it returns alveole, which is good, but it also returns potoroo, which is not. There is no easy way to improve this; you just have to hope there aren't too many false positives in the list. For strongly patterned words this is not much of a problem; for weakly patterned words the list is likely to be very long anyway.
You can filter the result a little, though, by specifying in this case that no letter should repeat three times. You can do this by putting \(.\).*\1.*\1 in the second search string, and specifying "don't match"; this reduces the matches from 69 to 49, and eliminates potoroo.


Finally, I have a chance with that one :) After I played much more complex and complicated contest missions, this one was like morning walk for a bread roll ;) The mission is very simple in almost every aspect, yet it is good and solid.
I don't know where/what the secret of this mission is, but somehow You did it, and You just did it well :) I like it very much (as little outdated b0.3 ;) You did something with pretty much nothing :) and it is a gift that few can boast of :) Found 5271/5271 of loot, zero blackjacks, zero 1st alarms, 36 min.
Just got a chance to look at my mission comments! Just wanted to say thanks for all the words of encouragement, I was super nervous to actually start learning wtf to do with this map editor BEAST known as Dromed, but I'm glad I finally got over that hurdle and started actually tinkering with it.
The mission, if you couldn't tell, was a sort of love letter to TTLG for some of the memes and some of the more "unique" posters of the board, being a lurker for quite some time now.


BEUMER troughed belt conveyors: > 400 m for horizontal curves, vertical curves . Long distances, large heights, small radii - the BEUMER Group system solutions are . In rough terrain, the transport of goods - for example by truck - involves.
May 16, 2018 . Transport belts are the easiest and cheapest method of automatic item transportation. They are the first tier among the three transport belts.
Mar 2, 2018 . The Underground belt is the first tier of underground belts. This type of belt is generally used to allow a transport belt to cross another . Belt transport system . Long handed inserter · Fast inserter.png . Stone brick.png.
The troughed belt conveyor is one of FEECO International's most commonly . The conveyor is typically half the length of the track rails; for example, a 100' long shuttle conveyor would have the ability to transport material throughout the length of a 200' storage facility. The most .. Gravel, Stone, Coal, Fine Ore). 100 lb.
A belt conveyor is included in nearly every solution we design for our customers in the waste recycling and bulk handling industry. . belt conveyor bottom ash transport . granite bulk handling conveyor · long belt conveyor system for coal.
A real all-rounder for transport systems are the conveyor belts, also called belt conveyors . In the past, long transport routes and insufficient technical possibilities . with tight roller pitch for the transport of complete stone packages Conveyors.
Conveyor transport plays such a central role in . To address this problem, offers Belt guiding systems that sense and . conveyor, but can also occur during transport. One risk .. 90 Shore A PU. Metric tonnes: 138,000. Hours: 600. Type of material: Granite. Material size: . The right cleaner lasts 5 times longer. Blade.
along the transport route. The SICON® conveyor belt remains closed from the feeding point to . For high conveyor capacities, long conveyor systems and high.
CS TRANSPORT SYSTEMS :: MS PLUS AUTOMATION :: T .
Our modular conveyor systems allow efficient on-site transport processes while . of this system is the links in the modular belt that is used for transport, which are . This means that the associated maintenance work and faults no longer arise,.
challenges in the driving of the 2.3 km long C line and the 4 km long S line of the project. Two 8.7 m . an integrated metropolitan transport system with 70 km of track and 66 .. removal is by continuous belt conveyor from the. TBM back-up to.
Van Stone introduced the first 100% stainless steel slider bed belt conveyor, the . the first raft launch conveyor, the first river lift system for guest transportation,.
Conveyor belts for marble and granite with heavy loads - Stone .
Sep 8, 2016 . Conveyor belts for marble and granite with heavy loads . PVC at the top surface provides less cuts on the belt and allows a long product life.
The NTOR belt conveyor blast machines are continuous pass-through units featuring a material transport system comprising a highly abrasion-resistant wire mesh belt. The system . Version for stone and similar materials. The mesh belt blast.
KFG-P 2000. Incline belt conveyor for mobile use and for vertical transport of small product. . Certified belt conveyor system for demanding cleanroom tasks.


text:
Mobilize Monday: What's at Stake in November, Reopening Schools, Breonna and Vanessa
wecanonline July 13, 2020 Mobilize Monday
Sisters,
There is so much going on in our country right now it is hard to keep up! I know it is a struggle not to feel overwhelmed, so WE CAN will continue to bring you posts that are short and to the point. We will also continue to show you at least one thing you can do every week to take a stand for our rights, freedom, and safety.
This week, WE CAN wants to remind you about some of the things at stake in the November election. Yes, we need to work to elect a president who has our best interests at heart, but we also need to pay close attention to the United States Senate, State Legislatures, school board elections, and all the folks who make decisions that affect our daily lives.
Take a look at this short video from the Washington Post about what's at stake in November. According to the Washington Post, "Voters won't just be electing a president this November, they'll be deciding which state lawmakers get to draw new congressional districts, an exercise that could reshape the political landscape for years to come."
Source: The Washington Post
Reopening Schools During Coronavirus Pandemic
The American Academy of Pediatrics added to the clusterf*ck surrounding the reopening of schools during the pandemic by basically agreeing with the 'administration' about reopening schools in the Fall. The AAP has since reversed its position and is joining with the American Federation of Teachers, National Education Association, and the School Superintendents Association saying that school districts should proceed with caution when reopening schools.
WE CAN invites you to share your thoughts about reopening schools. Help us to ensure that they are hearing from a representative group of parents and concerned citizens, not just those with money or some other motivation for reopening schools. Record a video today and encourage your friends and family to do the same! WE CAN will share the videos with officials at the U.S. Department of Education.
Demand Justice for Breonna Taylor
The office of Daniel Cameron, Attorney General for the Commonwealth of Kentucky and former aide to Senator Mitch McConnell (R-KY), is investigating the murder of Breonna Taylor. Please send a message to his office to let him know that we expect justice for Breonna and demand that his office pursue criminal charges against the four officers who fired 22 shots. Only one officer has been arrested so far and that is NOT ok.
Say Her Name – Vanessa Guillen
Source: United States Army
U.S. Army Specialist Vanessa Guillen reported being sexually harassed by her superiors and others on the Fort Hood Army base. As far as we can tell, her reports were largely ignored or written off, with one female officer, Air Force Lt. Col. Betsy Schoeller, saying sexual harassment is basically the price of admission for women into the good ole boy club.
Vanessa went missing in April and sadly her body was recently found leaving her family grieving the loss. The prime suspect committed suicide, but his alleged accomplice was arrested.
WE CAN will have details next week with specific actions you can take to demand justice for Vanessa!
Stephanie McGencey, WE CAN Founder
Breonna TaylorCoronavirusEducationVanessa Guillen


text:
Q: Writing test for a Gin API using testify results in a 404 HTTP response code I have attempted to write a test for my Gin API using testify.
Unfortunately, it responds with an unexpected HTTP 404 response code within the test.
When I execute the program I can reach the corresponding interface via curl and browser.
Why are my tests failing ?
Test code:
func (suite *statisticTestSuite) TestGetProjects() {
    suite.T().Log("TestGetAllProjects")

    recorder := httptest.NewRecorder()

    router := gin.Default()

    request, err := http.NewRequest(http.MethodGet, "/api/v1/statistics/projects", nil)
    request.Header = http.Header{"Content-Type": []string{"application/json"}}

    assert.NoError(suite.T(), err)

    router.ServeHTTP(recorder, request)

    data := make(map[string]interface{})
    data["projects"] = 3

    respBody, err := json.Marshal(gin.H{
        "code": 200,
        "msg":  "ok",
        "data": data,
    })

    fmt.Println(recorder.Code)
    fmt.Println(respBody)
}

A: You create a router without any handle. Add router.GET("/api/v1/statistics/projects", your handleFunc)
or
func TestHandle(t *testing.T) {
    recorder := httptest.NewRecorder()
    c, _ := gin.CreateTestContext(recorder)

    yourHandleFunc(c)

    fmt.Println(recorder.Code)

}


2007: Steve Jobs announced the first iPhone. No SDK! Everything is HTML5! Revolutionary. But what do we have now? Native Apps in native App Stores. I love HTML5 based games. What should I do if I want my game to be distributed on various App Stores? There are several possible solutions: Install the App using HTML5 WebApp Manifest and put your game on home screen. Write a "wrapper" app using a WebView.


text:
Q: It is true that $ \lim_{x \to a} f(x) = L \Leftrightarrow \lim_{x \to a^+} f(x) = \lim_{x \to a^-} f(x) = L $? $$ \lim_{x \to a} f(x) = L \Leftrightarrow \lim_{x \to a^+} f(x) = \lim_{x \to a^-} f(x) = L    $$
Do you think it is true? I need to know that to determine these two limits:
$ \lim_{x \to 0} x^{|\tan x|} \overset{??}{=} 1 $
$ \lim_{x \to 0} \frac{(\sin x - \tanh x)^2}{(e^x-1-\ln(1+x))^3} \overset{??}{=} \frac{1}{36} $
It is clear that it is fulfilled when $ x \to 0^+ $, but what about when $ x \to 0^- $?
Thank you all!

A: hint
For the first, you can only compute $\lim_{x\to 0^+}$ using
$$(\forall x>0)\; x^{|\tan(x)|}=e^{\frac{\sin(x)}{x\cos(x)}x\ln(x)}$$
and the well-known limits.
For the second, you will need Taylor expansion
$$e^x=1+x+\frac{x^2}{2}(1+\epsilon_1(x))$$
$$\ln(1+x)=x-\frac{x^2}{2}(1+\epsilon_2(x))$$
$$\sin(x)=x-\frac{x^3}{6}(1+\epsilon_3(x))$$
and
$$\tanh(x)=x-\frac{x^3}{3}(1+\epsilon_4(x))$$


Is Your Company Built for S.P.E.E.D.?
Is Your COMPANY Ready for a Culture of Accountability with Care™?
Sales are not where you want them. Productivity is low. Employee engagement has decreased. Employees are not being retained. Devoted clients are bailing on you. Correcting unacceptable work is the norm. No system to hold people accountable. Our workflow bottlenecks. Deadlines are not being met and it's costing us! Not keeping up with the competition.
We install the Accountability with Care™ System throughout your company so that everyone is working together to increase sales, productivity, employee engagement, employee retention and devoted clients/customers.
Fed up and ready for a new start today!
Alan has helped me triple my business in only five months. Working with Alan has been the most beneficial thing I have done for myself and my business.


The controversy comes because President Obama took to the airwaves almost immediately after the shooting and specifically said that he would politicize the tragedy to promote a gun control agenda.
The timing, as we've said from the beginning, is really suspect. I think it'll be up to the families to make that determination about whether he was sincere to reach out to them, but remember, this is the same man who told you that you could keep your physician.
The President is expected in Roseburg at 11 a.m. Pacific time, after flying up to Oregon from fundraising events in California. Protests are scheduled at the same time at several locations, according to a Defend Roseburg Facebook page.


How many times have you been working in the depths of your aircraft's fuselage, a flashlight in your mouth, and a work light burning a hole in your trousers (and not providing any light while it was doing it) and wished for better lighting? I have been a cave explorer on and off throughout my life, and I'll tel you what - it was just a good preparation for working in a light airplane's fuselage!
But fret no more! In a recent trip up to Glasair Aviation, I found an idea whose time has come - LED strip lights, intended for use as under cabinet lighting in kitchens and work rooms. These strips of LEDs are available all over the internet as kit of four to six lights, with thin cables to connect them. They are light, small, and compact - perfect for zip-tying or using a spot of velcro to hold them up near the ceiling, out of the way of tools and bodies. You can buy a set for as low as $30 at the local big-box store, and light the entire interior of whatever airplane you're working on. Just mount the switch near where you enter, and you'll have great lighting through the time you spend working back there.
Along with being out of the way, and generating lots of nice white light, they have the added benefits of generating very little heat, and not need to have their batteries replaced. A definite plus when it comes to modernizing your shop.
The peel and stick LED light rolls are even better. Cut to length and hook up to 12 volts. Rolls are 16 feet long and about half an inch wide. Very thin. You can use the whole roll or down to about 2 or 3 inches. Just made a 6 foot strip light for our Cub Scout Trailer. Solder connections are better than supplied plug. Best part, you can get a roll for about $11 dollars.


text:
L'Endymion d'Italie, Scille d'Italie ou Fausse jacinthe d'Italie (Hyacinthoides italica) est une espèce de plantes vivaces de la famille des Asparagacées.

Liens externes

Espèce d'Angiospermes (nom scientifique)
Asparagaceae
Plante décrite en 1944


text:
Wreck and Reference - Alien Pains
Wreck and Reference have released a surprise Guided by Voices tribute short album titled Alien Pains. Alien Pains is an homage to Guided By Voices' 1995 album Alien Lanes. Out now on all digital platforms. Wreck and Reference are an experimental music project from California formed in 2011 by Felix Skinner and Ignat Frege. Drawing upon the blown-out intensity of black metal and noise rock, they eschew traditional guitar-centric instrumentation to construct songs with digital samples, drums, and voice. Their latest release is Alien Pains, an homage to Guided By Voices' 1995 album Alien Lanes. Across four tracks, Alien Lanes mutates four GBV classics from blistering lo-fi pop into high-definition, noisy, monstrous tracks. Bob Pollard's lyrics, meditations on self-doubt, betrayal, and modern aviation find new life in Wreck and Reference's harrowing vision. To date, the band has released three full-length albums and three EPs. Their first work, an EP titled Black Cassette, showcased distorted fragments and synthesizers in angular, heavy songs, with themes of determinism and Cormac McCarthy-esque isolation. Their debut full-length Y̶o̶u̶t̶h̶ (2012) and their sophomore release Want (2014), described by Pitchfork as having "radical vision" and "boundless experimentation," represented dramatic expansions of their sonic palette. In 2016 Wreck and Reference released their third LP, Indifferent Rivers Romance End, perfecting their song craft and pushing their noisy, sample-based instrumentation to its limits. Once a solitary project, Wreck and Reference have recently begun collaborating with other experimental artists, including Ricky Eat Acid on See You Soon (2016) and photographer Wolfgang Tillmans, performing during his installation at the Tate Modern museum in London and producing a techno track for his Heute Will Ich Frei Sein EP (2018). Tour dates: 4.18 – London, UK – Shacklewell Arms 4.19 – Tilburg, NE – Roadburn Festival 4.20 – Ghent, BE – De Kolonie 4.21 – Leipzig, DE – Doom Over Leipzig Festival 4.22 – Berlin, DE – Urban Spree 6.11 – Los Angeles, CA – The Echo 6.12 – San Francisco, CA – Bottom of the Hill


Pioneers in the industry, we offer thread plug gauge from India.
Our Thread Plug Gauge widely utilized in various industries for respective industrial applications such as engineering and chemical. We fabricated the products using high grade of raw material based on latest technology and advanced mechanisms. These thread plug gauge is available in different sizes and dimensions can be customized as per the clients' specific requirements. These products clients can avail from us these products at reasonable price.
Sizes : Standard and special as per drawings.