text:
\section{Introduction}
\vspace{-10pt}
In the past few years, large-scale vision-language pre-trained (VLP) models, such as CLIP~\citep{clip}, ALIGN~\citep{align}, and BLIP~\citep{blip} have achieved remarkable success in open-world visual concept learning. These methods have brought new light but also pose a new question: how to efficiently adapt the knowledge from pretraining to the downstream tasks since these models are typical of massive sizes which are not feasible for normal users to re-train.

\begin{wrapfigure}{r}{9cm}
\vspace{-10px}
    \includegraphics[width=9cm]{top.pdf}
    \caption{The motivation that one category can be complementarily described in different views (An example of ``Brambling'').   }
    \label{fig:motivation}
\vspace{-10px}
\end{wrapfigure}
One of the conventional paradigms of utilizing pretrained knowledge is ``pre-training, fine-tuning'', which fixes the architecture of the pre-trained neural network and tunes its parameters using task-specific objective functions. Beyond fine-tuning the parameters, many recent methods~\citep{coop,cocoop} introduce the concept of prompt learning from the field of NLP to the vision domain and achieve striking performance gain for the few-shot visual classification. They fix the model parameters and instead learn suitable prompts by turning a template sentence into a set of learnable vectors. Then, these prompts are learned by minimizing the distance between the visual features and prompt-based language features.

Despite significant improvements over manual prompts, learning only a sentence is intuitively insufficient to represent a class.  One class can be described by many intrinsic characteristics and even extrinsic context relations. Thus, for one object, we may have multiple prompt candidates which focus on different attributes. As shown in Figure~\ref{fig:motivation}, we can describe the class ``Brambling'' in different views: such as the color of the wing, the color of the crown and eyes, the shape and color of the tail, and even the living environment information.
It motivates us to learn multiple prompts to comprehensively represent the class and thus facilitate classification.

The most natural solution is to directly learn multiple prompts by respectively matching each prompt with the visual features. However, it is the same as matching the mean of prompt features and the visual features. This solution is problematic since all prompts are encouraged to be closer to one single point and thus tend to learn the same characteristics. It contradicts our purpose to learn comprehensive prompts. To solve this problem, we tested adding some constraints to push away the prompt from each other, but found that this solution still fails to learn representative and comprehensive prompts. This solution treats the visual representation as one single point, and such a unified view of visual features ignores the fact that different prompts may only focus on one or a subset of characteristics.

To address this problem, in this paper, we propose Prompt Learning with Optimal Transport (\texttt{\textbf{PLOT}}), which applies optimal transport~(OT) to align the local visual features and multiple textual prompts. Optimal transport can calculate the distance between two distributions under the form of multiple sampling. In our prompt learning framework, we formulate local visual features and multiple prompts as the samplings of two discrete distributions and use OT to encourage fine-grained cross-modal matching.
Specifically, to obtain the local visual features with different semantic clues, we extract all feature maps as the visual representation instead of the single global representation.
Fortunately, we can easily obtain the visual feature maps from the visual encoder of CLIP by using all outputs of the multi-head self-attention layer~\citep{rao2021denseclip}.
Then the problem comes down to how to calculate the distance between two feature sets.

We solve this problem by introducing the optimal transport theory~\citep{villani2009optimal} and formulate the feature sets as a discrete probability distribution where each feature has an equal probability value.
Furthermore, to reduce the computational cost and avoid the extra model parameters, we learn the prompts with a two-stage optimization strategy.
At the first stage in the inner loop, we fix both visual and text features and optimize the optimal transport problem by a fast Sinkhorn distances algorithm~\citep{cuturi2013sinkhorn}. Then, in the outer loop, we fix all parameters of optimal transport and back-propagate the gradient to learn the prompts with different characteristics. Compared with conventional distance (such as Euclidean distance of mean features), optimal transport can align different visual features for each local prompt, which is more robust to the visual misalignment and tolerates well feature shift~\citep{emd}. It is because OT learns an adaptive transport plan to align features, which achieves fine-grained matching across two modalities. We conduct experiments on 11 datasets following the standard setting of CLIP~\citep{clip} and CoOp~\citep{coop} to evaluate our method. These experiments span the visual classification of generic objects, scenes, actions, fine-grained categories, and so on. The significant result improvement demonstrates that \textbf{\texttt{PLOT}}  can effectively learn representative and comprehensive prompts.

\vspace{-10pt}
\section{Related Work}
\vspace{-5pt}
\textbf{Optimal Transport}
The Optimal Transport~\citep{monge1781memoire} is initially introduced to solve the problem of how to reduce the cost when moving several items simultaneously.
Recently, OT theory has drawn wide attention in the machine learning and computer vision community by comparing distributions readily available to them under the form of feature sets~\citep{COTFNT}. Due to the brilliant property of distribution matching, OT has been applied in many theoretic and application tasks including
generative models~\citep{arjovsky2017wasserstein,salimans2018improving,zhao2020neural}, structural matching~\citep{chen2019improving,xu2020vocabulary,zhao2021towards,xu2019gromov} (e.g. sequence matching~\citep{chen2019improving} and graph matching~\citep{xu2019gromov}), and other distribution-based tasks (such as clustering~\citep{laclau2017co}, distribution estimation~\citep{boissard2015distribution}, and causal discovery~\citep{tu2022optimal}). In this paper, we use OT to align the features of vision and language modalities
by learning an adaptive transport plan~\citep{emd}.

\textbf{Vision-Language Pre-trained Models}
Vision-Language Pre-trained (VLP) models aim to explore the semantic correspondence between the vision and language modalities through large-scale pre-training. Recently, VLP models have achieved an exciting performance improvement in the zero-shot and few-shot visual recognition~\citep{clip,clip-adapter,coop,cocoop,vtclip}, which shows the great potential to promote open-world visual understanding with the help of language. One key part of learning VLP models is the self-supervised learning objective on two modalities.
The popular VLP objectives can be divided into reconstruction~\citep{visualbert,vln-bert,meter,vilt}, contrastive matching~\citep{clip,align,mural}, or the combination of both two~\citep{li2021align,vlmo,kamath2021mdetr}. Besides, recent progress in the field of VLP also benefits a lot from large-scale pair-wised datasets. For example, CLIP~\citep{clip} applies 400 million image-text pairs for contrastive learning, while ALIGN even exploits 1.8 billion data pairs. Beyond recognition, these VLP models also show great potential for other downstream applications, such as dense prediction~\citep{rao2021denseclip,zhou2021denseclip}, image generation~\citep{nichol2021glide,ramesh2022hierarchical,patashnik2021styleclip}, and action understanding~\citep{wang2021actionclip,tevet2022motionclip}.

\textbf{Prompt Learning}
Prompt learning is introduced from the field of NLP to efficiently adapt the large language model to downstream tasks. Different from the conventional ``pre-training, fine-tuning'' paradigm which initializes the pre-trained model and tunes the parameters of the network using downstream task-specific objective functions, prompt learning applies textual prompt to reformulate the downstream tasks as the original pretrained task~\citep{liu2021pre,petroni2019language}.
By the prompt, the domain shift between pretrained task and downstream application is reduced and thus the pretrained knowledge can be easier adapted to downstream tasks.
The concept of prompt learning~\citep{petroni2019language,radford2019language,poerner2019bert} begins from the success of GPT~\citep{radford2019language} series. Early prompt learning methods (such as Petroni~\emph{et al.}~\citep{petroni2019language} and Pörner~\emph{et al.}~\citep{poerner2019bert}) always manually create templates based on human prior knowledge.
Furthermore, some mining-based methods~\citep{jiang2020can} and gradient-based methods~\citep{autoprompt} are proposed to automatically search for appropriate templates. Beyond search in the discrete space, some methods~\citep{li2021prefix,tsimpoukelli2021multimodal,liu2021gpt} remove the constraint that the prompts are ``words'' and instead learn prompts in the continuous embedding space. Recently, CoOp~\citep{coop} and its extended version~\citep{cocoop} introduce prompt learning into open-world visual understanding to adapt the knowledge from the large-scale visual-language pretrained models and achieve great performance improvement on the few-shot visual recognition.
Compared with CoOp, our \textbf{\texttt{PLOT}}  method further improves prompt learning by introducing the optimal transport distance to learn multiple local prompts and achieves fine-grained vision-language matching.

\vspace{-10pt}
\section{Approach}
\vspace{-5pt}
In this section we first revisit the baseline CoOp~(\ref{sec:revisit CoOp}), review the preliminaries of optimal transport~(\ref{sec:Optimal Transport}), and then introduce our \textbf{\texttt{PLOT}}  (\ref{sec:PLOT}) to show how we learn multiple comprehensive prompts.
\vspace{-10pt}
\subsection{A Revisit of CoOp} \label{sec:revisit CoOp}
\vspace{-5pt}

CoOp~\citep{coop} is one of the pioneering methods to learn the prompts for using vision language pretrained knowledge (such as CLIP~\citep{clip}) for downstream open-world visual recognition. Different from CLIP which manually designs the prompt templates, CoOp sets a part of context words in the template as continuous learnable parameters which can be learned from the few-shot data.
Then the classification weights can be represented by the distance between the learned prompt and visual feature.

Specifically, given an image $\bm{x}$, a visual feature $\bm{f}=f(\bm{x})$ is obtained by
the visual encoder $f$ of CLIP.
Then, the textual prompt can be formulated as $\bm{t}_k = \{\bm{\omega}_1,\bm{\omega}_2,\hdots,\bm{\omega}_L, \bm{c}_k \} $, where $\bm{c}_k$ is the word embedding of the class name, $ \bm{\omega} =\{\bm{\omega}_l|_{l=1}^{L}\}$ are learnable vectors where each vector has the same dimension as the original word embedding and L is the length of context words.
With prompt $\bm{t}_k $ as the input, the text encoder $g$ outputs the textual feature as $\bm{g}_k=g(\bm{t}_k)$.
The final prediction probability is computed by the matching score as follows:
\begin{equation} \label{eq:predict}
p(y=k|\bm{x}) = \frac{\exp(\text{sim}(\bm{f},\bm{g}_k)/\tau)}{\sum_{k'=1}^{K}\exp(\text{sim}(\bm{f},\bm{g}_{k'})/\tau) },
\end{equation}
where $\text{sim}(\cdot,\cdot)$ denotes a metric function such as cosine similarity, and $\tau$ stands for the temperature of Softmax.
Then we can optimize the parameters of $\{\bm{vec}_l|_{l=1}^{L}\}$ with the cross-entropy loss between the prediction and the labeled target.

\begin{figure}[t]
    \centering
    \includegraphics[width=\textwidth]{framework.pdf}
    \caption{\textbf{The framework:} \textbf{\texttt{PLOT}}  first describes each category with multiple prompts and obtains a set of prompt features by text encoder. The image is also encoded as a set of local features. Then the optimal transport is used as the metric between prompts and visual features.}
    \label{fig:plot}
    \vspace{-10px}
\end{figure}
\vspace{-5pt}
\subsection{Optimal Transport} \label{sec:Optimal Transport}
\vspace{-5pt}
Optimal transport~(OT) distance is a widely used metric for the comparison of distributions. Here, we only focus on the discrete situation which is more related to our framework. Assuming we have two sets of points (features), the discrete distributions are formulated as:
\begin{equation} \label{eq:discrete}
U=\sum_{m=1}^{M}u_m\delta_{\bm{f}_m} \hspace{2em} \text{and} \hspace{2em} V=\sum_{n=1}^{N}v_n\delta_{\bm{g}_n},
\end{equation}
where $\bm{u}$ and $\bm{v}$ are the discrete probability vectors that sum to 1, and $\delta_{\bm{f}}$ is a Dirac delta function placed at support point $\bm{f}$ in the embedding space.
Then, the total distance is written as:
\begin{equation} \label{eq:cost}
<\bm{T},\bm{C}> = \sum_{m=1}^{M}\sum_{n=1}^{N}\bm{T}_{m,n}\bm{C}_{m,n}.
\end{equation}
We call $\bm{C}$ the cost matrix in which each point denotes the cost between $\bm{f}_m$ and $\bm{g}_n$, such as $\bm{C}_{m,n}= 1- \text{sim}(\bm{f}_m,\bm{g}_{n})$. While the $\bm{T} $ is called the transport plan, which is learned to minimize the total distance.
The optimization problem of optimal transport is formulated as:
\begin{equation} \label{eq:optimization}
\begin{aligned}
& d_{\text{OT}}(\bm{u},\bm{v}|\bm{C}) = \underset{\bm{T}}{\text{minimize}}
 <\bm{T},\bm{C}> \\
& \text{subject to}
~~~~~~\bm{T}\bm{1}_N = \bm{u},\; \bm{T}^\top \bm{1}_M = \bm{v},\; \bm{T} \in \mathbb{R}^{M \times N}_+.
\end{aligned}
\end{equation}

As directly optimizing the above objective is always time-consuming, we apply the Sinkhorn distance~\citep{cuturi2013sinkhorn} to use an entropic constraint for fast optimization.
The optimization problem with a Lagrange multiplier of the entropy
constraint is:
\begin{equation} \label{eq:Sinkhorn}
\begin{aligned}
& d_{\text{OT},\lambda}(\bm{u},\bm{v}|\bm{C})=\underset{\bm{T}}{\text{minimize}}
 <\bm{T},\bm{C}> - \lambda h(\bm{T})\\
& \text{subject to}
~~~~~~~\bm{T}\bm{1}_N = \bm{u},\; \bm{T}^\top \bm{1}_M = \bm{v},\;
\bm{T} \in \mathbb{R}^{M \times N}_+,
\end{aligned}
\end{equation}
where $h(\cdot) $ is entropy and $\lambda \geq 0$ is a hyper-parameter. Then we can have a fast optimization solution with a few iterations as:
\begin{equation} \label{eq:Sinkhorn_optimization}
\bm{T}^*= \text{diag}(\bm{u}^{(t)}) \exp(-\bm{C}/\lambda) \text{diag}(\bm{v}^{(t)}),
\end{equation}
where $t$ denotes the iteration and in each iteration
$\bm{u}^{(t)} =\bm{u}/\left((\exp(-\bm{C}/\lambda)\bm{v}^{(t-1)}\right)$ and  $\bm{v}^{(t)} =\bm{v}/\left((\exp(-\bm{C}/\lambda)^\top\bm{u}^{(t)}\right) $, with the initiation $\bm{v}^{(0)} = \bm{1}$.

\vspace{-5pt}
\subsection{Prompt Learning with Optimal Transport}
\label{sec:PLOT}
\vspace{-5pt}
In this subsection, we introduce the details of our \textbf{\texttt{PLOT}} , which learns multiple prompts to describe different characteristics of the category by minimizing the OT distance.

Specifically, as shown in Figure~\ref{fig:plot}, given an image $\bm{x}$, we first feed it to the visual encoder branch of CLIP. Apart from the global visual feature $\bm{f}$, we can also obtain a set of local features $\{\bm{f}_m|_{m=1}^{M}\}$. The visual encoder has a multi-head attention pooling layer in which the input is the combination of the global feature and a set of local features (feature map) and the output is a tensor with the shape $\mathbb{R}^{(H\times W+1)\times C}$, where $H$ and $W$ is the height and width of feature map and $C$ is the feature dimension.
Therefore, we can obtain $M=H\times W$ local features and a global feature. At the same time, for class $k$, we can initialize N local prompts as $\{\bm{t}_{k,n}|_{n=1}^{N}\} $ with learnable vectors $\{\bm{\omega}_{n}|_{n=1}^{N}\}$, where each is the same as the prompt in CoOp. With both visual and textual encoders, we can obtain local visual features $\bm{F}=\{\bm{f}_m|_{m=1}^{M}\} \in \mathbb{R}^{M\times C}$ and prompt features $\bm{G}_k=\{\bm{g}_n|_{n=1}^{N}\} \in \mathbb{R}^{N\times C}$.

In the inner loop, we learn the transport plan $\bm{T}$ with these fixed support sets $\bm{F},\bm{G}_k$, by minimizing the following OT distance to push $\bm{G}_k$ to $\bm{F}$:
\begin{equation} \label{eq:distance}
\begin{aligned}
 d_{\text{OT}}(k) = d_{\text{OT}}(\bm{u},\bm{v}|\bm{1}-\bm{F}^\top\bm{G}_k),
\end{aligned}
\end{equation}
where $\bm{C}=\bm{1}-\bm{F}^\top\bm{G}_k$ denotes that we use the cosine distance between $\bm{F}$ and $\bm{G}_k$ as the cost matrix. Then we can obtain the solution of transport plan $\bm{T}^*$ as Eq.~\ref{eq:Sinkhorn_optimization} and the final OT distance $d_{\text{OT}}(k)$.

Given the OT distance between $\bm{G}_k$ and $\bm{F}$, we reformulate the prediction probability as:
\begin{equation} \label{eq:predict_OT}
p_{\text{OT}}(y=k|\bm{x}) = \frac{\exp \left(\left(1-d_{\text{OT}}(k)\right)/\tau \right)}{\sum_{k'=1}^{K} \exp\left(\left(1-d_{\text{OT}}(k')\right)/\tau\right) }.
\end{equation}
In the outer loop, we fix the transport plan $\bm{T}^*$ and optimize $\{\bm{vec}_{l,n}|_{l=1,n=1}^{L,N}\}$ with cross entropy:
\begin{equation} \label{eq:loss}
L_{\text{CE}} =-\frac{1}{|\mathcal{X}|}\sum_{\bm{x} \in \mathcal{X}} \sum_{k=1}^{K} y_{\bm{x},k} p_{\text{OT}}(y=k|\bm{x}),
\end{equation}
where $\bm{y}_{\bm{x}}$ is a one-hot label vector. The detailed algorithm can be found in Appendix~\ref{app:ot}.

Though the optimization strategy of the optimal transport and prompts is two-stage, the whole training flow is end-to-end. It is because the transport plan is computed using a small number of matrix multiplications as a forward module. The gradients of these matrix multiplications are taped for back-propagation for end-to-end optimization, which makes the whole system fully differentiable (including the iterative algorithm) and easy to implement using an autograd library like PyTorch.
In the experiments, we found that it is natural and relatively easy to this optimization strategy.

\vspace{-10pt}
\section{Experiments}
\vspace{-5pt}
Extensive experiments are conducted to evaluate our method, including comparison with CoOp, ablation studies, parameter analysis extensibility analysis, computing cost analysis, and visualization.
\vspace{-10pt}
\subsection{Datasets \label{sec:datasets}}
\vspace{-5pt}
We followed the experimental settings in the CoOp~\citep{coop} for the few-shot learning evaluation. The experiments are conducted on the 11 visual recognition datasets, including Caltech101~\citep{fei2004learning}, DTD~\citep{cimpoi2014describing}, EuroSAT~\citep{helber2019eurosat}, FGVCAircraft~\citep{maji2013fine}, Flowers102~\citep{nilsback2008automated}, Food101~\citep{bossard2014food}, ImageNet~\citep{deng2009imagenet}, OxfordPets~\citep{parkhi2012cats}, StanfordCars~\citep{cars}, SUN397~\citep{xiao2010sun}, and UCF101~\citep{soomro2012ucf101}. These datasets span visual classification of generic objects, scenes, actions, fine-grained categories, and so on, which constitutes a comprehensive evaluation of our method. All experiments adopted the few-shot evaluation protocol used in CLIP~\citep{clip} and CoOp~\citep{coop}, where we respectively choose 1, 2, 4, 8, and 16 shots for model training and use the original test set for evaluation. Besides, we also evaluated the robustness of our method with domain shift. Following CoOp, we used the ImageNet as the source domain and evaluate our method with ImageNet-based robustness evaluation datasets including ImageNetV2~\citep{ImageNetv2}, ImageNet-Sketch~\citep{wang2019learning}, ImageNet-A~\citep{imagenet-a}, and ImageNet-R~\citep{imagenet-r}.
A detailed introduction of each dataset can be found in the appendix.

\vspace{-5pt}
\subsection{Implementation details}
\label{sec:details}
\vspace{-5pt}
We chose CoOp~\citep{coop} as our main competitor to evaluate our method. Compared with CoOp which only learns a global prompt for one class, our \textbf{\texttt{PLOT}}  method learns multiple local prompts and applies the OT distance for better fine-grained alignment. Besides, we also reported the performance of training a linear classifier with the CLIP~\citep{clip} features. It is also a widely-used strategy to adapt the pretrained knowledge for the downstream task~\citep{autoprompt}. We reproduced the performance of CoOp and the CLIP linear probe with the released official code.

The original CoOp method has different versions with different class token positions and parameter initialization strategies. As the performance gap among different versions is limited, we directly chose one of them as our baseline, where the token position is ``end'', the parameter initialization strategy is ``random'', and the length of learnable context tokens is set as 16.
Following the widely used setting in~\citep{coop,cocoop,clip-adapter,tip-adapter}, we also chose RN50~\citep{he2016deep} as the backbone network of the visual branch. All the code of our method is based on CoOp, which adopted the SGD optimizer with 0.002 initial learning rate, CosineAnnealingLR schedule, and a warmup trick with 1e-5 learning rate. Besides, we also followed the epoch strategy to train more epochs for more shots.

We apply $N = 4$ prompts for each category and use $M = 7\times 7$ due to the feature map size.  We set the hyper-parameters in the Sinkhorn distances algorithm~\citep{cuturi2013sinkhorn} as $\lambda=0.1$ for all the datasets.
We set the maximum iteration number of the inner loop as 100 and will early stop the iteration when the average absolute update value $\Lambda <0.01$. We initialize all values in the vector $v$ and $\mu$ as $1/N$ and $1/M$ respectively.
All models are conducted on the Pytorch~\citep{paszke2019pytorch} 1.7.1 and trained on 4 NVIDIA A100 GPUs. We repeated the experiments three times with different seeds and reported the average.

\begin{figure}
  \centering
  \includegraphics[width=0.92\linewidth]{results_cocoop.png}
  \caption{ The few-shot learning results on 11 datasets. We compare our \textbf{\texttt{PLOT}}  with CoOp, CoCoOp, and the Linear Probe method and observe the consistent and significant performance improvement on most datasets. (The average accuracy on all datasets is shown on the left top.)}
\label{fig: comparison_curve}
\vspace{-0.5cm}
\end{figure}
\vspace{-5pt}
\subsection{Comparison With CoOp}
\vspace{-5pt}
In this subsection, we compare our \textbf{\texttt{PLOT}}  with the baseline CoOp on the few-shot recognition and domain generalization tasks.

\textbf{Few-Shot Learning}
We summarized the experimental results in Figure~\ref{fig: comparison_curve} where the red line denotes our \textbf{\texttt{PLOT}}  method, the blue one denotes CoOp, the purple line denotes CoCoOp~\citep{cocoop}, and the green one is the CLIP linear probe. As the settings in the CoCoOp and CoOp are different, we re-run the CoCoOp method in the setting of CoOp.
We observed that all prompt learning methods outperform the linear probe method by a large margin. Besides, \textbf{\texttt{PLOT}}  can further outperform CoOp and CoCoOp on most of the datasets. Taking the average accuracy (at the left top) as the example, \textbf{\texttt{PLOT}}  respectively gained $3.03\%, 3.45\%, 2.13\%, 1.38\%, 0.61\%$ performance boost over CoOp at $1,2,4,8,16$ shots.
The performance gap reduces when shots increase. It is not surprising since both CoOp and \textbf{\texttt{PLOT}}  focus on utilizing the pre-trained knowledge, and the effect of pre-training diminishes given more training data.
Among all datasets, \textbf{\texttt{PLOT}}  achieves a larger improvement over CoOp on the FOOD101 and DTD datasets and achieves comparable performance on the StanfordCars datasets.
It may be because the discriminative characters in StanfordCars coincide with each other, such that one global prompt and one global visual feature can work well.
Note that we don't use the class-specific context, thus the performance on the fine-grained classification datasets is lower, e.g. the performance of both CoOp and \textbf{\texttt{PLOT}}  without class-specific context is lower than the linear probing on FGVCAircraft.
All these performance comparisons can serve as experimental evidence to demonstrate that multiple local prompts and the OT distance facilitate the prompt learning of vision-language models. The detailed accuracy can be found in the appendix.

\begin{wraptable}{r}{7.5cm}
\caption{Comparisons on robustness to domain shift.}
\vspace{-0.3cm}
\footnotesize
\centering
\newcolumntype{g}{>{\columncolor{Gray}}r}
\renewcommand\tabcolsep{1pt}
\label{tab:robustness}
    \begin{tabular}{lccccc}
    \toprule
    \multirow{2}{*}{\centering Method}
    & Source & \multicolumn{4}{c}{Target} \\ \cmidrule(lr){2-2} \cmidrule(lr){3-6}
    & ImageNet & -V2 & -Sketch & -A & -R \\
    \midrule
    CLIP + CoOp & 61.91 & 54.26 & 32.47 & 21.78 & 54.21 \\
    CLIP + \textbf{\texttt{PLOT}}  ($N\!=\!4$) & \textbf{63.01} & \textbf{55.11} & \textbf{33.00} & \textbf{21.86} & \textbf{55.61} \\
    \bottomrule
    \end{tabular}
\vspace{-0.2cm}
\end{wraptable}

\textbf{Domain generalization}
The robustness also plays a critical role in model applications since the real-world environment may have large domain shifts with the training data.
Therefore, we conducted a robustness evaluation to investigate the transferability of models learned by \textbf{\texttt{PLOT}} . Table~\ref{tab:robustness} summarizes the results of our \textbf{\texttt{PLOT}}  method and CoOp on four ImageNet-based robustness evaluation datasets. For both methods, we trained the models on ImageNet with 16 shots per class.  For \textbf{\texttt{PLOT}} , we set the number of prompts as $N=4$.
We can observe that \textbf{\texttt{PLOT}}  outperforms CoOp consistently on both source and target domains. These experimental results demonstrate that the performance improvement of our learning multiple prompts doesn't rely on single-domain overfitting.

\vspace{-5pt}
\subsection{Ablation Studies and More Analysis}
\label{sec:analysis}
\vspace{-5pt}

In this subsection, we conducted the ablation studies to investigate the effectiveness of different components, in order to answer the following questions.

\textbf{Q: Can we directly learn multiple prompts by matching the prompt ensemble with the global visual feature? } \textbf{A: No}.
As shown in Table~\ref{tab:ablation}, we report the performance of directly matching the prompt ensemble with the global visual feature (notated as ``G'') on three datasets including Caltech101, DTD, and FOOD101.
The performance improvement of this method over CoOp is limited and far lower than \textbf{\texttt{PLOT}} .
It may be because this ``G'' method is incentivized to learn the indistinguishable prompts, which contradicts our purpose to learn multiple comprehensive prompts.

\textbf{Q: Can ensemble methods that encourage the variety of prompts work well?
} \textbf{A: Not really}.
As shown in Table~\ref{tab:ablation}, we further apply two methods to encourage the
 variety of prompts and then use the ensemble to match the global feature.
In method ``V'', we add an objective function to add the distance between every two prompts as a regularization term.
In method ``E'', we use predefined different initializations to replace the random initialization, such as "a photo of a", "this is a photo", "this is a", and "one picture of a".
However, ``G+V'' did not achieve consistent improvement over the ``G''.
Despite the clear improvement brought by ``G+E'', our \textbf{\texttt{PLOT}}  showed
consistent superiority over ``G+E'', which further demonstrates the effectiveness of the OT distance.

\textbf{Q: Does the improvement mainly come from using all feature maps?}  \textbf{A: No}.
In \textbf{\texttt{PLOT}} , we apply all feature maps of the visual encoder branch, where each feature is a local embedding at one spatial position.
However, we demonstrate that the improvement of \textbf{\texttt{PLOT}}  does not only rely on using all feature maps. On the contrary, directly using the feature map to replace the global feature causes a large performance drop.
For example, on all three datasets, directly using the feature map (``M'' or ``M+V'') has an around $20\%$ 1 shot accuracy drop over using the global visual feature.
It is not surprising since the original CLIP model is trained by matching the global visual feature and language feature. Without using the OT method, the distance between the feature map and multiple textual prompts degenerates to the mean distance of each feature-prompt pair.

\begin{table*}[t]
\centering
 \caption{\textbf{Ablation studies on few-shot recognition}. \textbf{\texttt{PLOT}} : our defined model with $N=4$.
 CoOp: the baseline method.
 G: respectively matching the global visual feature and multiple textual prompts
 V: applying a constraint to add the variance of prompts.
 E: using different initializations as the ensemble:
 M: using the visual feature map instead of the global visual feature. }
 \label{tab:ablation}  \vspace{-5pt}
 \centering
 \small \renewcommand{\arraystretch}{1.0}
 \setlength{\tabcolsep}{3pt}
\begin{tabular} {l>{\columncolor{white}[1pt][\tabcolsep]}l>{\columncolor{white}[1pt][\tabcolsep]}ccccc}
\toprule
Dataset                  & Settings  & 1 shot  & 2 shots & 4 shots & 8 shots  & 16 shots\\ \midrule
\multirow{6}{*}{\centering Caltech101}
    &  \textbf{\texttt{PLOT}}    & \bm{$89.83\pm 0.33$}  & \bm{$90.67\pm 0.21$}  & \bm{$90.80\pm 0.20$}  & \bm{$91.54\pm 0.33$}  & \bm{$92.24\pm 0.38$}  \\
    &  CoOp   & $87.51\pm 1.02$  & $87.84\pm 1.10$  & $89.52\pm 0.80$  & $90.28\pm 0.42$  & $91.99\pm 0.31$  \\
    &  G     & $88.13\pm 0.36$  & $86.98\pm 1.25$  & $88.45\pm 0.79$  & $90.16\pm 0.22$  & $90.72\pm 0.18$  \\
    &  G+V    & $88.28\pm 0.43$  & $87.72\pm 1.25$  & $88.45\pm 0.30$  & $89.82\pm 0.20$  & $92.00\pm 0.13$  \\
    &  G+E    & $88.91\pm 0.38$  & $90.01\pm 0.22$  & $90.41\pm 0.20$  & $90.60\pm 0.10$  & $91.74\pm 0.42$  \\
    &  M      & $69.78\pm 1.75$  & $71.57\pm 1.59$  & $77.18\pm 2.16$  & $81.77\pm 0.47$  & $86.21\pm 0.20$  \\
    &  M+V    & $66.11\pm 8.29$  & $71.45\pm 3.98$  & $79.30\pm 3.96$  & $86.96\pm 0.78$  & $89.80\pm 0.17$  \\
                                    \midrul
\multirow{6}{*}{\centering DTD}
    &  \textbf{\texttt{PLOT}}    & \bm{$46.55\pm 2.62$}  & \bm{$51.24\pm 1.95$}  & \bm{$56.03\pm 0.43$}  & \bm{$61.70\pm 0.35$}  & \bm{$65.60\pm 0.82$}  \\
    &  CoOp   & $43.62\pm 1.96$  & $45.35\pm 0.31$  & $53.94\pm 1.37$  & $59.69\pm 0.13$  & $62.51\pm 0.25$  \\
    &  G      & $45.12\pm 1.69$  & $48.39\pm 2.08$  & $54.75\pm 0.48$  & $60.15\pm 0.70$  & $63.59\pm 0.76$  \\
    &  G+V    & $45.90\pm 2.00$  & $48.50\pm 0.99$  & $53.96\pm 0.48$  & $59.69\pm 1.01$  & $63.51\pm 0.66$  \\
    &  G+E    & $46.39\pm 1.00$  & $49.31\pm 0.56$  & $52.99\pm 0.60$  & $60.44\pm 1.64$  & $63.97\pm 0.48$  \\
    &  M      & $13.18\pm 4.57$  & $12.25\pm 3.86$  & $13.00\pm 4.73$  & $20.76\pm 5.42$  & $26.99\pm 1.98$  \\
    &  M+V    & $12.61\pm 5.93$  & $15.11\pm 1.81$  & $20.35\pm 1.33$  & $44.13\pm 2.39$  & $56.85\pm 0.54$  \\
\midrul
    \multirow{6}{*}{\centering FOOD101}
    &  \textbf{\texttt{PLOT}}    & \bm{$77.74\pm 0.47$}  &\bm{$77.70\pm 0.02$}  & \bm{$77.21\pm 0.43$}  & \bm{$75.31\pm 0.30$}  & \bm{$77.09\pm 0.18$}  \\
    &  CoOp   & $74.25\pm 1.52$  & $72.61\pm 1.33$  & $73.49\pm 2.03$  & $71.58\pm 0.79$  & $74.48\pm 0.15$  \\
    &  G      & $74.63\pm 0.11$  & $70.15\pm 0.49$  & $70.41\pm 0.46$  & $70.72\pm 0.98$  & $73.68\pm 0.46$  \\
    &  G+V    & $74.83\pm 0.31$  & $70.09\pm 0.85$  & $70.86\pm 0.22$  & $70.80\pm 0.68$  & $73.93\pm 0.35$  \\
    &  G+E    & $75.77\pm 0.62$  & $73.54\pm 0.88$  & $75.82\pm 0.44$  & $72.40\pm 0.50$  & $75.52\pm 0.33$  \\
    &  M      & $52.02\pm 4.86$  & $46.12\pm 1.46$  & $46.86\pm 1.39$  & $53.43\pm 0.88$  & $61.28\pm 0.23$  \\
    &  M+V    & $46.52\pm 1.15$  & $45.95\pm 2.66$  & $53.57\pm 0.83$  & $62.95\pm 0.37$  & $67.63\pm 1.11$  \\ \bottomrule
\end{tabular} \vspace{-5pt}
\end{table*}

\begin{table*}[t]
\centering
 \caption{Parameter analysis for the number of prompts}
 \label{tab:parameter}  \vspace{-5pt}
 \centering
 \small \renewcommand{\arraystretch}{1.0}
 \setlength{\tabcolsep}{3pt}
\begin{tabular} {l>{\columncolor{white}[1pt][\tabcolsep]}l>{\columncolor{white}[1pt][\tabcolsep]}ccccc}
\toprule
Dataset                  & Settings  & 1 shot  & 2 shots & 4 shots & 8 shots  & 16 shots\\ \midrule
\multirow{4}{*}{\centering Caltech101}
                                    & N=1 & $88.47\pm 1.15$  & $89.19\pm 0.39$  & $89.70\pm 0.38$  & $90.45\pm 0.24$  & $91.56\pm 0.14$  \\
                                    & N=2 & $88.86\pm 0.51$  & $89.60\pm 0.10$  & $90.60\pm 0.17$  & $91.25\pm 0.65$  & $91.89\pm 0.36$  \\
                                    & N=4 & \bm{$89.83\pm 0.33$}  & \bm{$90.67\pm 0.21$}  & $90.80\pm 0.20$  & \bm{$91.54\pm 0.33$}  & \bm{$92.24\pm 0.38$}  \\
                                    & N=8 & $89.74\pm 0.30$  & $90.18\pm 0.46$  &\bm{$91.02\pm 0.18$}  & $91.28\pm 0.28$  & $92.04\pm 0.29$  \\  \midrul
\multirow{4}{*}{\centering DTD}
                                    & N=1 & $43.91\pm 0.65$  & $48.21\pm 2.20$  & $53.69\pm 1.10$  & $58.90\pm 0.19$  & $62.85\pm 0.74$  \\
                                    & N=2 & $45.59\pm 2.46$  & $48.06\pm 1.92$  & $55.58\pm 1.71$  & $61.56\pm 0.17$  & $64.60\pm 0.92$  \\
                                    & N=4 & $46.55\pm 2.62$  & $51.24\pm 1.95$  & \bm{$56.03\pm 0.43$}  & $61.70\pm 0.35$  & \bm{$65.60\pm 0.82$}  \\
                                    & N=8 & \bm{$46.89\pm 1.94$}  & \bm{$51.87\pm 2.06$}  & $54.45\pm 0.48$  & \bm{$62.20\pm 0.56$}  & $65.25\pm 0.38$  \\ \midrul
    \multirow{4}{*}{\centering FOOD101}
                                    & N=1 & $75.96\pm 0.48$  & $76.12\pm 0.59$  & $77.11\pm 0.41$  & $76.56\pm 0.69$  & $77.43\pm 0.80$  \\
                                    & N=2 & $77.12\pm 0.49$  & $76.89\pm 0.23$  & $76.16\pm 0.52$  & $75.23\pm 0.69$  & $76.81\pm 0.50$  \\
                                    & N=4 & $77.74\pm 0.47$  & $77.70\pm 0.02$  & $77.21\pm 0.43$  & $75.31\pm 0.30$  & $77.09\pm 0.18$  \\
                                    & N=8 & \bm{$78.05\pm 0.15$}  & \bm{$78.19\pm 0.07$}  & \bm{$78.12\pm 0.17$}  & \bm{$76.63\pm 0.22$}  & \bm{$77.48\pm 0.12$}  \\  \bottomrule
\end{tabular} \vspace{-15pt}
\end{table*}

\textbf{Q: How many prompts are needed? } \textbf{A: 4 prompts are enough}
One important hyper-parameter in \textbf{\texttt{PLOT}}  is the number of prompts. To analyze the effect of the number of prompts, we conducted the experiments on three datasets with $1,2,4,8$ prompts. The results are summarized in the white part of Table~\ref{tab:parameter}.
We can observe that the performance obviously increases when adding the number of prompts from 1 to 4. For example, \textbf{\texttt{PLOT}}  (N=4) respectively obtains $1.36\%$, $2.64\%$, and $1.68\%$ 1-shot accuracy improvement over \textbf{\texttt{PLOT}}  (N=1) on three datasets. Besides, when we further increase the number of prompts, the improvement is not consistent. To balance the improvement and cost, we set $N=4$ as the default configuration of our \textbf{\texttt{PLOT}}  model. In the experiments, we tuned this hyper-parameter on the Caltech101 dataset and applied it to other datasets.

\textbf{Q: Can \textbf{\texttt{PLOT}}  benefit Adapter-based methods? } \textbf{A: Yes}.
Adapter-based methods~\citep{clip-adapter,tip-adapter} is another research direction of the efficient adaptation of pre-trained vision-language models. Different from the prompt learning that fixes the model parameters and tunes the language prompt, adapter-based methods~\citep{clip-adapter,tip-adapter} allow for fine-tuning a part of the network or adding an extra model for training. Recently, adapter-based methods also achieve good performance on few-shot visual recognition. Therefore, we want to explore whether our \textbf{\texttt{PLOT}}  method can benefit them, and how.

\begin{wraptable}{r}{8.7cm}
\caption{Comparison with Adapter-based method on ImageNet.}
\vspace{-0.2cm}
\footnotesize
\centering
\newcolumntype{g}{>{\columncolor{Gray}}r}
\renewcommand\tabcolsep{2pt}
\label{tab:adapter}
\begin{tabular}{lccccc}
\toprule
 Methods  & 1 shot  & 2 shots & 4 shots & 8 shots  & 16 shots\\ \midrule
    Tip-Adapter-F & 61.32 & 61.69 & 62.52 & 64.00 & 65.51 \\
    Tip-Adapter-F + OT &  61.44 & 61.98 & 62.86 & 64.13 & 65.76 \\
    Tip-Adapter-F + \textbf{\texttt{PLOT}}  & \textbf{62.27} & \textbf{64.31} & \textbf{63.89} & \textbf{65.04} & \textbf{66.17} \\
    \bottomrule
    \end{tabular} \vspace{-10pt}
\end{wraptable}

We apply the Tip-adapter-F~\citep{tip-adapter} as our baseline method, which learns a $Linear(d, N_{cls}\times K_{shots})$ model to describe one image by the similarity with all training samples, where $d$ is the dimension of visual feature, $N_{cls}$ is the number of categories (e.g. 1000 in ImageNet), and $K_{shots}$ is the number of shots. Then, the final similarity consists of the original distance between the visual feature and prompt ensembling and the new distance calculated by the learned feature and one-hot vector of labels (whose dimension is $(N_{cls}\times K_{shots}, N_{cls}) $). Please find details in Tip-adapter-F~\citep{tip-adapter}.
To introduce \textbf{\texttt{PLOT}}  to this framework, we first used the feature map to replace the global feature and then learned multiple linear models. As a result, with different local features and different linear models, we can obtain a $M \times N$ distance matrix and apply the Sinkhorn algorithm~\citep{cuturi2013sinkhorn} to calculate the OT distance. Furthermore, we can apply the learned prompts as co-partner of the ensembling prompt to refine the final similarity.
Table~\ref{tab:adapter} summarizes the few-shot recognition results of the original Tip-Adapter-F method and our adapter-based \textbf{\texttt{PLOT}}  methods on ImageNet. From this table, We observe that using the OT distance can improve the performance of the adapter-based method.

\textbf{Q: Can \textbf{\texttt{PLOT}}  benefit zero-shot learning? } \textbf{A: No}.
CLIP shows that manually designing the prompts can still achieve good performance. We obtain 7 prompts by prompt engineering on the ImageNet dataset and can further ensemble them to obtain \bm{$60.38\%$} top 1 accuracy. In this section, we replace the cosine distance between the global visual feature and prompt ensemble with the OT distance between the feature map and all 7 prompts. However, without any learning, the OT distance only obtains \bm{$58.78\%$} accuracy. It is a limitation of the \textbf{\texttt{PLOT}}  to still need few-shot data for optimization, which cannot be directly applied in the zero-shot setting. We argue there are two reasons why the OT distance does not work without learning: 1) prompt engineering selects prompts based on the global feature and cosine distance, instead of OT distance with feature map; 2) all these selected prompts are close to the global feature and lack the complementarity.

\textbf{Q: What is the extra computation time cost of PLOT over CoOp baseline?}
\textbf{A: Around \bm{$10\%$} inference speed and \bm{$5\%$} training time}.
Please see the detailed analysis in the appendix.

\begin{figure}
  \centering
  \includegraphics[width=0.77\linewidth]{visualization.pdf}
  \caption{Visualization. We provide the heatmaps of transport plan $\bm{T}$ related to each prompt on 4 categories in ImageNet. Different transport plans focus on different attributes of the object.}
\label{fig: visualization}
\vspace{-0.5cm}
\end{figure}

\vspace{-10pt}
\subsection{Visualization}
\vspace{-5pt}
In this subsection, we provide some visualization examples of the transport plans $\bm{T}$ related to different prompts (N=4).
We translate each transport plan into colorful heatmaps and resize them into their original size and combine them with the raw image.
As shown in Figure~\ref{fig: visualization}, we provide the heatmaps of 4 categories in ImageNet. We observe that different transport plans highlight different regions of the image, which demonstrates that the learned multiple prompts are complementary. For the class ``Brambling'', the prompts respectively focus on the head, tail, wing, and environment.
For ``Dog Sled'', the prompts are related to dogs, the sled, some ties, and the snow environment.  We also provide the interpretation of the learned prompt and the visualization of the false case. Please find them in Section \ref{sec: interpretation_prompts} and \ref{sec: false_case} in the appendix.

\vspace{-10pt}
\section{Conclusion}
\vspace{-5pt}
In this paper, we present a method, named \textbf{\texttt{PLOT}}, to learn multiple comprehensive prompts to describe diverse characteristics of one category. To avoid convergence to one point, we propose to apply the optimal transport to achieve the fine-grained alignment between both vision and language domains. We apply a two-stage optimization strategy where the inner loop fixes the prompts and learns the transport plan to calculate the cross-modality distance, and the outer loop uses this distance to optimize the prompt learner. We build our method on the base of CoOp and achieve significant improvement on the few-shot recognition task in various datasets, which demonstrates the advantage to learn multiple prompts instead of a single one.

\section{More Details of Optimal Transport}
\label{app:ot}

The Optimal Transport~\citep{monge1781memoire} is initially introduced to find a transportation plan to move simultaneously several items at a minimal cost, such as moving a pile of sand to fill all the holes. Recently, it is widely used for the comparison of distributions.
Mathematically, given two probability density function $U$ and $V$ over space $\mathcal{X} $ and $\mathcal{Y} $, the OT (Wasserstein) distance~\citep{thorpe2019introduction} can be defined as
\begin{equation} \label{eq:wasserstein}
D_{\text{OT}}(U,V) = \underset{\Gamma}{\inf}  \int_{\mathcal{X}\times\mathcal{Y} } \bm{C}(\bm{x},\bm{y}) d\gamma(\bm{x},\bm{y}),
\end{equation}
where $\bm{C}(\bm{x},\bm{y})$ is the cost between two points in the space $\mathcal{X}\times\mathcal{Y} $, and $\Gamma $ denotes the set of transport plans between support points $ \bm{x}$ and $ \bm{y}$ (e.g. $\gamma(\bm{x},\bm{y}) $).
We can regard two probability density functions $U$ and $V$ as piles and holes and $\bm{C}$ is the cost function of moving a unit of sand.

In our problem of multiple prompts learning, we formulate the sets of visual features and prompt features as two discrete distributions as
\begin{equation} \label{eq:discrete2}
U=\sum_{m=1}^{M}u_m\delta_{\bm{f}_m} \hspace{2em} \text{and} \hspace{2em} V=\sum_{n=1}^{N}v_n\delta_{\bm{g}_n},
\end{equation}
where $\bm{u}$ and $\bm{v}$ are the discrete probability vectorss that sum to 1, and $\delta_{\bm{f}}$ is a Dirac delta function placed at support point $\bm{f}$ in the embedding space. Given two support points $\bm{f}_m$ and $\bm{g}_n$, the cost function is written as $\bm{C} (\bm{f}_m,\bm{g}_n )= 1- \text{sim}(\bm{f}_m,\bm{g}_{n})= 1-\frac{\bm{f}_m^\top\bm{g}_{n}}{||\bm{f}_m||\cdot||\bm{g}_{n}||}$. For simply, in this discrete situation, $\bm{C} \in \mathbb{R}^{M\times N}$ is a cost matrix in which each point denotes the cost between $\bm{f}_m$ and $\bm{g}_n$.
Then, the total distance of these two distributions is written as:
\begin{equation} \label{eq:cost2}
<\bm{T},\bm{C}> = \sum_{m=1}^{M}\sum_{n=1}^{N}\bm{T}_{m,n}\bm{C}_{m,n},
\end{equation}
where the $\bm{T} \in \mathbb{R}^{M\times N} $ is a matrix of transport plan, which is learned to minimize the total distance. Each point $\bm{T}_{m,n}$ in $\bm{T} $ is a weight of local cost $\bm{C}_{m,n} $.

The optimization problem of optimal transport is formulated as:
\begin{equation} \label{eq:optimization2}
\begin{aligned}
& d_{\text{OT}}(\bm{u},\bm{v}|\bm{C}) = \underset{\bm{T}}{\text{minimize}}
 <\bm{T},\bm{C}> \\
& \text{subject to}
~~~~~~\bm{T}\bm{1}_N = \bm{u},\; \bm{T}^\top \bm{1}_M = \bm{v},\; \bm{T} \in \mathbb{R}^{M \times N}_+.
\end{aligned}
\end{equation}
These constraints of $\bm{T}$ are used to match its marginal distributions and original discrete distributions in Eq.~\ref{eq:discrete2}. In our framework, we
treat visual features $\bm{f}_m$ and prompt features $\bm{g}_n$ equally and thus $\bm{u} = \bm{1}_{M\times 1}/M$ and $\bm{v} = \bm{1}_{N\times 1}/N$.

As directly optimizing the above objective is always time-consuming, we apply the Sinkhorn distance~\citep{cuturi2013sinkhorn} to use an entropic constraint for fast optimization.
The optimization problem with a Lagrange multiplier of the entropy
constraint is:
\begin{equation} \label{eq:Sinkhorn-app-1}
\begin{aligned}
& d_{\text{OT},\lambda}(\bm{u},\bm{v}|\bm{C})=\underset{\bm{T}}{\text{minimize}}
 <\bm{T},\bm{C}> - \lambda h(\bm{T})\\
& \text{subject to}
~~~~~~~\bm{T}\bm{1}_N = \bm{u},\; \bm{T}^\top \bm{1}_M = \bm{v},\; \bm{T} \in \mathbb{R}^{M \times N}_+,
\end{aligned}
\end{equation}
where $h(\cdot) $ is entropy and $\lambda \geq 0$ is a hyper-parameter. Then we can have a fast optimization solution with a few iterations as:
\begin{equation} \label{eq:Sinkhorn-app-2}
\bm{T}^*= \text{diag}(\bm{u}^{(t)}) \exp(-\bm{C}/\lambda) \text{diag}(\bm{v}^{(t)}),
\end{equation}
where $t$ denotes iteration and in each iteration
$\bm{u}^{(t)} =\bm{u}/\left((\exp(-\bm{C}/\lambda)\bm{v}^{(t-1)}\right) $ and  $\bm{v}^{(t)} =\bm{v}/\left((\exp(-\bm{C}/\lambda)^\top\bm{u}^{(t)}\right) $, with the initiation $\bm{v}^{(0)} = \bm{1} $. The detailed algorithms of the training and testing processes are shown in Algorithms~\ref{Algorithm} and ~\ref{Algorithm2}.

\renewcommand{\algorithmicrequire}{\textbf{Input:}}
\renewcommand{\algorithmicensure}{\textbf{Output:}}
\renewcommand{\algorithmicrequire}{\textbf{Input:}}
\begin{algorithm}[tb]
\caption{\!\!\! \textbf{:} The training process of Prompt Learning with Optimal Transport}

\label{Algorithm}
    \begin{algorithmic}[1]
     \Require Training few-shot image data: $\mathbf{X}$ = $\{ \bm{x}\}$,
     pretrained CLIP model $f$ and $g$,
      number of prompts $N$, entropy parameter $\lambda$, maximum number of iterations in inner and outer loops $T_{in},T_{out}$.
    \Ensure  The parameters of prompts  $\{\bm{ \omega }_{n}|_{n=1}^{N}\}$
    \State {Initialize $\{\bm{ \omega }_{n}|_{n=1}^{N}\}$}
    \For {$t_{out}=1,2,\dots,T_{out}$ in the outer loop}
        \State{Obtain a visual feature set $\bm{F} \in \mathbb{R}^{M \times C} $ with the visual encoder $f(x)$; }
        \State{Generate prompt feature set $\bm{G}_k \in \mathbb{R}^{N \times C}$
        of each class with the textual encoder $\{g(t_k^n)\}|_{n=1}^{N}$;}
        \State{Calculate the cost matrix $\bm{C}_k=\bm{1}-\bm{F}^\top\bm{G}_k \in \mathbb{R}^{M \times N} $ of each class}
        \State{Calculate the OT distance with an inner loop:}
        Initialize the $\bm{v}^{(0)} = \bm{1} $, $\delta =0.01$ and $\Delta_{v}= \infty$
        \For{$t_{in}=1,2,\dots,T_{in}$}
        \State{Update $\bm{u}^{(t_{in})} =\bm{u}/((\exp(-\bm{C}/\lambda)\bm{v}^{(t_{in}-1)}) $}
        \State{Update $\bm{v}^{(t_{in})} =\bm{v}/((\exp(-\bm{C}/\lambda)^\top\bm{u}^{(t_{in})}) $}
        \State{Update $\Delta_{v} =\sum |\bm{v}^{(t_{in})}- \bm{v}^{(t_{in}-1)}|/N $}
        \If{$\Delta_{v}< \delta$}
        \State{break}
        \EndIf
        \EndFor
        \State{Obtain optimal transport plan  as $\bm{T}^*_k= \text{diag}(\bm{u}^{(t)}) \exp(-\bm{C}_k/\lambda) \text{diag}(\bm{v}^{(t)}),$}
        \State{Calculate the OT distance $d_{\text{OT}}(k) = <\bm{T}^*_k,\bm{C}_k >$}
        \State{Calculate the classification probability $p_{\text{OT}}(y=k|\bm{x})$ with the OT distance}
        \State{Update the parameters of prompts $\{\bm{ \omega }_{n}|_{n=1}^{N}\}$ with cross-entropy loss $L_{\text{CE}}$ }
    \EndFor
    \State \Return $\{\bm{ \omega }_{n}|_{n=1}^{N}\}$

    \end{algorithmic}
\end{algorithm}

\renewcommand{\algorithmicrequire}{\textbf{Input:}}
\renewcommand{\algorithmicensure}{\textbf{Output:}}
\renewcommand{\algorithmicrequire}{\textbf{Input:}}
\begin{algorithm}[tb]
\caption{\!\!\! \textbf{:} The inference process of Prompt Learning with Optimal Transport}

\label{Algorithm2}
    \begin{algorithmic}[1]
     \Require Testing image data: $\mathbf{X}$ = $\{ \bm{x}\}$, number of prompts $N$, number of classes $K$,
     learned prompts $\{\bm{t}_k^n|_{k=1,n=1}^{K,N}\}$,
      a frozen pretrained CLIP model including image encoder $f$ and text encoder$g$
    \Ensure  The classification of each image
    \For {$\bm{x}$ in $\mathbf{X}$ }
        \State{Obtain a visual feature set $\bm{F} \in \mathbb{R}^{M \times C} $ with the visual encoder $f(x)$; }
        \State{Generate prompt feature set $\bm{G}_k \in \mathbb{R}^{N \times C}$
        of each class with the textual encoder $\{g(t_k^n)\}|_{n=1}^{N}$;}
        \State{Calculate the cost matrix $\bm{C}_k=\bm{1}-\bm{F}^\top\bm{G}_k \in \mathbb{R}^{M \times N} $ of each class}
        \State{Calculate the OT distance with an inner loop:}
        Initialize the $\bm{v}^{(0)} = \bm{1} $, $\delta =0.01$ and $\Delta_{v}= \infty$
        \For{$t_{in}=1,2,\dots,T_{in}$}
        \State{Update $\bm{u}^{(t_{in})} =\bm{u}/((\exp(-\bm{C}/\lambda)\bm{v}^{(t_{in}-1)}) $}
        \State{Update $\bm{v}^{(t_{in})} =\bm{v}/((\exp(-\bm{C}/\lambda)^\top\bm{u}^{(t_{in})}) $}
        \State{Update $\Delta_{v} =\sum |\bm{v}^{(t_{in})}- \bm{v}^{(t_{in}-1)}|/N $}
        \If{$\Delta_{v}< \delta$}
        \State{break}
        \EndIf
        \EndFor
        \State{Obtain optimal transport plan  as $\bm{T}^*_k= \text{diag}(\bm{u}^{(t)}) \exp(-\bm{C}_k/\lambda) \text{diag}(\bm{v}^{(t)}),$}
        \State{Calculate the OT distance $d_{\text{OT}}(k) = <\bm{T}^*_k,\bm{C}_k >$}
        \State{Calculate the classification probability $p_{\text{OT}}(y=k|\bm{x})$ with the OT distance}
        \State \Return $k^* = \max\limits_{k} p_{\text{OT}}(y=k|\bm{x}) $
    \EndFor

    \end{algorithmic}
\end{algorithm}

\section{Dataset Details}\label{app:dataset}

The datasets we used in the experiments follow CoOp~\citep{coop}, which include 11 datasets for few-shot visual recognition and 4 ImageNet-based datasets for generalization (robustness) evaluation. The details of each dataset are shown in Table~\ref{tab:dataset}, including the number of classes, the sizes of training and testing sets, and original tasks.

\begin{table*}[t]
    \tabstyle{6pt}
    \caption{The detailed statistics of datasets used in experiments. }
    \label{tab:dataset}
    \resizebox{\textwidth}{!}{
    \begin{tabular}{lcccc}
    \toprule
Dataset                  & Classes  & Training size  & Testing size & Task \\ \midrule
Caltech101~\citep{fei2004learning} & 100 & 4,128 & 2,465& Object recognition \\
DTD~\citep{cimpoi2014describing}& 47 & 2,820 & 1,692 &  Texture recognition\\
EuroSAT~\citep{helber2019eurosat}& 10 & 13,500 & 8,100 & Satellite image recognition \\ FGVCAircraft~\citep{maji2013fine} & 100 & 3,334 & 3,333 & Fine-grained aircraft recognition\\
Flowers102~\citep{nilsback2008automated} & 102 & 4,093 & 2,463 & Fine-grained flowers recognition \\ Food101~\citep{bossard2014food} & 101 & 50,500& 30,300 & Fine-grained food recognition  \\ ImageNet~\citep{deng2009imagenet} & 1,000 & 1.28M & 50,000 & Object recognition \\ OxfordPets~\citep{parkhi2012cats} & 37  & 2,944 & 3,669 & Fine-grained pets recognition \\ StanfordCars~\citep{cars} & 196 & 6,509 & 8,041 & Fine-grained car recognition \\
SUN397~\citep{xiao2010sun}& 397& 15,880 & 19,850 & Scene recognition\\
UCF101~\citep{soomro2012ucf101}& 101 & 7,639 & 3,783 & Action recognition\\
\midrule
ImageNetV2~\citep{ImageNetv2} & 1,000 & - & 10,000 & Robustness of collocation  \\
ImageNet-Sketch~\citep{wang2019learning} & 1000 & - &50,889 & Robustness of sketch domain\\
ImageNet-A~\citep{imagenet-a}& 200 & - &7,500 &Robustness of adversarial attack\\
ImageNet-R~\citep{imagenet-r}& 200 & - &30,000&Robustness of multi-domains\\
    \bottomrule
    \end{tabular}
    }
\end{table*}

\section{Few-shot Recognition Accuracy}

In Section 4.3.1, we provide a line chart to show and compare the performance of \textbf{\texttt{PLOT}}  and CoOp. In this section, we provide detailed performance results on all 11 few-shot recognition datasets in Table~\ref{tab:few-shot}, where we use gray for our method and white for CoOp. To highlight, we respectively use dark cyan and light cyan to represent the performance of \textbf{\texttt{PLOT}}  and CoOp on the average of all 11 datasets. We repeat all experiments 3 times and report the mean and standard deviation in the table.

\begin{table*}[t]
\centering
 \caption{The few-shot visual recognition accuracy on 11 datasets.}
 \label{tab:few-shot}
 \centering
 \small \renewcommand{\arraystretch}{1.0}
 \setlength{\tabcolsep}{3pt}
\begin{tabular} {l>{\columncolor{white}[1pt][\tabcolsep]}l>{\columncolor{white}[1pt][\tabcolsep]}ccccc}
\toprule
Dataset                  & Methods  & 1 shot  & 2 shots & 4 shots & 8 shots  & 16 shots\\ \midrule
\multirow{2}{*}{\centering Caltech101}
    &  \cellcolor{Gray}\textbf{\texttt{PLOT}}    & \cellcolor{Gray}$89.83\pm 0.33$  & \cellcolor{Gray}$90.67\pm 0.21$  & \cellcolor{Gray}$90.80\pm 0.20$  & \cellcolor{Gray}$91.54\pm 0.33$  &\cellcolor{Gray} $92.24\pm 0.38$  \\
    &  CoOp   & $87.51\pm 1.02$  & $87.84\pm 1.10$  & $89.52\pm 0.80$  & $90.28\pm 0.42$  & $91.99\pm 0.31$  \\
    \midrul
\multirow{2}{*}{\centering DTD}
    &  \cellcolor{Gray}\textbf{\texttt{PLOT}}    & \cellcolor{Gray}$46.55\pm 2.62$  & \cellcolor{Gray}$51.24\pm 1.95$  & \cellcolor{Gray}$56.03\pm 0.43$  & \cellcolor{Gray}$61.70\pm 0.35$  & \cellcolor{Gray}$65.60\pm 0.82$  \\
    &  CoOp   & $43.62\pm 1.96$  & $45.35\pm 0.31$  &  $53.94\pm 1.37$  & $59.69\pm 0.13$  & $62.51\pm 0.25$  \\ \midrul
    \multirow{2}{*}{\centering EuroSAT}
    &   \cellcolor{Gray}\textbf{\texttt{PLOT}}    & \cellcolor{Gray}$54.05\pm 5.95$  & \cellcolor{Gray}$64.21\pm 1.90$  &\cellcolor{Gray} $72.36\pm 2.29$  & \cellcolor{Gray}$78.15\pm 2.65$  & \cellcolor{Gray}$82.23\pm 0.91$  \\
    &  CoOp   & $52.12\pm 5.46$  & $59.00\pm 3.48$  & $68.61\pm 3.54$  & $77.08\pm 2.42$  & $83.69\pm 0.47$  \\
    \midrul
    \multirow{2}{*}{\centering FGVCAircraft}
    &  \cellcolor{Gray}\textbf{\texttt{PLOT}}    & \cellcolor{Gray}$17.90\pm 0.09$  & \cellcolor{Gray}$18.94\pm 0.44$  &\cellcolor{Gray} $22.36\pm 0.42$  & \cellcolor{Gray}$26.17\pm 0.29$  & \cellcolor{Gray}$31.49\pm 0.89$  \\
    &  CoOp   & $8.59\pm 5.79$  & $16.52\pm 2.38$  & $20.63\pm 2.46$  & $26.63\pm 0.86$  & $31.43\pm 0.96$  \\
    \midrul
    \multirow{2}{*}{\centering Flowers102}
    &  \cellcolor{Gray}\textbf{\texttt{PLOT}}    & \cellcolor{Gray}$71.72\pm 0.97$  & \cellcolor{Gray}$81.19\pm 0.79$  & \cellcolor{Gray}$87.82\pm 0.20$  & \cellcolor{Gray}$92.43\pm 0.25$  & \cellcolor{Gray}$94.76\pm 0.34$  \\
    &  CoOp   & $67.98\pm 1.98$  & $77.58\pm 1.46$  & $86.10\pm 1.05$  & $91.27\pm 0.83$  & $94.49\pm 0.40$  \\
    \midrul
    \multirow{2}{*}{\centering FOOD101}
    & \cellcolor{Gray}\textbf{\texttt{PLOT}}    & \cellcolor{Gray}$77.74\pm 0.47$  &  \cellcolor{Gray}$77.70\pm 0.02$  &  \cellcolor{Gray}$77.21\pm 0.43$  & \cellcolor{Gray}$75.31\pm 0.30$  &  \cellcolor{Gray}$77.09\pm 0.18$  \\
    &  CoOp   & $74.25\pm 1.52$  &  $72.61\pm 1.33$  &  $73.49\pm 2.03$  &  $71.58\pm 0.79$  &  $74.48\pm 0.15$  \\
    \midrul
    \multirow{2}{*}{\centering ImageNet}
    &  \cellcolor{Gray}\textbf{\texttt{PLOT}}    & \cellcolor{Gray}$59.54\pm 0.16$  & \cellcolor{Gray}$60.64\pm 0.06$  & \cellcolor{Gray}$61.49\pm 0.23$  & \cellcolor{Gray}$61.92\pm 0.09$  & \cellcolor{Gray}$63.01\pm 0.13$  \\
    &  CoOp   & $56.99\pm 1.03$  & $56.40\pm 0.87$  & $58.48\pm 0.47$  & $60.39\pm 0.57$  & $61.91\pm 0.17$  \\
    \midrul
    \multirow{2}{*}{\centering OxfordPets}
    &  \cellcolor{Gray}\textbf{\texttt{PLOT}}    & \cellcolor{Gray}$87.49\pm 0.57$  & \cellcolor{Gray}$86.64\pm 0.63$  & \cellcolor{Gray}$88.63\pm 0.26$  & \cellcolor{Gray}$87.39\pm 0.74$  &\cellcolor{Gray} $87.21\pm 0.40$  \\
    &  CoOp   & $85.99\pm 0.28$  & $82.22\pm 2.15$  & $86.65\pm 0.97$  & $85.36\pm 1.00$  & $87.02\pm 0.89$  \\
    \midrul
    \multirow{2}{*}{\centering StanfordCars}
    &  \cellcolor{Gray}\textbf{\texttt{PLOT}}    & \cellcolor{Gray}$56.60\pm 0.36$  & \cellcolor{Gray}$57.52\pm 0.71$  & \cellcolor{Gray}$63.41\pm 0.29$  & \cellcolor{Gray}$67.03\pm 0.50$  & \cellcolor{Gray}$72.80\pm 0.75$  \\
    &  CoOp   & $55.81\pm 1.67$  & $58.41\pm 0.43$  & $62.74\pm 0.16$  & $67.64\pm 0.06$  & $73.60\pm 0.19$  \\
    \midrul
    \multirow{2}{*}{\centering SUN397}
    &  \cellcolor{Gray}\textbf{\texttt{PLOT}}    & \cellcolor{Gray}$62.47\pm 0.43$  & \cellcolor{Gray}$61.71\pm 0.65$  & \cellcolor{Gray}$65.09\pm 0.43$  & \cellcolor{Gray}$67.48\pm 0.04$  &\cellcolor{Gray} $69.96\pm 0.24$  \\
    &  CoOp   & $60.12\pm 0.82$  & $59.60\pm 0.76$  & $63.24\pm 0.63$  & $65.77\pm 0.02$  & $68.36\pm 0.66$  \\
    \midrul
    \multirow{2}{*}{\centering UCF101}
    &  \cellcolor{Gray}\textbf{\texttt{PLOT}}    & \cellcolor{Gray}$64.53\pm 0.70$  \cellcolor{Gray}& $66.83\pm 0.43$  & \cellcolor{Gray}$69.60\pm 0.67$  &\cellcolor{Gray} $74.45\pm 0.50$  & \cellcolor{Gray}$77.26\pm 0.64$  \\
    &  CoOp   & $62.13\pm 1.14$  & $64.05\pm 0.99$  & $67.79\pm 0.71$  & $72.71\pm 0.50$  & $76.90\pm 0.50$  \\
    \midrul
    \multirow{2}{*}{\centering Average}
    &  \cellcolor{DarkCyan}\textbf{\texttt{PLOT}}    &  \cellcolor{DarkCyan}$62.59\pm 1.13$  &  \cellcolor{DarkCyan}$65.23\pm 0.72$  &  \cellcolor{DarkCyan}$68.60\pm 0.52$  &  \cellcolor{DarkCyan}$71.23\pm 0.51$  &  \cellcolor{DarkCyan}$73.94\pm 0.54$  \\
    &  \cellcolor{LightCyan}CoOp   & \cellcolor{LightCyan}$59.56\pm 2.06$  & \cellcolor{LightCyan}$61.78\pm 1.39$  & \cellcolor{LightCyan}$66.47\pm 1.29$  & \cellcolor{LightCyan}$69.85\pm 0.69$  & \cellcolor{LightCyan}$73.33\pm 0.42$  \\
   \bottomrule
\end{tabular}
\end{table*}

\section{Computation Cost Evaluation }

As shown in Table~\ref{tab:time}, we provide the comparison of the training time and inference seed of the baseline method CoOp~\citep{coop} and our \textbf{\texttt{PLOT}}  with the different number of prompts. We report the one-epoch time training on the 1-shot setting of the Food101~\citep{bossard2014food} dataset and the number of images processed by the model in 1 second. Taking $N=4$ as an example, \textbf{\texttt{PLOT}}  only reduces the $9.2\%$ inference speed and requires an extra $4.9\%$ training time, which is acceptable given the performance improvement.

\begin{table*}[t]
    \tabstyle{9pt}
    \caption{The training and inference time comparison.}
    \label{tab:time}
	 \setlength{\tabcolsep}{4pt}
	\begin{tabular}{lccccc}
	 \toprule
Settings& CoOp  & \textbf{\texttt{PLOT}} (N=1)  & \textbf{\texttt{PLOT}} (N=2) & \textbf{\texttt{PLOT}} (N=4) & \textbf{\texttt{PLOT}} (N=8) \\ \midrule
Training Time (s) & 1.127 & 1.135 & 1.148 & 1.182 & 1.267 \\
Inference Time (images/s) & 719.1 & 714.4 & 690.7 & 653.0 & 519.8 \\
    \bottomrule
    \end{tabular}
\end{table*}

\section{The Interpretation of the Learned Prompts}
\label{sec: interpretation_prompts}

The learned prompts are difficult to be understood by humans since the parameters are optimized in the continuous space~\citep{coop}. CoOp proposes to use the word which is nearest to learned prompts in the embedding space to visualize the prompts. Following this manner, we show the nearest words of our learned prompts in Table~\ref{tab:prompts}.  Similar to CoOp, most words can not be directly understood by human logic.  However, we still find the relations between the learned prompts and the corresponding optimal transport plan. As shown in Figure 4 in the main paper, we can observe that the optimal transport plan for Prompt 1 always focuses on the ``head'', such as the head of ``brambling'', the head of ``rooster'', and even the head of ``aircraft carrier''.  It is because the word ``head'' is in Prompt 1.  Similarly, we can find that Prompt 4 prefers the white part of images, such as the white environment in the image of ``brambling'' and the snow in the image of ``dog sled''.  It demonstrates that the learned multiple prompts focus on different characteristics of categories.

\begin{table*}[t]
    \tabstyle{9pt}
    \caption{The nearest words for 16 context vectors of all $N=4$ prompts learned by \textbf{\texttt{PLOT}} . N/A means non-Latin characters. }
    \label{tab:prompts}
    \begin{tabular}{lcccc}
    \toprule
Number                  & Prompt 1  & Prompt 2  & Prompt 3 & Prompt 4\\ \midrule
1& ag & pa & trying & gaz\\
2& flint & as & field & \textbf{white} \\
3 & leaving & wit & N/A & t\\
4 & sot & l & icons & ario \\
5 & tint & N/A& eclub & safe  \\
6& tar & yl &indiffe&class\\
7 & attn & N/A & ts & represented\\
8 & 2  & job & cold & attend \\
9 & rollingstones & built & yeah & vie \\
10& N/A& brought & band  &recognized\\
11& N/A & or& love & old\\
12 & bel & j & late & stel  \\
13 & \textbf{head} & ag &industry & awhile\\
14& artifact &bad &N/A &ded\\
15& an &chie &across &these\\
16& 5 & in &actual &visiting\\
    \bottomrule
    \end{tabular}
\end{table*}

\section{Visualization of the Failure Cases}
\label{sec: false_case}

To better understand the method and further discover the reason of the failure cases. we visualize the attention maps of some failure cases. As shown in Figure ~\ref{fig:fail}, we showed two failure examples with class "2000 AM General Hummer" in the StanfordCars dataset.
During the training, we set the number of prompts as 4, but in these visualization results, we found that some of the learned prompts remarkably coincide with each other. These prompts can be roughly divided into two classes: Foreground and Background. For example, in both images,
prompts 2 (right top) and 3 (left down) focus on the foreground car, while the others focus on the background. It demonstrates that not all classes have multiple complementary attributes, which motivates us to go further to learn the dynamic local prompts numbers to reduce the computational load in the future.

\begin{figure}[H]
    \centering
    \includegraphics[width=\linewidth]{cars_2.pdf}
    \caption{Failure Visualization. We provide the heatmaps of transport plan T related to each prompt on 2 failure examples in the StanfordCars dataset.}
    \label{fig:fail}
\end{figure}

redpajama_set_name: RedPajamaArXiv


text:
Adjust Font |
RSS | Copyright & Reuse
U.S. State Department
Faith and Mental Health Help Shape the Integration of Muslim Refugees in Germany
UNHCR/G. Welters
Faith plays a significant role in the mental health and social integration of refugees and asylum seekers. Yet in Germany, concerns about Islam and migrants from Muslim-majority countries have complicated social cohesion efforts, particularly in the wake of the 2015-16 refugee and migration crisis.
Top 10 Migration Issues of 2020
IOM/Angela Wells
The COVID-19 pandemic had a sweeping impact on global migration and mobility in 2020. The year also saw further erosion of the humanitarian protection system, movement sparked by natural disasters, and other major developments. Check out our annual countdown of what we see as the most significant migration developments of the year.
Building Climate Resilience through Migration in Thailand
Cpl. Robert J. Maurer/U.S. Marine Corps
Migration can help build resilience against the encroaching effects of climate change. Instead of being passive victims of environmental degradation, individuals sometimes move to gain money, knowledge, and skills that can fortify their household of origin. Migrant workers from Thailand demonstrate how and under what conditions this process works.
Gage Skidmore
In Relatively Peaceful Tanzania, Climate Change and Migration Can Spur Conflict
B. Bannon/UNHCR
Climate-induced migration can lead to tensions and violence between host communities and new arrivals. This conflict can flare up at various levels, including among rural farmers and herders in relatively peaceful countries such as Tanzania.
Special Issue: Climate Change and Migration
Climate change promises to profoundly impact all aspects of human society. This special issue of the Migration Information Source and a companion podcast, Changing Climate, Changing Migration, examine how the effects of environmental change are shaping migration across the globe, now and in the future.
Whether as migrant-sending or migrant-receiving locations—or both—many countries have rich, complex international and internal migration histories. MPI's online journal, the Migration Information Source, offers profiles of more than 70 nations. Written by leading scholars, these profiles delve into countries' migration histories, demographics, policymaking, and more.
U.S. Spotlight
U.S. Policy Beat
Browse by Article TypeFeaturePolicy BeatCountry ProfileSpotlight
Browse by RegionAfrica (sub-Saharan)Asia and the PacificCentral America & the CaribbeanEurope-European UnionMiddle East & North AfricaNorth America-United States-Canada-MexicoSouth America
-Year20022003200420052006200720082009201020112012201320142015201620172018201920202021
On the Brink of Demographic Crisis, Governments in East Asia Turn Slowly to Immigration
With many countries in East Asia facing unfavorable demographic shifts in the form of aging populations, low fertility, and shrinking workforces, governments in 2016 continued to explore immigration as a potential policy solution. However, a tradition of cultural homogeneity and wariness among publics about increased immigration is leading policymakers to test the waters with very small steps.
While Mobility Comes under Assault in Europe, Other Regions Forge Ahead
In stark contrast to a Europe that is erecting new barriers and reinstituting border controls, other regions around the world are moving toward greater mobility for intraregional travelers and migrants. Regional blocs in South America and Southeast Asia have been working to ease intraregional movements of workers, and the African Union in 2016 launched a new biometric African passport.
Despite Hard-Line Immigration Campaign Pledges, Trump Policy Remains in Flux
Donald Trump has made a series of postelection statements suggesting he may backtrack on several campaign pledges on immigration, including building a wall across the entire U.S.-Mexico border and deporting all 11 million unauthorized immigrants. Still, his choice of top advisors points to a hard-line agenda. This Policy Beat article examines what some of the better known elements of Trump's immigration policy might look like.
The Impact of Mexican Maternal Migration on Children's Future Ambitions
While Mexican women account for a significant share of migration flows to the United States, there has been little focus on their movement and effects on children in Mexico. This article, based on survey data of children in Puebla, Mexico, explores the impact of maternal Mexican migration on educational experiences and aspirations of the children left behind.
Language Diversity and English Proficiency in the United States
Although the number of U.S. residents who speak a language other than English has grown in recent decades, the share of those who are Limited English Proficient (LEP) has fallen: 40 percent in 2015, compared to 44 percent in 1980—even as immigration rose rapidly. This article examines growing linguistic diversity in the country and sketches a profile of the LEP population, including size, location, and socioeconomic characteristics.
full archive >
Little Job Growth Makes Labor Migration and Remittances the Norm in Post-Soviet Armenia
For many Armenians, working abroad and sending money home has become the main way of coping with poverty and limited job prospects. Aleksandr V. Gevorkyan, Arkady Gevorkyan, and Karine Mashuryan examine recent labor migration flows, the growth in remittances, and concerns about the country's dependence on both.
Material Support to Terrorism — Consequences for Refugees and Asylum Seekers in the United States
Counterterrorism policies have had an unintended, negative effect on asylum and refugee resettlement in the United States. Swetha Sridharan of the Council on Foreign Relations explains the origins of the material-support bar, which groups it has affected, and how the U.S. government is addressing the problem.
Canada's Temporary Migration Program: A Model Despite Flaws
In response to an agricultural worker shortage over 40 years ago, Canada initiated a temporary migration program to brings workers from the Caribbean and later Mexico. But this "model" program also has its drawbacks, as Tanya Basok of the University of Windsor explains.
Causes of South-South Migration and Its Socioeconomic Effects
Migrants' networks and relatively small travel distances help explain migration from one developing country to another. Dilip Ratha and William Shaw of the World Bank look at these and other reasons for and effects of South-South migration.
How Los Angeles Deflected Mexican Immigrants to the American Heartland
In the 1990s, Mexican immigrants began to leave California, Texas, and Illinois for the so-called new settlement states where they had not previously resided. As Ivan Light of UCLA explains, their reasons for leaving or bypassing Los Angeles were both economic and political.
The search yielded 0 results
About MPI's Online Journal
The Migration Information Source provides fresh thought, authoritative data, and global analysis of international migration and refugee trends.
For more about the Source, click here.
Sign up for the Source Newsletter
Contact the Source
Advertise in the Source
Interested in quickly finding out what resources MPI and its online journal, the Migration Information Source, have by country? The country resources page allows users to select individual countries and see all the Source articles, MPI research, multimedia, and more that relates to an individual country. This is a great way to dive into everything MPI and the Source have to offer by country.
Immigration to the United States is the subject of significant debate, with questions ranging from the size of inflows, the role of immigrants in the labor market, and the nature of enforcement, to humanitarian admission policies and the sociodemographic characteristics of new arrivals. As such, it is important for the public to have access to accurate current and historical information. This article offers the top statistics on immigrants and immigration to the United States.
The Migration Data Hub
The Migration Data Hub showcases the most current demographic, social, and economic statistics about immigrants and immigration internationally and in the United States (including at the state level). Visit the Data Hub for current and historical data on immigrant and refugee populations, migration trends, and sociodemographic characteristics. From asylum recognition rates to remittance flows, the Data Hub offers easy access to top statistics.
Top 10 Migration Issues of the Year
Every year, the Migration Information Source compiles the Top 10 Migration Issues of the Year, assessing key developments globally regarding immigration developments, policies, and trends. Visit our Top 10 Library today to check out the top migration developments of the year from 2005 onward.
The Migration Information Source has published a series of special issues that focus on particular migration trends of note, among them "Migration in the Modern Chinese World," "Women and Migration," and a special issue on migration and development. Check out our Special Issue Library.
The Philippines: Beyond Labor Migration, Toward Development and (Possibly) Return
Naturalization Trends in the United States
The Obama Record on Deportations: Deporter in Chief or Not?
European Immigrants in the United States
Chinese Immigrants in the United States
Filipino Immigrants in the United States
Display Modal
The Migration Information Source's twice-monthly free newsletter provides fresh thought, authoritative data, and global analysis of international migration and refugee trends.

redpajama_set_name: RedPajamaCommonCrawl


text:
Q: Why and how is IE11 UA string spoofed when visiting specific sites? When visiting certain sites with IE11 (such as www.walgreens.com), the following UA string is emitted by the browser:
Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/7.0)

...but for most other sites the expected IE11 UA string is sent:
Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko

What is the underlying mechanism at work here?  Why is MSIE 9.0 being sent?  It doesn't seem like it has anything to do with the server response, since the request and its headers are fully sent before receiving a response (from what I understand).  It's not clear to me what information the browser is using to make this distinction.

A: Internet Explorer 10 and 11 have an option to download a list from a Microsoft server that tells IE to use specific compatibility settings for specific sites. You can view a version of this list here and will see that Walgreens.com is listed with the info to use compatibility to IE 9.
If you want to disable this behaviour on the client, disable the option "download compatibility list from Microsoft" in the Compatibility View Settings of Internet Explorer.
If you own the domain Walgreens.com, you have to contact Microsoft, so they take you from the list.

redpajama_set_name: RedPajamaStackExchange


Primary care is a term for a category of medical practice whereby each patient has a single physician who directs and coordinates the patient's health care. Primary care physicians are often general/family practitioners or internists and are responsible for the general care of their patients as well as the coordination of treatment provided by specialists. Primary care encourages relationship building between physician and patient, creating an environment of knowledge and trust and allowing for a history that helps both patient and physician to detect changes in health status or condition, early on. The concept is also fundamental to most insurance plans in that many of them require patients to see a primary care physician about a problem before visiting a specialist.
Family medicine is one of four subspecialties-pediatrics, family practice medicine, general medicine, and internal medicine-associated with primary care. Because they are licensed to treat all members of a family starting with children above age five and including women of child-bearing age, family medicine physicians undergo training in pediatrics and gynecology, as well a general internal medicine. Family medicine practitioners are required to complete an average of 30 hours of supplemental medical education training each year.
Many of the doctors associated with Roger Williams Medical Center are family practice physicians and have close working relationships with the center's specialists to ensure the utmost continuity of care.
Geriatric medicine is the specialty devoted to the study of old age and the disorders and health requirements of those above 60 years of age. Doctors who specialize in this branch of medicine are called geriatricians.
Often with a background in general or internal medicine, the generalist physician becomes a geriatrician by acquiring special training in the aging process and the diagnostic, therapeutic, preventive, and rehabilitative aspects of illness in the elderly. Geriatric medicine compares to pediatrics in terms of its understanding of age-specific issues and care. While seniors certainly could see non-specialized internists or generalists, they might be better served by geriatricians' special knowledge, just as a 5-year-old is better served by a pediatrician.
An example of the special skills required for geriatric care is the need to understand drug interactions of the multiple drugs that seniors may be taking and common problems of old age, including incontinence, bone breakage due to falls, Parkinson's disease, Alzheimer's disease and dementias. To cater to the special needs of the elderly, geriatricians may provide care outside of the office, for example, in patients' homes and nursing homes. They also may utilize non-medical resources such as community social services. In addition, because they have first-hand knowledge of their patients' capabilities they may counsel family members about such issues as the ability to drive a car and the need for nursing care.
Internal medicine deals with the function of the internal organs, such as the liver and lungs, plus the diagnosis and treatment of associated problems. Internists are often primary care physicians, providing the first contact for an unspecified internal illness or problem. They may diagnose and treat the problem themselves, or work in conjunction with a specialist for more focused diagnosis and treatment.


text:
Petroleum Equipment & Services Association Elects New Board of Directors
Cameron's Gary Halverson, FMC Technologies' Johan Pfeiffer and Gardner Denver's Saeid Rahimian to lead organization during the 2015-2016 year.
Houston, Texas (PRWEB) April 30, 2015
The Petroleum Equipment & Services Association (PESA) has elected Gary Halverson as chairman, Johan Pfeiffer as vice chairman and Saeid Rahimian as first vice president for the 2015-2016 Board of Directors. The leaders were announced at the PESA Annual meeting on April 24, 2015.
"Since PESA was founded in 1933, our members have evolved with the energy industry, developing new technologies and contributing to exponential job growth," said Halverson. "In the coming year, I look forward to working with PESA's board, staff and membership to further the understanding of the vital part the service and supply sector plays in the energy industry."
Halverson joined Cameron in 1978 and has held numerous management positions throughout his career in the U.S. and overseas. He is currently the president, drilling & production systems and senior vice president of Cameron International. A graduate of Odessa College, he previously served as vice chairman on the executive committee of PESA.
Prior to assuming the role of vice president of Surface Technologies for FMC Technologies Inc., Pfeiffer was vice president of Global Surface Wellhead. He is a graduate of the Swiss Institute of Technology, with a master's degree in international studies from the University of Pennsylvania's Lauder Institute and an MBA from the University of Pennsylvania's Wharton School.
Rahimian is the president and chief executive officer of Gardner Denver's Energy Group, and corporate vice president. He obtained his undergraduate degree from the University of Evansville and a master of science in mechanical engineering from The Ohio State University. In addition, he is a graduate of the Harvard Business School Advanced Management Program.
"Under the board's leadership, PESA is developing a stronger voice for the energy industry's service, supply and manufacturing organizations," explained PESA President Leslie Shockley Beyer. "Our member companies' achievements in job creation, technological innovation and energy security continue to be a critical contribution to the U.S. economy. It has been a pleasure working with outgoing chairman Paul Coppinger and I look forward to continuing this momentum with Gary, Johan and Saeid as we face new challenges and opportunities for our sector of the industry and our members."
Outgoing Chairman Paul Coppinger, divisional managing director of Weir Oil & Gas, will remain active on the board as immediate past chairman. For the past year, he has worked with PESA to rebrand the organization and expand advocacy efforts on behalf of the membership.
About PESA
The Petroleum Equipment & Services Association (PESA) is the unified voice for the energy industry's service, supply and manufacturing organizations, advocating and supporting continued achievements in job creation, technological innovation and economic stability. Today, PESA's more than 200 companies represent a diverse global workforce of 1.3 million that generates more than $277 billion in annual revenue. To learn more, visit www.pesa.org.
What is Hydraulic Fracturing?
Congressional Leadership Dialogues


I don't know what it is about Blackpool but I just adore the place. Where some may see poverty, drunkenness and an economy hit hard, like most British seaside towns, resulting in a widespread drug problem, I see sun, sea, sand and a chance for my toddler to have fun.
I have lovely memories of being taken to Blackpool as a child and yes, it may be tacky, but I love the flashing lights of the arcade, the decadence of the old trams, the tradition of the illuminations and the thrill of the Pleasurebeach. There are so many different things to do when you arrive in Blackpool and one day is simply not enough! We recently visited for the day for the first time in nearly a year and we sadly ran out of time. Next time we are definitely going to book into one of the many hotels Blackpool has to offer, and make a weekend of it.
Having a toddler means that the Blackpool I enjoyed in the past may not be so relevant any more. Gone are the days of spending afternoons in the local watering holes drinking cheap lager; spending a few hours in the Pleasurebeach testing out all the thrilling rides and spending hours on end in Primark looking for the perfect outfit and leaving with ten dresses, a full set of jewellery, two pairs of shoes and god knows how many vest tops.
These days you are more likely to find us doing things which are way more fun than wasting money boozing and frittering the cash away on clothes. Here are my top seven things to do in Blackpool with a toddler.
There are literally hundreds of arcades down the Golden Mike but Coral Island has always been and remains to this day my favourite. Is it the amazing pirate monorail which floats above the arcade itself? The flashing lights of the best machines in Blackpool? Or maybe it's the location, right opposite Central Pier and only slightly further down from Blackpool Tower. In any case, Coral Island is not to be missed. We spent ages throwing two pence pieces into the machine, desperately trying to get rid of them so we could move on, but continually winning more; I won a teddy on the Kentucky Derby (my secret talent, built up from many games in Blackpool as a child!) and Dex had a go on the Bob the Builder ride on. Such a magical place!
Stepping onto North Pier is like taking a step back in time. It's certainly a lot less busy than Central Pier and we loved the music piping out of the speakers dotted along the side. (Think sixties classics like Jimmy Ruffin and Buddy Holly) What's more there's miniature railway train (I did mention Dex loves the choo choos didn't I?) which silently glides up and down the pier hooting whenever somebody gets in the way. Walking up and down the pier on a hot sunny day let's you cool off a little while taking in the sights of the Tower and the Pleasurebeach beyond.
A trip to the seaside is not complete without devouring an ice cream on the promenade. We found a lovely place to sit just outside of Central Pier who had the most deliciously creamy ice cream available in either a tub or in a cone.
Donkey rides have been a tradition on Britain's seasides since 1890 and are still going on Blackpool beach. Available just outside Central Pier on the seafront, the donkey rides cost a few quid but make one small toddler very happy to see a real life donkey and have a little ride.
A trip to the seaside is not complete without a visit to one of the local cafes to have fish and chips (or sausage and chips and mine and Dexter's cases) The chips have to be slathered in salt and vinegar (okay, not so much salt for the toddler) and I always have mine with gravy because I'm a true notherner and Dex has his with ketchup because he's a toddler. Nom, nom, nom!


text:
Based on your enjoyment of "Project Hail Mary" by Andy Weir… You're likely* to like:
Jazz Bashara is a criminal. Well, sort of. Life on Artemis, the first and only city on the moon, is tough if you're not a rich tourist or an eccentric billionaire. So smuggling in the occasional harmless bit of contraband barely counts, right? Not when you've got debts to pay and your job as a porter barely covers the rent. Everything changes when Jazz sees the chance to commit the perfect crime, with a reward too lucrative to turn down. But pulling off the impossible is just the start of her problems, as she learns that she's stepped square into … (Goodreads)
World War Z: An Oral History of the Zombie War
It has been nearly twenty years since the start of the apocalyptic worldwide pandemic known as the Zombie War, and about ten years since the war has ended in humanity's victory. The framing device for the novel follows "Max Brooks", author of the Zombie Survival Guide (referred to simply as "the civilian survival guide" in this book) and agent of the United Nations Postwar Commission, as he travels the world interviewing survivors of this zombie plague. The exact origin of the zombie plague is unknown, but the first cases of what became the global pandemic began in China . It … (Wikipedia)
The Way of Kings
by Brandon Sanderson
The story rotates between the points of view of Kaladin, Shallan Davar, Szeth-son-son-Vallano, Dalinar Kholin, and several other minor characters, who lead seemingly unconnected lives. Szeth, a Shin man cast out by his people and condemned to obey his constantly changing masters, is sent to assassinate the king of one of the world's most powerful nations, Alethkar. As the story progresses, he continuously changes hands, doing his best to hide the fact that he possesses an Honorblade, a mythical blade used by the Heralds that can cut through any material. He also possesses access to powers that are no longer … (Wikipedia)
Words of Radiance
Years ago, Szeth-son-son-Vallano, the Assassin in White, was sent by the Parshendi to assassinate the Alethi king Gavilar Kholin (for reasons not yet revealed to the reader). This murder resulted in the Vengeance Pact among the highprinces of Alethkar and the War of Reckoning against the Parshendi. Now Szeth is active again, and is sent by King Taravangian of Kharbranth, to kill Highprince Dalinar Kholin (brother of the late King Gavilar). Kaladin, once a slave and bridgeman on the Shattered Plains, is given command of the royal bodyguards to protect Dalinar and his family (including King Elokhar) from perils and … (Wikipedia)
by N.K. Jemisin
In a prologue, an extraordinarily powerful orogene discusses the sad state of the world and laments the oppression of his race. He then uses his enormous power to fracture the entire continent along its length, threatening to cause the worst Fifth Season in recorded history. The story then follows three female orogenes across the Stillness from different time periods: Essun, Damaya, and Syenite. Essun is a middle-aged woman with two young children living in a small southern comm named Tirimo. Secretly, she is an orogene, a human with the ability to manipulate earth and stone by absorbing or redirecting heat … (Wikipedia)
The Dark Forest
This is the second novel in "Remembrance of Earth's Past", the near-future trilogy written by the China's multiple-award-winning science fiction author, Cixin Liu. In The Dark Forest , Earth is reeling from the revelation of a coming alien invasion — four centuries in the future. The aliens' human collaborators have been defeated but the presence of the sophons, the subatomic particles that allow Trisolaris instant access to all human information, means that Earth's defense plans are exposed to the enemy. Only the human mind remains a secret. This is the motivation for the Wallfacer Project, a daring plan that grants … (Goodreads)
Oathbringer
In Oathbringer , the third volume of the New York Times bestselling Stormlight Archive, humanity faces a new Desolation with the return of the Voidbringers, a foe with numbers as great as their thirst for vengeance. Dalinar Kholin's Alethi armies won a fleeting victory at a terrible cost: The enemy Parshendi summoned the violent Everstorm, which now sweeps the world with destruction, and in its passing awakens the once peaceful and subservient parshmen to the horror of their millennia-long enslavement by humans. While on a desperate flight to warn his family of the threat, Kaladin Stormblessed must come to grips … (Goodreads)
Ready Player Two
by Ernest Cline
Nine days after winning OASIS founder James Halliday's contest and the arrest of Nolan Sorrento, Wade Watts is informed about a new technology that Halliday created and never released to the general public. Called "Oasis Neural Interface" (ONI), this technology allows users to not only experience the OASIS with all five senses, but also record their experiences in real life. Wade quickly becomes addicted to the thrills offered by ONI and reveals its existence to the other members of the High Five. Both Shoto and Aech agree to release the product while Art3mis believes that it would only exacerbate the … (Wikipedia)
by Martha Wells
No, I didn't kill the dead human. If I had, I wouldn't dump the body in the station mall. When Murderbot discovers a dead body on Preservation Station, it knows it is going to have to assist station security to determine who the body is (was), how they were killed (that should be relatively straightforward, at least), and why (because apparently that matters to a lot of people—who knew?) Yes, the unthinkable is about to happen: Murderbot must voluntarily speak to humans! Again! … (Goodreads)
The Hero of Ages
The Hero of Ages is the prophesied savior of the Terris people, foretold to find and give up the power at the Well of Ascension , in a selfless act to save the world from the Deepness. However, the Terris prophecies proved to have been altered, in a ruse allowing the powerful force named Ruin to escape imprisonment. Ruin wanted to destroy the world instantaneously but his power was too weak, as part of it had been taken and hidden by the opposing force, Preservation, long ago. Freedom from the Well of Ascension enabled Ruin to directly affect the world … (Wikipedia)
Children of Time
The book's plot involves a planet inhabited by evolved spiders uplifted by human scientist Avrana Kern, and their much later discovery by the last humans alive in the universe. ,, ,, ,, The work plays off the contrast between the societal development of the spiders and the barbaric descent of the starship crew of the last humans. ,, … (Wikipedia)
The first full-length novel in Martha Wells' ,New York Times ,and ,USA Today ,bestselling Murderbot Diaries series. An ,Amazon's, Best of the Year So Far Pick Named a Best of 2020 Pick for NPR | ,Book Riot ,,| Polygon , ,,,", I caught myself rereading my favorite parts... and I can't recommend it enough." — New York Times You know that feeling when you're at work, and you've had enough of people, and then the boss walks in with yet another job that needs to be done right this second or the world will end, but all you want to … (Barnes & Noble)
Leviathan Falls
by James S.A. Corey
The Laconian Empire has fallen, setting the thirteen hundred solar systems free from the rule of Winston Duarte. But the ancient enemy that killed the gate builders is awake, and the war against our universe has begun again. In the dead system of Adro, Elvi Okoye leads a desperate scientific mission to understand what the gate builders were and what destroyed them, even if it means compromising herself and the half-alien children who bear the weight of her investigation. Through the wide-flung systems of humanity, Colonel Aliana Tanaka hunts for Duarte's missing daughter. . . and the shattered emperor himself. … (Goodreads)
The Kaiju Preservation Society
,The Kaiju Preservation Society, is John Scalzi's first standalone adventure since the conclusion of his ,New York Times, bestselling Interdependency trilogy. When COVID-19 sweeps through New York City, Jamie Gray is stuck as a dead-end driver for food delivery apps. That is, until Jamie makes a delivery to an old acquaintance, Tom, who works at what he calls "an animal rights organization." Tom's team needs a last-minute grunt to handle things on their next field visit. Jamie, eager to do anything, immediately signs on. What Tom doesn't tell Jamie is that the animals his team cares for are not here … (Goodreads)
"As a heartless killing machine, I was a complete failure." In a corporate-dominated spacefaring future, planetary missions must be approved and supplied by the Company. Exploratory teams are accompanied by Company-supplied security androids, for their own safety. But in a society where contracts are awarded to the lowest bidder, safety isn't a primary concern. On a distant planet, a team of scientists are conducting surface tests, shadowed by their Company-supplied 'droid — a self-aware SecUnit that has hacked its own governor module, and refers to itself (though never out loud) as "Murderbot." Scornful of humans, all it really wants is … (Goodreads)
Wool Omnibus
by Hugh Howey
The story of Wool takes place on a post-apocalyptic Earth. ,, Humanity clings to survival in the Silo, a subterranean city extending one hundred forty-four stories beneath the surface. The series initially follows the character of Holston, the sheriff of the Silo, with subsequent volumes focusing on the characters of Juliette, Jahns, and Marnes. An ongoing storyline of the series is the focus on the mystery behind the Silo and the secrets it holds. The mystery is eventually revealed at the end of book five; Shift, which encompasses books six through eight, comprise a prequel to the series. Book nine, … (Wikipedia)
Death's End
The story begins during the Fall of Constantinople , describing a prostitute who gains the power of retrieving objects and cutting out human organs without penetrating their enclosures. Constantine XI tasks her with killing Mehmed II , but her powers are mysteriously lost, and she is killed by Byzantine soldiers in retaliation. Next, the story shifts to the timeframe of the beginning of The Three Body Problem . The physicist Yang Dong, daughter of Ye Wenjie, having discovered her mother's conspiracy with Trisolaris, and witnessing the disruption of the particle-accelerators, is eventually driven to suicide. Prior to her death, she … (Wikipedia)
The Stone Sky
Following the events of ,The Obelisk Gate, , the former inhabitants of Castrima-under are moving north after damage by rival comm Rennanis has compromised the mechanisms of the geode and made it uninhabitable. Essun, who has been in a coma since opening the Obelisk Gate, awakens to find that her arm has turned to stone as a consequence of the massive magical energies of the Gate. She is nursed back to health, and finds that the Moon is approaching the closest point in its long, elliptical orbit, meaning that she has only a short time to return it to a … (Wikipedia)


text:
The 113th Big Game: Cardinal Look To Take The Axe Back From Golden Bears
By Avinash Kunnath@avinashkunnath Nov 17, 2010, 11:57pm PST
Share All sharing options for: The 113th Big Game: Cardinal Look To Take The Axe Back From Golden Bears
Last year, a highly touted Stanford Cardinal team was hovering on the edge of a Rose Bowl berth and coming off of two impressive victories of the two powers in the Pac-10, upsetting the soon-to-be Pasadena-bound Oregon Ducks and steamrolling the USC Trojans in Los Angeles. The Cardinal seemed to be the overwhelming favorites in their matchup with their Bay Area rivals, riding their Heisman candidate Toby Gerhart and up-and-coming quarterback in Andrew Luck to take over as the Bay Area's best.
But Cal turned the tables on them, as both Shane Vereen and Kevin Riley outperformed their counterparts. Vereen rushed for 193 yards on 42 carries and three touchdowns, and Riley threw 17 of 31 for 235 passing yards, one touchdown and one interception, as Cal controlled the football and kept the Cardinal from touching the ball for vast stretches of the game, converting 11 of 19 third downs. While Gerhart had a solid 136 yards on 20 carries, four touchdowns and a beastly 29 yard catch, almost half of those yards came on his opening score and he was held in check for most of the game after Stanford went up 14-0. The Cardinal defense proceeded to give up 31 of the next 38 points and Luck could never get going, completing only 10 of 30 passes and 157 yards. Most damningly, Luck got the Cardinal to the red zone down six with under two minutes left, but threw the game-clinching interception to Mike Mohamed. The loss ended the Cardinal's faint Rose Bowl chances, and the Axe went back to Berkeley.
Now, the Big Game shifts back to Strawberry Canyon, and things seem to look even tougher for Cal to score the upset. Despite the loss of Gerhart, the Cardinal have not let up, starting their season 9-1 behind their Heisman-contending quarterback and an offensive line that is mighty difficult to shed blocks against. Other than struggles with Arizona St. last week and getting blown out by Oregon in Autzen after letting a 21-3 lead, Stanford's offense has been fairly unstoppable, racking up 466 total yards a game (14th in the nation), 24.9 first downs (tied for 10th in the nation) and 39.8 points per game (9th in the nation).
The Cal defense was great last week in holding down Oregon, but can they do it two weeks in a row? The special teams appears to have regressed back to last year's horrible form and the offense could not get anything going after the opening drive behind new starting quarterback Brock Mansion, forcing Vereen to keep the offense moving. The Cardinal defense has greatly improved from last season and has proven they can grind with almost anyone. It could come down to the home-field advantage for Cal; the Golden Bears are 4-1 at home and have seen offenses (no matter how diverse or impressive) grind to a halt in Memorial.
What will happen this week? Anything goes in the Big Game.
Stanford Rolls to 48-14 Victory After Late Cal Touchdown
Pac-10 College Football Scores and Standings: Cal, Washington, Oregon State, UCLA Battling For Bowl Lives