text:
\section{Introduction}
The recent advances in large-scale vision and language models (\textit{e.g.}, CLIP \cite{clip}, ALIGN \cite{align}) have set impressive milestones in several computer vision tasks. The contrastively trained foundation models comprising vision and text encoders can capture semantically insightful concepts in a joint embedding space. As a result, these models offer excellent zero-shot generalization performance in downstream tasks without relying on task-specific supervision, thanks to the availability of instructional text prompts (`\texttt{A photo of a [\textit{CLS}]}'). However, an optimal
prompt design is not trivial. Consequently, recent studies \cite{coop, cocoop, prod} focus on optimizing prompts in a data-driven manner. Despite its success, prompt learning is restricted to training data distribution
and is susceptible to \emph{domain shift} \cite{cocoop}.

\begin{figure}
    \centering
    \includegraphics[width=0.49\columnwidth]{images/teaser_r50.pdf}
    \includegraphics[width=0.49\columnwidth]{images/teaser_vit.pdf}

    \caption{We compare the performance of different DG techniques for the DomainNet \cite{domain_net} dataset for both the ResNet-50 and ViT backbones. We find that Zero-shot CLIP lags far behind state-of-the-art traditional DG models like SWAD \cite{swad}, highlighting its lack of generalizability. However, using tokens referring to the domain identifiers in the prompts improves baseline CLIP's performance. Finally, \textsc{StyLIP} outperforms the previous best prompting techniques \cite{prograd, clip-adapter} substantially, highlighting the importance of style and content disentanglement in the prompts for DG tasks.}

    \label{fig:teaser1}
\end{figure}
Nonetheless, domain shift is natural in real-world applications, and it is a well-known fact that deep learning models are vulnerable to differences in data distributions between training and test sets \cite{batchnorm}. Furthermore, the domain shift is generally unpredictable, as in many applications, we may not have access to the target domain in advance. Therefore, the problem of Domain Generalization (DG) \cite{dg-survey, l2g, mldg} has been investigated. In DG, one or multiple datasets
from different labelled source domains are considered to
learn a domain-generic representation space that can be
evaluated on novel target domains.
Traditionally, DG techniques utilize a vision encoder
trained solely on image data \cite{l2a-ot, ddaig, mixstyle}. In this regard, the superiority of the foundation models in reducing the semantic gap using cross-modal knowledge has motivated researchers to use these models coupled with prompt engineering \cite{clip, coop}, although their applicability for practical DG settings is yet to be systematically explored.

In this paper, we follow this research trend, but different from existing prompting methods \cite{coop, cocoop, prod, prograd}, which evaluate the generalization capabilities of CLIP on datasets where the domain shift is limited (\textit{e.g.} variants of ImageNet \cite{krizhevsky2012imagenet}), we study a more challenging setting where the visual appearance of images varies significantly from different domains.
In this regard, Fig. \ref{fig:teaser1} shows the average multi-source DG performance of different methods on DomainNet \cite{domain_net}. Zero-shot CLIP \cite{clip} performs poorly compared to the best traditional DG model in the literature \textit{i.e.} SWAD \cite{swad}, by approx. $2.5 \%$. Interestingly, CLIP's accuracy increases by almost $3 \%$ when a domain-conditional prompt is utilized (`\texttt{A [Domain] of a [\textit{CLS}]}'), advocating the suitability of a more representative prompt for DG. However, \textit{domain-level annotations may not always be available, and the static domain name is not a representative of the style properties which characterize the domains\footnote{We use \textit{domain} and \textit{style}, and \textit{object} and \textit{semantic} interchangeably}} \cite{mancini2018boosting}.

Specialized prompt-tuning (Fig. \ref{fig:teaser2}) techniques like \cite{amortized, prograd, cocoop} improve CLIP's performance (Fig. \ref{fig:teaser1}). However, such approaches are suboptimal for DG, as it is unclear whether the prompts refined from random vectors \cite{cocoop, prograd} can encode the domain knowledge effectively. In this respect, Zhang \textit{et al.} \cite{amortized} proposes a domain-prompt initialization strategy from the batch statistics of the visual features, but \cite{amortized} considers domain-level supervision and overlooks to capture important lower-level style characteristics. Another recent endeavour \cite{niu2022domain} considers prompt learning from CLIP without utilising the source domains' visual samples but considering the domain knowledge.
It is evident from these discussions that there is a research gap in learning prompts while accounting for unknown domain shifts in the absence of explicit domain identifiers. In this regard, we argue that leveraging the visual features in such a situation is imperative. Another important aspect is to dynamically equip the prompts with object-level variations, incorporating visual feature distributions into the prompts, which is found to partially aid in cross-domain generalization tasks \cite{cocoop}.
This motivates us to address the following research question: \textit{can we leverage CLIP's vision backbone to encode the object style and semantic content information for learning domain and instance aware prompts to tackle DG?}

We propose \textsc{StyLIP}, a novel domain-invariant prompt tuning strategy for CLIP to address the problems mentioned above. To this end, we seek to allow the prompts to learn the support for a given class concept better by conditioning them on the domain and content information derived from the visual space of pre-trained CLIP.
In this respect, \textsc{StyLIP} takes advantage of the well-established hypothesis that the instance-wise feature statistics corresponding to the intermediate feature layers of an image encoder describe the domain information \cite{instyle} at multiple scales. Following this, we propose to extract the mean and standard deviation from the intermediate feature map outputs of CLIP's vision encoder and pass them through a set of \textit{style projectors} to learn domain-specific tokens in the prompts. As opposed to existing models like \cite{coop, cocoop, prograd}, which learn the prompt token embeddings from the
ad-hoc sentences like \texttt{This is the Photo}, using the different scales of the style features certainly helps in learning an improved domain-aware prompts, as they provide a better prompt initialization than the ad-hoc sentences. Subsequently, the class prompt embeddings are obtained through CLIP's text encoder.

In addition, we propose to equip the obtained prompt embeddings with the content information of the images, so they can capture the object-level variations well and do not overfit the training classes. This is specifically important when DG is performed from a set of base (training) classes to a set of novel (test) classes.
Zhou \textit{et al.}  \cite{cocoop} proposes to add high-level semantic image features obtained from CLIP's vision encoder to the prompt tokens to tackle this issue. However, while \cite{cocoop} considers the global data distributions of the base and novel classes identical, in this paper, we are interested in a setting where the distributions of base and novel classes are deemed to differ.
Although previous studies have tacked this issue under the notion of \textit{learning unknown classes from unknown domains} \cite{cumix}, we propose to solve the same under the hood of generalizable prompt learning.
Hence, a more transferable visual content embedding of the images is preferable, something the high-level semantic features considered in \cite{cocoop} may not offer.
To achieve this goal, we propose to combine the visual feature responses obtained from different layers of CLIP's vision encoder and aggregate these multilevel representations through a \textit{content projector}. The idea is to encode mid- to high-level image characteristics, which are more generic across categories \cite{zheng2016good}. As opposed to \cite{cocoop}, we propose to aggregate these visual features with the final prompt embeddings through a learnable fusion network since adding the same visual information to all the prompt tokens may induce redundancy in the token features.

\paragraph{Contributions.} We highlight our major contributions as: \\
- We introduce \textsc{StyLIP}, a domain-agnostic prompt learning strategy that leverages CLIP's frozen vision encoder to extract the domain and content information from an image and deploy them in prompt learning through light-weight learnable projectors;\\
- We propose to learn the individual prompt tokens from the visual style features obtained at different scales, thus allowing our prompts to focus on the style and class semantic information separately without any domain-level supervision. Similarly, we propose a novel way to integrate the multi-scale visual content information into the prompt embeddings. This takes care of the overfitting issue while making the prompt schema generic across domains.\\
- We showcase the performance of \textsc{StyLIP} for multiple datasets on five major DG tasks: i) single-source and multi-source DG, ii) cross-dataset DG, iii) in-domain base to novel class DG, and iv) cross-domain base to novel class DG, a novel task we introduce in the context of prompting. Experimentally, \textsc{StyLIP} outperforms the competitors in all tasks at least by $0.2-4 \%$.
To our knowledge, ours is the first attempt to extensively study the DG problem using CLIP.

\section{Related Works}
\label{sec:related_works}
\paragraph{Domain generalization}There exist several flavours of the DG problem. For example, multi-source DG \cite{l2a-ot, style-neophile, lostdg} considers the presence of multiple training domains, as opposed to single-source DG \cite{sdg1, sdg2}. Typically, most DG techniques tackle the domain shift problem in a closed-set setting (homogeneous DG) where the label set is shared across all the domains. Alternatively, some methods deal with the more challenging heterogeneous DG \cite{hetdg1, hetdg2} that uses different labels between the source and target domains.

\begin{figure}[t]
    \centering
    \includegraphics[scale=0.58]{images/domain_teaser.pdf}

    \caption{A depiction of prompt tuning of a foundation model on source domains and evaluated on the novel target domain.}

    \label{fig:teaser2}
\end{figure}
Earlier studies on DG proposed learning domain-invariant representations by considering data from multiple source domains and introducing domain alignment losses~\cite{li2020domain, wang2021respecting,jia2020single, MMD}.
Others explored self-supervised learning~\cite{jigen}, ensemble learning~\cite{xu2014exploiting}, domain-specific networks~\cite{mancini2018best} and meta-learning ~\cite{learningtolearn}.
However, the limited number of training domains available in these approaches may negatively affect DG performance.
Therefore, domain augmentation approaches \cite{sfa, l2a-ot, style-neophile, mixstyle} were introduced to generate novel pseudo-domains.
In this regard, single-source DG models \cite{sdg1, sdg2, sdg3} proposed synthesising novel diversified styles by perturbing the source domain using entropy maximization, meta-learning, and adversarial learning. Differently, approaches for heterogeneous DG \cite{hetdg3,hetdg4, l2a-ot} were derived to improve the generalizability of the trained model for novel tasks.

More recently, \cite{amortized} analyzed the performance of CLIP \cite{clip} on data drawn from multiple domains and proposed a domain prompt generator where domain information is inferred by averaging the batch-wise visual features obtained from CLIP's vision encoder.
In addition to using the domain information, the method in \cite{amortized} has a few issues: i) it proposes to capture the domain information from the final layer of the vision encoder, which is known to encode more semantic information than domain-specific artifacts, ii) it typically performs well with the large batch size and may overfit for small batches since it is not feasible to obtain an unbiased estimate of style from a small number of feature vectors.
A couple of recent works \cite{niu2022domain, li2022learning} proposed to learn domain invariant prompts either by exploiting the text-based source domain knowledge or by leveraging the image patches to learn the prompt input to the ViT models, similar to the idea of visual prompt tuning (VPT) \cite{vpt}.
\textit{We would like to emphasize that our \textsc{StyLIP} is different from \cite{niu2022domain, amortized, li2022learning} in two aspects: the consideration of style features at different visual encoder levels to learn the individual prompt tokens, and the judicious exploration of multi-scale visual content features in prompt learning, since both the ideas turning out to be fruitful to solve the different DG tasks. Experimentally, \textsc{StyLIP} shows more prominence in five different DG settings, while the evaluations of \cite{niu2022domain, amortized} are limited. Also, our prompting is aimed towards dual-stream foundation models, as opposed to \cite{vpt, li2022learning}, which focus on learning the prompt inputs to the ViT}.

Another line of recent research \cite{cumix} introduced the problem of visual recognition for unknown domains and classes, which combines DG with the notion of zero-shot learning \cite{zs_good_bad}. Subsequent works proposed the use of multimodal information \cite{chandhok2021structured} or disentangled feature learning \cite{mangla2021context} for this purpose. \textit{Our proposed experimental setup for a base to novel class generalization is similar to this setting; however, we are interested in analyzing the performance of the prompt learning techniques for foundation models in this respect, contrary to the more ad-hoc models mentioned above}.

\paragraph{Prompt tuning for vision-language models}
The foundation models trained to learn semantically rich visual representations by exploiting text information have become mainstream in
language processing \cite{lee2018pre,radford2019language} and computer vision \cite{bommasani2021opportunities,bossard2014food,helber2019eurosat}. The key to the success of these models is the design of task-centric textual descriptions for visual data \cite{henaff2020data,huynh2020fine}.
While earlier prompting strategies were primarily manual, later works relied on prompt learning.
For instance, CoOp \cite{coop} proposed to optimize unified and class-specific prompts in the continuous space through back-propagation.
To partially tackle the CoOp's lack of generalization, CoCoOp \cite{cocoop} suggested input-conditioned prompt learning. While \cite{coop, cocoop} learned projectors for textual input, CLIP-adapter \cite{clip-adapter} proposed to fine-tune feature adapters in both visual and language branches.
ProGrad \cite{prograd} followed a similar approach to CoCoOp, ensuring that the network does not forget the knowledge learned from the foundation model. In \cite{tpt}, \textit{test time prompt tuning} (TPT) was proposed where consistency among multiple views of the same image is used as a supervision signal for prediction.
Probabilistic and variational models have also been used to learn prompt distributions to match the spreads of the visual features \cite{prod, varprompt}. Finally, MaPle \cite{maple} proposed to enhance the compatibility between both the encoders of CLIP at different levels. \textit{Differently, in this paper, we introduce the notion of domain-aware prompting by judiciously leveraging the visual information from CLIP to address the problem of domain shift}.

\begin{figure*}
    \centering
    \includegraphics[width=\linewidth]{images/method_stylip.pdf}

    \caption{\textsc{StyLIP} learns to generate a discriminative joint embedding space for the image and prompt embeddings. The prompts are conditioned both on the image style and the content information, which are extracted from the vision encoder $f_v$ and adapted for the target task through the projectors $\{\mathcal{P}_m\}_{m=1}^M, \{\mathcal{B}_l\}_{l=1}^L$ and $\mathcal{P}_C$, respectively. In particular, given the image $x$, we consider the style features corresponding to layer $m$ of $f_v$ to learn the $m^{th}$ token of the prompt, $c_m(x)$, through the style projector $P_m$. Similarly, we combine the multi-scale content features for $x$ by aggregating the feature responses from multiple layers of $f_v$ through $\mathcal{P}_C$. Before aggregation, the bottleneck networks $\{\mathcal{B}_l\}$ are used to reduce the dimensionality of the features from the different layers.
    Finally, a fusion projector $\mathcal{P}_A$ is introduced to compute the classification weights. We also list the existing relevant prompt tuning techniques (CoOp \cite{coop}, CoCoOp \cite{cocoop} and, DPL \cite{amortized}) to highlight how \textsc{StyLIP} is different. $v_m$ is the $m^{th}$ prompt learner from a random vector, and $X_b^d$ defines a batch of samples for domain $d$. Typically, the image information is added with the learned prompt in DPL and CoCoOp. Best viewed in colour. }

    \label{fig:architecture}
\end{figure*}

\section{Proposed Methodology}
\label{sec:Proposed_Methodology}
\subsection{Problem and notation}
The DG problem involves $\mathcal{N}$ labelled source domains $\mathcal{S}^i = \{x_i^k, y_i^k\}_{k=1}^{n_i} \approx P^{\mathcal{S}^i}_{data}$, $1 \leq i \leq \mathcal{N}$, where $x_i \in \mathcal{X}^i$, $y_i \in \mathcal{Y}$, and $P^{\mathcal{S}^i}_{data}$ denote the input data, label, and the joint distribution concerning the data and the label space, respectively. Furthermore, $P^{\mathcal{S}^i}_{data} \neq P^{\mathcal{S}^j}_{data}$ $\forall i,j \in \{1, 2, \cdots, \mathcal{N}\}$, indicating that the source domains are mutually distinct. We call the setting single-source DG if $\mathcal{N}=1$, else it is known as multi-source DG.
The goal is to train a model $f:\mathcal{X} \rightarrow \mathcal{Y}$ given $\mathcal{S} = \{\mathcal{S}^i\}_{i=1}^{\mathcal{N}}$, which is expected to generalize for a novel target domain $\mathcal{S}^{\mathcal{N}+1} = \{x_t^k, y_t^k\}_{k=1}^{n_t}$ unseen during training with $x_t \in \mathcal{X}^t$ and $y_t \in \mathcal{Y}^t$ and $P^t_{data}$ denotes the target distribution which is different from the source distributions. Typically, we consider a closed-set setting where $\mathcal{Y} \cup \mathcal{Y}^t = \mathcal{Y} \cap \mathcal{Y}^t$. Also, for the base to new class generalization setting, we consider $\mathcal{Y} \cap \mathcal{Y}^t = \emptyset$.

\subsection{The \textsc{StyLIP} model}
\label{sec:details}
In this section, we introduce \textsc{StyLIP}, a novel approach for DG based on CLIP \cite{clip}.
\textsc{StyLIP} leverages CLIP's frozen vision encoder ($f_v$) and text encoder ($f_t$), trained on a large volume of image-text pairs (see Fig. \ref{fig:architecture}). $f_v$ that transforms an input image into a feature embedding vector can be implemented with different architectures: in our experiments (see Section \ref{sec:results}), we consider ResNet-50 (RN50) \cite{resnet}, and ViT-B/16 \cite{vit}. $f_t$ is built upon a Transformer \cite{transformer}: it is provided with an input of a sequence of word tokens and converts them into a vectorized representation.

As stated, \textsc{StyLIP} seeks to utilize the multi-scale visual features extracted from different levels of $f_v$ to estimate the style and content primitives and further channel them in learning a generic prompt space regarding a concept. Typically, high-level representations of the deepest layer of a vision encoder tend to capture the abstract object semantics suitable for classification but suffer from a lack of description of local patterns like oriented edges or local shapes \cite{zheng2016good}. Therefore, the set of characteristics obtained from multiple levels is deemed more transferable between tasks than the high-level features alone. Similarly, the instance-wise feature statistics calculated from multiple layers of the encoder capture different levels of style, e.g., the texture in the top layers usually has larger granularity than those in the bottom layers \cite{babenko2014neural}.

To model a continuous prompt embedding space using these multilevel visual features, \textsc{StyLIP} (see Fig. \ref{fig:architecture}) adopts a set of projector networks on top of $f_v$ and $f_t$: a set of $M$ style projectors $\{\mathcal{P}_m\}_{m=1}^M$ to encode domain characteristics into $M$ prefix tokens $\{c_m\}_{m=1}^M$, a content projector $\mathcal{P}_C$ to encode feature responses from all the $L$ encoder layers of $f_v$ after reducing their dimensions using bottleneck layers $\{\mathcal{B}_l\}_{l=1}^L$,
and a fusion projector $\mathcal{P}_A$.
We discuss the structure of the proposed projectors in detail below.

\paragraph{Embedding style into prompts.} For calculating the style features, let us consider the vector $\mathcal{F}_l(x) = [\vec{\mu_l}(x); \vec{\sigma_l}(x)]$ denoting the channel-wise mean and standard deviation of the feature map outputs from the $l^{th}$ layer ( $1 \leq l \leq L$) of $f_v$, also indicated as $f_v^l(x)$. Here,
$[-;-]$ denotes the concatenation operation. Specifically, if $f_v^l(x)$ is of dimensions $W \times H \times C$ (height, width, and depth dimensions), the statistics corresponding to the $c^{th}$ feature map $f_v^{l_c}, $ ($\mu_l^c, \sigma_l^c$), are calculated as:
\begin{equation}
\label{mu_sigma1}
\begin{aligned}
    \centering
    \mu_l^c &= \frac{1}{WH} \underset{w,h=1}{\overset{W,H}{\sum}} f_v^{l_c}(x)_{w,h}
\end{aligned}
\end{equation}

\begin{equation}
\label{mu_sigma2}
\begin{aligned}
    \centering
    \sigma_l^c &= \sqrt{\underset{w,h=1}{\overset{W,H}{\sum}} (f_v^{l_c}(x)_{w,h} - \mu_l^c)^2}
\end{aligned}
\end{equation}

In the simplest case, when the context length $M$ equals the number of encoder layers $L$, we seek to learn the $m^{th}$ context vector $c_m$ from $\mathcal{F}_m(x)$.
Considering that the dimensions of $\mathcal{F}_m(x)$s are inconsistent and
to appropriately input $\mathcal{F}_m$ into the text encoder $f_t$, we deploy the style projectors $\{\mathcal{P}_m\}_{m=1}^M$ and compute $c_m(x) = \mathcal{P}_m(\mathcal{F}_m(x))$, \textit{i.e.} the $m^{th}$ context vector for the text prompt. We define
\begin{equation} \centering
t_y = [c_1(x)] [c_2(x)] \cdots [c_M(x)] [CLS_y]
\label{eq:prompt}
\end{equation}
as the prompt for $(x, y)$ where $[CLS_y]$ is the word embedding of label $y$. Finally, $f_t$ generates the embedding $f_t(t_y)$.

However, the context length in prompting is a hyper-parameter, meaning an $M$ different from $L$ may be preferred for a given task. To incorporate this flexibility in our prompt learning, we consider aggregation or replication of representations from $\{\mathcal{F}_l(x)\}_{l=1}^L$ depending on whether $M < L$ or $M > L$, respectively. (see Section \ref{sec:Experimenatl_protocol}).

\vspace{1mm}

 \paragraph{Image-conditioned prompt embeddings}The paradigm of $t_y$ considers the information of the visual style of the images, but a static class embedding $CLS_y$ for all images with the label $y$ may limit its versatility. To further generalize the prompt embeddings, we propose to supplement $t_y$ with content image information. As discussed, we extract multiple complementary visual characteristics associated with an image by aggregating the multilevel feature responses obtained from the $L$ blocks of $f_v$.

One naive way to combine this multilevel information is by flattening the feature maps of individual blocks, followed by concatenation. However, this leads to a very high-dimensional vector representation compared to the dimensionality of $t_y$, undermining the effects of $t_y$ in the final classification weights. As a result, the contrastive task may lead to triviality. We propose reducing the feature maps' dimensions before concatenation as a remedy. This also shrinks the size of the inputs to $\mathcal{P}_C$, thus controlling its number of learnable parameters and the amount of information exchanged by the two encoders.

 Precisely, given the $l^{th}$-layer feature maps $f^l_v(x)$, we perform $1 \times 1$ convolution followed by flattening using $\mathcal{B}_l$ to reduce the channel depth of $f^l_v(x)$ from original $C$ to $\hat{C} << C$, resulting in $\mathcal{B}_l(f_v^l(x)) \in \mathbb{R}^{WH\hat{C} \times 1}$.
 Finally, we concatenate the $\mathcal{B}(f_v^l(x))$s to obtain $\hat{f}_v(x)$:

 \begin{equation}
 \centering
 \hat{f}_v(x) = [\mathcal{B}_1(f_v^1(x));\mathcal{B}_2(f_v^2(x));\cdots; \mathcal{B}_L(f_v^L(x))]
 \end{equation}

 The content projector $\mathcal{P}_C$ learns the combined image embedding $\mathcal{P}_C(\hat{f}_v(x))$ through a linear transformation.
\vspace{1.5mm}

To generate the classification weights for a given $(x, y)$, we first concatenate $\mathcal{P}_C(\hat{f}_v(x))$ with $f_t(t_y)$ and transform the aggregated information through the fusion projector $\mathcal{P}_A$ to obtain $\hat{t}(x, y)$ as follows:

\begin{equation}
    \centering
    \hat{t}(x, y) = P_A([\mathcal{P}_C(\hat{f}_v(x));f_t(t_y)])
\end{equation}

\subsection{Training and inference}

 The projectors are trained using a contrastive loss $\mathcal{L}_{con}$ between $\hat{t}(x, y)$ and the image features obtained from the final embedding layer of $f_v$, i.e., $f_v(x)$, as follows:

 \begin{equation}
      \centering
     \mathcal{L}_{con} = \underset{\substack{\{\mathcal{P}_m\}_{m=1}^M, \\\{\mathcal{B}_{l}\}_{l=1}^L, \mathcal{P}_C, \mathcal{P}_A}}{\arg \min} \underset{{(x, y) \sim P^{\mathcal{S}}_{data}}}{\mathbb{E}}\ - \log (p(\hat{t}(x, y)\|x))
 \end{equation}
\noindent where $P^{\mathcal{S}}_{data}$ is the joint data distribution of $\mathcal{S}$ and
\begin{equation}
     \centering
    p(\hat{t}(x^k, y^k)\|x^k) = \frac{e^{\delta(\hat{t}(x^k, y^k), f_v(x^k))/\tau}}{\underset{n \in \mathcal{Y}}{\sum}e^{\delta(\hat{t}(x^k,n), f_v(x^k))/ \tau}}
\end{equation}
$\delta$ defines the cosine similarity and $\tau$ is the temperature hyperparameter.
The contrastive loss synergistically maximizes the similarity between the image and the correct class prompt embeddings while minimizing the similarity between the image and all the opposing classes.

During inference, we calculate the compatibility between $f_v(x_t)$ and the prompt embeddings for all classes in $\mathcal{Y}^t$. The class with the highest compatibility is selected as:

\begin{equation}

    \hat{y}_t = \underset{n \in \mathcal{Y}^t}{\arg \max} \: p(\hat{t}(x_t,n)\|x_t)

\end{equation}

\section{Experimental Results}
\label{sec:results}

\subsection{Experimental Protocol}
\label{sec:Experimenatl_protocol}
\paragraph{Datasets}We evaluate \textsc{StyLIP} over five benchmark datasets for multi-source and single-source DG, which are described as follows: (1) \textbf{Office-Home} \cite{officehome} - It consists of 15,500 images coming from 65 classes covering four domains, namely, Art, Clipart, Product, and Real. (2) \textbf{PACS} \cite{dbadg} - Includes 9991 images consisting of seven classes that are spread across four domains, Artpaint, Cartoon, Sketch, and Photo.
(3) \textbf{VLCS} \cite{vlcs} - It was prepared by combining images from four image classification datasets, i.e., PASCAL VOC 2007 \cite{pascalvoc}, Caltech \cite{caltech}, LabelMe \cite{labelme}, and Sun \cite{sundataset}. It consists of images from five classes, Bird, Car, Chair, Dog, and Person. (4) \textbf{Digits-DG} \cite{l2a-ot} - This dataset is designed in the combination of handwritten digit recognition datasets, namely, MNIST \cite{mnist}, MNIST-M \cite{mnist-m}, SVHN \cite{svhn}, and SYN \cite{mnist-m}. (5) \textbf{DomainNet} \cite{domain_net} - It consists of images from six distinct domains, including real, painting, clipart, quickdraw, infograph, and sketch. Each domain has 48K - 172K images (600K in total) categorized into 345 classes.

We further analyse the performance of \textsc{StyLIP} for cross dataset generalization, where \textsc{StyLIP} is trained on ImageNet \cite{krizhevsky2012imagenet} and tested on 10 other different datasets, including Caltech101 \cite{caltech}, OxfordPets \cite{oxford}, StanfordCars \cite{stanford}, Flowers102 \cite{flower}, Food101 \cite{food}, FGVCAircraft \cite{aircraft}, SUN397 \cite{sundataset}, DTD \cite{dtd}, EuroSAT \cite{helber2019eurosat} and UCF101 \cite{ucf}.

\paragraph{Model implementation, training, and evaluation protocols}We implement the projectors in $(\mathcal{P}_C, \mathcal{P}_A, \{\mathcal{P}_m\}_{m=1}^M)$ as single dense layers. We train the model with Adam optimizer \cite{kingma2014adam} with a learning rate of $2e-2$ and betas $(0.9, 0.999)$.
 We consider a context length of four for all the experiments following \cite{coop, cocoop}. For RN50, we consider the feature map outputs from the four convolution stages to extract the style and content features, hence $L=M=4$. For ViT-B/16, we obtain the embedding outputs from the $L=12$ encoder layers. We further average the features for every three consecutive layers of $f_v$ in a bottom-up manner without overlap to generate four intermediate feature representations, which are subsequently used to produce four distinct domain information vectors to be passed to $\{\mathcal{P}_m\}_{m=1}^4$.
 We fix the number of output channels of the bottleneck $\hat{C}=3$ using cross-validation, where $10 \%$ images from each source domain are treated as the validation set. We further ablate $\hat{C}$ in Section \ref{sec:ablation} to check various other architecture choices.
Finally, we consider a mini-batch size of $4$ for DomainNet and Office-Home, while it is $8$ for the other datasets, and we train the model for $10$ epochs. We report the average top-1 classification performance on $\mathcal{S}^{\mathcal{N}+1}$ over three different executions of \textsc{StyLIP}.

\paragraph{Baselines}We consider three types of methods for comparison to check the generalizability of pre-trained CLIP features and that of the prompting strategies.
Our baseline is \textbf{zero-shot CLIP} with the prompt as `\texttt{A Photo of a [CLS]}'. We also include \textbf{domain name} in the prompt as `\texttt{A [Domain] of a [\textit{CLS}]}'.

We use CLIP features to train a linear classifier, which we term \textbf{Linear Probing}. Furthermore, we deploy these features in conjunction with three traditional models for learning in the presence of domain shift: \textbf{CORAL} \cite{coral}, \textbf{DANN} \cite{dann}, and \textbf{CROSSGRAD} \cite{cross-grad}. From the traditional DG literature, we report the performance of \textbf{SWAD} \cite{swad}.

Furthermore, we choose to compare \textsc{StyLIP} with existing prompt learning techniques including \textbf{CoOp} \cite{coop}, \textbf{CoCoOp} \cite{cocoop}, \textbf{CLIP-Adapter} \cite{clip-adapter}, \textbf{DPL} \cite{amortized}, \textbf{ProGrad} \cite{prograd}, \textbf{VPT} \cite{vpt}, \textbf{CSVPT} \cite{li2022learning} and, \textbf{TPT} \cite{tpt}, respectively.

Finally, we evaluate three variants of \textsc{StyLIP} intending to ablate the individual components of our approach: i)  $\{\mathcal{P}_m\}_{m=1}^M$ are trained from random vectors (similar to CoOp), but we consider the multi-scale content feature learning of \textsc{StyLIP} through $(\{\mathcal{B}_l\}_{l=1}^L, \mathcal{P}_A, \mathcal{P}_C)$, respectively. (\textbf{\textsc{StyLIP}-con}). This establishes the importance of including the visual style information in the prompt tokens over learning the tokens from randomized inputs. ii) the model without content features and $(\{\mathcal{B}_l\}_{l=1}^L, \mathcal{P}_C)$ but with $\{\mathcal{P}_m\}_{m=1}^M$ (\textbf{\textsc{StyLIP}-sty}). This is to verify the fact whether the inclusion of the content features add any value in terms of better generalization, and iii) the version of \textsc{StyLIP} where the features of the deepest layer of $f_v$ are used for the content branch (\textbf{\textsc{StyLIP}*}). This is to assess the importance of the multi-scale content features over the semantic high-level visual content properties.

\begin{table*}[!ht]

    \scriptsize{
    \begin{center}

    \caption{Comparison of our proposed \textsc{StyLIP} with the state-of-the-art methods on PACS, VLCS, Digits-DG, Office-Home, and DomainNet datasets for multi-source DG in terms of mean leave-one-out performance. $\dagger$ uses a different backbone than CLIP. (In $\%$) $^\#$\tablefootnote{Methods are trained for a large number of epochs such as VPT \cite {vpt} trains for $100$ epochs, whereas our proposed \textsc{StyLIP} is trained only with $10$ epochs.}\hl{Methods perform fine-tuning the visual backbones.}}
    \scalebox{0.73}{
    \begin{tabular}{llccccc}
    \toprule

    \multicolumn{1}{c}{\textbf{Backbone}} &\multicolumn{1}{l}{\textbf{Method}} & \multicolumn{1}{c}{\textbf{PACS}}& \multicolumn{1}{c}{\textbf{VLCS}}& \multicolumn{1}{c}{\textbf{Off.Home}}&\multicolumn{1}{c}{\textbf{Dig.DG}}&\multicolumn{1}{c}{\textbf{Dom.N.}}\\
    \midrule

         \multirow{17}{*}{\rotatebox{90}{CLIP RN50}} & SWAD$^{\dagger}$ \cite{swad} &88.10 &  79.10& 70.60&-&46.50  \\

        \cmidrule(lr){2-7}
         &  Lin. Probing & 91.65& 79.48 & 70.17 &62.22&46.10\\
        ~ & CORAL \cite{coral} &92.17 &  81.40& 70.65&61.04&41.31\\
        ~ & DANN \cite{dann}& 92.43 &  81.67& 70.78&62.39&41.97\\
        ~ & CROSSGRAD \cite{cross-grad} & 91.56 & 79.63&    70.47&62.98&45.64\\
        &ZS-CLIP \cite{clip}& 90.32 & 76.43& 66.75&56.41&44.18\\
       & ZS-CLIP + DN&  91.86 & 77.70&67.93&58.30&47.50\\

        ~ & CoOp \cite{coop}& 92.28 & 81.87& 71.65&73.11& 49.71\\
        ~ & CoCoOp \cite{cocoop} & 91.64& 82.30 & 71.93&74.58& 50.16\\
        ~ & CLIP-Adapt. \cite{clip-adapter}&  92.08 & 82.35 & 72.18&73.79&50.25 \\
        ~ & DPL \cite{amortized} & 91.96 & 82.12 & 72.54&74.33&50.38\\

        ~ & ProGrad \cite{prograd} & 92.01 & 82.23& 71.85&74.45&50.27\\
        ~ & TPT \cite{tpt} & 92.16 & 82.39 &72.07&74.68&50.30\\

         &\cellcolor{LightCyan}\textsc{StyLIP}-con&	\cellcolor{LightCyan}92.35&	\cellcolor{LightCyan}84.07&	\cellcolor{LightCyan}73.89&\cellcolor{LightCyan}75.90&\cellcolor{LightCyan}51.43\\
        ~ & \cellcolor{LightCyan}\textsc{StyLIP}-sty  & \cellcolor{LightCyan}92.96  &\cellcolor{LightCyan}84.39& \cellcolor{LightCyan}74.22&\cellcolor{LightCyan}76.21&\cellcolor{LightCyan}51.80\\

        &	\cellcolor{LightCyan}\textsc{StyLIP}*&  \cellcolor{LightCyan}92.47&	\cellcolor{LightCyan}83.60&	\cellcolor{LightCyan}73.56&	\cellcolor{LightCyan}75.81&\cellcolor{LightCyan}51.63\\

        ~ & \cellcolor{LightCyan}\textsc{StyLIP} & \cellcolor{LightCyan}\textbf{93.59} & \cellcolor{LightCyan}\textbf{84.83} & \cellcolor{LightCyan}\textbf{74.80} & \cellcolor{LightCyan}\textbf{76.49}& \cellcolor{LightCyan}\textbf{52.29} \\ \midrule
        \multirow{14}{*}{\rotatebox{90}{CLIP ViT-B/16}} & Lin. Probing  & 96.54  & 82.63& 80.43 &70.15&57.46\\
        ~ & CROSSGRAD \cite{cross-grad} &96.40  &83.76  & 80.55 &70.83&57.60\\
        & ZS-CLIP \cite{clip}&95.81 & 80.57& 78.57&65.79&54.08\\
        & ZS-CLIP + DN&96.30  &81.94 &79.10 &67.41&56.95\\

        ~ & CoOp \cite{coop}& 97.00 & 82.98&	81.12 &76.41&59.52\\
        ~ & CoCoOp \cite{cocoop}& 96.73 & 83.59&	80.70&78.49&59.68\\
        ~ & CLIP-Adapt. \cite{clip-adapter}&96.41 & 84.32&82.23&77.86&59.90\\
        ~ & DPL \cite{amortized} & 97.07 & 83.99& 83.00 &77.32&59.86\\
        ~ & ProGrad \cite{prograd}&96.50 &83.82& 82.46 &78.26&59.65\\
        ~ & TPT \cite{tpt} & 96.99 &83.72&82.45&78.51&59.87 \\
        ~ & MIRO \cite{miro} &95.80 &83.60 &82.30 & - &57.20\\
        ~ & \hl{VPT}$^\#$\cite{vpt} &97.20 &84.90 &85.20 & - &59.80 \\
        ~ &\hl{CSVPT}$^\#$\cite{csvpt} &97.30 &84.90 &85.00 & - &60.00 \\

         &\cellcolor{LightCyan}\textsc{StyLIP}-con & \cellcolor{LightCyan}96.82&	\cellcolor{LightCyan}85.61& 	\cellcolor{LightCyan}83.90&\cellcolor{LightCyan}80.63&\cellcolor{LightCyan}61.51\\
         & \cellcolor{LightCyan}\textsc{StyLIP}-sty & \cellcolor{LightCyan}97.25  & \cellcolor{LightCyan}86.27&  \cellcolor{LightCyan}84.18 &\cellcolor{LightCyan}80.91&\cellcolor{LightCyan}61.77\\

         &	\cellcolor{LightCyan}\textsc{StyLIP}*&  \cellcolor{LightCyan}97.11&	\cellcolor{LightCyan}85.88&	\cellcolor{LightCyan}83.41&	\cellcolor{LightCyan}80.56&\cellcolor{LightCyan}61.39\\

         & \cellcolor{LightCyan}\textsc{StyLIP}&\cellcolor{LightCyan}\textbf{98.05}&\cellcolor{LightCyan}\textbf{86.94}  &\cellcolor{LightCyan}\textbf{84.63} &\cellcolor{LightCyan}\textbf{81.38}&\cellcolor{LightCyan}\textbf{62.02}\\
         \bottomrule
    \end{tabular}}
    \label{tab_PACS}
    \end{center}}

\end{table*}

\begin{table}[!ht]
    \centering
    \scriptsize{
    \caption{Comparison of \textsc{StyLIP} with CLIP-based state-of-the-art methods for single-source DG on PACS, VLCS, and Office-Home datasets in terms of mean leave-all-but-one-domain-out performance. (In $\%$) }
    \scalebox{0.68}{
    \begin{tabular}{clccc}
    \toprule
   \multirow{2}[1]{*}{\textbf{Backbone}}&\multirow{2}[1]{*}{\textbf{Method}}&\multicolumn{1}{c}{\textbf{PACS}}&\multicolumn{1}{c}{\textbf{VLCS}}&\multicolumn{1}{c}{\textbf{Office Home}}\\
          \cmidrule(lr){3-3}\cmidrule(lr){4-4}\cmidrule(lr){5-5}

       \midrule

       \multirow{9}{*}{CLIP RN50}  &Lin. Probing & 85.67 &69.42&65.99\\
          &CROSSGRAD \cite{cross-grad}&  86.19&71.35&66.47\\
         &CoOp \cite{coop}&  89.88&74.04&69.04\\
        &CoCoOp \cite{cocoop}  & 88.69&74.80&69.48\\
        &CLIP-Adapter \cite{clip-adapter} & 88.86&75.31&69.29\\
         &DPL \cite{amortized} &89.24&74.86&69.10\\
         &Prograd \cite{prograd}& 88.51&75.40&69.49\\
         &TPT \cite{tpt}& 88.93 & 75.02 & 69.58 \\

        &\cellcolor{LightCyan}\textsc{StyLIP} &\cellcolor{LightCyan}\textbf{92.61} & \cellcolor{LightCyan}\textbf{77.18}&\cellcolor{LightCyan}\textbf{71.60}\\
         \midrule

           \multirow{9}{*}{CLIP ViT-B/16}  &Lin. Probing & 89.85 &76.15&77.71\\
          &CROSSGRAD \cite{cross-grad}&  90.68&76.89&78.30\\
         &CoOp \cite{coop}&  95.59&80.10&80.44\\
        &CoCoOp \cite{cocoop}  & 94.92&80.44&81.19\\
        &CLIP-Adapter \cite{clip-adapter} & 94.60&80.27&80.86\\
         &DPL \cite{amortized} &94.70&80.58&80.79\\
         &Prograd \cite{prograd}& 94.82&80.38&81.37\\
         &TPT \cite{tpt}&95.14 & 80.57 &  81.43\\

        &\cellcolor{LightCyan}\textsc{StyLIP} &\cellcolor{LightCyan}\textbf{97.03} & \cellcolor{LightCyan} \textbf{82.90}&\cellcolor{LightCyan}\textbf{83.89}\\
        \bottomrule
    \end{tabular}}

    \label{tab:ssdg}
    }
\end{table}

\begin{table*}[!ht]

\centering
\scriptsize{
    \centering
    \caption{Analysis of the generalization from base to new classes across domains. We show results on Office-Home and DomainNet with \textit{ClipArt} acting as the source domain, while others denote the target. The model is trained (backbone CLIP ViT-B/16) using $16$ shots from the base classes. (In $\%$)}
    \scalebox{0.64}{
    \begin{tabular}{lcccccccccccc}
    \toprule
     \multirow{4}{*}{\textbf{Method}}&\multicolumn{5}{c}{\textbf{Office-Home}}&\multicolumn{7}{c}{\textbf{DomainNet}}\\\cmidrule(lr){2-6}\cmidrule(lr){7-13}
    &\multicolumn{1}{c}{\textbf{Base}}&\multicolumn{4}{c}{\textbf{New}}&\multicolumn{1}{c}{\textbf{Base}}&\multicolumn{6}{c}{\textbf{New}}\\\cmidrule(lr){2-2}\cmidrule(lr){3-6}\cmidrule(lr){7-7}\cmidrule(lr){8-13}
        ~ & \textit{\textbf{Clip Art}} & \textbf{Art} & \textbf{Clip Art} & \textbf{Product} & \textbf{Real World} &\textit{\textbf{Clip Art}}&\textbf{Clip Art}&\textbf{Infograph}&\textbf{Painting}&\textbf{Quick Draw}&\textbf{Real}&\textbf{Sketch}\\

        \midrule
        CLIP \cite{clip}  & 78.12 &62.01 & 77.78 & 87.52 & 88.02 &78.00&76.55&49.80&70.84&17.56&\textbf{88.11}&66.54\\
        CoOp \cite{coop}& 82.60 &70.60 & 82.23 & 90.44 & 87.21 &82.79&75.60&48.60&71.38&20.90&85.19&67.39\\
        CoCoOp \cite{cocoop}& 82.64 &71.00 &83.61 & 92.12 & 89.19 &82.85&77.40&52.61&72.06&20.80&88.00&68.12\\
         CLIP-Adapter \cite{clip-adapter}& 80.00 &73.19 & 83.00 & 92.11 & 89.53 &80.51&76.33&51.70&71.81&20.15&87.30&67.60\\
         DPL \cite{amortized}&  82.20 &71.54 & 82.80 &92.37 & 89.15 &82.35&76.49&52.10&71.88&20.30&87.54&67.73\\
         ProGrad \cite{prograd}& 82.41 &72.00 &83.29 &92.11&89.58&83.00&77.50  &51.44  &72.16  & 20.86 &87.11&67.05\\
        \cellcolor{LightCyan}\textsc{StyLIP}*& \cellcolor{LightCyan}83.90&\cellcolor{LightCyan}73.48 &\cellcolor{LightCyan}85.07 &\cellcolor{LightCyan}92.60&\cellcolor{LightCyan}90.77&\cellcolor{LightCyan}84.19&\cellcolor{LightCyan}77.62  &\cellcolor{LightCyan}52.80  &\cellcolor{LightCyan}73.00  &\cellcolor{LightCyan} 21.10 &\cellcolor{LightCyan}87.54&\cellcolor{LightCyan}68.29\\

       \cellcolor{LightCyan}\textsc{StyLIP}&\cellcolor{LightCyan}\textbf{84.33} & \cellcolor{LightCyan}\textbf{74.60} & \cellcolor{LightCyan}\textbf{87.25}  & \cellcolor{LightCyan}\textbf{93.00} & \cellcolor{LightCyan}\textbf{91.42} &\cellcolor{LightCyan}\textbf{84.90}&\cellcolor{LightCyan}\textbf{78.14}&\cellcolor{LightCyan}\textbf{53.09}&\cellcolor{LightCyan}\textbf{73.60}&\cellcolor{LightCyan}\textbf{21.69}&\cellcolor{LightCyan}87.90&\cellcolor{LightCyan}\textbf{68.61}\\ \bottomrule
    \end{tabular}}\label{tab:b2n}}

\end{table*}

\label{sec:Results}
\subsection{Comparison with state-of-the-art}
We discuss the experimental comparisons of \textsc{StyLIP} with the literature in the following order of the DG tasks: i) multi-source DG, ii) single-source DG, iii) cross-domain base to novel class DG, iv) in-domain base to novel class DG and, v) cross-dataset DG respectively. We follow the leave-one-domain-out evaluation protocol for multi-source DG where all the domains except one are considered source domains while the model is to be verified on the held-out target domain. For single-source DG, we train the model on one domain and test it on the remaining domains (leave-all-but-one-domain-out). For a fair comparison with the literature, we use the benchmark train-test splits for the multi-source and single-source DG experiments, while the few-shot experiments with $16$-shots are used for the remaining DG settings. However, we have showcased the sensitivity of \textsc{StyLIP} with respect to the number of training samples in Fig. \ref{fig_context}.

\paragraph{Multi-source DG}We report the comparative results in terms of mean leave-out-out performance for PACS, VLCS, Office-Home, Digits-DG, and DomainNet in Table \ref{tab_PACS} for both the visual backbones (RN50 and ViT-B/16). As expected, better performances are obtained with ViT-B/16 to RN50 with an improvement of $\approx 10 \%$ on the larger and more complex datasets, \textit{i.e.} Office-Home and DomainNet.
We observe that the performance of zero-shot CLIP is consistently poorer than that of the other methods. For instance, zero-shot CLIP produces average target generalization accuracies of $78.57 \%$ and $54.08 \%$ on Office-Home and DomainNet when used with the ViT-B/16 backbone inferior to the performance of Linear Probing by at least $1.5 \%$. This demonstrates the inability of the naive CLIP-based without any prompt learning method to handle significant domain variations. The use of the learnable classifier in Linear Probing shows improvements but is still sub-optimal.
Domain alignment techniques like DANN and CORAL produce mostly comparable performance with Linear Probing, outperforming the baseline CLIP in most cases; however, Linear Probing largely outperforms (by more than $4 \%$) CORAL and DANN on DomainNet for the ResNet-50 backbone.
Unlike these techniques, models based on prompt learning, such as CoOp, CoCoOp, CLIP-Adapter, ProGrad, VPT and TPT, typically improve upon the other comparative techniques considerably, thanks to their more intelligent prompting approaches, which are aligned to the DG task to some extent. Finally, DPL and CSVPT, the prompt tuning model tailor-made for DG, produce equivalent results with the other prompt learning models ($59.8 \%$ on DomainNet using the ViT-B/16 backbone).

Finally, \textsc{StyLIP} outperforms all competitors, including DPL and CSVPT, for all datasets and vision backbones, producing state-of-the-art results. In particular, the performance measures of \textsc{StyLIP} on VLCS, Digits-DG, and DomanNet are $86.94 \%$, $81.38 \%$, and $62.03 \%$, respectively, using ViT-B/16, which are better than the others by at least $3 \%$.

The performance of \textsc{StyLIP}-con is poorer than the \textsc{StyLIP} by $\approx 0.5-1.3 \%$, whereas \textsc{StyLIP}-sty, is marginally better than \textsc{StyLIP}-con, but inferior to \textsc{StyLIP}. However, both \textsc{StyLIP}-sty and \textsc{StyLIP}-con perform comparably or better than the other prompting methods. The apparent problem with these variants of \textsc{StyLIP} is that they capture partial visual properties that lead to sub-optimal prompt learning, while \textsc{StyLIP} fully utilizes both styles and content information of the images, thus reducing the gap between visual and semantic spaces. Finally, \textsc{StyLIP} outperforms \textsc{StyLIP}*, thanks to the multi-scale content features, which are more generalizable than the deeper semantically oriented visual representations. We show the detail domain-wise results on the multi-source DG setup in the appendix.

\paragraph{Single-source DG}Single-source DG is more challenging than multi-source DG since data from only one domain is used during training to learn the generalizable features. At the same time, the model is evaluated on multiple, diverse targets. We consider three datasets, in this case, PACS, VLCS, and Office-Home, and report the average leave-all-but-one-domain-out in Table \ref{tab:ssdg} over all possible domain combinations. We observe that \textsc{StyLIP} is capable of convincingly outperforming the other prompting techniques for all data sets, approximately by $1.4-2.5 \%$ following multi-source DG experiments (Table \ref{tab_PACS}), thus producing the new state-of-the-art results for single-source DG. In particular, for Office-Home, where the domains are very different, and many classes are fine-grained, \textsc{StyLIP} shows impressive performance for all combinations of domains and beats the next best by $2.46 \%$. This further advocate that the prompts outputted by \textsc{StyLIP} can effectively adapt to the domain characteristics of novel domains and are very expressive of the visual contents. We show the detail domain-wise results on the single-source DG setup in the appendix.
\begin{table*}[!ht]

\scriptsize{
    \centering
    \caption{Comparison with state-of-the-art methods on base-to-new generalization. \textsc{StyLIP} shows better generalization performance over existing methods on 11 different recognition datasets on $16$-shots and a context length of four. HM represents the harmonic mean. (In $\%$)}
    \scalebox{0.86}{
    \begin{tabular}{lcccllcccllccc}

     \multicolumn{4}{c}{(a) \textbf{Average over 11 datasets}}&\multicolumn{4}{c}{(b) ImageNet}&\multicolumn{4}{c}{(c) Caltech101}\\\cmidrule(lr){1-4}\cmidrule(lr){5-8}\cmidrule(lr){9-12}

 &\multicolumn{1}{c}{Base}&\multicolumn{1}{c|}{New}&\multicolumn{1}{c}{HM}&&\multicolumn{1}{c}{Base}&\multicolumn{1}{c|}{New}&\multicolumn{1}{c}{HM}&&\multicolumn{1}{c}{Base}&\multicolumn{1}{c|}{New}&\multicolumn{1}{c}{HM}\\\cmidrule(lr){1-4}\cmidrule(lr){5-8}\cmidrule(lr){9-12}

  CLIP \cite{clip}&\multicolumn{1}{c}{ 69.34}&\multicolumn{1}{c|}{ 74.22}&\multicolumn{1}{c}{71.70 }&CLIP \cite{clip}&\multicolumn{1}{c}{ 72.43}&\multicolumn{1}{c|}{68.14 }&\multicolumn{1}{c}{70.22 }&CLIP \cite{clip}&\multicolumn{1}{c}{96.84 }&\multicolumn{1}{c|}{ 94.00}&\multicolumn{1}{c}{ 95.40}\\

    CoOp \cite{coop}&\multicolumn{1}{c}{ 82.69}&\multicolumn{1}{c|}{ 63.22}&\multicolumn{1}{c}{ 71.66}&CoOp \cite{coop}&\multicolumn{1}{c}{76.47 }&\multicolumn{1}{c|}{67.88 }&\multicolumn{1}{c}{71.92 }&CoOp \cite{coop}&\multicolumn{1}{c}{ 98.00}&\multicolumn{1}{c|}{89.81 }&\multicolumn{1}{c}{ 93.73}\\
CoCoOp \cite{cocoop}&\multicolumn{1}{c}{80.47 }&\multicolumn{1}{c|}{71.69 }&\multicolumn{1}{c}{ 75.83}&CoCoOp \cite{cocoop}&\multicolumn{1}{c}{ 75.98}&\multicolumn{1}{c|}{70.43 }&\multicolumn{1}{c}{ 73.10}&CoCoOp \cite{cocoop}&\multicolumn{1}{c}{ 97.76}&\multicolumn{1}{c|}{93.81 }&\multicolumn{1}{c}{95.84 }\\
MaPLe \cite{maple}&\multicolumn{1}{c}{ 82.28}&\multicolumn{1}{c|}{75.14 }&\multicolumn{1}{c}{ 78.55}&MaPLe \cite{maple}&\multicolumn{1}{c}{76.66 }&\multicolumn{1}{c|}{ 70.54}&\multicolumn{1}{c}{ 73.47}&MaPLe \cite{maple}&\multicolumn{1}{c}{ 97.74}&\multicolumn{1}{c|}{94.36 }&\multicolumn{1}{c}{96.02 }\\\cmidrule(lr){1-4}\cmidrule(lr){5-8}\cmidrule(lr){9-12}

\cellcolor{LightCyan}\textsc{StyLIP}&\multicolumn{1}{c}{\cellcolor{LightCyan}\textbf{83.22}}&\multicolumn{1}{c|}
{\cellcolor{LightCyan}{\textbf{75.94}}}&\multicolumn{1}{c}{\cellcolor{LightCyan}\textbf{79.41}}&
\cellcolor{LightCyan}\textsc{StyLIP}&\multicolumn{1}{c}{\cellcolor{LightCyan}\textbf{77.15}}&\multicolumn{1}{c|}{\cellcolor{LightCyan}\textbf{71.34}}&\multicolumn{1}{c}
{\cellcolor{LightCyan}\textbf{74.13}}&
\cellcolor{LightCyan}\textsc{StyLIP}&\multicolumn{1}{c}
{\cellcolor{LightCyan}\textbf{98.23} }&\multicolumn{1}{c|}{\cellcolor{LightCyan}\textbf{94.91}}&\multicolumn{1}{c}
{\cellcolor{LightCyan}\textbf{96.54}}\\
\cmidrule(lr){1-4}\cmidrule(lr){5-8}\cmidrule(lr){9-12}
&&&&&&&&&&&\\
 \multicolumn{4}{c}{(d) OxfordPets}&\multicolumn{4}{c}{(e) StanfordCars}&\multicolumn{4}{c}{(f) Flowers102}\\\cmidrule(lr){1-4}\cmidrule(lr){5-8}\cmidrule(lr){9-12}

 &\multicolumn{1}{c}{Base}&\multicolumn{1}{c|}{New}&
 \multicolumn{1}{c}{HM}&&\multicolumn{1}{c}{Base}&
 \multicolumn{1}{c|}{New}&\multicolumn{1}{c}{HM}&&
 \multicolumn{1}{c}{Base}&\multicolumn{1}{c|}{New}&
 \multicolumn{1}{c}{HM}\\\cmidrule(lr){1-4}\cmidrule(lr){5-8}\cmidrule(lr){9-12}

  CLIP \cite{clip}&\multicolumn{1}{c}{ 91.17}&
  \multicolumn{1}{c|}{97.26 }&\multicolumn{1}{c}{ 94.12}&
  CLIP \cite{clip}&\multicolumn{1}{c}{63.37 }&
  \multicolumn{1}{c|}{\textbf{74.89} }&\multicolumn{1}{c}{68.65 }&
  CLIP \cite{clip}&\multicolumn{1}{c}{72.08 }&\multicolumn{1}{c|}{\textbf{77.80}}&\multicolumn{1}{c}{74.83 }\\

    CoOp \cite{coop}&\multicolumn{1}{c}{93.67 }&
    \multicolumn{1}{c|}{ 95.29}&\multicolumn{1}{c}{ 94.47}&
    CoOp \cite{coop}&\multicolumn{1}{c}{ \textbf{78.12}}&
    \multicolumn{1}{c|}{60.40 }&\multicolumn{1}{c}{68.13}&
    CoOp \cite{coop}&\multicolumn{1}{c}{\textbf{97.60} }&
    \multicolumn{1}{c|}{ 59.67}&\multicolumn{1}{c}{74.06 }\\

CoCoOp \cite{cocoop}&\multicolumn{1}{c}{ 95.20}&
\multicolumn{1}{c|}{ 97.69}&\multicolumn{1}{c}{ 96.43}&
CoCoOp \cite{cocoop}&\multicolumn{1}{c}{70.49 }&
\multicolumn{1}{c|}{ 73.59}&\multicolumn{1}{c}{72.01 }&
CoCoOp \cite{cocoop}&\multicolumn{1}{c}{ 94.87}&
\multicolumn{1}{c|}{71.15 }&\multicolumn{1}{c}{81.71 }\\

MaPLe \cite{maple}&\multicolumn{1}{c}{95.43 }&
\multicolumn{1}{c|}{97.76 }&\multicolumn{1}{c}{ 96.58}&
MaPLe \cite{maple}&\multicolumn{1}{c}{72.94 }&
\multicolumn{1}{c|}{74.00 }&\multicolumn{1}{c}{73.47 }&
MaPLe \cite{maple}&\multicolumn{1}{c}{95.92 }&
\multicolumn{1}{c|}{72.46 }&\multicolumn{1}{c}{ 82.56}\\
\cmidrule(lr){1-4}\cmidrule(lr){5-8}\cmidrule(lr){9-12}

\cellcolor{LightCyan}\textsc{StyLIP}&\multicolumn{1}{c}{ \cellcolor{LightCyan}\textbf{95.96}}&\multicolumn{1}{c|}
{\cellcolor{LightCyan}\textbf{98.14} }&\multicolumn{1}{c}{ \cellcolor{LightCyan}\textbf{97.04}}&\cellcolor{LightCyan}
\textsc{StyLIP}&\multicolumn{1}{c}
{\cellcolor{LightCyan}75.19}&\multicolumn{1}{c|}
{ \cellcolor{LightCyan}74.46}&\multicolumn{1}{c}
{\cellcolor{LightCyan}\textbf{74.82} }&\cellcolor{LightCyan}
\textsc{StyLIP}&\multicolumn{1}{c}
{\cellcolor{LightCyan}96.54}&\multicolumn{1}{c|}
{\cellcolor{LightCyan}73.08}&\multicolumn{1}{c}
{\cellcolor{LightCyan}\textbf{83.19} }\\
\cmidrule(lr){1-4}\cmidrule(lr){5-8}\cmidrule(lr){9-12}
&&&&&&&&&&&\\
 \multicolumn{4}{c}{(g) Food101}&\multicolumn{4}{c}{(h) FGVCAircraft}&\multicolumn{4}{c}{(i) SUN397}\\\cmidrule(lr){1-4}\cmidrule(lr){5-8}\cmidrule(lr){9-12}

 &\multicolumn{1}{c}{Base}&\multicolumn{1}{c|}{New}
 &\multicolumn{1}{c}{HM}&&\multicolumn{1}{c}{Base}
 &\multicolumn{1}{c|}{New}&\multicolumn{1}{c}{HM}
 &&\multicolumn{1}{c}{Base}&\multicolumn{1}{c|}{New}
 &\multicolumn{1}{c}{HM}\\
 \cmidrule(lr){1-4}\cmidrule(lr){5-8}\cmidrule(lr){9-12}

  CLIP \cite{clip}&\multicolumn{1}{c}{90.10 }&
  \multicolumn{1}{c|}{ 91.22}&\multicolumn{1}{c}{90.66 }&
  CLIP \cite{clip}&\multicolumn{1}{c}{ 27.19}&
  \multicolumn{1}{c|}{\textbf{36.29}}&\multicolumn{1}{c}{31.09 }&
  CLIP \cite{clip}&\multicolumn{1}{c}{ 69.36}&
  \multicolumn{1}{c|}{ 75.35}&\multicolumn{1}{c}{ 72.23}\\

    CoOp \cite{coop}&\multicolumn{1}{c}{ 88.33}&
    \multicolumn{1}{c|}{ 82.26}&\multicolumn{1}{c}{ 85.19}&
    CoOp \cite{coop}&\multicolumn{1}{c}{ 40.44}&
    \multicolumn{1}{c|}{22.30 }&\multicolumn{1}{c}{28.75 }&
    CoOp \cite{coop}&\multicolumn{1}{c}{80.60 }&
    \multicolumn{1}{c|}{65.89 }&\multicolumn{1}{c}{ 72.51}\\

CoCoOp \cite{cocoop}&\multicolumn{1}{c}{90.70 }&
\multicolumn{1}{c|}{ 91.29}&\multicolumn{1}{c}{90.99}&
CoCoOp \cite{cocoop}&\multicolumn{1}{c}{ 33.41}&
\multicolumn{1}{c|}{ 23.71}&\multicolumn{1}{c}{27.74 }&
CoCoOp \cite{cocoop}&\multicolumn{1}{c}{79.74 }&
\multicolumn{1}{c|}{ 76.46}&\multicolumn{1}{c}{ 78.27}\\

MaPLe \cite{maple}&\multicolumn{1}{c}{90.71 }&
\multicolumn{1}{c|}{ 92.05}&\multicolumn{1}{c}{91.38 }&
MaPLe \cite{maple}&\multicolumn{1}{c}{37.44 }&
\multicolumn{1}{c|}{35.61 }&\multicolumn{1}{c}{ 36.50}&
MaPLe \cite{maple}&\multicolumn{1}{c}{ 80.82}&
\multicolumn{1}{c|}{78.70 }&\multicolumn{1}{c}{ 79.75}\\
\cmidrule(lr){1-4}\cmidrule(lr){5-8}\cmidrule(lr){9-12}

\cellcolor{LightCyan}\textsc{StyLIP}&
\multicolumn{1}{c}{\cellcolor{LightCyan}\textbf{91.20} }&
\multicolumn{1}{c|}{ \cellcolor{LightCyan}\textbf{92.48}}&
\multicolumn{1}{c}{\cellcolor{LightCyan}\textbf{91.84}}&
\cellcolor{LightCyan}\textsc{StyLIP}&
\multicolumn{1}{c}{\cellcolor{LightCyan}\textbf{37.65}}&
\multicolumn{1}{c|}{\cellcolor{LightCyan}35.93}&
\multicolumn{1}{c}{\cellcolor{LightCyan}\textbf{36.77}}&
\cellcolor{LightCyan}\textsc{StyLIP}&
\multicolumn{1}{c}{\cellcolor{LightCyan}\textbf{82.12}}&
\multicolumn{1}{c|}{\cellcolor{LightCyan}\textbf{79.95} }&
\multicolumn{1}{c}{\cellcolor{LightCyan}\textbf{81.02}}\\
\cmidrule(lr){1-4}\cmidrule(lr){5-8}\cmidrule(lr){9-12}
&&&&&&&&&&&&\\
 \multicolumn{4}{c}{(j) DTD}&\multicolumn{4}{c}{(k) EuroSAT}&\multicolumn{4}{c}{(l) UCF101}\\\cmidrule(lr){1-4}\cmidrule(lr){5-8}\cmidrule(lr){9-12}

 &\multicolumn{1}{c}{Base}&\multicolumn{1}{c|}{New}&
 \multicolumn{1}{c}{HM}&&\multicolumn{1}{c}{Base}&
 \multicolumn{1}{c|}{New}&\multicolumn{1}{c}{HM}&&
 \multicolumn{1}{c}{Base}&\multicolumn{1}{c|}{New}&
 \multicolumn{1}{c}{HM}\\
 \cmidrule(lr){1-4}\cmidrule(lr){5-8}\cmidrule(lr){9-12}

  CLIP \cite{clip}&\multicolumn{1}{c}{ 53.24}&
  \multicolumn{1}{c|}{ 59.90}&\multicolumn{1}{c}{56.37 }&
  CLIP \cite{clip}&\multicolumn{1}{c}{ 56.48}&
  \multicolumn{1}{c|}{ 64.05}&\multicolumn{1}{c}{60.03}&
  CLIP \cite{clip}&\multicolumn{1}{c}{70.53 }&
  \multicolumn{1}{c|}{77.50 }&\multicolumn{1}{c}{73.85 }\\

    CoOp \cite{coop}&\multicolumn{1}{c}{79.44 }&
    \multicolumn{1}{c|}{41.18 }&\multicolumn{1}{c}{54.24 }&
    CoOp \cite{coop}&\multicolumn{1}{c}{92.19 }&
    \multicolumn{1}{c|}{ 54.74}&\multicolumn{1}{c}{ 68.69}&
    CoOp \cite{coop}&\multicolumn{1}{c}{ 84.39}&
    \multicolumn{1}{c|}{ 56.05}&
    \multicolumn{1}{c}{67.46 }\\

CoCoOp \cite{cocoop}&\multicolumn{1}{c}{ 77.01}&
\multicolumn{1}{c|}{56.00 }&\multicolumn{1}{c}{64.85 }&
CoCoOp \cite{cocoop}&\multicolumn{1}{c}{87.49 }&
\multicolumn{1}{c|}{ 60.04}&\multicolumn{1}{c}{71.21 }&
CoCoOp \cite{cocoop}&\multicolumn{1}{c}{ 82.33}&
\multicolumn{1}{c|}{ 73.45}&\multicolumn{1}{c}{77.64 }\\

MaPLe \cite{maple}&\multicolumn{1}{c}{ 80.36}&
\multicolumn{1}{c|}{ 59.18}&\multicolumn{1}{c}{68.16 }&
MaPLe \cite{maple}&\multicolumn{1}{c}{94.07 }&
\multicolumn{1}{c|}{73.23 }&\multicolumn{1}{c}{ 82.35}&
MaPLe \cite{maple}&\multicolumn{1}{c}{ 83.00}&
\multicolumn{1}{c|}{78.66 }&\multicolumn{1}{c}{80.77 }\\
\cmidrule(lr){1-4}\cmidrule(lr){5-8}\cmidrule(lr){9-12}

\cellcolor{LightCyan}\textsc{StyLIP}&\multicolumn{1}{c}{ \cellcolor{LightCyan}\textbf{81.57}}&\multicolumn{1}{c|}{\cellcolor{LightCyan}\textbf{61.72}}&\multicolumn{1}{c}{ \cellcolor{LightCyan}\textbf{70.27}}&\cellcolor{LightCyan}\textsc{StyLIP}&\multicolumn{1}{c}{\cellcolor{LightCyan}\textbf{94.61}}&\multicolumn{1}{c|}{\cellcolor{LightCyan}\textbf{74.06}}&\multicolumn{1}{c}{\cellcolor{LightCyan}\textbf{83.08}}&\cellcolor{LightCyan}\textsc{StyLIP}&\multicolumn{1}{c}{\cellcolor{LightCyan}\textbf{85.19}}&\multicolumn{1}{c|}{\cellcolor{LightCyan}\textbf{79.22}}&\multicolumn{1}{c}{\cellcolor{LightCyan}\textbf{82.10}}\\

\cmidrule(lr){1-4}\cmidrule(lr){5-8}\cmidrule(lr){9-12}
    \end{tabular}}
    \label{tab:b2n_image}}
\end{table*}

\paragraph{Generalizing across novel domains and categories}
One of the highlights of pre-trained CLIP is its ability to generalize to novel classes. Existing prompt tuning works \cite{coop, cocoop, clip-adapter, prograd} have shown an improvement in performance in this setting, where the base and novel classes are selected from the same distribution. In this paper, we move one step forward and compare the generalization performance of the prompting techniques for CLIP when the base and novel classes originate from different visual domains, namely source and target, respectively. We consider Office-Home \cite{officehome} and DomainNet \cite{domain_net} and divide the object categories randomly and evenly to define the base and novel classes.
In Table \ref{tab:b2n}, we consider \textit{ClipArt} as the source domain, and the rest as targets. Similarly in Table \ref{app_tab:b2n_3}, we consider the domain \textit{Product} for Office-Home and the domain \textit{Painting} for DomainNet as the source domains, respectively.

As we observe in Table \ref{tab:b2n}, \textsc{StyLIP} outperforms the other prompting techniques in nine out of ten cases by $\approx 0.3-4 \%$ while generalizing to novel classes from both the source and the target domains, respectively. For the \textit{Real} domain of DomainNet, \textsc{StyLIP} lags \cite{cocoop} by a mere $0.21 \%$. \textsc{StyLIP} is less prone to overfitting to the classes of the source domain due to the better transferability offered by our model through multi-scale feature embedding. To validate this claim, we repeat this experiment using the model \textsc{StyLIP}*, which deals with only the high-level visual encodings.
Confirming our hypothesis, we find that the performance of \textsc{StyLIP}* is consistently poorer than \textsc{StyLIP} for all cases ($\approx 0.2-1.2 \%$).

\paragraph{In-domain base to novel class generalization}In addition to the cross-domain generalization to novel categories, we show the performance of \textsc{StyLIP} on the $11$ datasets \cite{coop} where the base and novel classes are divided for each dataset to define the source and the target domains. A context length of four and $16$ samples per class are considered for training the model. We collate the results in Table \ref{tab:b2n_image}, which shows that \textsc{StyLIP} beats the state-of-the-art, CoOp \cite{coop}, CoCoOp \cite{cocoop}, and the recent state-of-the-art MaPLe \cite{maple} convincingly by more than $0.9 \%$ on average H-score (H-score is the harmonic mean of the base-class and novel-class accuracies). Specifically, \textsc{StyLIP} is better than CoOp and CoCoOp by $\approx 8 \%$ and $4 \%$, respectively. We observe that \textsc{StyLIP} is able to beat the others both for the base as well as novel classes. This is important since the existing methods are mostly found to boost the performance of novel classes at the cost of decreasing base class performance.
\\
\paragraph{Generalization across datasets}Having demonstrated the generalizability of \textsc{StyLIP} for different visual domains, we further show that \textsc{StyLIP} can transfer knowledge across datasets. The target task can change from object classification to texture recognition in this setting. Following the literature \cite{coop}, we perform prompt learning using $16$-shots from the $1000$ classes of ImageNet (source) and test on the other $10$ datasets (target). On the source domain, \textsc{StyLIP} beats the recent \cite{maple} by almost $1.6 \%$ (Table \ref{fig:dgad}). In contrast, for more specialized target datasets, such as DTD, EuroSAT and FGVAircrafts, \textsc{StyLIP} beats the other competitors. For the fine-grained datasets (Caltech, OxfordPets, Flowers, etc.), \textsc{StyLIP} shows improvements up to $2 \%$, exhibiting much stronger transferability.

\begin{table*}[!ht]
    \centering
    \caption{Comparison of \textsc{StyLIP} with the prompt benchmark methods for generalization across datasets. We train the model on ImageNet using $16$-shots with CLIP ViT-B/16 and test on $10$ other datasets. (In $\%$)}
    \scalebox{0.77}{
    \begin{tabular}{llccccccccccc}
    \toprule
        \multicolumn{1}{c}{\textbf{Method}} & \multicolumn{1}{c}{\textbf{Source}} & \multicolumn{11}{c}{\textbf{Target}}  \\ \cmidrule(lr){2-2} \cmidrule(lr){3-13}

        & {ImgNet.} & {C101} & {Pets} & {Cars} & {Flowers} & {Food} & {Aircraft} & {Sun397} & {DTD} & {EuroSAT} & {UCF101} & {Average} \\
        \midrule
        CoOp \cite{coop} & 71.51 & 93.70 & 89.14 & 64.51 & 68.71 & 85.30 & 18.47 & 64.15 & 41.92 & 46.39 & 66.55 & 63.88 \\
        CoCoOp \cite{cocoop}& 71.02 & 94.43 & 90.14 & 65.32 & 71.88 & 86.06 & 22.94 & 67.36 & 45.73 & 45.37 & 68.21 & 65.74\\
        MaPLe \cite{maple}& 70.72&	93.53&	90.49&	65.57&	72.23&	86.20&	24.74&	67.01&	46.49&	48.06&	68.69&	66.30\\

\cellcolor{LightCyan}\textsc{StyLIP}&\cellcolor{LightCyan}\textbf{72.30}&\cellcolor{LightCyan}\textbf{95.45}&	\cellcolor{LightCyan}\textbf{91.60}&	\cellcolor{LightCyan}\textbf{67.09}&	\cellcolor{LightCyan}\textbf{72.36}&	\cellcolor{LightCyan}\textbf{88.60}&	\cellcolor{LightCyan}\textbf{25.21}&	\cellcolor{LightCyan}\textbf{68.11}&	\cellcolor{LightCyan}\textbf{47.86}&	\cellcolor{LightCyan}\textbf{48.22}&	\cellcolor{LightCyan}\textbf{69.30}	&\cellcolor{LightCyan}\textbf{67.38}\\
\bottomrule

    \end{tabular}}
    \label{fig:dgad}

\end{table*}
\subsection{Ablation analysis}\label{sec:ablation}
\paragraph{Analysis of multi-scale image features}
To check the efficacy of multi-scale features in \textsc{StyLIP}, we experiment with PACS and VLCS with RN50, where we progressively consider more encoder layer outputs of $f_v$ to define the style and content features for $M=L$. For example, we first consider the outputs from the final convolution block of RN50, which means we have a context length of one, and the content features from only one layer are utilized, followed by the outputs of the top two blocks (context length is two and content features corresponding to two layers are considered), and so on. We observe that the model concerning all the convolution blocks provides the best performance, and the general trend shows using more layers is better (Fig. \ref{fig:intermediate}).
\begin{table}[!ht]
    \centering
    \small{
    \caption{Comparison of our proposed \textsc{StyLIP} with the state-of-the-art methods, including their extensions with multi-scale features on the meta networks for PACS, VLCS, and Office-Home datasets for multi-source DG in terms of mean leave-one-out performance. (In $\%$)}
    \scalebox{0.73}{
    \begin{tabular}{clccc}
    \toprule
   \multirow{2}[1]{*}{\textbf{Backbone}}&\multirow{2}[1]{*}{\textbf{Method}}&\multicolumn{1}{c}{\textbf{PACS}}&\multicolumn{1}{c}{\textbf{VLCS}}&\multicolumn{1}{c}{\textbf{Office Home}}\\
          \cmidrule(lr){3-3}\cmidrule(lr){4-4}\cmidrule(lr){5-5}

       \midrule

       \multirow{5}{*}{CLIP RN50}
         &CoOp \cite{coop}&  89.88&74.04&69.04\\
         &MS-CoOp &90.43 &74.61 &69.87 \\
        &CoCoOp \cite{cocoop}  & 88.69&74.80&69.48\\
        &MS-CoCoOp &89.15 &75.90 &69.72 \\
        &\cellcolor{LightCyan}\textsc{StyLIP} &\cellcolor{LightCyan}\textbf{92.61} & \cellcolor{LightCyan}\textbf{77.18}&\cellcolor{LightCyan}\textbf{71.60}\\
         \midrule

           \multirow{5}{*}{CLIP ViT-B/16}
         &CoOp \cite{coop}&  95.59&80.10&80.44\\
         &MS-CoOp &95.86 &80.32 &80.77 \\
        &CoCoOp \cite{cocoop}  & 94.92&80.44&81.19\\
        &MS-CoCoOp &95.14 &81.06 &81.93 \\
        &\cellcolor{LightCyan}\textsc{StyLIP} &\cellcolor{LightCyan}\textbf{97.03} & \cellcolor{LightCyan} \textbf{82.90}&\cellcolor{LightCyan}\textbf{83.89}\\
        \bottomrule
    \end{tabular}}

    \label{tab:ms}
    }
\end{table}
In another experiment, we fix $M=1$ considering the deepest layer style features, and we progressively use content features from multiple layers ($L$ is varied from $1-4$). Similarly, we fix $L=4$, and vary $M$ from $1$ to $4$. As we anticipate, the content features have less significance if the context length is already small and corresponds to a limited amount of style features (as opposed to Fig. \ref{fig_context} where average style features from all the layers are used for $M=1$). Nonetheless, using more content features with increasing $L$ shows an increasing trend in the performance (Fig \ref{fig:M_and_L}).

\begin{table*}[!ht]

\centering
\scriptsize{
    \centering
    \caption{{\color{red}}Analysis of the multi-scale features on the  meta networks for the generalization from base to new classes across domains. We show results on Office-Home \textit{ClipArt} acting as the source domain, while others denote the target. The model is trained (backbone CLIP ViT-B/16) using $16$-shots from the base classes. (In $\%$)}
    \scalebox{0.9}{
    \begin{tabular}{lccccc}
    \toprule
     \multirow{4}{*}{\textbf{Method}}&\multicolumn{5}{c}{\textbf{Office-Home}}\\\cmidrule(lr){2-6}
    &\multicolumn{1}{c}{\textbf{Base}}&\multicolumn{4}{c}{\textbf{New}}\\\cmidrule(lr){2-2}\cmidrule(lr){3-6}
        ~ & \textit{\textbf{Clip Art}} & \textbf{Art} & \textbf{Clip Art} & \textbf{Product} & \textbf{Real World} \\
        \midrule

        CLIP \cite{clip}  & 78.12 &62.01 & 77.78 & 87.52 & 88.02 \\
        CoOp \cite{coop}& 82.60 &70.60 & 82.23 & 90.44 & 87.21 \\
        MS-CoOp & 83.47 &71.66 & 82.50 & 91.29 & 87.66 \\
        CoCoOp \cite{cocoop}& 82.64 &71.00 &83.61 & 92.12 & 89.19 \\
        MS-CoCoOp & 83.59 &72.25 &85.09 & 92.30& 89.67 \\
         \cellcolor{LightCyan}\textsc{StyLIP}&\cellcolor{LightCyan}\textbf{84.33} & \cellcolor{LightCyan}\textbf{74.60}& \cellcolor{LightCyan}\textbf{87.25}& \cellcolor{LightCyan}\textbf{93.00} & \cellcolor{LightCyan}\textbf{91.42} \\ \bottomrule
    \end{tabular}}\label{tab:ms_b2n}}

\end{table*}

Finally, integrating the multi-scale content features into the prompt embedding through the fusion network $\mathcal{P}_A$ reduces redundancy in the tokens than the addition-based approach followed in \cite{amortized, cocoop} and outputs more expressive prompts. This is validated in Tab. \ref{tab:ms}, where we train CoOp and CoCoOp with the multi-scale content features in place of the deepest semantic features in the respective meta networks. We find \textsc{StyLIP} to outperform these modified CoOp and CoCoOp versions for PACS, VLCS, and Office-Home, respectively. This clearly suggests that our notion of using the style features for learning the tokens indeed plays an important role. We also ablate the effect of multi-scale content features in training the CoOp and CoCoOp for the cross-domain base to new class generalization on Office-Home (Table \ref{tab:ms_b2n}) and DomainNet (Table \ref{app_tab:b2n} datasets, and we find that \textsc{StyLIP} outperforms CoOp and CoCoOp by at least 0.7$\%$.

\begin{figure}[t]
    \centering
    \includegraphics[width=0.49\columnwidth]{images/M4_L14.pdf}
    \includegraphics[width=0.49\columnwidth]{images/L1_M14.pdf}

    \caption{Effect of the number of encoder layers of $f_v$ in \textsc{StyLIP} considering RN50 and context length, i.e., M=1 and $L$ is varied from $1-4$ and L=4 and $M$ is varied from $1-4$ for multi-source DG on PACS, VLCS and OfficeHome datasets.}

    \label{fig:M_and_L}
\end{figure}

\begin{figure}[ht!]
\centering

   \includegraphics[width=\columnwidth]{images/context_length.pdf}

   \includegraphics[width=\columnwidth]{images/training_samples.pdf}

\caption{Sensitivity of prompting techniques (with ViT-B/16) on the context length ($M$) and the number of training samples per class. We show the average performance for multi-source DG on PACS dataset.}
\label{fig_context}
\end{figure}

\paragraph{Context length $(M)$}As we mention in Fig. \ref{fig_context}, we evaluate the effects of different context lengths for multi-source DG on PACS using the ViT backbone.
We find that \textsc{StyLIP} outperforms the other techniques, including \cite{cocoop,coop,prograd, amortized, clip-adapter} for context lengths of $1$, $4$, $6$, $12$, and $16$. To generate the style primitives for $M=16$, we choose to replicate the feature statistics vectors for the final four encoder layers, i.e., $\mathcal{F}_{9-12}(x)$, in addition to that of the original $12$ layers and feed them to $\{\mathcal{P}_m\}_{m=1}^{16}$.
A context of $4$ provides the optimal performance for \textsc{StyLIP}. We further find that a longer context length drastically deteriorates the performance of \cite{cocoop, prograd}, while \textsc{StyLIP} performs consistently across all context lengths. This is because each context vector corresponds to the style information extracted from a specific layer of $f_v$; therefore, some style aspects are always present in the prompt embeddings. \\
\paragraph{Sensitivity to the number of training samples for the conventional DG setup}To assess the robustness of \textsc{StyLIP} versus the number of training samples for the conventional DG setting, we train the single-source DG model on PACS while varying the number of training samples per class in the range $[1, 5, 10, 16, All]$. As shown in Fig. \ref{fig_context}, the DG performance of \cite{coop, cocoop} degrades in the low-data regime, while \cite{amortized, prograd, clip-adapter} shows comparatively better performance. Finally, \textsc{StyLIP} maintains its superior performance for very few training samples and shows improvements with more shots. This further confirms that \textsc{StyLIP} is resistant to overfitting.

\begin{table*}[t]
    \centering
    \caption{Ablation analysis of \textsc{StyLIP} for multi-source DG on PACS and Office-Home using ViT-B/16 backbone. (In $\%$)}
    \scalebox{0.8}{
    \begin{tabular}{l|cc}
    \toprule

        \multicolumn{1}{l}{\textbf{Baselines}}&\multicolumn{1}{c}{\textbf{PACS}}&\multicolumn{1}{c}{\textbf{Off.Home}}\\
        \midrule

       \multicolumn{1}{l}{ Late Fusion Projector (max-pool)}&95.33&82.10\\
        \multicolumn{1}{l}{Late Fusion Projector (average pool)}&93.21&81.33\\
        \midrule
        \multicolumn{1}{l}{Depth of $\{\mathcal{P}_m\}_{m=1}^M$ and $\mathcal{P}_C$ (2 Layers)}&97.28&84.47\\
        \multicolumn{1}{l}{Depth $\{\mathcal{P}_m\}_{m=1}^M$ and $\mathcal{P}_C$ (3 Layers)}&97.43&84.40\\
        \midrule
        \multicolumn{1}{l}{$\{\mathcal{P}_m\}_{m=1}^M$ Only ($\mu$)}&96.81&83.26\\
        \multicolumn{1}{l}{$\{\mathcal{P}_m\}_{m=1}^M$ Only ($\sigma$)}&97.00&83.42\\
        \midrule
        \multicolumn{1}{l}{$\{\mathcal{B}_l\}_{l=1}^L$ (GAP over the spatial dimensions of the feature-maps)}&97.64&83.99\\
        \multicolumn{1}{l}{$\{\mathcal{B}_l\}_{l=1}^L$ (only flatten)}&97.30&83.57\\
        \multicolumn{1}{l}{$\{\mathcal{B}_l\}_{l=1}^L$ (conv 1x1 with $\hat{C}=2$)}&97.57&83.79\\
        \multicolumn{1}{l}{$\{\mathcal{B}_l\}_{l=1}^L$ (conv 1x1 with $\hat{C}=4$)}&97.92&84.33\\
        \multicolumn{1}{l}{$\{\mathcal{B}_l\}_{l=1}^L$ (conv 1x1 with $\hat{C}=16$)}&96.44&83.10\\
        \midrule
        \multicolumn{1}{l}{\cellcolor{LightCyan}\textsc{StyLIP} ($\hat{C}=3$)}&\cellcolor{LightCyan}\textbf{98.05}&\cellcolor{LightCyan}\textbf{84.63}\\
        \bottomrule

    \end{tabular}}
    \label{tab:ablation}

\end{table*}

\paragraph{Depth of style and content projectors}To check the sensitivity of \textsc{StyLIP} on the depth of $\{\mathcal{P}_m\}_{m=1}^M$ and $\mathcal{P}_C$, we consider cases of multi-source DG where the projectors are two-layers and three-layers deep, respectively, in PACS and Office-Home (Table \ref{tab:ablation}). We find that performance decreases marginally with increasing depth: $0.6-0.8 \%$ for PACS and $0.2-0.4 \%$ for Office-Home than \textsc{StyLIP} with linear projectors, suggesting \textsc{StyLIP} is indeed lightweight.\\
\paragraph{Learnable vs. non-learnable $\mathcal{P}_A$}Typically, $\mathcal{P}_C$ and $f_t$ produce feature embeddings of similar dimensions; hence, one way to fuse them in $\mathcal{P}_A$ is through element-wise feature pooling. In this regard, we use the max and average feature pooling strategies and observe in Table \ref{tab:ablation} that such aggregations affect the performance, reducing the multi-source DG accuracies on PACS and Office-Home by $\approx 2-3 \%$ in max pooling and $\approx 4-5 \%$ in average pooling than \textsc{StyLIP}. \\
\paragraph{Analysis of style features}Typically, the mean and std. of the feature maps together are known to capture the visual style information. To validate the same, we study the model's performance with either mean or std. being used as input to the style projectors. In this regard, we see a decrease in the performance of $1-2 \%$ compared to \textsc{StyLIP}, suggesting the importance of both statistical estimates. Interestingly, we see better accuracy when only std. is used for context learning than only mean (Table \ref{tab:ablation}).\\
\paragraph{Sensitivity to the depth of the bottleneck layer $\hat{C}$}We consider different $\hat{C}$ values in the range ${2, 3, 4, 16}$ to see the effects of the bottleneck dimensions in the final accuracy (Table \ref{tab:ablation}). While $\hat{C}=3$ provides the best performance, we see the numbers decreasing from $\hat{C}=4$ onwards, finally producing a dip of almost $1.5 \%$ for $\hat{C}=16$. Besides, we consider the scenario where $1 \times 1$ convolutions are not used, and we perform global average pooling (GAP), or directly flatten the feature maps and then concatenate. Both options perform poorly compared to \textsc{StyLIP} by $0.5-1 \%$.

\begin{figure}
    \centering
    \includegraphics[width=0.47\columnwidth]{images/PACS.pdf}
    \includegraphics[width=0.47\columnwidth]{images/VLCS.pdf}

    \caption{Effect of number of encoder layers of $f_v$ in \textsc{StyLIP} considering RN50 for multi-source DG on PACS and VLCS.}

    \label{fig:intermediate}
\end{figure}

\paragraph{t-SNE Visualization.}
We show the quality of the generated prompts by showing the domain-wise prompt distributions in Fig. \ref{fig:tsne}. We observe that the prompt embeddings obtained from \cite{amortized} overlap and lose the discriminative properties, whereas \textsc{StyLIP} can produce a cleaner embedding space. This suggests the superiority of our style projectors over the prompt learner of \cite{amortized}.

\begin{figure}[h!]
    \centering
    \includegraphics[width=7.6cm]{images/tsne.pdf}
    \caption{t-SNE visualization of the prompt embedding outputs from $f_t$ for DPL \cite{amortized} and \textsc{StyLIP} for multi-source domain generalization on PACS dataset. \textsc{StyLIP} can provide a more discriminative domain-discriminative embedding space. The legends $(0,1,2)$ denote the different source domains.}
    \label{fig:tsne}
\end{figure}

\paragraph{Computation Complexity.}
We run our model on NVIDIA RTX 3090 Ti with 24 GB card. Tab. \ref{tab:compute} represents the comparison of computational complexity between different prompting techniques (CoOp \cite{coop}, CoCoOp \cite{cocoop}, and MaPLe \cite{maple}) in terms of GFLOPS relative to CoOp. MaPLe requires $0.12\%$ more computational overhead than CoOp and CoCoOp, whereas \textsc{StyLIP} needs $0.18 \%$ more resources than MaPLe, but \textsc{StyLIP} outperforms state-of-the-art MaPLe on the cross-dataset generalization (average over 11 datasets) approximately by $1.2\%$.

\begin{table}[!ht]
    \centering
    \caption{Increase in compute w.r.t. CoOp and CoCoOp.}
    \scalebox{1}{
    \begin{tabular}{cccc}
    \toprule
     CoOp \cite{coop} & CoCoOp \cite{cocoop} & MaPLe \cite{maple} & \textsc{StyLIP} \\
    \midrule

    1$\times$ & 1$\times$ & +0.12$\%$& +0.18$\%$ \\

    \bottomrule
    \end{tabular}}

    \label{tab:compute}
\end{table}

\section{Takeaways}
\label{sec:Conclusions}
In this paper, our objective is to direct the pre-trained large-scale foundation models to solve the DG tasks in the presence of an unaccountable domain shift. Consequently, we introduce a domain-agnostic prompt learning strategy for CLIP, \textsc{StyLIP}, which integrates the multi-scale visual style and content information obtained from CLIP's frozen vision encoder in the prompt learning process, enhancing its generalizability. We rigorously evaluate \textsc{StyLIP} on multiple cross-domain inference tasks, and \textsc{StyLIP} is found to produce state-of-the-art performance consistently.
We hope our study of task-generalizable prompt learning will open up new research avenues in computer vision. Future directions may consider using different foundation models for domain-aware prompt learning and using the proposed idea for structured prediction tasks like semantic segmentation and depth estimation, to name a few.
\section*{Acknowledgement}
Thanks to Mainak Singha and Ankit Jha for conducting the extensive experiments. Also, we thank Shirsha Bose, Enrico Fini, Biplab Banerjee, and Elisa Ricci for the scientific discussions and their valuable feedback in shaping the research and the articles.
\clearpage
\begin{appendices}

\section{Ablation Studies}\label{secA1}
\begin{itemize}
    \item Table \ref{app_tab_PACS} and Table \ref{app_tab_DigitDG} show the detailed version of Table \ref{tab_PACS}, where we consider the domain-wise multi-source DG performance and compare our proposed \textsc{StyLIP} with the referred SOTA methods, and \textsc{StyLIP} beats them in all the domains of the mentioned datasets.

    \item Similar to multi-source DG setup, we show the detailed domain-wise experimental results for single-source DG setup in Table \ref{tab:app_ssdg} for PACS, VLCS, and Office-Home datasets.

    \item In Table \ref{app_tab:b2n_3}, we show the base to new class generalization results by considering the domain \textit{Product} and the domain \textit{Painting} as the source domain and the rest as the target domains for Office-Home and DomainNet datasets, respectively. We observe that \textsc{StyLIP} outperforms referred literature by a fair margin.

    \item Finally, we showcase the effect of multi-scale content features on the meta networks for the generalization base to new classes across domains in Table \ref{app_tab:b2n}. Here, we compare our proposed \textsc{StyLIP} with CoOp \cite{coop} and CoCoOp \cite{cocoop} by training them with multi-scale (MS) content features instead of deepest semantic features in the respective meta networks. We find that multi-scale content features help generate better learnable prompts for the respective prompting methods.
\end{itemize}
\begin{table*}[!ht]

    \scriptsize{
    \begin{center}

    \caption{Comparison of our proposed \textsc{StyLIP} with the state-of-the-art methods on PACS, VLCS and Office Home datasets.}
    \scalebox{0.58}{
    \begin{tabular}{llccccccccccccccc}
    \hline

    \multirow{2}{*}{\textbf{Backbone}} &\multirow{2}{*}{\textbf{Method}} & \multicolumn{5}{c}{\textbf{PACS}}& \multicolumn{5}{c}{\textbf{VLCS}}& \multicolumn{5}{c}{\textbf{Office Home}}\\
    \cmidrule(lr){3-7}\cmidrule(lr){8-12}\cmidrule(lr){13-17}
         &  & \textbf{Art P.} & \textbf{Cartoon} & \textbf{Photo} & \textbf{Sketch} & \textbf{Avg.} & \textbf{LableMe} & \textbf{Sun}& \textbf{Caltech} & \textbf{Pascal} & \textbf{Avg.}& \textbf{Art} & \textbf{Clip A.} & \textbf{Real W.} &\textbf{Prod.} & \textbf{Avg.}  \\
         \midrule

         \multirow{17}{*} {CLIP RN-50} & SWAD \cite{swad} & - & - & - & - & 88.10 & -& - & -&- & 79.10& -& -& -& -& 70.60\\ \cmidrule(lr){2-17}

         ~ & Lin. Probing & 91.29 & 90.92 & 99.02 & 85.37 & 91.65 & 63.37 & 79.20 & 98.96 & 76.39 & 79.48 & 69.54 & 49.70 & 80.06 & 81.39 & 70.17 \\
         ~ & CORAL \cite{coral} & 92.04 & 92.91 & 98.54 & 85.19 & 92.17 & 61.75& 84.09 & 98.84&80.92 & 81.40& 70.31& 50.60& 80.19& 81.50& 70.65\\
        ~ & DANN \cite{dann}& 92.77 & 93.09 & 98.60 & 85.26 & 92.43 & 67.64 & 82.01 & 97.37 & 79.66 & 81.67 & 70.36& 50.77&80.23  &81.75 &70.78  \\
         ~ & CROSSGRAD \cite{cross-grad}& 91.46 & 92.16 &99.20  & 83.40 &91.56 & 63.42&76.98  & 99.01 & 79.11 & 79.63&  70.20& 50.11&  80.49&  81.07&  70.47\\
        ~ & ZS-CLIP \cite{clip}& 89.36 & 93.61 & 98.56 & 79.73 & 90.32 & 61.45 & 76.63 & 96.06 & 71.56 & 76.43& 67.81& 44.22& 78.56 &76.41 & 66.75\\
        ~ & ZS-CLIP + DN & 92.72 & 95.05 & 99.24  & 80.39 & 91.86 & 61.56 & 74.71 & 98.82 & 75.71 & 77.70 & 68.82 & 45.44 &  80.82 &  76.64 & 67.93\\

        ~ & CoOp \cite{coop}& 91.86 & 92.85 & 99.25 & 85.17 & 92.28  & 67.95 & 81.04 & 98.92 & 79.55 & 81.87 & 71.70 & 51.40 & 81.96 & 81.52 & 71.65 \\
        ~ & CoCoOp \cite{cocoop}& 90.89 & 92.50 & 98.21 & 84.96 & 91.64  & 68.50 & 81.76 & 99.03 & 79.90 & 82.30 & 71.55& 51.61 & 82.25 & 82.30 & 71.93 \\
        ~ & CLIP-Adapt. \cite{clip-adapter}& 92.57 & 92.03 & 99.31 & 84.41 & 92.08 & 70.84 & 81.22 & 99.03 & 78.31 & 82.35 & 71.83 & 52.19 & 82.40 & 82.28 & 72.18 \\
        ~ & DPL \cite{amortized}& 92.95 & 93.44 & 99.13 & 82.31 & 91.96  & 70.51 & 81.00 & 98.94 & 78.01 & 82.12 & 71.90 & 52.55 & 82.67 & 82.93 & 72.54\\

        ~ & ProGrad \cite{prograd}& 92.40 &92.65 & 99.31 &83.70 & 92.01 & 70.09 & 81.22 & 99.07 & 78.42 & 82.23& 71.61 & 51.80 &  82.09&  82.00& 71.85\\
        ~ & TPT \cite{tpt} & 92.70 & 93.30 & 99.07 & 83.55 & 92.16 & 70.87 & 81.41 & 99.06 & 78.22 & 82.39& 71.83 & 52.16 & 82.40 & 81.90 & 72.07 \\

         &\cellcolor{LightCyan}\textsc{StyLIP}-con&\cellcolor{LightCyan}92.55&	\cellcolor{LightCyan}93.39&	\cellcolor{LightCyan}99.26&	\cellcolor{LightCyan}84.20&	\cellcolor{LightCyan}92.35&\cellcolor{LightCyan}72.49&	\cellcolor{LightCyan}83.01&	\cellcolor{LightCyan}99.01	&\cellcolor{LightCyan}81.75&	\cellcolor{LightCyan}84.07&\cellcolor{LightCyan}73.61&\cellcolor{LightCyan}54.21&	\cellcolor{LightCyan}83.67&	\cellcolor{LightCyan}84.09&	\cellcolor{LightCyan}73.89\\
        ~ & \cellcolor{LightCyan}\textsc{StyLIP}-sty & \cellcolor{LightCyan}92.30& \cellcolor{LightCyan}93.75 & \cellcolor{LightCyan}99.42& \cellcolor{LightCyan}86.35 & \cellcolor{LightCyan}92.96 & \cellcolor{LightCyan}72.88 & \cellcolor{LightCyan}83.20 & \cellcolor{LightCyan}99.11 & \cellcolor{LightCyan}82.37 & \cellcolor{LightCyan}84.39& \cellcolor{LightCyan}73.86 & \cellcolor{LightCyan}54.70 & \cellcolor{LightCyan}83.91 & \cellcolor{LightCyan}84.40 & \cellcolor{LightCyan}74.22\\
        ~ & \cellcolor{LightCyan}\textsc{StyLIP}* &\cellcolor{LightCyan}92.70&\cellcolor{LightCyan}93.25&\cellcolor{LightCyan}99.31&\cellcolor{LightCyan}84.65& \cellcolor{LightCyan}92.47&\cellcolor{LightCyan}72.10&\cellcolor{LightCyan}82.00&\cellcolor{LightCyan}99.01&\cellcolor{LightCyan}81.32&\cellcolor{LightCyan}83.60 &\cellcolor{LightCyan}73.30&\cellcolor{LightCyan}53.45&\cellcolor{LightCyan}83.87&\cellcolor{LightCyan}83.61&\cellcolor{LightCyan}73.56\\

        ~ & \cellcolor{LightCyan}\textsc{StyLIP} & \cellcolor{LightCyan}\textbf{93.71} & \cellcolor{LightCyan}\textbf{94.20} & \cellcolor{LightCyan}\textbf{99.48} & \cellcolor{LightCyan}\textbf{86.98} & \cellcolor{LightCyan}\textbf{93.59} & \cellcolor{LightCyan}\textbf{73.25} & \cellcolor{LightCyan}\textbf{84.18} & \cellcolor{LightCyan}\textbf{99.11} & \cellcolor{LightCyan}\textbf{82.80} & \cellcolor{LightCyan}\textbf{84.83} & \cellcolor{LightCyan}\textbf{74.60} & \cellcolor{LightCyan}\textbf{55.18} & \cellcolor{LightCyan}\textbf{84.30} & \cellcolor{LightCyan}\textbf{85.11} & \cellcolor{LightCyan}\textbf{74.80} \\ \midrule

        \multirow{14}{*}{CLIP ViT-B/16} & Lin. Probing & 97.60 &98.95 &99.88 & 89.73 & 96.54 & 68.12 & 83.62 & 99.21 & 79.57 & 82.63& 81.55 & 65.70 & 87.14 & 87.32 & 80.43 \\
        ~ & CROSSGRAD \cite{cross-grad} & 96.85 & 98.82 &99.65  & 90.31 &96.40 & 69.57&82.80  &99.26  &83.40  &83.76 & 81.28 & 66.43&  86.91& 87.60 & 80.55 \\

        ~ & ZS-CLIP \cite{clip}&96.92 & 98.81 & 99.79 & 87.71&95.81 & 68.79& 80.83 & 98.50 & 74.16 & 80.57& 79.46&63.08 & 86.46 &85.27 & 78.57\\

        ~ & ZS-CLIP + DN & 97.65 & 98.92 & 99.82 & 88.81 & 96.30 & 71.03 & 82.23 & 98.61 & 75.89 & 81.94 & 80.06 & 63.56 &  86.92 &  85.84 & 79.10\\

        ~ & CoOp \cite{coop}& 97.60 & 98.58 & 99.94 & 91.89 & 97.00 & 67.78 & 84.05 & 98.52 & 81.59 & 82.98& 80.08&68.99	&86.96	&88.44&	81.12 \\
        ~ & CoCoOp \cite{cocoop}& 97.10 & 97.98 & 99.83 & 92.00 & 96.73 & 72.24 & 84.79 & 99.22 & 78.10 & 83.59&  79.60	&69.35&	86.32&	87.51&	80.70\\
        ~ & CLIP-Adapt. \cite{clip-adapter}& 97.36 & 98.77 & 99.88 & 89.63 & 96.41 & 70.70 & 84.64 & 98.59 & 83.35 & 84.32& 82.76 & 70.08 & 88.02 & 88.04&82.23\\
        ~ & DPL \cite{amortized}& 96.71 & 98.50 & 99.46 & 93.59 & 97.07 & 71.21 & 83.45 & 99.24 & 82.05 & 83.99& 82.94 & 71.80 & 88.59 & 88.65 & 83.00 \\
        ~ & ProGrad \cite{prograd}& 96.80 &98.96 & 99.59 &90.64 & 96.50 & 71.30 & 83.09& 99.41 & 81.50 & 83.82& 82.57 & 70.20& 88.60& 88.49& 82.46 \\
        ~ & TPT \cite{tpt} & 96.59 & 98.35 & 99.46 & 93.50 &96.99 & 71.05 & 83.20 & 99.20 & 81.44 & 83.72& 82.40 & 70.63 & 88.71 & 88.05 & 82.45 \\

         &\cellcolor{LightCyan}\textsc{StyLIP}-con&\cellcolor{LightCyan}97.46 & \cellcolor{LightCyan}98.99 & \cellcolor{LightCyan}99.94 & \cellcolor{LightCyan}90.87 & \cellcolor{LightCyan}96.82& \cellcolor{LightCyan}74.35&	\cellcolor{LightCyan}84.43&	\cellcolor{LightCyan}99.30&	\cellcolor{LightCyan}84.36&	\cellcolor{LightCyan}85.61& \cellcolor{LightCyan}84.30&	\cellcolor{LightCyan}71.90&	\cellcolor{LightCyan}89.49&	\cellcolor{LightCyan}89.91&	\cellcolor{LightCyan}83.90\\
        ~ &\cellcolor{LightCyan}\textsc{StyLIP}-sty &\cellcolor{LightCyan}97.71 & \cellcolor{LightCyan}98.90 & \cellcolor{LightCyan}99.88 & \cellcolor{LightCyan}92.51 & \cellcolor{LightCyan}97.25 & \cellcolor{LightCyan}75.02 & \cellcolor{LightCyan}85.69 & \cellcolor{LightCyan}99.26 & \cellcolor{LightCyan}85.10 & \cellcolor{LightCyan}86.27& \cellcolor{LightCyan}84.51 &\cellcolor{LightCyan}72.29  & \cellcolor{LightCyan}89.50 & \cellcolor{LightCyan}90.42 & \cellcolor{LightCyan}84.18 \\
        ~ & \cellcolor{LightCyan}\textsc{StyLIP}* &\cellcolor{LightCyan}97.64&\cellcolor{LightCyan}98.81&\cellcolor{LightCyan}99.70&\cellcolor{LightCyan}92.28& \cellcolor{LightCyan}97.11&\cellcolor{LightCyan}74.95&\cellcolor{LightCyan}85.22&\cellcolor{LightCyan}99.26&\cellcolor{LightCyan}84.10& \cellcolor{LightCyan}85.88&\cellcolor{LightCyan}83.79&\cellcolor{LightCyan}71.66&\cellcolor{LightCyan}89.25&\cellcolor{LightCyan}88.96&\cellcolor{LightCyan}83.41\\

        ~ & \cellcolor{LightCyan}\textsc{StyLIP}& \cellcolor{LightCyan}\textbf{98.62} & \cellcolor{LightCyan}\textbf{99.02} & \cellcolor{LightCyan}\textbf{99.96} & \cellcolor{LightCyan}\textbf{94.58} & \cellcolor{LightCyan}\textbf{98.05}& \cellcolor{LightCyan}\textbf{75.60} & \cellcolor{LightCyan}\textbf{86.72} & \cellcolor{LightCyan}\textbf{99.46} & \cellcolor{LightCyan}\textbf{85.99} & \cellcolor{LightCyan}\textbf{86.94}  & \cellcolor{LightCyan}\textbf{84.93} & \cellcolor{LightCyan}\textbf{72.61} & \cellcolor{LightCyan}\textbf{90.35} & \cellcolor{LightCyan}\textbf{90.64} & \cellcolor{LightCyan}\textbf{84.63} \\ \hline
    \end{tabular}}
    \label{app_tab_PACS}
    \end{center}}
\end{table*}

\begin{table*}[!ht]
    \centering
    \scriptsize{
    \caption{Comparison of our proposed \textsc{StyLIP} with state-of-the-art methods on DigitDG and DomainNet datasets.}
    \scalebox{0.7}{
    \begin{tabular}{clcccccccccccc}
    \toprule
    \multirow{2}{*}{\textbf{Backbone}} &\multirow{2}{*}{\textbf{Method}} & \multicolumn{5}{c}{\textbf{DigitDG}}& \multicolumn{7}{c}{\textbf{DomainNet}}\\
    \cmidrule(lr){3-7}\cmidrule(lr){8-14}
        &  & \textbf{M.} & \textbf{MM.} & \textbf{SV.} & \textbf{SY.} &\textbf{Avg.}&\textbf{ClipA.}&\textbf{Paint.}&\textbf{Real}&\textbf{Info.}	&\textbf{Quk.D.}	&\textbf{Sket.}	&\textbf{Avg.}\\
        \midrule

        \multirow{17}{*}{CLIP RN-50} & SWAD & - & - & - & - & -&-&-&-&-&-&-&46.50 \\\cmidrule(lr){2-14}

        ~ & Lin. Probing & 78.80 & 55.22 & 43.43 & 71.41 & 62.22 &54.63&	48.22&	74.15	&39.62	&11.28	&48.70	&46.10\\
        ~ & CORAL \cite{coral} & 76.20 & 55.29 & 41.53 & 71.14 & 61.04&49.50&43.28&68.10&36.41&6.38&44.20&41.31 \\
        ~ & DANN \cite{dann} & 78.35& 56.00 & 42.87 & 72.33 & 62.39 &49.96&45.11&67.83&36.60&6.70&45.60&41.97 \\
        ~ & CROSSGRAD \cite{cross-grad} &79.24 & 56.89 &44.30 &71.50 &62.98 &54.18&47.60&73.46&39.17&11.10&48.41& 45.64\\
        ~ & ZS-CLIP \cite{clip}& 72.15 & 50.98 & 39.31 & 63.20 &56.41& 51.70&46.35&73.40&38.56&8.24&46.84&44.18 \\
        ~ & ZS-CLIP + DN & 73.36 & 52.74 & 41.62 & 65.48 &58.30& 55.08&49.67&76.81&41.72&11.62&50.12&47.50 \\

        ~ & CoOp \cite{coop}& 92.91 & 65.54 & 54.80 & 79.19 & 73.11 &57.42	&53.50&	75.92	&42.50	&13.81	&55.09	&49.71\\
        ~ & CoCoOp \cite{cocoop}& 93.20 & 66.47 & 57.61 & 81.03 & 74.58 &58.22	&54.37&	76.83	&42.60&13.72&55.23&50.16\\
        ~ & CLIP-Adapt. \cite{clip-adapter}& 92.63 & 66.10 & 55.94 & 80.35& 73.79 &58.11&	54.59&	77.15&	42.31&	14.00&	55.40&	50.25\\
        ~ & DPL \cite{amortized}& 92.60 & 66.76 & 57.31 & 80.64 & 74.33 &58.25&55.00&76.91&42.75&13.95&55.43&50.38\\
        ~ & ProGrad \cite{prograd}& 92.75& 66.30 & 57.50 & 81.28 & 74.45 &57.91&55.27&76.52&42.87&13.71&55.35&50.27\\
         ~ & TPT \cite{tpt} & 93.00 & 66.54 & 57.90 & 81.29 & 74.68 &57.43&56.99&76.30&42.01&13.95&55.10&50.30\\

	&	\cellcolor{LightCyan}\textsc{StyLIP}-con&\cellcolor{LightCyan}93.12&	\cellcolor{LightCyan}68.00&	\cellcolor{LightCyan}61.29&	\cellcolor{LightCyan}\cellcolor{LightCyan}81.10&	\cellcolor{LightCyan}75.90  & \cellcolor{LightCyan}58.92&	\cellcolor{LightCyan}56.84&	\cellcolor{LightCyan}78.33&	\cellcolor{LightCyan}43.72&	\cellcolor{LightCyan}13.41&	\cellcolor{LightCyan}57.36&	\cellcolor{LightCyan}51.43\\
         ~ & \cellcolor{LightCyan}\textsc{StyLIP}-sty & \cellcolor{LightCyan}93.18& \cellcolor{LightCyan}68.50 & \cellcolor{LightCyan}61.70 &\cellcolor{LightCyan}81.45 &\cellcolor{LightCyan}76.21 & \cellcolor{LightCyan}60.10&\cellcolor{LightCyan}57.05 & \cellcolor{LightCyan}78.42&\cellcolor{LightCyan}43.97 &\cellcolor{LightCyan}13.82 &\cellcolor{LightCyan}57.40 &\cellcolor{LightCyan}51.80\\

         ~ & \cellcolor{LightCyan}\textsc{StyLIP}*& \cellcolor{LightCyan}92.90 & \cellcolor{LightCyan}67.09 & \cellcolor{LightCyan}61.80 & \cellcolor{LightCyan}81.45 & \cellcolor{LightCyan}75.81 &\cellcolor{LightCyan}59.80&\cellcolor{LightCyan}57.00&\cellcolor{LightCyan}78.10&\cellcolor{LightCyan}43.87&\cellcolor{LightCyan}13.71&\cellcolor{LightCyan}57.30&\cellcolor{LightCyan}51.63\\
        ~ & \cellcolor{LightCyan}\textsc{StyLIP}& \cellcolor{LightCyan}\textbf{93.45} & \cellcolor{LightCyan}\textbf{68.87} & \cellcolor{LightCyan}\textbf{62.01} & \cellcolor{LightCyan}\textbf{81.63} & \cellcolor{LightCyan}\textbf{76.49} &\cellcolor{LightCyan}\textbf{60.85}&\cellcolor{LightCyan}\textbf{57.30}&\cellcolor{LightCyan}\textbf{78.95}&\cellcolor{LightCyan}\textbf{44.38}&\cellcolor{LightCyan}\textbf{14.60}&\cellcolor{LightCyan}\textbf{57.66}&\cellcolor{LightCyan}\textbf{52.29}\\ \midrule

        \multirow{14}{*}{CLIP ViT-B/16} & Lin. Probing &90.42 & 62.65 & 51.70 & 75.83 & 70.15&72.19	&59.37&	82.40&	50.11&	15.90&	64.80&	57.46 \\
         ~ & CROSSGRAD \cite{cross-grad} & 90.66& 63.18 &53.11 &76.40 & 70.83&72.30&59.65&82.32&50.70&15.87&64.69& 57.60\\

        ~ & ZS-CLIP \cite{clip}& 84.80 & 59.33 & 48.60 & 70.41 & 65.79 &68.06&	56.59	&79.45&	46.03&	14.08&	60.28&	54.08\\
        ~ & ZS-CLIP + DN & 86.44 & 61.06 & 50.18 & 71.96 & 67.41 &71.23&	58.74 &81.33& 49.16& 16.39&	64.84&	56.95\\
        ~ & CoOp \cite{coop} & 94.37 & 68.01 & 60.00 & 83.24 & 76.41 &75.68& 61.15&	83.90&	52.88&	17.40&	66.10&59.52	\\
        ~ & CoCoOp \cite{cocoop}& 95.55 & 70.30 & 62.59 & 85.51 & 78.49&75.35&	61.80&	84.26	&52.99	&17.42	&66.24	&59.68\\
        ~ & CLIP-Adapt. \cite{clip-adapter}& 94.95 & 69.76 & 62.09 & 84.66 & 77.86 &76.04	&61.78&	84.59&53.10	&17.00	&66.87	&59.90\\

        ~ & DPL \cite{amortized}& 94.44 & 67.38 & 62.68 & 84.79 & 77.32 &75.81&62.00&84.31&52.76&17.20&67.10&59.86\\
        ~ & ProGrad \cite{prograd}& 94.97& 70.21 & 63.10 & 84.77 & 78.26&75.00 &63.01&83.80&51.72&16.99&67.40&59.65\\
         ~ & TPT \cite{tpt} &94.67&71.20&63.50&84.70&78.51& 75.08 & 62.41 & 83.95 & 52.40 & 17.39&68.00&59.87 \\
         &\cellcolor{LightCyan}\textsc{StyLIP}-con&\cellcolor{LightCyan}96.31	&\cellcolor{LightCyan}74.11&	\cellcolor{LightCyan}66.20&	\cellcolor{LightCyan}85.91&	\cellcolor{LightCyan}80.63&\cellcolor{LightCyan}78.19	&\cellcolor{LightCyan}63.71	&\cellcolor{LightCyan}\cellcolor{LightCyan}85.50	&\cellcolor{LightCyan}55.63	&\cellcolor{LightCyan}18.00	&\cellcolor{LightCyan}68.02	&\cellcolor{LightCyan}61.51\\
         ~ & \cellcolor{LightCyan}\textsc{StyLIP}-sty &\cellcolor{LightCyan}96.50 &\cellcolor{LightCyan}73.82  &\cellcolor{LightCyan}66.50  & \cellcolor{LightCyan}86.83& \cellcolor{LightCyan}80.91& \cellcolor{LightCyan}78.42&	\cellcolor{LightCyan}64.29&	\cellcolor{LightCyan}86.11&	\cellcolor{LightCyan}55.35&	\cellcolor{LightCyan}18.23&	\cellcolor{LightCyan}68.19&	\cellcolor{LightCyan}61.77\\
         ~ & \cellcolor{LightCyan}\textsc{StyLIP}*& \cellcolor{LightCyan}96.35 & \cellcolor{LightCyan}73.82 & \cellcolor{LightCyan}65.90 & \cellcolor{LightCyan}86.15 & \cellcolor{LightCyan}80.56 &\cellcolor{LightCyan}77.63&\cellcolor{LightCyan}64.50&\cellcolor{LightCyan}85.00&\cellcolor{LightCyan}55.33&\cellcolor{LightCyan}17.90&\cellcolor{LightCyan}68.00&\cellcolor{LightCyan}61.39\\

        ~ & \cellcolor{LightCyan}\textsc{StyLIP}& \cellcolor{LightCyan}\textbf{96.73}& \cellcolor{LightCyan}\textbf{74.90} & \cellcolor{LightCyan}\textbf{66.39} & \cellcolor{LightCyan}\textbf{87.51} & \cellcolor{LightCyan}\textbf{81.38} &\cellcolor{LightCyan}\textbf{78.48}&\cellcolor{LightCyan}\textbf{64.61}&\cellcolor{LightCyan}\textbf{86.34}&\cellcolor{LightCyan}\textbf{55.86}&\cellcolor{LightCyan}\textbf{18.45}&\cellcolor{LightCyan}\textbf{68.33}&\cellcolor{LightCyan}\textbf{62.02}\\ \bottomrule
    \end{tabular}
    \label{app_tab_DigitDG}
       }}
\end{table*}

\begin{table*}[!ht]
    \centering
    \scriptsize{
    \caption{Comparison of \textsc{StyLIP} with CLIP-based state-of-the-art methods for single-source DG on PACS, VLCS, and Office-Home datasets in terms of mean leave-one-out performance. (In $\%$) }
    \scalebox{0.58}{
    \begin{tabular}{clccccccccccccccc}
    \toprule
   \multirow{2}[1]{*}{\textbf{Backbone}}&\multirow{2}[1]{*}{\textbf{Method}}&\multicolumn{5}{c}{\textbf{PACS}}&\multicolumn{5}{c}{\textbf{VLCS}}&\multicolumn{5}{c}{\textbf{Office Home}}\\
          \cmidrule(lr){3-7}\cmidrule(lr){8-12}\cmidrule(lr){13-17}

         &  & \textbf{Art P.} & \textbf{Cartoon} & \textbf{Photo} & \textbf{Sketch} & \textbf{Avg.} & \textbf{LableMe} & \textbf{Sun}& \textbf{Caltech} & \textbf{Pascal} & \textbf{Avg.}& \textbf{Art} & \textbf{Clip A.} & \textbf{Real W.} &\textbf{Prod.} & \textbf{Avg.}  \\

       \midrule

       \multirow{9}{*}{CLIP RN50} &Lin. Probing & 91.63&88.75&84.10&78.21&85.67 &60.31&72.20&64.95&80.22&69.42&61.47&65.88&63.70&72.90&65.99\\
          &CROSSGRAD \cite{cross-grad}& 92.06&89.68&83.67&79.35&86.19&64.92&72.69&66.13&81.65&71.35&61.79&66.45&64.12&73.50&66.47\\
         &CoOp \cite{coop}& 92.84 &90.77&88.93&86.96&89.88&68.38&75.51&68.01&84.27&74.04&63.76&68.99&66.52&76.88&69.04\\
        &CoCoOp \cite{cocoop}  &92.69&91.13&86.29&84.66&88.69&69.14&76.00&68.44&85.60&74.80&63.20&69.30&67.30&77.56&69.48\\
        &CLIP-Adapter \cite{clip-adapter} &92.66&91.50&88.04&87.25& 88.86&71.91&76.34&67.80&85.31&75.31&63.70&69.26&66.89&77.30&69.29\\
         &DPL \cite{amortized} &92.10&91.43&88.61&84.80&89.24&69.57&76.11&68.29&85.45&74.86&63.42&69.57&66.60&76.84&69.10\\
         &Prograd \cite{prograd}& 92.59&91.35&85.33&84.76&88.51&70.63&76.88&68.56&85.51&75.40&63.77&69.41&67.23&77.56&69.49\\
         &TPT \cite{tpt}&92.23&91.00&87.10&85.39& 88.93 &70.90&75.61&68.48&85.11& 75.02 &63.58&69.70&67.81&77.23& 69.58 \\

        &\cellcolor{LightCyan}\textsc{StyLIP} &\cellcolor{LightCyan}\textbf{93.85}&\cellcolor{LightCyan}\textbf{93.09}&\cellcolor{LightCyan}\textbf{91.57}&\cellcolor{LightCyan}\textbf{91.93}&\cellcolor{LightCyan}\textbf{92.61} &\cellcolor{LightCyan}\textbf{72.69}&\cellcolor{LightCyan}\textbf{79.20}&\cellcolor{LightCyan}\textbf{70.33}&\cellcolor{LightCyan}\textbf{86.50}& \cellcolor{LightCyan}\textbf{77.18}&\cellcolor{LightCyan}\textbf{64.81}&\cellcolor{LightCyan}\textbf{71.56}&\cellcolor{LightCyan}\textbf{69.10}&\cellcolor{LightCyan}\textbf{80.93}&\cellcolor{LightCyan}\textbf{71.60}\\
         \midrule

           \multirow{9}{*}{CLIP ViT-B/16}  &Lin. Probing & 92.66&92.29&90.90&83.55&89.85 &71.99&81.05&70.31&81.24&76.15&74.15&78.39&76.93&81.36&77.71\\
          &CROSSGRAD \cite{cross-grad}&93.72&92.44&91.30&85.26&  90.68&73.76&81.88&70.45&81.46&76.89&74.80&78.91&77.60&81.89&78.30\\
         &CoOp \cite{coop}&97.26&95.00&94.16&95.94&  95.59&79.03&83.16&73.00&85.20&80.10&76.85&81.21&79.53&84.15&80.44\\
        &CoCoOp \cite{cocoop}  &96.89&95.13&93.78&93.87& 94.92&79.81&83.59&73.42&84.92&80.44&77.22&81.50&81.36&84.71&81.19\\
        &CLIP-Adapter \cite{clip-adapter} &96.72&94.29&92.23&95.18& 94.60&79.47&83.20&73.30&85.10&80.27&76.94&81.37&80.70&84.44&80.86\\
         &DPL \cite{amortized} &97.09&95.57&92.18&93.97&94.70&79.22&84.59&73.66&84.86&80.58&76.71&81.33&80.75&84.39&80.79\\
         &Prograd \cite{prograd}& 97.38&94.73&92.50&94.66&94.82&79.51&84.27&73.50&84.22&80.38&77.81&81.58&81.42&84.68&81.37\\
         &TPT \cite{tpt}&97.61&94.50&93.44&95.01&95.14 &79.60&84.10&73.59&85.00& 80.57 &78.05&81.70&81.26&84.71&  81.43\\

        &\cellcolor{LightCyan}\textsc{StyLIP} &\cellcolor{LightCyan}\textbf{98.54}&\cellcolor{LightCyan}\textbf{96.76}&\cellcolor{LightCyan}\textbf{96.09}&\cellcolor{LightCyan}\textbf{96.71}&\cellcolor{LightCyan}\textbf{97.03} &\cellcolor{LightCyan}\textbf{82.56}&\cellcolor{LightCyan}\textbf{86.21}&\cellcolor{LightCyan}\textbf{75.80}&\cellcolor{LightCyan}\textbf{87.02}& \cellcolor{LightCyan}\textbf{82.90}&\cellcolor{LightCyan}\textbf{81.12}&\cellcolor{LightCyan}\textbf{83.45}&\cellcolor{LightCyan}\textbf{84.70}&\cellcolor{LightCyan}\textbf{86.29}&\cellcolor{LightCyan}\textbf{83.89}\\
        \bottomrule
    \end{tabular}}

    \label{tab:app_ssdg}
    }
\end{table*}

\begin{table*}[!ht]

\centering
\scriptsize{
    \centering
    \caption{Analysis of the generalization from base to new classes across domains. We show results on Office-Home with \textit{Product} acting as the source domain, while others denote the target. For DomainNet, we consider \textit{Painting} as the source domain and the rest used as target domains. The model is trained (backbone CLIP ViT-B/16) using $16$-shots from the base classes. (In $\%$)}
    \scalebox{0.65}{
    \begin{tabular}{lcccccccccccc}
    \toprule
     \multirow{4}{*}{\textbf{Method}}&\multicolumn{5}{c}{\textbf{Office-Home}}&\multicolumn{7}{c}{\textbf{DomainNet}}\\\cmidrule(lr){2-6}\cmidrule(lr){7-13}
    &\multicolumn{1}{c}{\textbf{Base}}&\multicolumn{4}{c}{\textbf{New}}&\multicolumn{1}{c}{\textbf{Base}}&\multicolumn{6}{c}{\textbf{New}}\\\cmidrule(lr){2-2}\cmidrule(lr){3-6}\cmidrule(lr){7-7}\cmidrule(lr){8-13}
        ~ & \textit{\textbf{Product}} & \textbf{Art} & \textbf{Clip Art} & \textbf{Product} & \textbf{Real World} &\textit{\textbf{Painting}}&\textbf{Clip Art}&\textbf{Infograph}&\textbf{Painting}&\textbf{Quick Draw}&\textbf{Real}&\textbf{Sketch}\\
        \midrule

        CLIP \cite{clip}  & 92.42 &62.01 &77.78 & 87.52 & 88.02 &72.09&76.55&49.80&70.84&17.56&88.11&66.54\\
        CoOp \cite{coop}& 94.66 &71.82 & 84.39 & 91.50&88.83&79.82&71.27&45.18&71.45&14.55&82.08&62.94\\
        CoCoOp \cite{cocoop}& 95.85 & 71.48 &83.64 & 89.90 & 87.99 &75.37&72.51&48.21&74.46&18.11&85.89&66.95\\
         CLIP-Adapter \cite{clip-adapter}& 94.62 &73.67 & 84.57 & 93.30 & 90.95 &79.68&74.99&48.91&75.00&18.14&87.20&66.40\\
         DPL \cite{amortized}& 95.31 & 71.42 &84.91 &92.67 &88.37 &78.52&73.47&49.05&73.62&17.83&86.82&66.58\\
         ProGrad \cite{prograd}& 96.75 &72.13 &85.29 &91.14&89.50&79.66&74.80  &47.15&73.86  &18.67 &85.61&66.79\\

       \cellcolor{LightCyan}\textsc{StyLIP}&\cellcolor{LightCyan}\textbf{97.69} & \cellcolor{LightCyan}\textbf{73.91} & \cellcolor{LightCyan}\textbf{87.30} & \cellcolor{LightCyan}\textbf{94.25} & \cellcolor{LightCyan}\textbf{91.88} &\cellcolor{LightCyan}\textbf{80.28}&\cellcolor{LightCyan}\textbf{76.72}&\cellcolor{LightCyan}\textbf{51.25}&\cellcolor{LightCyan}\textbf{76.57}&\cellcolor{LightCyan}\textbf{19.32}&\cellcolor{LightCyan}\textbf{88.53}&\cellcolor{LightCyan}\textbf{67.51}\\ \bottomrule
    \end{tabular}}\label{app_tab:b2n_3}}

\end{table*}

\begin{table*}[!ht]

\centering
\scriptsize{
    \centering
    \caption{Analysis of the multi-scale features on the meta networks for the generalization from base to new classes across domains. We show results on DomainNet with \textit{ClipArt} acting as the source domain, while others denote the target. The model is trained (backbone CLIP ViT-B/16) using $16$-shots from the base classes. (In $\%$)}
    \scalebox{0.7}{
    \begin{tabular}{lccccccc}
    \toprule
     \multirow{4}{*}{\textbf{Method}}&\multicolumn{7}{c}{\textbf{DomainNet}}\\\cmidrule(lr){2-8}
    &\multicolumn{1}{c}{\textbf{Base}}&\multicolumn{6}{c}{\textbf{New}}\\\cmidrule(lr){2-2}\cmidrule(lr){3-8}
        ~ & \textit{\textbf{Clip Art}}&\textbf{Clip Art}&\textbf{Infograph}&\textbf{Painting}&\textbf{Quick Draw}&\textbf{Real}&\textbf{Sketch}\\

        \midrule

        CoOp \cite{coop} &82.79&75.60&48.60&71.38&20.90&85.19&67.39\\
        MS-CoOp &83.43&75.91&49.15&71.80&20.92&86.02&67.77\\

        CoCoOp \cite{cocoop} &82.85&77.40&52.61&72.06&20.80&88.00&68.12\\
        MS-CoCoOp &84.02&77.58&52.39&72.45&20.67&87.51&68.30\\

       \cellcolor{LightCyan}\textsc{StyLIP}&\cellcolor{LightCyan}\textbf{84.90}&\cellcolor{LightCyan}\textbf{78.14}&\cellcolor{LightCyan}\textbf{53.09}&\cellcolor{LightCyan}\textbf{73.60}&\cellcolor{LightCyan}\textbf{21.69}&\cellcolor{LightCyan}87.90&\cellcolor{LightCyan}\textbf{68.61}\\ \bottomrule
    \end{tabular}}\label{app_tab:b2n}}

\end{table*}

\end{appendices}
\clearpage

redpajama_set_name: RedPajamaArXiv