text:
France finds wreckage of submarine which disappeared in 1968(1/5)
The AsterX submarine drone returns on board the Antea research vessel during the second phase of the search for the wreckage of the Minerve submarine, in this image provided by the Marine Nationale on July 22, 2019. The French Navy has found the wreckage of its 'Minerve' submarine, which disappeared off the coast of Toulon in the Mediterranean Sea in 1968, leading to the loss of 52 lives. (Photo/Agencies)
Launching of the AsterX submarine drone from the Antea research vessel during the second phase of the search for the wreckage of the Minerve submarine, in this image provided by the Marine Nationale on July 22, 2019. The French Navy has found the wreckage of its 'Minerve' submarine, which disappeared off the coast of Toulon in the Mediterranean Sea in 1968, leading to the loss of 52 lives. (Photo/Agencies)
Staff of the Hydrographic & Oceanographic Service of the Navy (SHOM) analyse the data sent in by the AsterX submarine drone aboard the Antea research vessel during the second phase of the search for the wreckage of the Minerve submarine, in this image provided by the Marine Nationale on July 22, 2019. The French Navy has found the wreckage of its 'Minerve' submarine, which disappeared off the coast of Toulon in the Mediterranean Sea in 1968, leading to the loss of 52 lives. (Photo/Agencies)
The AsterX submarine drone returns on board the Antea research vessel during the second phase of the search for the wreckage of the Minerve submarine, in this image provided by the Marine Nationale on July 22, 2019. The French Navy has found the wreckage of its 'Minerve' submarine, which disappeared off the coast of Toulon in the Mediterranean Sea in 1968, leading to the loss of 52 lives.
In pics: horse lassoing in north China's Inner Mongolia
Grave of 'real life Asterix' discovered at building site
France finds wreckage of submarine which disappeared in 1968
Trainees attend courses to become 1st batch of female bullet train drivers in China
Horse lassoing in north China's Inner Mongolia
Chinese soldiers in training for international games
First 25 companies start trading on sci-tech innovation board
Grandpa crafts unique wooden school bag
Pingtan cross-strait bridge fully connected
Rain forces closure of Mogao Grottoes again
Maginificant Jiaozhou Bay Bridge
Calgary Zoo awaits new panda cub


Kalyani subdivision is an administrative subdivision of the Nadia district in the state of West Bengal, India.
Nadia district is part of the large alluvial plain formed by the Ganges-Bhagirathi system. The plains spread southwards from the head of the delta. The Kalyani subdivision has the Bhagirathi on the west, with Hooghly district lying across the river. Topographically, Kalyani subdivision is part of the Ranaghat-Chakdaha Plain, the low-lying area found in the south-eastern part of the district. The area slopes southwards. The area had large forests. The huge influx of East Bengali refugees that took place in the district immediately after the partition of India and the steady influx ever since paved way for conversion of forest into agricultural land.
Kalyani subdivision has 4 police stations, 2 community development blocks, 2 panchayat samitis, 27 gram panchayats, 245 mouzas, 219 inhabited villages 4 municipalities and 13 census towns. The municipalities are: Kalyani, Chakdaha, Haringhata, and Gayespur.The census towns are: Punglia, Darappur, Lalpur, Chanduria, Simurali, Priyanagar, Jangal, Madanpur, Saguna, Kulia, Simhat, Subarnapur and Digha.
The subdivision has its headquarters at Kalyani.
Chakdaha block consists of 17 gram panchayats, viz. Chanduria–I, Hingara, Saguna, Tatla–I, Chanduria–II, Kancharapara, Sarati, Tatla–II, Dewli, Madanpur–I, Silinda–I, Dubra, Madanpur–II, Silinda–II, Ghetugachhi, Rautari and Simurali.
Haringhata block consists of ten gram panchayats, viz. Berohi–I, Haringhata–I, Kastodanga–II, Nagarukhra–I, Berohi–II, Haringhata–II, Mollabelia, Fatepur, Kastodanga–I and Nagarukhra–lI.
Note: Primary schools include junior basic schools; middle schools, high schools and higher secondary schools include madrasahs; technical schools include junior technical schools, junior government polytechnics, industrial technical institutes, industrial training centres, nursing training institutes etc.; technical and professional colleges include engineering colleges, medical colleges, para-medical institutes, management colleges, teachers training and nursing training colleges, law colleges, art colleges, music colleges etc. Special and non-formal education centres include sishu siksha kendras, madhyamik siksha kendras, centres of Rabindra mukta vidyalaya, recognised Sanskrit tols, institutions for the blind and other handicapped persons, Anganwadi centres, reformatory schools etc.
University of Kalyani, is an affiliating, teaching and research university, established at Kalyani in 1960.
Bidhan Chandra Krishi Viswavidyalaya, an agricultural university, was established at Mohanpur in 1974.
Kalyani Government Engineering College, was established at Kalyani in 1995.
JIS College of Engineering, an autonomus institute, was established at Kalyani in 2000.
Indian Institute of Science Education and Research, an autonomous institute, was established at Mohanpur in 2006.
College of Medicine & JNM Hospital, a medical college, was established at Kalyani in 2009. It is affiliated with West Bengal University of Health Sciences.
Indian Institute of Information Technology, Kalyani, one of the twenty IIITs in the country, was established at Kalyani in 2014.
Kalyani Mahavidyalaya was established at Kalyani in 1999. Affiliated to the University of Kalyani, it offers honours courses in Bengali, English, history, economics,sociology, education, geography, computer science, molecular biology with bio-technology, micro biology, mathematics, chemistry, statistics and accountancy.
Chakdaha College was established at Chakdaha in 1972. Affiliated to the University of Kalyani, it offers honours courses in Bengali, English, Sanskrit, economics, history, political science, geography, physics, chemistry, mathematics, botany, zoology and accountancy.
Haringhata Mahavidyalaya was established at Haringhata in 1986. Affiliated to the University of Kalyani, it offers honours courses in Bengali, English, geography, education, history, political science and accountancy.
Snehansu Kanta Acharya Institute of Law was established at Kalyani in 2004. The institute is named after Snehansu Kanta Acharya, a legal luminary and a Marxist politician.
The table below (all data in numbers) presents an overview of the medical facilities available and patients treated in the hospitals, health centres and sub-centres in 2014 in Nadia district.
Haringhata Reserved for SC Haringhata community development block, and Dewli and Hingara gram panchayats of Chakdaha community development block.
^ "District Census Handbook, Nadia, 2011, Series 20, Part XII A" (PDF). Pages 13,14. Directorate of Census Operations, West Bengal. Retrieved 28 May 2017.
^ a b "District Statistical Handbook 2014 Nadia". Table 2.2, 2.4(a). Department of Statistics and Programme Implementation, Government of West Bengal. Archived from the original on 29 July 2017. Retrieved 16 May 2017.
^ a b "District Statistical Handbook 2014 Nadia". Table 2.1. Department of Statistics and Programme Implementation, Government of West Bengal. Archived from the original on 29 July 2017. Retrieved 16 May 2017.
^ "C.D. Block Wise Primary Census Abstract Data(PCA)". West Bengal – District-wise CD Blocks. Registrar General and Census Commissioner, India. Retrieved 16 May 2017.
^ "Directory of District, Subdivision, Panchayat Samiti/ Block and Gram Panchayats in West Bengal". Bardhaman - Revised in March 2008. Panchayats and Rural Development Department, Government of West Bengal. Retrieved 16 May 2017.
^ "District Statistical Handbook 2014 Nadia". Tables 2.1, 2.2. Department of Statistics and Programme Implementation, Government of West Bengal. Archived from the original on 29 July 2017. Retrieved 17 May 2017.
^ "Nadia District Police". Police Unit. West Bengal Police. Retrieved 17 May 2017.
^ "C.D. Block Wise Primary Census Abstract Data(PCA)". 2011 census: West Bengal – District-wise CD Blocks. Registrar General and Census Commissioner, India. Retrieved 16 May 2017.
^ "Directory of District, Subdivision, Panchayat Samiti/ Block and Gram Panchayats in West Bengal". Bardhaman - Revised in March 2008. Panchayats and Rural Development Department, Government of West Bengal. Retrieved 17 May 2017.
^ a b c "District Statistical Handbook 2014 Nadia". Basic data: Table 4.4, 4.5, Clarifications: other related tables. Department of Statistics and Programme Implementation, Government of West Bengal. Archived from the original on 29 July 2017. Retrieved 26 May 2017.
^ "University of Kalyani". KU. Retrieved 26 May 2017.
^ "Bidhan Chandra Krishi Viswavidyalaya". BCKV. Retrieved 26 May 2017.
^ "Kalyani Government Engineering College". KGEC. Retrieved 26 May 2017.
^ "JIS College of Engineering". JIS college. Retrieved 26 May 2017.
^ "Indian Institute of Science Education and Research Kolkata". IISER Kolkata. Retrieved 26 May 2017.
^ "College of Medicine & J.N.M. Hospital, West Bengal University of Health Sciences". West Bengal University of Health Sciences. Retrieved 26 May 2017.
^ "Indian Institute of Information Technology, Kalyani". IIIT Kalyani. Retrieved 26 May 2017.
^ "Kalyani Mahavidyalaya". KM. Retrieved 26 May 2017.
^ "Kalyani Mahavidyalaya". College Admission. Retrieved 26 May 2017.
^ "Chakdaha College". CC. Retrieved 26 May 2017.
^ "Chakdaha College". College Admission. Retrieved 26 May 2017.
^ "Haringhata Mahavidyalaya". HM. Retrieved 26 May 2017.
^ "Haringhata Mahavidyalaya". College Admission. Retrieved 26 May 2017.
^ "Snehansu Kanta Acharya Institute of Law". SKAIL. Retrieved 26 May 2017.
^ "District Statistical Handbook 2014 Nadia". Table 3.1, 3.3. Department of Statistics and Programme Implementation, Government of West Bengal. Archived from the original on 29 July 2017. Retrieved 2 March 2017.
^ a b c "Health & Family Welfare Department". Health Statistics. Government of West Bengal. Retrieved 30 May 2017.
^ "Delimitation Commission Order No. 18, 15 February 2006" (PDF). West Bengal. Election Commission of India. Retrieved 24 May 2017.
This page was last edited on 24 April 2019, at 06:31 (UTC).


text:
\section{Introduction}\label{sec:introduction}}
\else
\section{Introduction}
\label{sec:introduction}
\fi

\IEEEPARstart{T}{his} demo file is intended to serve as a ``starter file''
for IEEE Computer Society journal papers produced under \LaTeX\ using
IEEEtran.cls version 1.8b and later.
I wish you the best of success.

\hfill mds

\hfill August 26, 2015

\subsection{Subsection Heading Here}
Subsection text here.

\subsubsection{Subsubsection Heading Here}
Subsubsection text here.

\section{Conclusion}
The conclusion goes here.

\appendices
\section{Proof of the First Zonklar Equation}
Appendix one text goes here.

\section{}
Appendix two text goes here.

\ifCLASSOPTIONcompsoc

  \section*{Acknowledgments}
\else

  \section*{Acknowledgment}
\fi

The authors would like to thank...

\ifCLASSOPTIONcaptionsoff
  \newpage
\fi

\section{Introduction}
This demo file is intended to serve as a ``starter file''
for IEEE Computer Society conference papers produced under \LaTeX\ using
IEEEtran.cls version 1.8b and later.
I wish you the best of success.

\hfill mds

\hfill August 26, 2015

\subsection{Subsection Heading Here}
Subsection text here.

\subsubsection{Subsubsection Heading Here}
Subsubsection text here.

\section{Conclusion}
The conclusion goes here.

\ifCLASSOPTIONcompsoc

  \section*{Acknowledgments}
\else

  \section*{Acknowledgment}
\fi

The authors would like to thank...

\section{Introduction}
Few-shot learning refers to the task of learning a new concept with only a few labeled samples,
which is inspired by human learning ability.
As labeling is often
expensive in real scenarios, few-shot learning has become an important and widely studied problem.
However, with little supervision information, learning to recognize new classes is challenging because directly training the model from a few samples may overfit.
A common idea in few-shot learning is to train the model from base classes with sufficient samples to get prior knowledge and then migrate to the novel classes with a few examples.
Existing studies on few-shot image classification can be roughly divided into three categories, namely fine-tuning based methods, data-augmentation based methods and meta-learning based methods.
Among them, the most widely studied is meta-learning \cite{hospedales2021meta}, which acquires the abstract learning ability to generalize to new classes by learning meta-knowledge from a set of different meta tasks.
Although significant progress has been made in few-shot learning,
most of existing few-shot learning methods require the network to be pre-trained in a supervised manner on a large amount of labeled data of base classes.
As a result, the current few-shot learning methods have limited generalization ability and is impractical in the real world
due to the shortage of supervised data.\par

\begin{figure}[t]
\centering
\includegraphics[width=1.0\linewidth]{Fig3.pdf}
\caption{
SgVA-CLIP vs. previous VLP-based few-shot learning methods.
(a) Previous VLP-based few-shot learning methods focus on enhancing the cross-modal alignment.
(b) SgVA-CLIP makes a comprehensive consideration of discriminative task-specific vision feature space and cross-modal feature space.}
\label{intro}
\end{figure}

Recently, self-supervised learning \cite{chen2020simple,he2022masked} has emerged as a possible solution to alleviate the dependency on large-scale labeled data.
Self-supervised learning exploits pretext tasks to mine supervised information from large-scale unsupervised data, thereby learning rich implicit priors and latent representations.
With the development of self-supervised learning,
Vision-Language Pre-trained models (VLPs),
e.g., CLIP \cite{radford2021learning}, ALIGN \cite{jia2021scaling}, and Florence \cite{yuan2021florence}, attract more and more attention due to its significant performance in a variety of downstream tasks, such as image classification and visual question answering.
VLPs like CLIP can provide effective visual and semantic knowledge of open-world concepts that are learned on large-scale image-text pairs, laying a good generalization foundation of few-shot image classification.\par

VLPs have been successfully applied to few-shot image classification
with the help of carefully designed text prompts \cite{radford2021learning}, which can change the discrete class labels into language sentences.
For example, CLIP model \cite{radford2021learning} learns vision and language representations by aligning the image and text in a cross-modal space, which allows images to be correctly classified via image-text similarity.
There are also VLPs-based few-shot learning methods that focus on enhancing the image-text alignment.
Context Optimization (CoOp) \cite{zhou2022learning} is proposed to improve the text embedding of CLIP by soft prompt engineering.
CLIP-Adapter \cite{gao2021clip} fine-tune the image representation by adjusting an extra bottleneck layer.
ProGrad \cite{BeierZhu2022PromptalignedGF} proposes Prompt-aligned Gradient to prevent prompt tuning from forgetting the the general
knowledge learned from VLPs.

Existing methods only consider the image-text alignment when transferring the VLPs to solve few-shot image classification.
Although relying on the image-text similarity can well capture the visual and semantic knowledge learned by the pre-trained model,
it is sometimes unreliable to recognize objects without comprehensively considering the specific discriminative visual information of the few-shot task.
The reason is that to learn a good image-text alignment model, the pre-trained VLPs may neglect detailed visual information that is difficult to describe by language sentences.
However, the neglected visual information is probably important for learning an effective classifier when the labeled samples are insufficient.

To address above problem, we propose a new framework, named Semantic-guided Visual Adapting (SgVA), which can effectively extend vision-language
pre-trained models (e.g., CLIP) to produce discriminative task-specific visual features with the guidance of the fine-grained cross-modal knowledge learned by the pre-trained model.
The adapted visual features can well complement the cross-modal features to improve few-shot image classification.
Figure \ref{intro} shows the main idea of our work.
Specifically,
our method is extended from the pre-trained CLIP.
Given labeled support images and unlabeled query images, we firstly extract the visual features for the images from the output before the cross-modal projection layer of the CLIP model, which are referred as pre-trained visual features. And cross-modal features for both the images and the prompted texts of the class labels are extracted from the output of the cross-modal projection layer.
Next, we map the pre-trained visual features to task-specific visual features by a visual adapting layer.
We update the visual adapting layer on the few-shot samples by a vision-specific contrastive loss and cross-modal contrastive loss
with the help of vision prototypes and cross-modal prototypes that are obtained by averaging the corresponding sample features of a given class.
Moreover, we adopt an implicit distillation loss to utilize the fine-grained cross-modal knowledge (i.e., relative similarities between samples and prototypes in the cross-modal space) learned by the pre-trained model to guide the updating of the vision adapter.
Finally, we infer the class label for a given query sample by jointly considering its distance to vision-specific prototypes and cross-modal prototypes.
\par

Our contributions are summarized as follows.
We propose a new framework of semantic-guided visual adapting,
which flexibly extends the vision-language pre-trained models (e.g., CLIP) to produce discriminative task-specific visual features by jointly using vision-specific contrastive loss, cross-modal contrastive loss, and implicit knowledge distillation.
We obtain new state-of-the-art results in few-shot image classification by comprehensively considering the sample relations based on both the adapted visual features and the cross-modal features,
which demonstrates a strong complementarity between the two kinds of feature space and also provides a promising direction for future research.

\section{Related Work}
This section reviews three topics closely related to our work in terms of few-shot learning, prototype networks and knowledge distillation.

\subsection{Few-shot Learning}
Few-shot learning aims to learn a model that can recognize new classes with a few training samples.
The widely studied conventional few-shot learning methods include {fine-tuning \cite{vedaldi2014convolutional,shen2021partial}, data/feature augmentation~\cite{kumar2019closer,yoon2019tapnet},
and meta learning \cite{hospedales2021meta,sun2019meta}}.
Recently,
Vision-language pre-trained models (VLPs) (e.g. CLIP \cite{radford2021learning} and ALIGN \cite{jia2021scaling}) have been applied to few-shot learning by transferring the powerful representation ability.
In order to realize data-efficient fine-tuning, CoOp \cite{zhou2022learning} improves the ability of image-text alignment through continuous prompt optimization,
and CLIP-Adapter \cite{gao2021clip} designs lightweight feature adapters to explore simple fine-tuning.
After that, Tip-Adapter \cite{zhang2021tip}, a training-free method, is proposed to save computational resources.
Different from them, VT-CLIP \cite{zhang2021vt} improves the interaction of image and text branches of CLIP by cross-modal module.
Other works like MUST \cite{li2022masked} and UPL \cite{huang2022unsupervised} think about unsupervised learning.
Besides, WiSE-FT \cite{wortsman2022robust} and CoCoOp \cite{zhou2022conditional} consider both the accuracy of target distribution and robustness to distribution shifts.
Unlike the above methods that focus on image-text contrastive learning, we extend the
pre-trained CLIP to learn more discriminative visual features that can well complement the cross-modal features in few-shot learning.

\subsection{Prototype Networks}
Prototype network \cite{snell2017prototypical} is proposed in 2017 to solve the problem of few-shot classification,
which aims at learning a metric space where query samples can be accurately classified by calculating the distances between queries and prototypes.
Compared with other few-shot learning methods, Prototype network reflects a simpler inductive bias, which is beneficial in the case of limited data.
Owing to the potential of this paradigm, many variations have been developed since then.
Chen \emph{et al}. \cite{chen2020new} found that introducing an extra pre-training phase on the entire base classes could improve performance, but it leads to poor generalization ability.
Early prototype networks only employ visual information, but increasingly there are approaches to explore how semantic knowledge can enhance the performance.
For example, Chen \emph{et al}. \cite{xing2019adaptive} learned semantic knowledge from unsupervised corpora, and proposed an adaptive modality mixing mechanism to combine the visual and semantics knowledge, showing improvements in few-shot learning.
Frederik \emph{et al}. \cite{pahde2021multimodal} mapped text data to the visual embedding space with the help of a generative model, and then designed a strategy to combine the real and generated features through the nearest neighbor algorithm.
Instead of only forming a single metric space as in existing prototype networks, we construct two metric spaces including visual and cross-modal spaces to comprehensively conduct the few-shot learning.

\subsection{Knowledge Distillation}
Knowledge Distillation (KD) \cite{bucilua2006model,hinton2015distilling} means transferring the knowledge from the pre-trained complex model (teacher model) to a simpler structured network (student model).
Owing to its superior performance in knowledge transferring and model enhancement, KD is widely used in model compression and transfer learning.
In the process of KD, the output of teacher model is used as the supervision signal to train the student model through the distillation loss.
And the optimization target is to make the class-level probability distribution of the student model match the probability output of the teacher model.
In terms of model compression,
DistillBert \cite{sanh2019distilbert} and TinyBert \cite{jiao2019tinybert} use KD to explore smaller and faster models for language representation learning.
In addition to model compression, KD also plays an important role in knowledge transferring between different modalities.
For example,  the vision-language distillation framework DistillVLM \cite{fang2021compressing} is proposed to improve vision-language tasks like image captioning and VQA.
Hafner \emph{et al}. \cite{hafner2022cross} propose a novel cross-modal distillation method for robust person re-identification, which transfers knowledge from RGB images to depth images.
Besides, in \cite{cho2020speech}, semantic knowledge is transmitted from language model to a spoken language understanding module, so that the deficiency of speech data can be alleviated.
The above works show the benefits of KD in both self-supervised learning and multi-modal alignment.
Different from existing methods, we propose a knowledge distillation which can implicitly transfer the fine-grained relation knowledge learned in the cross-modal space to the vision space by conducting the distillation in the cross-modal space.
\par

\section{The Proposed Method}
\subsection{Problem Definition}

{
In conventional few-shot learning,
sufficient samples of base classes and a few samples of novel classes are given.
The goal is to learn a good classifier that can generalize well on test samples of the novel classes.
In the popularly studied meta-learning mechanism, a series of meta-tasks (episodes) are created for training and testing.
For each meta-task in the training phase, N (Way) classes and K (Shot) samples for each class are randomly selected from the base classes to make up the Support Set $S = \left \{ \left ( x_{i}, y_{i}   \right )  \right \}_{i = 0}^{N\times K}$, where $x_{i}$ denotes the sampled image, and $y_{i}$ is the label of $x_{i}$.
Other M samples of the N (Way) classes
are randomly selected to form the Query Set $Q = \left \{ \left ( x_{i}, y_{i}   \right )  \right \}_{i = 0}^{M}$.
In the test phase, the labeled support set and the unlabeled query set are created for novel classes.

It is worth noting that since the pre-trained vision and language model, e.g., CLIP, can provide a good foundation model for the few-shot learning, the base classes are not indispensable in this work.
Without base classes,
following CLIP~\cite{radford2021learning}, the support set and query set used for model learning are randomly selected from the given training samples of novel classes.}

\subsection{Framework Overview}

\begin{figure*}[t]
\centering
\includegraphics[width=1.0\linewidth]{Fig1.pdf}
\caption{
Overview of the proposed network.
}
\label{model_overview}
\end{figure*}
The proposed framework, namely Semantic-guided Visual Adapting (SgVA), aims to learn discriminative task-specific visual features with the guidance of the cross-modal knowledge learned by the pre-trained CLIP model.
The adapted visual features can well complement the cross-modal features in the few-shot image classification.
As shown in Figure \ref{model_overview}, labeled support images and unlabeled query images are given.
The class label is transformed into text input by L-length learnable prompt vectors as in \cite{zhou2022learning}.
The images and texts go through the frozen image and text encoders respectively to generate the pre-trained visual features ($\boldsymbol {x_v}$) and text features ($\boldsymbol {x_t}$), which are then mapped to $\boldsymbol {x_{c\_v}}$ and
$\boldsymbol {x_{c\_t}}$ in the cross-modal space through different projection layers.
The useful fine-grained sample relations are included in the relative distances in the cross-modal feature space.
Meanwhile,
the pre-trained visual features ($\boldsymbol {x_v}$) are mapped to task-specific visual features ($\boldsymbol {x_a}$) by a visual adapting layer.
To fully exploit sample relations in the vision space and cross-modal space,
we build vision prototypes and cross-modal prototypes that are obtained by averaging the corresponding support sample features of a given class.
Based on the prototypes, we update the visual adapting layer on the few-shot samples by a vision-specific contrastive loss, a cross-modal contrastive loss and a knowledge distillation loss.
The knowledge distillation utilizes the fine-grained distance relationship of cross-modal features to guide the learning of the visual adapting.
In the inference process, test images can be recognized by jointly considering its distance to the vision-specific prototypes and cross-modal prototypes.

\subsection{Visual Adapting Layer}
The SgVA is a visual adapting framework that is not only constrained by the vision-specific contrastive loss in the unimodal visual space but also guided by the fine-grained knowledge in the cross-modal space, thus producing more discriminative visual features in the few-shot scenario.
The adapting layer consists of a two-layer Multi-layer Perceptron (MLP) and an adaptive residual connection.
The new feature $New_v(\boldsymbol{x_v})$ acquired in the adapting layer can be represented as:
\begin{equation}
New_{v}(\boldsymbol{x_v}) =  \mathrm{ReLU}(\boldsymbol{x_v}^{T} W_1)W_2, \label{new_feature} \end{equation}
where $\boldsymbol{x_v}$ denotes the pre-trained visual feature, $W_1$ and $W_2$ are learnable weights of the two fully connected layers, and ReLU is activation function.
Then, the new feature is added to the original visual feature by an adaptive residual connection:
\begin{equation} \boldsymbol{x_a} =  [New_{v}(\boldsymbol{x_v}), \boldsymbol{x_v}]W_a, \label{xa_form}\end{equation}
where $\boldsymbol{x_a}$ is the adapted visual feature, and the $W_a\in \mathbb{R}^{2}$ is a learnable weight vector. $[]$ denotes the operation of combining the vectors as a matrix.

\subsection{Cross-modal Contrastive Loss}
Firstly, cross-modal prototypes ($\boldsymbol {p_{c\_t}}$) of $N$ classes are calculated by averaging the cross-modal text features of the support samples of each class. Since the text embedding is the same for the same class, $\boldsymbol {p_{c\_t}}$ is represented by \begin{equation}
\mathbf{\boldsymbol{p}_{c\_t}}(k)=\boldsymbol {x_{c\_t}}(k),
\label{cross-modal prototype}
\end{equation}where $\boldsymbol {x_{c\_t}}(k)$ denotes the cross-modal text feature of the $k_{th}$ class.

Then, for every query sample $x$, the cosine similarity scores between the cross-modal visual feature  ($\boldsymbol {x_{c\_v}}$) and the cross-modal prototypes ($\boldsymbol {p_{c\_t}}$) are calculated.
The cosine similarity is scaled by a temperature parameter $\tau$, and normalized into a probability distribution via Softmax.
Then, the cross-modal contrastive loss $L_{cl \_i2t}$ is defined as the cross entropy loss over the probability distribution:
\begin{equation}
L_{cl \_i2t}=-log\frac{exp (< \boldsymbol{x_{c\_v}}, \boldsymbol{{p_{c\_t}}_{(+)}} > /\tau)}{ {\textstyle \sum_{j=0}^{N}} exp (<\boldsymbol{{x_{c\_v}}}, \boldsymbol{p_{c\_t}}(j) > /\tau)},
\label{i2t}
\end{equation}
where $\tau$ is a temperature parameter learned by CLIP \cite{radford2021learning}.
And $\boldsymbol{{p_{c\_t}}_{(+)}} $ denotes the the cross-modal prototype of the positive class,
$< \cdot , \cdot >$ denotes cosine similarity, and $N$ is the number of classes.
The useful implicit knowledge is included in the relative distances between cross-modal features.

\subsection{Vision-specific Contrastive Loss}
For every query sample $x$, the cosine similarity scores between the adapted visual feature ($\boldsymbol {x_{a}}$) and the vision prototypes ($\boldsymbol {p_{a}}$) of $N$ classes are calculated by: \begin{equation}
\mathbf{\boldsymbol{p}_{a}}(k)=\frac{1}{\left|S(k)\right|} \sum_{\left({x_{i}}, y_{i}\right) \in S(k)} \boldsymbol {x_{a}}(i),
\label{vision prototype}
\end{equation}where $S(k)$ denotes the set of all the support samples of class $k$.
The purpose of vision-specific contrastive loss is to maximize the cosine similarity between $\boldsymbol{x_{a}}$ and
the positive prototype
while minimizing the cosine similarity between $\boldsymbol {x_{a}}$ and negative prototypes.
Formally, the vision-specific contrastive loss $L_{cl \_i2i}$ is calculated by:
\begin{equation}
L_{cl \_i2i}=-log\frac{exp (< \boldsymbol{x_{a}}, \boldsymbol{{p_{a}}_{(+)}} > /\tau)}{ {\textstyle \sum_{j=0}^{N}} exp (<\boldsymbol{{x_{a}}}, \boldsymbol{p_{a}}(j) > /\tau)}.
\label{i2i}
\end{equation}

\subsection{Implicit Knowledge Distillation}

With the vision-specific contrastive loss,
we can already make the samples of the same class close to each other, and the samples of different classes far from each other in the task-specific unimodal vision space.
However, the vision contrastive loss cannot provide the fine-grained relations between different samples.
For example, for two negative samples of a query sample, although we can utilize the vision contrastive loss to push both of them far away from the query sample, it is unclear which negative sample is more distant from the query sample.
This kind of fine-grained sample relations are important for learning discriminative visual features especially when the samples are insufficient.
As a remedy,
we consider to take advantage of the relative sample relationships produced by the pre-trained CLIP model, which cannot be achieved by contrastive learning alone.
It is worth noting that when compared with the visual data, the semantic meaning of a given class is more likely to be shared by the pre-trained CLIP model and the downstream few-shot task.
Therefore, we adopt a knowledge distillation loss
to guide the learning of the vision adapter
by the fine-grained relationships of cross-modal features.

\begin{figure}[t]
\centering
\includegraphics[width=0.5\textwidth]{Fig2.pdf}
\caption{Interpretation of the knowledge distillation between task-specific vision space and cross-modal space. The adapted visual feature $\boldsymbol{x_{a}}$ is mapped to the cross-modal space as a proxy representation $\boldsymbol{x_{c\_a}}$. The knowledge distillation is performed in the cross-modal space by matching the fine-grained sample-prototype relations of the proxy representation $\boldsymbol{x_{c\_a}}$ and the cross-modal visual feature $\boldsymbol{x_{c\_v}}$.}
\label{fig2}
\end{figure}

Figure \ref{fig2} gives an illustration of the knowledge distillation loss.
By mapping the task-specific visual feature $\boldsymbol {x_{a}}$ to the cross-modal space, a proxy representation of it in the cross-modal space, namely $\boldsymbol {x_{c\_a}}$ is obtained.
Then, we can utilize the fine-grained relations of the cross-modal visual feature $\boldsymbol {x_{c\_v}}$ to constrain the proxy representation $\boldsymbol {x_{c\_a}}$.
This means that we can use distances between samples obtained in the cross-modal space as extra supervision to implicitly guide the learning of the vision adapting layer.
The knowledge distillation is performed in the cross-modal space, and the gradients are backpropagated to the visual adapting layer.

More specifically, for a cross-modal visual feature $\boldsymbol {x_{c\_v}}$,
we calculate its distances to each of the cross-modal text prototypes as a teacher:\begin{equation}
d_{teacher}(k)=< \boldsymbol{x_{c\_v}}, \boldsymbol{{p_{c\_t}}}(k) > /\tau,
\label{logits_i2t}
\end{equation}
where $\tau$ is a temperature parameter learned by the pre-trained CLIP.
The distances between $\boldsymbol {x_{c\_a}}$ and each of the cross-modal text prototypes $\boldsymbol{p_{c\_t}}$
are calculated as a student:
\begin{equation}
d_{student}(k)=< \boldsymbol{x_{c\_a}}, \boldsymbol{{p_{c\_t}}}(k) > /\tau.
\label{logits_newi2t}
\end{equation}
Finally, the knowledge distillation loss is defined as:\begin{equation}
\begin{small}
\begin{aligned}
L_{KD} \!=\! -\frac{1}{N} \sum_{k=0}^{N}s(d_{teacher}(k)/\tau_{2})log(s(d_{student}(k))/\tau_{2})),
\end{aligned}
\end{small}
\label{loss_kd}
\end{equation}
where $\tau_{2}$ is a hyperparameter,
and $s$ denotes Softmax.

\subsection{Optimization}
Our framework can be learned in an end-to-end form for few-shot image classification.
The framework is optimized by jointly considering vision-specific contrastive loss, cross-modal contrastive loss, and knowledge distillation loss.
The overall loss is defined as:
\begin{equation}
L = L_{cl \_i2t} + L_{cl \_i2i} + L_{KD}.
\label{overall_loss}
\end{equation}

\subsection{Inference}
For a given test sample, we firstly compute its distance to the vision-specific prototypes and cross-modal prototypes.
Then, we adaptively fuse the results predicted in the vision space and the cross-modal space.
The fusion module is Multinomial Naive Bayes without extra parameters, which outputs the prediction class $\hat{y}$ with the maximum posterior probability for the input of $<\boldsymbol{x_{c\_v},  p_{c\_t}}>$ and $<\boldsymbol{x_a,  p_a}>$.

\begin{table*}[t]
\scriptsize
\centering
\caption{Comparison with state-of-the-art few-shot learning methods (with base classes) on miniImagenet and tieredImagenet.}
\setlength{\tabcolsep}{3.15mm}
\renewcommand\arraystretch{1.3}
\begin{tabular}{l|c|cc|cc}
\toprule[1pt]
 &  & \multicolumn{2}{c|}{miniImagenet (\%)} & \multicolumn{2}{c}{tieredImagenet (\%)} \\ \cline{3-6}
\multirow{-2}{*}{Method} & \multirow{-2}{*}{\begin{tabular}[c]{@{}c@{}}Vision\\ Backbone\end{tabular}} & \multicolumn{1}{c|}{5-way 1-shot} & 5-way 5-shot & \multicolumn{1}{c|}{5-way 1-shot} & 5-way 5-shot \\ \hline
EPNet+SSL & WRN-28-10 & \multicolumn{1}{c|}{79.22 ± 0.92} & 88.05 ± 0.51 & \multicolumn{1}{c|}{83.69 ± 0.99} & 89.34 ± 0.59 \\
EASY & 3×ResNet12 & \multicolumn{1}{c|}{83.02 ± 0.23} & 88.57 ± 0.12 & \multicolumn{1}{c|}{84.29 ± 0.24} & 89.76 ± 0.14 \\
PT+MAP & {\color[HTML]{000000} WRN/DenseNet121} & \multicolumn{1}{c|}{{\color[HTML]{000000} 82.92 ± 0.26}} & {\color[HTML]{000000} 88.82 ± 0.13} & \multicolumn{1}{c|}{{\color[HTML]{000000} 85.67 ± 0.26}} & {\color[HTML]{000000} 90.45 ± 0.14} \\
cluster-FSL & WRN-28-10 & \multicolumn{1}{c|}{85.74 ± 0.76} & 90.18 ± 0.43 & \multicolumn{1}{c|}{82.63 ± 0.79} & 89.16 ± 0.35 \\
HCTransformers & ViT-S & \multicolumn{1}{c|}{74.62 ± 0.20} & 89.19 ± 0.13 & \multicolumn{1}{c|}{79.57 ± 0.20} & 91.72 ± 0.11 \\
PEMnE-BMS* & DenseNet121 & \multicolumn{1}{c|}{85.54} & 91.53 & \multicolumn{1}{c|}{86.07 ± 0.25} & 91.09 ± 0.14 \\
CLIP\_LP+LN & {\color[HTML]{000000} ViT-B/16} & \multicolumn{1}{c|}{{\color[HTML]{000000} 92.08}} & {\color[HTML]{000000} 97.94} & \multicolumn{1}{c|}{{\color[HTML]{000000} -}} & {\color[HTML]{000000} -} \\
P\textgreater{}M\textgreater{}F (with ext. data) & ViT-B/16 & \multicolumn{1}{c|}{95.30} & 98.40 & \multicolumn{1}{c|}{-} & - \\
\textbf{SgVA-CLIP (ours)} & {\color[HTML]{000000} ViT-B/16} & \multicolumn{1}{c|}{{\color[HTML]{000000} \textbf{97.95 ± 0.19}}} & {\color[HTML]{000000} \textbf{98.72 ± 0.13}} & \multicolumn{1}{c|}{{\color[HTML]{000000} \textbf{95.73 ± 0.37}}} & {\color[HTML]{000000} \textbf{96.21 ± 0.37}} \\ \bottomrule[1pt]
\end{tabular}
\label{table1}
\end{table*}

\begin{table*}
\scriptsize
\caption{The benefit of knowledge distillation.}
\centering
\setlength{\tabcolsep}{1.85mm}
\renewcommand\arraystretch{1.3}
\begin{tabular}{l|c|c|c|c|c|c|c}
\toprule[1pt]
 &  &  &  &  &  &  &  \\
\multirow{-2}{*}{Method} & \multirow{-2}{*}{$L_{KD}$} & \multirow{-2}{*}{\begin{tabular}[c]{@{}c@{}}Vision\\ Backbone\end{tabular}} & \multirow{-2}{*}{\begin{tabular}[c]{@{}c@{}}ImageNet (\%)\\ 16-shot\end{tabular}} & \multirow{-2}{*}{\begin{tabular}[c]{@{}c@{}}SUN397 (\%)\\ 16-shot\end{tabular}} & \multirow{-2}{*}{\begin{tabular}[c]{@{}c@{}}Vision\\ Backbone\end{tabular}} & \multirow{-2}{*}{\begin{tabular}[c]{@{}c@{}}ImageNet (\%)\\ 16-shot\end{tabular}} & \multirow{-2}{*}{\begin{tabular}[c]{@{}c@{}}SUN397 (\%)\\ 16-shot\end{tabular}} \\ \hline
Vision-specific w/o KD & × &  & 63.61 & {\color[HTML]{343434} 69.00} &  & {\color[HTML]{000000} 50.11} & {\color[HTML]{000000} 61.70} \\
\textbf{Vision-specific w/ KD} & \checkmark &  & \textbf{66.95 (+3.34)} & {\color[HTML]{343434} \textbf{71.22 (+2.22)}} &  & \textbf{58.75 (+8.64)} & \textbf{66.56 (+4.86)} \\ \cline{1-2} \cline{4-5} \cline{7-8}
SgVA-CLIP w/o KD & × &  & 72.94 & 76.12 &  & 64.42 & 71.24 \\
\textbf{SgVA-CLIP w/ KD} & \checkmark & \multirow{-4}{*}{ViT-B/16} & {\color[HTML]{000000} \textbf{73.30 (+0.36)}} & {\color[HTML]{000000} \textbf{76.42 (+0.30)}} & \multirow{-4}{*}{ResNet50} & {\color[HTML]{000000} \textbf{65.70 (+1.28)}} & {\color[HTML]{000000} \textbf{71.99 (+0.75)}} \\   \bottomrule[1pt]
\end{tabular}
\label{table3}
\end{table*}

\begin{figure*}[h]

\centering
\includegraphics[width=1\linewidth]{Fig4.pdf}
\caption{Comparison with state-of-the-art few-shot learning methods (without base classes) on 11 datasets.}
\label{Fig4}
\end{figure*}

\section{Experiment and Results}
\subsection{Datasets}
For the standard meta-learning scenario, we choose miniImagenet \cite{NIPS2016_90e13578} and tieredImagenet \cite{ren2018meta}, which are common benchmarks for few-shot learning.
The purpose is to validate the generalization ability of our model on novel classes.
The miniImagenet dataset contains 100 classes sampled from ILSVRC-2012 \cite{russakovsky2015imagenet}, and is split to 64, 16, 20 classes for training, validation, and testing respectively.
Similarly, the tieredImagenet dataset includes 608 classes sampled from ILSVRC-2012, and is divided into 351, 97, 160 classes for training, validation, and testing respectively.
The setting of tieredImagenet is more challenging, because the base classes and novel classes come from different super categories. \par

For the scenario without base classes, we follow CLIP \cite{radford2021learning} and CoOp \cite{zhou2022learning} to select 11 image classification datasets to evaluate the performance, namely ImageNet \cite{deng2009imagenet}, StanfordCars \cite{krause20133d}, UCF101 \cite{soomro2012ucf101}, Caltech101 \cite{fei2004learning}, Flowers102 \cite{nilsback2008automated}, SUN397 \cite{xiao2010sun}, DTD \cite{cimpoi2014describing}, EuroSAT \cite{helber2019eurosat}, FGVCAircraft \cite{maji2013fine}, OxfordPets \cite{parkhi2012cats}, and Food101 \cite{bossard2014food}.
These datasets cover a series of diverse visual tasks including the classification of general objects, scenes, actions, and fine-grained categories.

\subsection{Implementation Details}
For the conventional setting of few-shot learning,
we evaluate our model on two widely used datasets, i.e., miniImagenet and tieredImagenet.
We train the overall framework on base classes for 100 epochs with 5-way 1-shot/5-shot tasks.
And in each episode, 15 query images per class are randomly sampled.
In test phase, we randomly 600 episodes on novel classes, and report the mean accuracy together with the 95\% confidence interval.
For the setting without base classes,
we follow the few-shot evaluation protocol adopted in CLIP~\cite{radford2021learning} to evaluate the model performance on 11 datasets, and set up 1, 2, 4, 8, 16 shots from all the classes to train the model and then test it on full test set.
The optimizer is SGD with momentum of 0.9 and weight decay of 0.0005.
The temperature in implicit knowledge distillation is set to 5, and the length of prompt vectors is set to 4.
We conduct all experiments on a single Nvidia V100 GPU.

\subsection{Baselines}
For the miniImagenet and tieredImagenet datasets, we compare our method with 8 baselines, including PEMnE-BMS* \cite{hu2022squeezing}, HCTransformers \cite{he2022attribute}, CLIP\_LP+LN \cite{kim2021adapt}, P\textgreater{M}\textgreater{F} \cite{hu2022pushing}, cluster-FSL \cite{ling2022semi}, PT+MAP \cite{hu2021leveraging}, EPNet \cite{rodriguez2020embedding} and EASY \cite{bendou2022easy} which are state-of-the-art methods on miniImagenet and tieredImagenet.
For the other 11 datasets, we compare our method with 5 baselines, namely Zero-shot CLIP \cite{radford2021learning}, CoOp \cite{zhou2022learning} and CLIP-Adapter \cite{gao2021clip}, Tip-Adapter-F \cite{zhang2021tip}, VT-CLIP \cite{zhang2021vt}, which are state-of-the-art few-shot learning methods based on VLP models.
For fair comparison, we follow their way of dataset splitting and adopt classification accuracy as the evaluation metric.

\subsection{Performance Comparison}
Table \ref{table1} shows the results
of our SgVA-CLIP and other state-of-the-art methods on miniImagenet and tieredImagenet.
Compared to the other methods,
our SgVA-CLIP achieves new state-of-the-art performance on both miniImagenet and tieredImageNet.
Although the P\textgreater{M}\textgreater{F} \cite{hu2022pushing} uses extra data, SgVA-CLIP still outperforms it by 2.65\% and 0.32\% respectively on the 5-way 1-shot and 5-way 5-shot tasks of miniImagenet.
Compared with CLIP\_LP+LN \cite{kim2021adapt}, which uses the same pre-trained parameters of CLIP as ours, SgVA-CLIP exceeds it by 5.87\% and 0.78\% respectively on the miniImagenet.
Besides, on the tieredImageNet, SgVA-CLIP surpasses the second best result by 9.66\% and 4.49\% respectively.
It is worth noting that the improvement of our method is larger when the number of shots is fewer, which demonstrates its generality capability in few-shot learning.\par

Figure \ref{Fig4} shows the comparison results of our method
and 5 recent baselines over 11 datasets with 1, 2, 4, 8, 16 shots.
For fair comparison, we adopt the vision backbone ResNet50 from CLIP as in \cite{zhou2022learning,gao2021clip,zhang2021tip,zhang2021vt}.
Compared with CoOp \cite{zhou2022learning}, our method achieves considerable improvement.
CoOp only considers the cross-modal information extraction from a perspective of prompt learning, but neglects the vision-specific information that is more discriminative.
By comprehensively exploiting the vision information and the cross-modal information, our method outperforms CoOp on 11 datasets.
Typically, the average performance gains over all datasets with 1, 2, 4, 8, 16 shots are 6.45\%, 5.07\%, 4.19\%, 3.87\% and 3.32\% respectively. Besides, the average gains over all the shots has reached 7.9\% and 7.09\% respectively on the Food101 and FGVAircraft dataset. \par

{
Compared with ProGrad \cite{BeierZhu2022PromptalignedGF}, which proposes a prompt-aligned gradient updating scheme, our SgVA-CLIP achieves significant performance and exceeds it by 2.21\% on average. Typically, the mean performance gains over 11 datasets with 1, 2, 4, 8, 16 shots are 2.15\%, 2.24\%, 1.93\%, 2.21\% and 2.53\% respectively.
Although a novel gradient updating strategy is proposed from the perspective of overcoming the improperly biased tuning, ProGrad gains limited improvement and cannot capture the more discriminative visual information.
}

Compared with CLIP-Adapter \cite{gao2021clip}, which is a visual adapter based on CLIP, our SgVA-CLIP surpasses it on most datasets.
The average performance gains over CLIP-Adapter across all shots on the 11 datasets is 1.97\%. By training an extra visual bottleneck layer, CLIP-Adapter can enhance the alignment of the visual features with the text features.
However, it only considers the visual information that is related to text.
Different from these two methods, our SgVA-CLIP proposes a semantic-guided adapting mechanism, which produces more discriminative visual features that can well complement the cross-modal features, showing a more promising perspective for few-shot learning.\par

The overall experiment results
demonstrate that the implicit knowledge distillation can promote the visual adapting and produce more effective task-specific visual features for few-shot learning.

\subsection{Ablation Study}

\textbf{Influence of the implicit knowledge distillation.}
To evaluate the effectiveness of the adopted knowledge distillation, we conduct experiment on ImageNet and SUN397.
The results are shown in Table \ref{table3}.
We can observe that with ResNet50 as the vision backbone, KD improves the accuracy of vision-based model by 8.64\% on ImageNet and 4.86\% on SUN397.
And KD enhances the SgVA-CLIP by 1.28\% and 0.75\% respectively.
We also note that when ResNet50 is used as the vision backbone, the performance gain of KD is greater than using the ViT-B/16 backbone.

\begin{table}[t]
\centering
\caption{Ablation study of direct and implicit knowledge distillation.}
\renewcommand\arraystretch{1.3}
\scriptsize
\centering
\setlength{\tabcolsep}{2.2mm}
\begin{tabular}{c|c|c|c|c}
\toprule[1pt]
\begin{tabular}[c]{@{}c@{}}Knowledge\\ Distillation\end{tabular} & \begin{tabular}[c]{@{}c@{}}Vision\\ Backbone\end{tabular} & \begin{tabular}[c]{@{}c@{}}ImageNet (\%)\\ 16-shot\end{tabular} & \begin{tabular}[c]{@{}c@{}}SUN397 (\%)\\ 16-shot\end{tabular} & Average (\%) \\
\hline
Direct & \multirow{2}{*}{ViT-B/16} & 68.98 & 75.70 & 72.34 \\
Implicit &  & \textbf{73.30} & \textbf{76.42} & \textbf{74.86} \\
\hline
Direct & \multirow{2}{*}{ResNet50} & 59.88 & 66.38 & 63.13 \\
\textbf{Implicit} &  & \textbf{65.64} & \textbf{71.99} & \textbf{68.82} \\
\bottomrule[1pt]
\end{tabular}
\label{table4}
\end{table}

\textbf{Implicit knowledge distillation vs. direct distillation.}
The direct distillation uses the Kullback-Leibler divergence to
match the sample relations in the cross-modal space
and the sample relations in the vision space.
As shown in Table \ref{table4}, compared with direct distillation, the proposed implicit knowledge distillation has an average performance gain of {2.52\% and 5.69\% respectively on ViT-B/16 and ResNet50 backbone}.
Because in the direct distillation,
the distribution gap between the two spaces may have a negative impact on the distillation.

\begin{table}[t]
\scriptsize
\centering
\caption{The complementarity between the vision space and the cross-modal space.}
\renewcommand\arraystretch{1.3}
\setlength{\tabcolsep}{2pt}
\begin{tabular}{c|c|c|c|c|c|c}
\toprule[1pt]
\multirow{2}{*}{Method} & \multirow{2}{*}{\begin{tabular}[c]{@{}c@{}}Vision \\ Backbone\end{tabular}} & \multirow{2}{*}{$L_{cl\_i2t}$} & \multirow{2}{*}{$L_{cl\_i2i}$} & \multicolumn{1}{l|}{\multirow{2}{*}{$L_{KD}$}} & ImageNet & SUN397 \\ \cline{6-7}
 &  &  &  & \multicolumn{1}{l|}{} & 16-shot & 16-shot \\ \hline
Vision-specific Prediction & ViT-B/16 & × & \checkmark & × & 63.61 & 69.00 \\
Cross-modal Prediction & ViT-B/16 & \checkmark & × & × & 71.48 & 73.50 \\
\textbf{Fused Prediction} & ViT-B/16 & \checkmark & \checkmark & × & \textbf{72.94} & \textbf{76.12} \\ \bottomrule[1pt]
\end{tabular}
\label{table2}
\end{table}

\textbf{Complementarity between the vision space and the cross-modal space.}
We compare the classification results on ImageNet and SUN397 obtained using the visual features and/or the cross-modal features.
The results are shown in Table \ref{table2}, which demonstrate that
comprehensively considering the visual feature and the cross-modal feature is better than simply using the one of them.
For the 16-shot learning task, fusing the results predicted from the vision features and the cross-modal features will increase the accuracy by 1.46\% and 2.62\% on ImageNet and SUN397 when compared with the cross-modal results, and the accuracy is increased by 9.33\% and 7.12\% respectively when compared with the vision-based results.\par

\subsection{The results of SgVA-CLIP on  different vision backbones}
The results in Figure 4 of the main text are based on the backbone ResNet50 for fair comparison with other methods, but SgVA-CLIP is also effective on other vision backbones.
Considering that only CoOp \cite{zhou2022learning} did a comprehensive analysis experiment of ViT-B/16, ViT-B/32, ResNet50 and ResNet101 backbones, we report more comparison results with CoOp as shown in Table \ref{table7}.
SgVA-CLIP surpasses CoOp by {2.74\%, 1.86\%, 1.53\% and 1.25\%} on average respectively on ViT-B/16, ViT-B/32, ResNet50 and ResNet101 backbones.

\begin{table}[t]
\renewcommand\arraystretch{1.05}
\scriptsize
\centering
\caption{Comparison of different vision backbones.}
\renewcommand\arraystretch{1.1}
\begin{tabular}{c|c|c|c|c}
\toprule[1pt]
\multirow{2}{*}{\begin{tabular}[c]{@{}c@{}}Vision\\Backbone\end{tabular}} & \multirow{2}{*}{Method} & ImageNet (\%) & SUN397 (\%) & \multirow{2}{*}{Average (\%)} \\
\cline{3-4}
 &  & 16-shot & 16-shot &  \\
\hline
\multirow{2}{*}{ResNet50} & CoOp & 62.95 & 69.26 & 66.11 \\
 & \textbf{SgVA-CLIP} & \textbf{65.70} & \textbf{71.99} & \textbf{68.85} \\
\hline
\multirow{2}{*}{ResNet101} & CoOp & 66.60 & 71.19 & 68.90 \\
 & \textbf{SgVA-CLIP} & \textbf{68.51} & \textbf{73.00} & \textbf{70.76} \\
\hline
\multirow{2}{*}{ViT-B/32} & CoOp & 66.85 & 72.38 & 69.62 \\
 & \textbf{SgVA-CLIP} & \textbf{68.26} & \textbf{74.04} & \textbf{71.15} \\
\hline
\multirow{2}{*}{ViT-B/16} & CoOp & 71.92 & 75.29 & 73.61 \\
 & \textbf{SgVA-CLIP} & \textbf{73.30} & \textbf{76.42} & \textbf{74.86} \\
\bottomrule[1pt]
\end{tabular}
\label{table7}
\end{table}

\subsection{Parameter analysis of the temperature $\tau_{2}$ in distillation}
To analyze the effect of the temperature $\tau_{2}$ in distillation,
we conduct experiments on ImageNet and SUN397 with different settings.
It is worth noting that the tenperature $\tau$ used in the cross-modal contrastive loss and vision-specific contrastive loss is outside the scope of parameter analysis because it is a pre-trained parameter of CLIP.
The temperature $\tau_{2}$ controls the smoothness of the soft labels.
As shown in Table \ref{table5}, we obtain the best performance when $\tau_{2}$ is 5.

\begin{table}[t]
\centering
\caption{Effect of temperature in knowledge distillation.}
\renewcommand\arraystretch{1.1}
\setlength{\tabcolsep}{1.64mm}
\scriptsize
\begin{tabular}{c|c|c|c|c}
\toprule[1pt]
\begin{tabular}[c]{@{}c@{}}Distillation\\ Temperature $\tau_{2}$\end{tabular} & \begin{tabular}[c]{@{}c@{}}Vision\\ Backbone\end{tabular} & \begin{tabular}[c]{@{}c@{}}ImageNet (\%)\\ 16-shot\end{tabular} & \begin{tabular}[c]{@{}c@{}}SUN397 (\%)\\ 16-shot\end{tabular} & Average (\%) \\
\hline
5 & \multirow{5}{*}{ResNet50} & 65.64 & \textbf{71.99} & \textbf{68.82} \\
10 &  & \textbf{65.70} & 71.86 & 68.78 \\
15 &  & 65.28 & 71.58 & 68.43 \\
20 &  & 64.78 & 71.49 & 68.14 \\
25 &  & 64.63 & 71.37 & 68.00 \\ \bottomrule[1pt]

\end{tabular}
\label{table5}
\end{table}

\begin{table}[t]
\renewcommand\arraystretch{1.05}
\scriptsize
\centering
\caption{Effect of the hidden dimension in the
visual adapting layer.}
\setlength{\tabcolsep}{1.5mm}
\begin{tabular}{l|c|c|c|c}
\toprule[1pt]
\multirow{2}{*}{\begin{tabular}[c]{@{}c@{}}Vision\\  Backbone\end{tabular}} & \multirow{2}{*}{\begin{tabular}[c]{@{}c@{}}Hidden\\ Dimension\end{tabular}} & ImageNet (\%) & SUN397 (\%) & \multirow{2}{*}{Average (\%)} \\ \cline{3-4}
 &  & 16-shot & 16-shot &  \\ \hline
\multirow{5}{*}{ResNet50} & 512 & 65.28 & 71.19 &  68.24\\
& 1024 & 65.16 & 71.37 & 68.27 \\
 & 2048 & 65.53 & 71.86 & 68.70 \\
 & \textbf{4096} & \textbf{65.64} & \textbf{71.99} & \textbf{68.82} \\
 & 8192 & 65.48 & 71.96 & 68.72 \\ \bottomrule[1pt]
\end{tabular}
\label{table6}
\end{table}

\subsection{Parameter analysis of the hidden dimension in the visual adapting layer}
Table~\ref{table6} shows the results of using different {hidden dimensions of the visual adapter layer}.
We observe that either too small or too large
dimension will deteriorate the performance and the best adapter dimension is 4096, which is able to preserve enough visual information without redundancy.
Therefore, we set the dimension to 4096 in the experiment.

\subsection{Visualization of the distribution of the vision-specific features}
{In Figure \ref{fig5}, we sample 10 classes} and display the distribution of the vision-specific features obtained by SgVA-CLIP and the cross-modal features obtained by CLIP.

The visualization results show that vision-specific features are more discriminative than the cross-modal features, and thus it is important to consider the adapted visual features in few-shot classification.

\begin{figure}[t]
\centering
\includegraphics[width=0.99\linewidth]{Fig5.pdf}
\caption{Visualization of the distribution of the vision-specific features and cross-modal features}
\label{fig5}
\end{figure}

\section{Conclusion}
We present SgVA-CLIP, a new VLP-based few-shot classification approach,
which can comprehensively consider unimodal vision correlation and cross-modal image-text correlation.
SgVA-CLIP focuses on the contrastive learning in two spaces and knowledge distillation between them, so that fine-grained cross-modal knowledge sharing can promote the learning of discriminative task-specific unimodal vision representations.
With the CLIP model frozen and only a few external parameters updated, the representation ability of CLIP can be quickly migrated to downstream classification tasks by a few labeled data.
According to the experimental results, SgVA-CLIP outperforms competitive baselines on 13 datasets under different few-shot settings.
In future work, we will combine SgVA-CLIP with other efficient tuning methods and explore the application of SgVA-CLIP in more downstream tasks.

\ifCLASSOPTIONcaptionsoff
  \newpage
\fi

\bibliographystyle{ieeetr}

\section{Introduction}\label{sec:introduction}}

\IEEEPARstart{T}{his} demo file is intended to serve as a ``starter file''
for IEEE Computer Society journal papers produced under \LaTeX\ using
IEEEtran.cls version 1.8b and later.
I wish you the best of success.

\hfill mds

\hfill August 26, 2015

\subsection{Subsection Heading Here}
Subsection text here.

\subsubsection{Subsubsection Heading Here}
Subsubsection text here.

\section{Conclusion}
The conclusion goes here.

\appendices
\section{Proof of the First Zonklar Equation}
Appendix one text goes here.

\section{}
Appendix two text goes here.

\ifCLASSOPTIONcompsoc

  \section*{Acknowledgments}
\else

  \section*{Acknowledgment}
\fi

The authors would like to thank...

\ifCLASSOPTIONcaptionsoff
  \newpage
\fi

\section{Introduction}
\IEEEPARstart{T}{his} demo file is intended to serve as a ``starter file''
for IEEE Communications Society journal papers produced under \LaTeX\ using
IEEEtran.cls version 1.8b and later.
I wish you the best of success.

\hfill mds

\hfill August 26, 2015

\subsection{Subsection Heading Here}
Subsection text here.

\subsubsection{Subsubsection Heading Here}
Subsubsection text here.

\section{Conclusion}
The conclusion goes here.

\appendices
\section{Proof of the First Zonklar Equation}
Appendix one text goes here.

\section{}
Appendix two text goes here.

\section*{Acknowledgment}

The authors would like to thank...

\ifCLASSOPTIONcaptionsoff
  \newpage
\fi

\section{Introduction}
\IEEEPARstart{T}{his} demo file is intended to serve as a ``starter file''
for IEEE \textsc{Transactions on Magnetics} journal papers produced under \LaTeX\ using
IEEEtran.cls version 1.8b and later.
I wish you the best of success.

\hfill mds

\hfill August 26, 2015

\subsection{Subsection Heading Here}
Subsection text here.

\subsubsection{Subsubsection Heading Here}
Subsubsection text here.

\section{Conclusion}
The conclusion goes here.

\appendices
\section{Proof of the First Zonklar Equation}
Appendix one text goes here.

\section{}
Appendix two text goes here.

\section*{Acknowledgment}

The authors would like to thank...

\ifCLASSOPTIONcaptionsoff
  \newpage
\fi

redpajama_set_name: RedPajamaArXiv


text:
Company Experiences
Trends in F&B
The Sustainability of Beef (part 1/2)
By ESCP Turin Campus December 2, 2020 January 11, 2021
Written by: Tommaso Cremonini and Michelangelo Serpico (IFBM students, intake 2020)
We are Michelangelo and Tommaso, and as students at ESCP Business School, we take sustainability very seriously. As future citizens of the world we look towards the future, and we strongly believe that sustainability is the basic building block to creating better companies that can shape the communities in which they exist.
As we believe that this vision cannot become a reality without the support of the people in those communities, we give the same importance to communication; and, as so often happens with complex topics, it's important to simplify in order to reach more people.
However, in the case of food sustainability, we see all too often that the necessary simplification is transformed into brutal generalization and falsification of information, which by no means can contribute to our fight.
As students of the Master in International Food & Beverage Management we give particular importance to this topic, and today we would like to speak more specifically about meat sustainability in Europe, a subject that is very frequently subject to negative distortions when portrayed to consumers.
That being said, in our small quest for truth, the first point to make regards deforestation…
Deforestation in Europe
Deforestation is always closely associated with meat production, and is intrinsic to the nature of the system itself.
To let the data speak for itself, according to the FAO, nowadays forests cover 43% of the European continent.
The number of green areas is increasingly expanding in Europe, and this is also due to the use of intensive farming, which leads us to our second topic…
The Bad Name of Intensive Farming
Intensive farming is often seen as the enemy in the eyes of many consumers; without going into the whole issue with its many relevant points of view, it's relatively easy to see that from an environmental point of view intensive farming has many positive aspects.
In particular, according to the FAO, 14.5 % of C02 emissions for which humans are responsible are originated by cattle breeding. However, the majority of these come from extensive farming methods adopted in South America, Asia and Africa; in Italy for example, according to a recent ISPRA report of 2017, the total contribution of the livestock sector is of about 4.4%; this is mainly due to a higher level of technology and the intensive use of resources.
If ideas around the impact of livestock farming are not always clearly communicated, the same is true about the use of resources, which leads us to the next topic…
Food or Feed?
Another very common misconception is that a large part of the feed used for livestock would be better used as food for humans.
However the FAO clearly estimates that 86% of cattle feed is made up of plant material which is not edible for humans. This is because grazers have a digestive system that is capable of using cellulose and lignin as sources of energy; these cannot be metabolized in the human digestive system. Other sources are waste from the production of human food, such as sunflower seed oil and many soy based products.
This may open up a discussion on the role of livestock farming in a circular economy, which would require a dedicated article; however it is worth looking thinking about the water footprint and how new technology is shaping the sector, which we will do in the next blog post.
To read more about:
forestry in Europe: click here
emissions and livestock: click here
livestock in Italy: click here
the Food-Feed debate: click here
Tags: beef ifbm IFBM students Sustainability Trends
Published by ESCP Turin Campus
View all posts by ESCP Turin Campus
Previous Entry La casa del prosciutto – Alberti 1906
Next Entry The Sustainability of Beef (part 2/2)
I agree to the Privacy Policy. I authorise Fondazione E.S.M.I. to use my personal data for marketing purposes related to their activities and services, in particular as it concerns educational opportunities in line with my interests.
© COPYRIGHT ESCP 2022 - Privacy Policy - Cookies
Fondazione European School of Management Italia P.IVA 09052310019 - REA 1181184
Sede Legale Corso Unione Sovietica 218bis, Torino. E-mail: info.it@escp.eu
© © ESCP Business School - World's First Business School
Berlin· London· Madrid· Paris· Turin· Warsaw

redpajama_set_name: RedPajamaCommonCrawl


text:
More of the Monkees
As you can imagine, the sudden and unexpected death of Michael Nesmith yesterday threw any number of plans sideways last night as I scrambled to get something of substance posted before bedtime.
We expect to see many more comments in the days to come ... but wanted to get some of this posted yet this afternoon so fans still have enough advance notice to see it.
For starters, Me-TV will be offering a 4-episode Monkees block tomorrow (Sunday, December 12th) as a tribute to the late Mike Nesmith, who passed away on Friday at the age of 78. (How cool that three of those four episodes are the ones I mentioned in my memoriam piece Friday night! I still love the episode when Michael takes on The Devil in a court of law to prove that Peter could play the harp on his own … but that probably is more of a Petercentric episode in the long run.)
More details below as to when to tune in. (kk)
R.I.P. Michael Nesmith of the Monkees
The singer-guitarist was 78. Watch a special tribute to Nesmith with four episodes of The Monkees this Sunday at 2P | 1C. The episodes include "I Have a Little Song Here," "Monkee Mayor," "The Monkees in Texas," and Nesmith's personal favorite episode, "Fairy Tale."
R.I.P. Michael Nesmith of the Monkees (metv.com)
Then, next weekend (beginning at Noon Eastern on Saturday, December 18th) The Decades Channel will run EVERY episode of The Monkees television series ... all 58 of them ... straight through 6 am on Monday, December 20th.
You can check the schedule here to find your favorite episodes. (kk)
Schedule (decades.com)
Much as I suspected, Michael Nesmith was determined to finish The Monkees' Farewell Tour no matter what … one final, last harrah.
Micky Dolenz told Rolling Stone Magazine:
Despite his condition, you could never, never have talked him out of the tour. We had every opportunity to stop the tour. You don't need any excuses these days. We had the option all the time to cancel dates, and we did cancel a couple for various reasons. We could have canceled more, but he was absolutely determined to finish that tour.
The last show of the tour is pretty much a blur to me. Our last conversation probably took place that night, but I don't recall it specifically. We did hug that night on the stage. And the whole tour was very emotional for me. I knew it was pretty unlikely that, for whatever reason, we'd ever be doing this again. That was both our attitudes for us. We went out with flair.
I don't really see myself as the last Monkee, but it's the end of an era. That's for sure. That's what happens. None of us are going to get out of this alive.
--Micky Dolenz
We've said many times before over the years that The Monkees are very fortunate as being "frozen in time" … because they have never really been off the air since their TV show first debuted in September of 1966, every generation to come along since has discovered the crazy antics and the incredible catalog of hits. Their music continues to sell and is well-represented on oldies radio. When Michael Nesmith died last Friday (just three weeks shy of his 79th birthday … a birthday he shared with former bandmate Davy Jones, who we lost in 2012) and Peter Tork passed in 2019 (almost seven years to the day of Jones' passing), it left only Micky Dolenz as a surviving member of the group. (Micky has been the long-standing voice of the group, and always seems to have been working, either in conjunction with some combination of the other three or as a very successful solo act. He already has a 2022 tour planned with former Rascal Felix Cavaliere.)
Still, this loss really hits home with me. It was The Monkees that made me want to play music. By the time they came along, the music of The Beatles had changed, becoming more sophisticated and studio oriented. In fact (although we didn't know it at the time), The Beatles played their last live show just three weeks before The Monkees premiered on television. It was a lot easier for me as a 13 year old to relate to the happy and catchy tunes of The Pre-Fab Four at the time than it was to try and follow along the new mystic ways of the band that inspired them. (C'mon … The Beatles were growing mustaches for God's sake!!!)
So yes, the closing of the gap affected me greatly with the passing of Nez.
I've been listening to Monkees music ever since. (And by that I mean even more so than I used to!) I wore those first four albums out in 1967! (And I can't tell you how many copies … or how many configurations … I've gone thru since!)
Thank you all for the music you gave us … it totally shaped my life in my all-time favorite year in music. (kk)
You can relive 1967, day by day, in our Forgotten Hits tribute here:
FORGOTTEN HITS: December 30th (forgottenhits60s.blogspot.com)
[Just continue to click the NEWER POST link at the bottom of each page and it will take you, day by day, thru our 1967 tribute]
Thanks for your piece on Michael Nesmith ... I loved reading your words!
David Salidor
I ran into a neighbor this morning with whom I often have music conversations. He was 11 when The Monkees TV show premiered in 1966. When I mentioned Mike Nesmith's passing, he asked me who that was. When I told him, the first thing out of his mouth was "Oh, I never paid them much attention, they were just actors who didn't play their own instruments." What do you do with people like that? Besides send them a link to Forgotten Hits, which I did just now.
If a musical act is judged by the number of smiles it's put on my face, then The Monkees rank as Number One. Their music is only "bad" (as that 1967 article reports Mike saying) if one doesn't have an ear for melody, harmony, hooks, and first-rate production and instrumentation. Throw in some pretty deft wordplay and (especially in Mike's case when he took songwriting duties) --profundity--and you've got first rate material on those first five albums.
I don't love ALL of Nesmith's post-Monkee material, but there is much to appreciate there and I loved showcasing The First National Band when I had my radio show.
Kent, your last paragraph before "We love you" summed up my feelings beautifully so I won't repeat your wonderful words except to say thank you.
Now when do we talk about that Rock and Roll Hall Of Fame Induction Ceremony???
Sam Tallerico
lostoldies@yahoo.com
Another VERY timely mention ...
(Unfortunately, it may already have aired by the time you see this ... but we're running it anyway just in case)
Chuck Buell here on a Big Football Game and a Big Forgotten Hit.
Creedence Clearwater Revival founder and Army Veteran, John Fogerty, will play his song, "Fortunate Son," with the U.S. Army's Six-String Soldiers band as part of "Songs of Service," a 30-minute TV Musical Special scheduled to air on select CBS stations TODAY, after the Army-Navy football game.
U.S. Army Photo of John Fogerty in 1967
prior to his becoming a supply clerk.
As Forgotten Hitters know, the song became an anthem for young men who didn't have the class or educational background that allowed them to skip military service during the Vietnam War era. Interestingly, because Fogerty was a bit "pissed off" about that situation, he became inspired one day to write "Fortunate Son," which he completed in about twenty minutes.
https://www.youtube.com/watch?v=2KYRF8vMsJ8
The special is confirmed to air in the New York City and Philadelphia CBS stations. If you live elsewhere, check your local listings. You also can watch the show on the "We Are The Mighty" YouTube Channel.
CB ( which stands for "Creedence Boy!" )
Saying Goodbye to Nez
The news was heartbreaking ... but certainly not a surprise ...
Other than it coming so fast after the conclusion of their Farewell Tour
(which sadly takes on a whole new meaning now within the context of Michael Nesmith's passing)
He did not look well when we saw him here in Chicago this past November ... frail and fragile ... yet still the consummate entertainer when it came to giving the fans what they wanted (and mustering up the effort to do so.)
The Mike Nesmith catalog within The Monkees was significant ... especially when one considers that the entire concept was a manufactured image, tailor made to succeed with an audience of teenage girls and perfectly crafted pop songs that would appeal to millions ... which, of course, they did.
And because every album by the band included at least a couple of Nez-tunes, he reaped HUGE benefits in the way of songwriting royalties when these records sold well into the millions of copies. (The fact that he also inherited millions more when his mother, the inventor of Liquid Paper, passed away only sweetened the pot.) Suffice to say that after The Monkees split up and Michael Nesmith ran his personal course fronting his own First National Band, there was more than enough money leftover to create movies and experimental (and very successful) video projects like "Elephant Parts" and (later) "Television Parts." Books and music anthologies continued ... and while Michael sat out most of The Monkees reunions that at other times brought together his former bandmates Micky Dolenz, Davy Jones and Peter Tork, he was still able to lead an influential and high profile life outside the public eye. (Michael developed his own music video production company, years before MTV launched to concept. Through his company The Video Ranch, Nesmith developed "Pop Clips," later selling the intellectual property to Time Warner, who then used the concept to develop and launch MTV.) His own videos were very inventive ... but he also produced clips for many other artists, including country stars Roseanne Cash and Jim Stafford. Many of these clips ran on his short-lived (but excellent) television series, "Television Parts."
Some of the material he contributed was given to Micky Dolenz to sing ... tracks like "Mary Mary" and "The Girl I Knew Somewhere," both of which were EXTREMELY popular cuts from early in their career ... but Michael also took the spotlight (adding a bit of country flavor to the band, thanks to his Texas roots) with tracks like "Good Clean Fun" (which opened the show on the farewell tour), "Listen To The Band," "Circle Sky" (from the film "Head"), "Love Is Only Sleeping," "What Am I Doing Hangin' 'Round" (written by Boomer Castleman and Michael Martin Murphey), "You Told Me," "You Just May Be The One," "Sunny Girlfriend," "Papa Gene's Blues," "Sweet Young Thing" and "The Kind Of Girl I Could Love." (Not to mention the fact that he COMPLETELY blew me away with his track "Don't Call On Me" from The Monkees' "Pisces, Aquarius, Capricorn and Jones, Ltd." LP, a HUGE #1 Hit from late 1967 that also happens to be my all-time favorite by the group. Nez becomes "lounge singer" Michael Nesmith on this one and executes it to perfection.)
Don Kirshner had lined up a Who's Who of songwriters, including Gerry Goffin and Carole King (with whom Nesmith shared songwriting credit on "Sweet Young Thing"), Neil Diamond, Neil Sedaka, the songwriting teams of Tommy Boyce and Bobby Hart, Jeff Barry and Ellie Greenwich, Barry Mann and Cynthia Weil, Hank Medress, Jay Siegel, Mitch and Phil Margo, David Gates, Nilsson, John Stewart, Paul Williams and others. Collectively, they kept The Monkees at the top of the charts with perfectly crafted pop tunes for two solid years.
(Dolenz told the story during the farewell tour about how Michael once presented his song "Different Drum" to the television show producers Bob Rafelson and Bert Schneider and musical director Don Kirshner, who turned it down flat, dismissing it by saying "That's not a Monkees song." Nesmith responded, "Well, I'm a Monkee ... and I wrote it ... so how is it not a Monkees song?") They still refused to let the band record it (although there is a VERY brief segment in one of the episodes where Mike and Peter play strolling musicians, passing the table where Davy and his date are dining, singing a short snippet of the tune.) When it became clear that Kirsner was never going to let The Monkees record the tune, Nez gave it to an up-and-coming, young female singer named Linda Ronstadt ... and it launched her career, reaching The Top 20 in early 1968. (Micky Dolenz also covered the tune earlier this year on his "Dolenz Sings Nesmith" album.)
The conflict with Kirshner continued. When The Monkees (led by Nesmith who, with Peter Tork, were the two real musicians in the group) demanded more artistic control (including being able to choose their own material as well as playing on their own records), the falling out between The Pre-Fab Four and Music Director Don Kirshner ended up in the group's favor, with the firing of Kirshner. (During one confrontation, Nesmith put his fist through a wall and then told Kirshner "That could have been your face." They did not end on good terms!)
If Davy was the heartthrob and Micky was the nut and Peter was the dumb one, then Michael was the smart one and quite often, the voice of reason ... whether he was negotiating with The Mayor ... or The Devil, he always seemed to be the only adult in the room. One episode had him fighting to protect his song "Gonna Buy Me A Dog," (a song he didn't even write!) from an unscrupulous publisher ... and in another favorite episode of mine, Mike played both the narrator and the Fairy Princess. (If you haven't seen it, don't ask ... just watch the "Fairy Tale" episode and enjoy it for its comedic genius.)
His post-Monkees solo career produced four charting singles, "Joanne" (#13 in Record World, 1970 - but only #21 in Billboard) being the biggest. His follow-up release, "Silver Moon," made The Top 40 in both Cash Box (#28) and Record World, but fell short (#42) in Billboard. His songs were covered by everyone from Linda Ronstadt and the Stone Poneys to The Nitty Gritty Dirt Band to Andy Williams to Susanna Huffs.
I had the pleasure of meeting him once, a few years back, when The Monkees performed as a trio shortly after the passing of Davy Jones at The Star Plaza in Merrillville, Indiana. Michael wore these incredibly sparkly shoes on stage for the concert ... but refused to put them back on backstage for a photo. Instead, I had to settle for a shot of me and Mike (with the top of his head cut off!!!)
I am SO glad we were able to make it to their Farewell Tour ... I'd be beside myself now if I had missed it. It was one last chance to say goodbye to the music and the artists that were such a HUGE part of my life growing up in the '60's.
Part of me feels like Michael fulfilled his ultimate fantasy ... his whole life he dreamed of being a successful musician ... and that dream came true a million times over. Despite obvious recent health issues (keep in mind that he also suffered a heart attack a couple of years before Covid side-tracked the tour all-together), he STILL came out (at significant risk to himself) to do the farewell tour with Micky. Micky had just released a tribute album to Mike's music ("Dolenz Sings Nesmith"), released to universally critical acclaim. I prefer to think that Nez went out on a high note ... he completed the tour, took his final bow ... and then he went home to die. I believe he finally came to accept his legacy with the fans, rather than downplay what he often referred to as "just one chapter in my career."
We love you, Michael. You made a HUGE impact on my life ... and I have been (and will continue to be) eternally grateful. Thank you for the music. (kk)
From Micky Dolenz ...
I'm heartbroken.
I've lost a dear friend and partner.
I'm so grateful that we could spend the last couple of months together doing what we loved best – singing, laughing, and doing shtick.
I'll miss it all so much. Especially the shtick.
Rest in peace, Nez.
I know you understand why I'm in mourning right now. I look forward to reading the Forgotten Hits comments on Nez. There is so much I want to say ... will email more this weekend. I also wanted to tell you again how much I enjoyed reading all of the Dolenz / Nesmith concert reviews.
Bummer ... we lost Mike Nesmith today.
I know you were a huge Monkees fan ... glad you got the chance to see him.
The Wool Hat has left us, leaving only Micky Dolenz as an original Monkee. I know you've been saying that Mike has been in poor health ... but it still hurts to lose him just the same.
Does this mean that Micky will now have to sing his self-prophesized version of "The Monkees Theme" ... "Hey, Hey, I'm a Monkee" ... from this point forward?
And poor Andrew Sandoval ...
It took an extra year and a half to get his INCREDIBLE "Day By Day" Monkees book out into the market ... and now it's already out of date! (Another Covid casualty!)
You'll find a beautiful tribute ...
In Memoriam: Michael Nesmith (1942-2021) - The Second Disc
Oh no ... not The Nez!!!
You likely already know, but sad to say, the wool hat has been silenced:
https://news.yahoo.com/michael-nesmith-monkees-singer-songwriter-171520208.html
"With infinite love we announce that Michael Nesmith has passed away this morning in his home, surrounded by family, peacefully and of natural causes," his family said in an issued statement ...
https://www.rollingstone.com/music/music-news/monkees-michael-nesmith-dead-1270079/
Upon kicking Don Kirshner from controlling the Monkees' music in 1967, this ran in the Lincoln paper here.
The Nez always told things as he saw them.
I've now got some of Shelly's blues.
https://www.youtube.com/watch?v=bVUHWFY0w1Y
Clark Besch
I just heard the tragic news about Mike Nesmith's physical passing. Really hits home.
I first encountered Mike in 1966 when the Monkees held their first press conference. Maybe there were two of them. One in Hollywood at the Columbia Pictures studio or in Burbank at the Columbia ranch. My mother Hilda worked from 1965-1968 for Raybert Productions who did "The Monkees" television series and helped type their scripts and was in the stenography pool on the Columbia lot. I first met Henry Diltz on "The Monkees" set as he was the second season staff photographer.
Mike was always the "country guy" to me. I think he was introduced initially as Michael Blessing when we met or he had just changed his name. He had a southern accent. That was really unique to a 15 year old child of Hollywood.
As you well know, long before it was fashionable, I've always touted the Monkees in my various books and articles the last 48 years. Gary Strobl went deep into his archives and provided me a couple of Nesmith photos with Jimi Hendrix on tour that are exhibited in the recent "Jimi Hendrix: Voodoo Child" book done with my brother Kenneth.
I've always felt Mike's pioneering work in music videos and helping establish the country rock sound of Southern California has been overlooked.
Harvey Kubernik
As a final gesture of my parting tribute to Mike, let me ask once again that we all work to the very best of our efforts to SAVE THE TEXAS PRAIRIE CHICKEN!!! (kk)
The Rolling Stones played a brief tribute concert to their fallen bandmate Charlie Watts on Monday, December 6th, at a small London club (Ronnie Scott's Jazz Club to be exact, which was Charlie's favorite.) The event was hosted by Charlie's granddaughter Charlotte and other members of the Watts family.
The venue was packed with 200 invited guests, among them former Stones bassist Bill Wyman (who did not perform with the band.) Mick Jagger, Keith Richards and Ronnie Wood, all of whom attended with their wives, were joined on stage by No Filter tour drummer Steve Jordan and played two R&B standards (because, as you know, they are essentially just a blues cover band) … "Shame Shame Shame" and "Down The Road Apiece." (kk)
Joe Walsh will be hosting his fifth annual VetsAid Concert on December 18th. ZZ Top and Ringo Starr have already been announced as performers … and more special guests are expected. Funds raised go directly to charities supporting veterans and their families and have raised over $1.5 million so far.
Tickets for this year's VetsAid live streaming show are on sale now and prices start at just $15. Viewers will be able to watch the premiere on Dec. 18 plus re-airings through Dec. 25. (kk)
It is being reported that Adele will earn TWO MILLION DOLLARS PER SHOW for her four-month residency in Las Vegas at the Caesar's Palace Colosseum. The shows kick off in January and run thru April … and the ticket prices are just as astounding … ranging from just under $3000 per ticket to upwards of $6000 per ticket. (You know, even if I had a great night at the tables and won $25,000, I don't think I'd spend HALF of it to see Adele!!! At that point, it might make more sense to plan a trip to London to see the ABBA Avatars ... and at least get a really cool vacation out of it!!!) kk
Here's a pretty cool Eddie Van Halen keepsake …
You can preorder it now thru Amazon.com (kk)
So, Kent, I never played with Legos when I was a kid. I only stepped on them in the dark late at night in my bare feet on the way to the refrigerator when my small sons had left theirs out after THEY had played with them! YOWWW!!!
Still, I find the creative designs the Lego Folks come up with rather fascinating! The "Beatles Rooftop Concert" that Clark Besh shared with us in Forgotten Hits yesterday is one of them! Pretty interesting!
And THESE weren't around when I was a Young Dad!
Padded slippers to create a barrier between your feet and the sharp-edged Legos left strewn around the floor!
Can you imagine?!
Thanx, Clark!
< Buell >
The official Cat Stevens-centric soundtrack to the cult film favorite "Harold And Maude" is FINALLY getting a commercial release here in The States on February 11th. (I LOVE this movie … can't even begin to count how many times I've seen it … and finally having ALL of the Cat Stevens songs in one place is a dream come true! And, they'll be interspersed with dialog from the film as well!!!) kk
1. Don't Be Shy
2. Dialogue 1 (I Go to Funerals)
3. On The Road to Find Out
4. I Wish, I Wish
5. Tchaikovsky's Concerto No.1 in B
6. Dialogue 2 (How Many Suicides)
7. Marching Band / Dialogue 3 (Harold Meets Maude)
8. Miles From Nowhere
9. Tea for the Tillerman
10. I Think I See the Light
11. Dialogue 4 (Sunflower)
12. Where Do the Children Play?
13. If You Want to Sing Out, Sing Out - Ruth Gordon and Bud Cort
14. Strauss' Blue Danube
15. Dialogue 5 (Somersaults)
16. If You Want to Sing Out, Sing Out
17. Dialogue 6 (Harold Loves Maude)
18. Trouble
19. If You Want to Sing Out, Sing Out (ending)
That A&M Documentary we told you about is airing now on Epix … check your local listings or talk to your cable provider to catch this mega-interesting series! (kk)
And the soundtrack to last year's "Summer Of Soul" concert documentary will be available soon, too. Focusing more on what we saw on screen, the track list represents what the film makers have offered up so far …
But the hope remains that we might get to see more from each of the musical acts that appeared at this unique festival.
In the meantime, you can enjoy:
Summer of Soul (...or When the Revolution Could Not Be Televised) Original Motion Picture Soundtrack (Legacy, 2022)
(Amazon U.S. / Amazon U.K. / Amazon Canada)
* digital bonus track
1. Uptown - The Chambers Brothers (a)
2. Why I Sing the Blues - B.B. King (e)
3. Don't Cha Hear Me Callin' to Ya - The 5th Dimension (a)
4. Aquarius/Let the Sunshine In (The Flesh Failures) - The 5th Dimension (a)
5. My Girl - David Ruffin (c)
6. Oh Happy Day - The Edwin Hawkins Singers (a)
7. It's Been a Change - The Staple Singers (b)
8. Precious Lord Take My Hand - The Operation Breadbasket Orchestra & Choir feat. Mahalia Jackson & Mavis Staples (b)
9. I Heard It Through the Grapevine - Gladys Knight & The Pips (c)
10. Watermelon Man - Mongo Santamaria (d)
11. Together - Ray Barretto (d)
12. Hold On, I'm Comin' - Herbie Mann (d)
13. Sing a Simple Song - Sly & The Family Stone (a)
14. Everyday People - Sly & The Family Stone (a)
15. Africa - Abbey Lincoln & Max Roach (a) *
16. Backlash Blues - Nina Simone (e)
17. Are You Ready - Nina Simone (e)
All tracks recorded live at the Harlem Cultural Festival, Mount Morris Park, Harlem, NY on 6/29/1969 (a), 7/13/1969 (b), 7/20/1969 (c), 7/27/1969 (d) and 8/17/1969 (e)
Last Man Standing: Suge Knight and the Murders of Biggie & Tupac Coming to Starz This Month ~
Latest Nick Broomfield Documentary Explores New Evidence and Witnesses Indicating LAPD Played a Key Role
Click here to view the trailer
U.S. audiences will be able to view Last Man Standing: Suge Knight and the Murders of Biggie & Tupac this month on the Starz channel. The film is a new Nick Broomfield- directed documentary that is a companion to his earlier (2002) Tupac & Biggie and premieres December 23. The film includes newly sourced footage taken at the time of the rap icons' killings as well as evidence that suggests direct involvement of elements of the Los Angeles Police Department in the murder of Biggie Smalls. The documentary, produced by Compton resident Pam Brooks who had earlier worked with Broomfield on his Tales of The Grim Sleeper (2014), is a compelling look at gang culture as embodied and advanced by Death Row Records kingpin Suge Knight.
Broomfield stated, "After finishing Biggie & Tupac in 2002, I became increasingly troubled by the treatment of LAPD detective Russell Poole, who had come up with evidence that LAPD officers were involved in the hit on Biggie Smalls." Poole, who suffered a massive heart attack in 2015 and died during a meeting at LA Sheriff's Department headquarters, claimed that Knight, affiliated with the Mob Piru's faction of the Bloods street gang, had LAPD officers in his employ and that they played a direct role in the murder of Notorious B.I.G. Broomfield explains, "It was a tragic end that Russell didn't deserve. However, a few years later people stated talking more freely after Suge Knight was sentenced."
Knight is serving a 28-year sentence for manslaughter in connection with the hit and run death of music executive Terry Carter in the parking lot of a Compton fast food restaurant. That location was used in the filming of Straight Outta Compton, the 2015 biopic that tells the story of the rise of N.W.A, the legendary Compton-based rap group. Knight's personal saga. from early childhood through his current incarceration is detailed in Last Man Standing. He will not be eligible for parole until 2034. Broomfield told The Independent, "Now that Suge Knight's behind bars, a lot of people are coming forward that were, frankly, frightened of getting killed before."
Last Man Standing: Suge Knight and the Murders of Biggie & Tupac offers eyewitness accounts of the rise of Death Row Records and Knight's consolidation of power. Tupac Shakur had been convicted of a sex abuse charge in connection with a gang rape that took place in New York in 1993 and was sentenced to four and a half years in prison. It was Suge Knight who facilitated his release from New York's Clinton Correctional Facility and his move from Interscope to Death Row Records. Thereafter, Shakur was totally loyal to Knight and the Mob Piru Bloods.
The film presents a chilling account of the events surrounding the murders of both Tupac in Las Vegas and Biggie in Los Angeles almost exactly six months later, and the role Suge Knight played in those events. The film was released earlier this year in Britain where it was critically lauded. MOJO called it "gripping viewing and one of Broomfield's best" while Uncut characterized it as a "terrifying portrait of the maelstrom Knight sent whirling and the lives it consumed."
In an interview with The Independent, producer Pam Brooks explored the film's underlying theme. "It's about closure, basically," she stated. "It was crucial how Suge Knight crewed up with the LAPD and that just shows you how lowlife they are for a little bit of change. They make good money, but it just shows you how it's about control. It's all about control."
Nick Broomfield is a prolific filmmaker, having directed more than three dozen documentaries dating back to the 1970s. He won BAFTA's Flaherty Documentary Award for Soldier Girls (1981), was the recipient of the Amsterdam Film Festival's Amnesty International - DOEN Award for Aileen: Life and Death of a Serial Killer(2003) and took home a FOCAL Award for Marianne & Leonard: Words of Love (2019). Aileen was the basis for the film Monster that earned Charlize Theron the 2004 Academy Award for Best Actress. Other of Broomfield's films that have been nominated for or received major awards include, among others, Battle for Haditha (2007), Ghosts (2006) and Kurt & Courtney (1998) and Whitney Can I Be Me (2017).
Frank B. sent us this newspaper clipping about a push for a street name change …
New Orleans urged to switch street name from Confederate to music legend | New Orleans | The Guardian
kk …
I'm Glad They're Not Naming It After His "MOTHER-IN-LAW."
https://www.theguardian.com/us-news/2021/dec/04/new-orleans-street-allen-toussaint-robert-e-lee
Tom Cuddy sent us this link to an article about the new Illinois Rock and Roll Hall Of Fame, recapping the events from earlier this past summer.
Hopefully, next year will be a better and safer year to finally check out these facilities.
https://patch.com/illinois/joliet/joliets-new-il-rock-roll-museum-inducting-1st-hall-fame
We've been saluting 1971 all year long in Forgotten Hits (and will pay tribute to 1972 next year with weekly surveys from around the country … assuming we can ever find a Vermont chart, that is!!!) …
So we wanted to share this Best Classic Bands 1971 Highlights piece, which takes you thru the whole year, listing all of the key events (including a couple that WE even missed!!!) 50 Years Ago: 1971 in Rock Music | Best Classic Bands
And Ultimate Classic Rock has chosen their Top 40 All-Time Favorite Singer / Songwriter Albums … and it's quite a list. (How many of these do you have in YOUR collection?)
Top 40 Singer-Songwriter Albums (ultimateclassicrock.com)
In the December, 2021 issue of "Shindig!" magazine Grahame Bent raves about "Jimi Hendrix: Voodoo Child" in a 4-star review. "The fact that this spectacularly illustrated 'Voodoo Child' comes from the combined pens of Harvey and Kenneth Kubernik undeniably brings a certain cache. Written primarily from an American viewpoint, aside from its fantastic visuals, the use of colour and eye-catching layouts, one of the book's strongest points is the sheer volume of interviewees and sources quoted in the text, which effectively voices the narrative."
Stephen Peeples interviews the Kubernik brothers on their new "Jimi Hendrix: Voodoo Child" book (which I SHOULD be starting any day now, Harvey!!!)
Jimi Hendrix 'Voodoo Child' Bio: Kubernik Brothers Q&A - Stephen K. Peeples (stephenkpeeples.com)
Harvey and Kenneth Kubernik do an excellent job of telling the legendary tale of the late great Jimi Hendrix. Through utilizing important figures in Hendrix's life, the inclusion of rare photography and interviews, and the Kubernik's own mastery of rock history this book is truly visceral experience.
--Laura Bialobos
The Kubernik Brothers are being interviewed TODAY (December 10th) on KPFK-FM at 5 pm Eastern … more details below …
Tune in Friday, December 10th, at 2 pm Pacific Standard Time to KPFK 90.7 FM Los Angeles in the Southern California Region. Co-Authors Harvey and Kenneth Kubernik will be the guest interview subjects, discussing their latest offering, (just published), "Jimi Hendrix: Voodoo Child" (Sterling Publishing) with "Pacifica Performance Showcase" Host Donna Walker. The program can also be heard nationally and globally on the internet at https://www.kpfk.org . The Kubernik brothers will be heard in the 2:00 - 3:00 pm time slot, Pacific Standard Time. Their appearance will be part of the station's fund drive.
https://www.kpfk.org/on-air/pacifica-performance-showcase
(Simulcast at kpfk.org and available for "Play" or "Download" via kpfk.org "Audio Archives" for approx. two months after the show airs. 1st half-hour is archived under "Arts in Review" and second half-hour is archived under "Pacifica Performance Showcase.")
And here's a link to Harvey's own piece on the infamous Rolling Stones / Altamont incident …
The Rolling Stones' US Tour Gimme Shelter Documentary Interviews (musicconnection.com)
And don't miss your chance to celebrate this weekend ...
From Chuck Buell, keeping his eye out for these, perhaps lesser known, Special Frantic Final End-of-the-Year Holidays for us!
He points out that ...
This Sunday, December 12th, is "National Ding-a-Ling" Day!
A Day established to encourage us all to reconnect during this Holiday Season with people we once talked to often but not so much anymore and people we haven't heard from in a while. It might be an old Forgotten Hitter, classmate, co-worker, old lover (you're on your own with that one!) or a neighbor from years ago who would love to hear the "Ding-a-Ling" of a phone call from you! You know, "Ding-a-ling-a-ling-a-ling!!
But, hey, OK! Because THIS is what you thought it stood for, here, against my better judgement, is Chuck Berry!
(Sheesh!)
CB ( which stands for the "Other" Chuck B! )
Watch for a brand new WEEKEND COMMENTS PAGE tomorrow ...
In Forgotten Hits!
More Breakfast With The Beatles (Part Two)
Breakfast With The Beatles
Phil Nee Talks To Billy Hinsche
THE 1971 SUPER CHARTS: Week Ending December 11th


text:
Region East Flanders
Talks in Ghent to get under way
The first steps have been taken to try find a way out of the impasse in the formation of a new coalition in the East Flemish city of Ghent. The Liberals and Christian democrats have agreed to talks with the progressive cartel made up of the greens and the socialists. The greens and the socialist will also meet with other parties.
Tue 16 Oct 2018 11:50
The cartel and the liberal/Christian democrat block have let it be known this morning that they will get around the table to discuss how progress can be made.
After Sunday's elections the Mieke Van Hecke's Christian Democrats said that they back the liberal Mathias De Clercq in his ambition to become Ghent's new Mayor. Mr De Clercq's liberals were the big winners in Sunday election and are now the biggest single party on the council. However, the progressive cartel, made up of two parties: the socialists and the green has more seats and as such believes that it should be leading the coalition talks.
The liberals and Christian democrats hope to break the cartel and install Mr De Clercq as Mayor. However, the cartel is talking about the possibility of a "a green Mayor". The cartel has "invited all democratic parties" for talks this afternoon.
Who will be Mayor?
Whoever takes the lead in the negotiations also decides who will be Mayor. The cartel has 21 seats and the cartel's leading green candidate has high hopes that Ghent new Mayor will be a green.
However, with 15 seats the liberals are the biggest single party and straight after the election result was announced Mr Clercq claimed the position of Mayor was his. Then negotiators intend to discuss content first before discussing who will be given what post. One thing that is certain though "the cartel will continue as one".
A scene from April last year... 2020 Pierre Crom
Government considering further restrictions for travellers
Mon 18 Jan 16:15
archive photo Radio 2
Corona measures extended till 1 March
Get your Trappist delivered to your door!
Tue 19 Jan 15:44