\section{Introduction}
Globular clusters (GCs) are ideal laboratories to study stellar evolution at different phases starting from main sequence (MS), sub-giant branch (SGB), red giant branch (RGB), horizontal branch (HB), asymptotic and post-asymptotic giant branch (PAGB), and finally the white dwarf (WD) phase. GCs are the oldest and the most massive stellar aggregates known to exist in our Galaxy, consisting of simple and multiple stellar populations \citep{2011MNRAS.415..643W, 2013ApJ...778..186V, 2004MSAIS...5..105B, 2007ApJ...661L..53P, 2008A&A...490..625M, 2008ApJ...673..241M, 2008ApJ...672L..29Y, 2009ApJ...697L..58A, 2009Natur.462..480L, 2009IAUS..258..233P, 2010sf2a.conf..319M, 2012ApJ...760...39P, 2015AJ....149...91P} with single stars, binary stars as well as multiple systems. Hence, they provide the best platform to study the exotic populations such as blue straggler stars (BSSs), cataclysmic variables, low mass X-ray binaries (LMXB) etc., which demand binary formation pathways. However, recent studies have shown that almost all the GCs consist of multiple stellar populations.

Ultraviolet (UV) study of evolved populations in GCs is very important because a few critical evolutionary phases, found in the GCs, are brighter in UV than in optical wavelength. These are the main contributors to the luminosity of GCs in the UV.  These UV bright stars are hot and mostly consists of HB stars, PAGB stars, WDs, and BSSs \citep{1972A&A....18..390Z, 1983PASP...95..256H}. The spectral energy distribution (SED) of these hot stars peaks at shorter wavelengths mainly in the near-ultraviolet (NUV) or far-ultraviolet (FUV). Note that MS stars and RGB stars, which make up the majority of stars in a GC, are faint at such short wavelengths. Hence, the UV images are less crowded than the optical images.
Recent studies of GCs  \citep{2003ApJ...588..464F, 2010ApJ...710..332D, 2010ApJ...722..158H, 2016ApJ...830..139P,2017MNRAS.469..267D, 2017ApJ...839...64R} have highlighted the importance of UV CMDs to identify and study the properties of UV bright stellar populations.
The HB stars are core helium burning low mass stars with hydrogen rich envelope surrounding the core. The core mass of stars lying on the HB is approximately $\sim0.5M_{\odot}$ (\citealp{1970ApJ...161..587I}). The HB itself consists of sub-populations separated by gaps, namely, the red horizontal branch (RHB), the blue horizontal branch (BHB) and the extreme horizontal branch (EHB). The RHB stars are cooler than the RR Lyrae instability strip with their effective temperatures ranging from $\sim5,000-6,200$ K whereas the BHB stars are hotter than the RR Lyrae instability strip and cooler than the EHB stars. The effective temperature of the BHB stars are expected to lie between $\sim8,000-20,000$ K. EHB stars are core helium burning stars with an envelope too thin to sustain hydrogen burning. These stars have effective temperature greater than 20,000 K and are expected to lie at the end of the blue tail of the HB in the optical CMD. UV photometry helps to confirm the presence of the EHB stars and separate them from bright BSSs, as optical colours become degenerate at the effective temperature of these stars \citep{2000ApJ...530..352D, 2001ApJ...562..368B, 2008ApJ...677.1069D}. Parameters and reasons behind the  distribution of stars along the HB in optical CMDs are not yet understood.
At first, it was thought that only metallicity of the cluster affected the HB morphology, but later through observations it was found that parameters other than metallicity, such as age of the cluster and/or He abundance, can affect the colour distribution of the HB stars (\citealp{2009Ap&SS.320..261C}). This problem is known as second parameter problem, first mentioned by Sandage \& Wallerstein \citep{1960ApJ...131..598S, 1967ApJ...150..469S, 1967AJ.....72...70V}. \cite{2014ApJ...785...21M} investigated the relation between HB morphology and various properties in GCs. They suggested that age and metallicity are the main global parameters of the HB morphology of GCs, while the range of He abundance within a GC is the main nonglobal parameter.\\

In clusters with the HB morphology covering a wide range in optical colour,
a number of discontinuities or jumps have been detected, although the visibility of these gaps change with the band passes utilized (\citealp{2016ApJ...822...44B}).
Three prominent such discontinuities or gaps known so far are "Grundahl jump" (G-jump) which lies within the BHB at $\sim$ 11500 K (\citealp{1999ApJ...524..242G}), "Momany jump" (M-jump) lies within the EHB at $\sim$ 23000 K \citep{2002ApJ...576L..65M, 2004A&A...420..605M} and the gap between the EHB and the blue-hook stars, covering $\sim$ 32000-36000 K (\citealp{2001ApJ...562..368B}). The gaps in the HB distribution are discernible in clusters hosting sufficient number of BHB and EHB stars and these gaps are found to be consistent in colour (effective temperature) across the clusters \citep{1998ApJ...500..311F, 2016ApJ...822...44B}. Various explanations are given for the presence of these gaps in HB distribution by (\citealp{2001PASP..113.1162M}), such as diverging evolutionary paths, mass loss on RGB phase, distinctions in CNO or rotation rates, dynamical interactions, atmospheric processes, He mixing in RGB stars, and statistical fluctuations.
\cite{2016ApJ...822...44B} characterized these HB features in 53 GCs, including NGC 1261. They created colour-colour plots for all selected clusters. They found less than four HB stars blue-ward of the G-jump in NGC 1261 (see Figure 5 in \citealp{2016ApJ...822...44B}). \cite{2012AJ....143..121S} constructed UV CMDs (FUV$-$NUV vs FUV) for 44 galactic globular clusters using GALEX data. They observed that the HB stars follow a diagonal sequence, unlike the horizontal distribution in optical and  its slope mainly depends on the bolometric correction effects.\\

EHB stars are intriguing objects to study as they are one of the important sources of the UV upturn at wavelengths shorter than 2300 {\AA} in the spectra of elliptical galaxies \citep{1990ApJ...364...35G,1995ApJ...442..105D,2011ApJ...740L..45C,2012ApJ...747...78B}. The origin of EHB stars in GCs is still not clear. One of the formation scenario suggested for EHB stars is enhanced mass loss on the RGB evolutionary phase. \cite{1976ApJ...204..488M} and \cite{1987fbs..conf..435T} proposed that the EHB stars are the product of dynamical interactions inside binary systems in clusters. He-mixing is also a possible scenario to explain the formation of EHB stars in clusters (\citealp{1997ApJ...474L..23S}). Through convection, helium is mixed into the outer layers of the envelope, which in turn, increase the envelope He abundance. The enhanced He abundance causes the tip of the RGB to have higher luminosity than in normal case. The higher luminosity leads to high mass loss on the RGB. Thus, He mixing is responsible for producing a bluer HB morphology. Other possible formation scenarios for EHB stars in clusters are early hot-flasher, late hot-flasher and He enrichment \citep{2001ApJ...562..368B,2010ApJ...718.1332B,2012ASPC..452...23B, 2015MNRAS.449.2741L, 2016PASP..128h2001H}.\\

  BSSs occupy a region above the turn-off (TO) point in the colour-magnitude diagram (CMD) of star clusters, where no stars are expected on the basis of standard stellar evolution, if we assume that all cluster stars are coeval. It has been suggested that BSSs are the product of stellar collisions or mass exchange in close binary systems \citep{1964MNRAS.128..147M, 1976ApL....17...87H}. Stellar collisions formation scenario for BSSs dominates in high-density environment whereas other formation scenarios dominate in a low-density environment.\\

Here, we study the southern galactic GC NGC 1261 located in constellation Horologium at a distance of 17.2 kpc (\citealp{2019RMxAA..55..337A}) and metallicity \big[Fe/H\big] = -1.27 dex (\citealp{2009A&A...508..695C}). Since this cluster is away from the galactic disc (b = $-52\degree.13$), it experiences almost negligible or no reddening. The age of this cluster is estimated to be 10.75 $\pm$ 0.25 Gyr (\citealp{2013ApJ...775..134V}), 11.5 $\pm$ 0.5 Gyr (\citealp{2010ApJ...708..698D}) and 12.6 $\pm$ 1 Gyr (\citealp{2013A&A...558A..53K}). In this study, the adopted age of this cluster is 12.6 Gyr to generate the isochrones. In this paper, we present the results of a UV imaging study of  NGC 1261 in six filters (2 FUV and 4 NUV), using the Ultra-violet Imaging Telescope (UVIT) on-board ASTROSAT. We used the proper motion estimated using HST and Gaia DR2 data to select cluster members in the inner and outer region of the cluster, respectively. We detect the HB and a few UV bright stars among the HB population. We determine the properties of the HB stars by analysing the SEDs to shed light on the formation and evolution of bright HB stars.\\

This paper is laid out as follows. Section~\ref{sec:2} describes the observations and data reduction. In Section~\ref{sec:3}, the UV and Optical CMDs are presented. We describe the properties of bright HB stars derived from the UVIT photometry along with HST, Gaia and Ground based photometry in Section~\ref{sec:4}.
All the results are discussed in Section~\ref{sec:5}. Our results are summarised and concluded in Section~\ref{sec:6}.

\section{Observations and Data Reduction}
\label{sec:2}
NGC 1261 was observed with UVIT on 26 August 2017 in two FUV and four NUV filters.
UVIT is one of the payloads in ASTROSAT, the first Indian Space Observatory. UVIT can perform simultaneous imaging in three channels: FUV(130-180nm), NUV(200-300nm) and VIS(320-550nm), in a circular field of view of diameter $\sim28{\arcmin}$. The spatial resolution (FWHM) is better than $1.8{\arcsec}$ for the FUV and NUV channels, and it is $2.2{\arcsec}$ for the VIS channel. The VIS channel is used only for drift correction. Each channel (FUV, NUV and VIS) consists of a set of selectable filters having different wavelength range. The intensified CMOS detectors are used in photon counting mode for the two UV channels and integration mode for the VIS channel. Further details about UVIT and calibration results can be found in \citep{2016SPIE.9905E..1FS, 2017JApA...38...28T}. The primary photometric calibration for all FUV and NUV channels were accomplished using observations of HZ4, a white dwarf spectrophotometric standard star (\citep{2017AJ....154..128T}).
The magnitude system adopted for UVIT filters is the AB magnitude system, and hence the estimated magnitudes will be in this system.\\

ASTROSAT has an orbit of 90 min., and UVIT observes only the night part of the orbit. The observation is, therefore, about 20 min or less per orbit (after the overheads). The UVIT instrument takes data over several orbits to complete the required exposure time. The images are created for each orbit using a customised software package, CCDLAB (\citealp{2017PASP..129k5002P}), by correcting for the geometric distortion, flat field illumination, and spacecraft drift. They are then aligned and combined to create a science ready image. The photometry is carried out on these final science ready images.

 The details of observations and photometry of NGC 1261 UVIT images are given in Table~\ref{tab:1}. The UVIT image created using one NUV N279N and one FUV F172M filter is shown in Figure~\ref{fig:1}. In this figure, blue and yellow colours represent UVIT FUV and NUV detections respectively. In the FUV, we are able to resolve stars up to the center of the cluster, but in the case of NUV images crowding is present in the central regions. Isolated stellar sources in the UVIT images have FWHM $\sim1.2{\arcsec}$ and $\sim1.5{\arcsec}$ in NUV and FUV channels, respectively.\\

 The PSF photometry was performed on all the UVIT images using DAOPHOT package in IRAF (\citealp{1987PASP...99..191S}). The various steps to carry out crowded field photometry are following. First stars are identified in the images using DAOFIND package in DAOPHOT. Then, we computed the magnitudes of stars
using DAOPHOT task PHOT.
The model point spread function (PSF) was generated by choosing isolated and moderately bright stars.The generated model PSF is then applied to all the detected stars using ALLSTAR task in DAOPHOT to obtain the PSF fitted magnitudes.
A curve-of-growth analysis technique was carried out to estimate aperture correction value in each filter and applied it to the PSF generated magnitudes. Finally, the saturation correction (For more details, see \citep{2017AJ....154..128T}) was done to obtain final magnitudes in each filter. The instrumental magnitudes have been calibrated to AB magnitude system by using zero-point magnitudes reported in the calibration paper \citep{2017AJ....154..128T}.
The Figure~\ref{fig:2} shows the PSF fit error plots for all filters as a function of magnitude. We detect stars as faint as 21 magnitude in NUV and 22 magnitude in FUV with typical errors 0.2 mag and 0.3 mag, respectively.

 \begin{figure}
    \centering

	\includegraphics[scale=0.4]{image.png}
    \caption{UVIT image of NGC 1261 obtained by combining images in NUV (N279N) and FUV (F172M) channels.}
    \label{fig:1}
\end{figure}
\begin{table}
	\hspace{1.5 cm}
	\caption{Details of observations and photometry of NGC 1261}
	\label{tab:1}
	\begin{tabular}{cccccc}
		\hline
		\hline
		 Filter & $\lambda_{mean}$ & $\Delta\lambda$  & ZP  & Exposure & No. of \\
		  & ({\AA}) & ({\AA}) & (AB mag) & Time (sec) & stars \\
		\hline
		 F169M & 1608 & 290 & 17.45 & 1746 & 486 \\
		 F172M & 1717 & 125 & 16.34 & 6662 & 2181 \\
		 N219M & 2196 & 270 & 16.59 & 2847 & 2610 \\
		 N245M & 2447 & 280 & 18.50 & 740 & 4755 \\
	 	 N263M & 2632 & 275 & 18.18 & 1022 & 6037\\
		 N279N & 2792 & 90  & 16.50 & 3831 & 9363 \\
		\hline
	\end{tabular}
\end{table}

\begin{figure}
   \hspace*{-0.7cm}

	\includegraphics[height=9.5cm, width=10cm]{image1.png}
     \caption{PSF fit errors as a function of magnitude for the UVIT observations of NGC 1261. The top two panels show the plot for two FUV filters, whereas the bottom 4 panels show for the NUV filters.}
    \label{fig:2}
\end{figure}

\section{The UV and Optical Colour-Magnitude Diagrams}
\label{sec:3}
Colour-Magnitude Diagrams (CMDs) are a very important tool to identify different evolutionary sequences in star clusters. As we have observed this cluster in 4 NUV and 2 FUV filters of UVIT, a large number of CMDs using various filter combinations are possible.
In order to identify stars in the inner region (within a diameter $\sim3.4'$), we have used HST UV legacy survey catalog of GCs (\citealp{2018MNRAS.481.3382N}) to cross-match with UVIT detected sources. Most likely members of this cluster in the inner region have proper motion probability more than $90\%$ as mentioned by \cite{2018MNRAS.481.3382N}. In order to cross-match UVIT detected sources with HST data, first we selected stars with proper motion membership probability more than $90\%$. In HST, the filters F606W and F814W are proxy to V and I bands.
To cross match HB stars and BSSs, we have used specific colour and magnitude range in the optical HST CMD. In the F606W$-$F814W vs F814W HST CMD, we have used $-$0.1<F606W$-$F814W<0.7 and 15<F814W<20.5 range for HB stars and  0<F606W$-$F814W<0.4 and  17.3<F814W<19.5 range for BSSs. Two stars appearing at the faint end of the blue tail of HB in optical CMD shown in Figure~\ref{opthstcmd} are also present in the colour-colour plot created for this cluster by \cite{2016ApJ...822...44B} (see their Figure 5). The Vega magnitude system used in HST is converted into the AB magnitude system, in order to adopt the same magnitude system\footnote{\url{http://waps.cfa.harvard.edu/MIST/model_grids.html##zeropoints}}. The magnitude system used to create optical CMDs in the inner and outer region is the Vega magnitude system, but both photometric systems are different. \cite{2001AJ....122.2587C}, \cite{2016AJ....152...55S} and \cite{2019RMxAA..55..337A} have studied the variability of stars in  this cluster. To identify the variable stars such as RR Lyrae and Sx Phe, we have cross-matched UVIT data with variable star catalog from \cite{2019RMxAA..55..337A}. In NUV bands, our sample of HB stars is not complete in the inner region as we are unable to resolve stars in the inner $1'$ diameter region due to crowding whereas, outside this region, we detect about 90\% stars compared to the HST. The total number of detected HB, BSSs and variable stars in each UVIT filter is tabulated in Table~\ref{tab:2}. The optical and UV-optical CMDs are shown in Figure~\ref{opthstcmd} and \ref{optuvhstcmds}, respectively. The photometric error bars shown in Figures~\ref{optuvhstcmds} and \ref{optuvgaiacmds} are the median of the photometric errors of stars at a selected magnitude ranges. In the UV-optical CMDs, we observe that the HB stars no longer follow horizontal sequence as found in the optical CMDs, rather it follows a diagonal sequence with less spread \citep{2007A&A...474..105B, 2008ApJ...677.1069D, 2011MNRAS.410..694D, 2009MNRAS.394L..56D, 2010ApJ...710..332D,   2019MNRAS.482.1080S, 2017AJ....154..233S, 2017ApJ...839...64R}. However, note that the sequence of BSSs, if diagonal, remains unaltered in both UV-optical and optical CMDs.\\

\begin{figure}
   \hspace{-0.9cm}

   \includegraphics[scale=0.5]{optical_cmd_n2_Basti_isocz.pdf}
     \caption{Optical CMD of NGC 1261 for the region covered by the HST.  Black dots represent the HST detected stars with proper motion membership probability more than $90\%$. only HB and BSS stars detected in the NUV N279N filter and cross-matched with HST catalog are shown in different colours, which are explained in the figure. The variable stars such as RR Lyrae and SX Phe are also shown in figure. The solid grey lines denote updated BaSTI-IAC model isochrone for 12.6 Gyr and \big[Fe/H\big] = -1.27 dex. The solid grey line on the HB locus is the zero-age HB (ZAHB) and the dotted one represents the terminal-age HB (TAHB), where the star has completed 99$\%$ of its He-burning lifetime. We also show the BSS model line which is extension of the zero age main sequence (ZAMS).}
    \label{opthstcmd}
\end{figure}
\begin{figure*}
   \hspace*{-1.4cm}

\includegraphics[height=20cm,width=20cm]{opt_uv_hst_Basti_isocz.pdf}
     \caption{UV-optical CMDs of NGC 1261 after cross-matching HST data with UVIT data in 4 NUV and 2 FUV filters. The meaning of different colours and symbols is shown in the panels. The photometric errors in magnitude and colour are also shown in each panel. The grey lines denote the updated BaSTI-IAC model isochrone for 12.6 Gyr and \big[Fe/H\big] = $-$1.27 dex. The solid and dotted grey lines are ZAHB and TAHB, respectively.}
    \label{optuvhstcmds}
\end{figure*}

 The optical and UV-optical CMDs are overlaid with updated BaSTI-IAC isochrones (\citealp{2018ApJ...856..125H}). The updated BaSTI-IAC\footnote{\url{http://basti-iac.oa-abruzzo.inaf.it/}} isochrones are generated for an age 12.6 Gyr (\citealp{2013A&A...558A..53K}), a distance modulus of 16.21 mag (\citealp{2019RMxAA..55..337A}) and a metallicity \big[Fe/H\big] = $-$1.27 dex (\citealp{2009A&A...508..695C}) with helium abundance Y = 0.247, \big[$\alpha$/H\big] = 0, including overshooting, diffusion, and mass loss efficiency parameter $\eta$ = 0.3. The BaSTI-IAC model also provides HB model, which includes zero age HB (ZAHB), post-ZAHB tracks and end of the He phase known as terminal age HB (TAHB) with or without diffusion for a particular mass range. We generated the ZAHB and TAHB tracks for a metallicity \big[Fe/H\big] = $-$1.27 dex including diffusion. The BaSTI-IAC model does not provide the BSS model line. In order to define the location of BSSs in CMDs, we used BaSTI isochrones \citep{2004ApJ...612..168P} generated using FSPS code of \cite{2009ApJ...699..486C}, \cite{2010ApJ...712..833C}. The observed UVIT stellar magnitudes are corrected for reddening and extinction. We have adopted a reddening E(B$-$V) of 0.01 mag from (\citealp{2013A&A...558A..53K}) and the ratio of total-to-selective extinction as $R_v$ = 3.1 from (\citealp{1958AJ.....63..201W}) for the Milky Way. The extinction co-efficient in visible is $A_V$ = 0.031 mag. The $A_V$ is used to determine extinction co-efficients $A_\lambda$ for all pass bands using the reddening relation of \cite{1989ApJ...345..245C}. The BS sequence shown in optical as well as in UV CMDs is the extension of the zero-age main sequence (ZAMS). It should roughly follow the ZAMS up to twice the turn-off mass if BSSs are the product of mergers between two MS stars. The MS turn-off mass in NGC 1261 is approximately 0.8M$_{\odot}$. The brightest and faintest part of the BS sequence shown in all CMDs corresponds to 1.6M$_{\odot}$ and 1.1M$_{\odot}$, respectively.\\
 \begin{figure}
 \hspace*{-0.9cm}

\includegraphics[scale=0.5]{grd_gaia_opt_cmd_n2_uvit_Basti_isocz.pdf}
     \caption{Optical CMD of NGC 1261 for the region outside the HST field. HB and BSS stars that are detected in the NUV N279M UVIT filter and cross-matched with ground based photometric data (\protect\citealp{2010A&A...516A..23K}) and Gaia data are marked with different colors. Rest of the stars shown with black dots are cross-matched ground based data with Gaia data. Other details are same as in Figure~\protect\ref{opthstcmd}.
    \label{optgaiacmd}
\end{figure}

 \begin{figure*}
\hspace*{-1.4cm}

\includegraphics[height=20cm,width=20cm]{opt_uv_grd_gaia_uvit_Basti_isocz.pdf}
     \caption{UV-optical CMDs of NGC 1261 after cross-matching ground based photometric (\protect\citealp{2010A&A...516A..23K}) and Gaia data with UVIT data in 4 NUV and 2 FUV filters. Rest of the details are same as in figure~\protect\ref{optuvhstcmds}.

     }
    \label{optuvgaiacmds}
\end{figure*}

 In Figure~\ref{optuvhstcmds}, the top two panels display the FUV-optical CMDs, generated using the F169M and F172M filters. The overlaid isochrone helps in defining the location of HB stars and BSSs, which span a large range in colour and magnitude, when compared to the optical CMDs, suggesting FUV-optical CMDs have better resolution in colour at a given magnitude. The HB stars bluer and redder to the RR Lyrae stars are detected in the FUV, suggesting that the BHB and the RHB stars are detected. The HB sequence is well aligned with the isochrone suggesting that the predicted and observed FUV magnitudes match well. In the FUV, the BSSs are found to span a large range in magnitude, for a given colour, and vice versa. The rest of the 4 panels show the NUV-optical CMDs. In NUV-optical CMDs in Figure~\ref{optuvhstcmds}, we find that the BSSs are as hot as blue HB stars and also they span a wide range in colour and magnitude in all the NUV CMDs. We have detected full HB population in all NUV-optical CMDs. The HB population appears as a tight sequence more-or-less aligned with the isochrone. In the case of CMDs using N245M, N263M and N279N filters, the red end of the HB is fainter than the isochrone. As the photometric errors are also large at this limiting magnitude, our data is only suggestive. We also note that same stars are fainter than the isochrone in both CMDs where N263M and N279N magnitudes are shown.
 In all the CMDs, we have detected two stars at the blue extreme end of the HB. These stars are quite separated from the observed HB sequence and are likely to be very hot HB stars, as suggested by their UV-optical colour.\\

 \begin{table}
	\centering
	\caption{Number of detected HB stars and BSSs in different UVIT filters. Here $N_{HB}$, $N_{BSS}$, $N_{RRL}$ and $N_{Sx Phe}$ represents number of detected HB, BSS, RRL and Sx Phe stars respectively. The number of stars detected in the outer region ($>$ 3.4$'$ diameter) are shown in parentheses. }
	\label{tab:2}
\begin{tabular}{cccccc}
		\hline
		\hline
		 Filter & $N_{HB}$ & $N_{BSS}$  & $N_{RRL}$ & $N_{Sx Phe}$ \\
		\hline
		 F169M & 59(9) & 41(1) & 10(1) & 1 \\
		 F172M & 97(11) & 55(2) & 15(3) & 1 \\
		 N219M & 196(40) & 71(8) & 19(4) & 2 \\
		 N245M & 216(46) & 75(7) & 19(4) & 2 \\
	 	 N263M & 222(47) & 62(8) & 19(4) & 1 \\
		 N279N & 221(46) & 70(7) & 18(3) & 3 \\
		\hline
	\end{tabular}
\end{table}

 To identify stars in the outer region of the cluster, UVIT detected stars were first cross-matched with Gaia proper motion membership data provided by \cite{2019MNRAS.488.3024B}, followed by cross-match with the ground based photometric data (\citealp{2010A&A...516A..23K}). Here, the photometric system adopted to create optical CMD is standard Johnson-Cousin photometric system. To generate UV-optical CMDs, we have converted the vega magnitude system into the AB magnitude system using photometric calibration mentioned in \citealp {2007AJ....133..734B}. Our sample of HB and BSSs detected with UVIT is 90\% complete in the outer region, when compared to the number of stars detected with Gaia. The 10\% of the stars, which are not detected, are fainter than the detection limit of the UVIT. Figures~\ref{optgaiacmd} and \ref{optuvgaiacmds} present the optical and UV-optical CMDs, respectively. We can clearly notice that less number of HB stars and BSSs are detected in the outer region of the cluster (outside the $3.4\arcmin$ diameter), when compared to the inner region. More stars are detected in the NUV when compared to the FUV. We detect only BHB stars in FUV pass bands and both BHB and RHB stars in all the NUV pass bands. We have also detected yellow straggler stars (YSS), identified based on their location in the optical CMD (figure~\ref{optgaiacmd}). We also detect very less number of BSSs in outer region as compared to inner region of the cluster. We notice that the detected BSSs are all redder than the predicted BSS line, though this could be an artefact due to the less number of detected BSSs.
 The comparison of the number of HB stars and BSSs detected in inner and outer region suggests that these stars may be segregated towards the center of the cluster.

\section{Spectral Energy Distributions of bright HB stars}
\label{sec:4}
Two extreme HB stars are detected in the central region. As these stars are well separated from the rest of the HB, we aim to check the evolutionary status of these stars by estimating their stellar parameters. It is also important to compare the properties of the BHB with these extreme HB stars. In order to estimate the parameters like effective temperature ($T_{eff}$), luminosity ($\frac{L}{L_{\odot}}$) and radius ($\frac{R}{R_{\odot}}$) of the UV bright HB stars, we have constructed their spectral energy distributions (SEDs). SEDs are constructed with the observed photometric data points from FUV-to-Optical and fitted with selected theoretical models as discussed below. We have used the virtual observatory tool, VOSA (VO Sed Analyser, \citealp{2008A&A...492..277B}) for SED analysis. VOSA utilises the filter transmission curves to calculate the synthetic photometry of the selected theoretical model. By using the fixed distance to the cluster, synthetic fluxes are scaled with the observed fluxes. After constructing the synthetic SED, it performs a $\chi^2$ minimisation test to compare the observed with the synthetic photometry to find the best fit parameters of the SED. The expression used to estimate reduced $\chi^2_{red}$ is
\begin{equation*}
\hspace*{2.0cm}
   \chi^2_{red} = \frac{1}{N-N_{f}} \displaystyle \sum_{i=1}^{N} {\frac{(F_{o,i} - M_{d}F_{m,i})^2}{\sigma_{o,i}^2}}
\end{equation*}
where N is the number of photometric data points, $N_{f}$ is the number of free parameters in the model, $F_{o,i}$ is the observed flux, $M_{d}F_{m,i}$ is the model flux of the star,
$M_{d} = \big(\frac{R}{D}\big)^2$ is the scaling factor corresponding to the star (where R is the radius of the star and D is the distance to the star) and $\sigma_{o,i}$ is the error in the observed flux. The number of photometric data points N for stars varies from 9 to 13 depending upon their detection in different available filters. The number of free parameters ($N_{f}$) used to fit SED are \big[Fe/H\big], log(g) and effective temperature. The radius of the stars were calculated by using scaling factor, $M_{d}$. As discussed below, \big[Fe/H\big] and log(g) are fixed and are not anymore free parameters in fitting the SED. \\

\begin{figure*}
\centering
\begin{subfigure}{0.5\textwidth}
  \centering
  \includegraphics[height=8.0cm,width=9.0cm]{SED_EHB1_Plot.pdf}
  \end{subfigure}%
\begin{subfigure}{0.5\textwidth}
  \centering
  \includegraphics[height=8.0cm,width=9.0cm]{SED_EHB2_Plot.pdf}
\end{subfigure}
\caption{Spectral energy distribution (SED) of EHB1 and EHB2 stars after correcting for extinction. The best fit parameters are shown in the figure.}
\label{seds}
\end{figure*}

We have used Kurucz stellar atmospheric models to generate SEDs \citep{1997A&A...318..841C, 2003IAUS..210P.A20C} for the bright HBs which cover the UV to IR wavelength range. We fixed the value of metallicity \big[Fe/H\big] = $-$1.5, close to the cluster metallicity and had given the range of effective temperature from 5000-50000 K and corresponding log $g$ from 3-5 dex for the adopted Kurucz models to fit the SED of HBs (see Figure 19 in \citealp{2001MNRAS.324..937P}).
We have combined 6 UVIT photometric data points  with 5 HST photometric data points from \cite{2018MNRAS.481.3382N} to generate SED for UV bright HB stars detected in the inner region of the cluster. For those detected in the outer region of the cluster, the photometric data points of UVIT (6 pass bands) with GAIA (3 pass bands) (\citealp{2018A&A...616A..12G}) and ground photometry (4 pass bands) (\citealp{2010A&A...516A..23K}) were combined. As errors in the HST measured flux are small which blows up the $\chi^2$ value for the HST data points, we assumed no error, which does not impact the fit parameters. In case, there are photometric data points with a zero observational error, VOSA assumes the biggest relative error present in the SED. VOSA makes use of Fitzpatrick reddening relation \citep{1999PASP..111...63F, 2005ApJ...619..931I} to correct for extinction in observed data points. The extinction corrected VOSA magnitudes are in a good agreement with that described in Section~\ref{sec:3}.\\

Best SED fits were obtained for 26 HB stars, out of which 2 are EHB stars and 24 are BHB stars. The SEDs of 2 EHB stars are shown in Figure~\ref{seds}. The estimated values of parameters  $T_{eff}$, $\frac{R}{R_{\odot}}$ and $\frac{L}{L_{\odot}}$ corresponding to the best fitting Kurucz model spectrum along with the errors for 26 HB stars are shown in Table~\ref{tab:3}.
In left panel of Figure~\ref{seds}, we notice that the observed data points are well fitted with the Kurucz model spectrum, but in the case of EHB2 star, three UVIT data points are not fitted with the model spectrum. There seems to be some amount of excess flux in these UVIT filters, as also reflected in figure \ref{optuvhstcmds}. We excluded these three data points to make the good fit of the observed data points to the Kurucz model spectrum. The high temperatures of EHB1 (31000 K) and EHB2 (31000 K) suggest that they belong to the class of EHB stars (\citealp{1986A&A...155...33H}). The BHB stars have a range of temperature, 8000 - 12750 K, with one star with 12750 K. The SEDs of these stars are presented in Appendix-A.\\

We also note from Table~\ref{tab:3} that the luminosity, $\frac{L}{L_{\odot}}$, is more or less constant for the HB stars (except for a few cases), whereas the $T_{eff}$ is found to increase with a corresponding decrease in radius.
As we have a good number of BHB stars, we estimate this relation assuming a constant luminosity.
The Figure~\ref{tempradvar} shows the variation of effective temperature of all bright HB stars with their corresponding radius. The best-fit relation to the observed points plotted in Figure~\ref{tempradvar} is :
\begin{equation*}
\hspace{2cm}
    T_{eff} = 0.5\sqrt{\frac{R}{R_{\odot}}} + 0.002
\end{equation*}
This relation corresponds to the well-known Stefan-Boltzmann law. Since the intrinsic luminosity of the BHB and EHB stars is not very different, so they should satisfy a quite similar $T_{eff}$-R relation.
We therefore extended the plot to higher $T_{eff}$ and included the EHB stars.
This relation is found to fit the EHB stars quite well as shown in the black curve in Figure~\ref{tempradvar}. The value of goodness of fit parameter for this curve-fit is 0.99, which is close to 1, hence, a very good fit. This fit also confirms the accuracy of the derived parameters from the SED fit.\\

In order to check the evolutionary status of EHB stars identified with UVIT, we have plotted the theoretical evolutionary tracks using the models presented by \cite{2019A&A...627A..34M}. The models used in their paper were computed by the extension of the PAGB evolutionary models by \cite{2016A&A...588A..25M}. We have selected the model with metallicity close to the cluster metallicity. The theoretical evolutionary tracks corresponding to different masses starting from the zero age horizontal branch (ZAHB) through to a point late in post-HB evolution or a point on the PAGB cooling track are shown in Figure~\ref{evotrack}. The terminal age HB (TAHB) representing the end of the HB phase is shown with dash-dotted line in Figure~\ref{evotrack}. We can see in Figure~\ref{evotrack} that all BHB stars are lying along the BHB tracks. Figure~\ref{evotrack} also shows that two EHB stars are indeed found along EHB tracks lying below the TAHB. So, these two stars do not belong to any of the  evolved class of stars, which lie on hot PAGB cooling sequence. This suggests that the two EHB stars share the properties of the BHB stars and therefore can be considered as the extreme extension of the HB. They are therefore still in the HB evolutionary phase. From the model, the masses of these two stars turn out to be approximately 0.5M$_{\odot}$ corresponding to the core mass of the stars lying on the HB.

\begin{table*}
\centering
     \caption{SED fit parameters of bright HB stars detected using UVIT data.}
    \label{tab:3}
    \large
	\makebox[0.79\linewidth]
	{
	\begin{tabular}{cccccccc}
		\hline
		\hline
		 Star ID & RA (deg) & DEC (deg) & $T_{eff}$ (K) & $\frac{L}{L_{\odot}}$  & $\frac{R}{R_{\odot}}$ & ${\chi}_{red}^2$ \\
		\hline
		 EHB1 & 48.04181 & -55.20597 & $31000 \pm 500$ & $38.92 \pm 1.09$ & $0.21 \pm 0.01$  & 2.7\\
		 EHB2 & 48.07732 & -55.22445 & $31000 \pm 500$ & $41.28 \pm 0.9$ & $0.22 \pm 0.01$ & 8.6 \\
		 BHB1 & 48.07928 & -55.22457 & $12750 \pm 125$ &  $44.26 \pm 1.64$ & $1.36 \pm 0.03$ & 4.6 \\
		 BHB2 & 48.09452 & -55.2267 & $11000 \pm 125$ & $43.46 \pm 1.96$ & $1.79 \pm 0.04$ & 6.9 \\
	 	 BHB3 & 48.08742 & -55.22956 & $11750 \pm 125$ &  $48.75 \pm 2.58$ & $1.69\pm 0.04$ & 2.9  \\
		 BHB4 & 48.06405 & -55.21413 & $11250 \pm 125$ & $42.77 \pm 1.87$  & $1.71 \pm 0.04$ & 3.6 \\
		 BHB5 & 48.07904 & -55.21753 & $11250 \pm 125$ & $45.97 \pm 2.25$ & $1.79 \pm 0.04$ & 4.3\\
		 BHB6 & 48.05424 & -55.21919 & $11000 \pm 125$ & $46.91 \pm 1.87$ & $1.89 \pm 0.04$ & 4.5 \\
		 BHB7 & 48.05896 & -55.21738 & $10500 \pm 125$ &  $49.12 \pm 2.51$ & $2.11 \pm 0.05$ & 7.3 \\
		 BHB8 & 48.08082 & -55.22729 & $10000 \pm 125$ & $46.70 \pm 2.09$ & $2.23 \pm 0.05$ & 6.3\\
		 BHB9 & 48.06626 & -55.21944 & $9500 \pm 125$ & $47.28 \pm 2.06$ & $2.52 \pm 0.06$ & 6.8\\
		 BHB10 & 48.06391 & -55.21627 & $9250 \pm 125$ &  $44.54 \pm 1.59$ & $2.59 \pm 0.06$ & 5.9\\
		 BHB11 & 48.04864 & -55.21555 & $9250 \pm 125$ & $56.64 \pm 1.89$ & $2.94 \pm 0.07$ & 4.2 \\
		 BHB12 & 48.05479 & -55.22369 & $8750 \pm 125$ & $48.88 \pm 1.41$ & $3.05 \pm 0.07$ & 3.6 \\
		 BHB13 & 48.10376 & -55.21723 & $8500 \pm 125$ & $49.55 \pm 1.57$ & $3.23 \pm 0.08$ & 5.0\\
		 BHB14 & 48.08487 & -55.22036 & $8750 \pm 125$ & $49.63 \pm 1.83$ & $3.08 \pm 0.07$ & 2.1\\
		 BHB15 & 48.06478 & -55.21766 & $8750 \pm 125$ & $49.42 \pm 1.64$ & $3.04 \pm 0.07$ & 5.5\\
		 BHB16 & 48.07389 & -55.22981 & $8250 \pm 125$ &  $49.79 \pm 1.69$& $3.47 \pm 0.08$ & 4.1\\
		 BHB17 & 48.04274 & -55.22656 & $8000 \pm 125$ &  $51.07 \pm 1.85$ & $3.73 \pm 0.09$ & 3.4 \\
		 BHB18 & 48.06466 & -55.21258 & $8000 \pm 125$ &  $50.06 \pm 1.55$ & $3.69 \pm 0.09$ & 5.0\\
		 BHB19 & 48.05484 & -55.18698 & $11500 \pm 125$ &  $46.69 \pm 3.98$ & $1.69 \pm 0.04$ & 5.7\\
		 BHB20 & 48.14102 & -55.20569 & $10000 \pm 125$ & $47.64 \pm 4.98$& $2.28 \pm 0.05$ & 4.6\\
		 BHB21 & 48.01035 & -55.31072 & $9750 \pm 125$ & $47.85 \pm 5.55$ & $2.42 \pm 0.06$ & 3.9 \\
		 BHB22 & 48.12378 & -55.25798 & $9500 \pm 125$ & $48.24 \pm 7.88$ & $2.54 \pm 0.06$ & 1.8 \\
		 BHB23 & 47.90097 & -55.23117 & $8750 \pm 125$ & $59.04 \pm 10.44$ & $3.32 \pm 0.07$ & 1.5\\
		 BHB24 & 48.14307 & -55.27254 & $8250 \pm 125$ & $51.27 \pm 13.41$ & $3.54 \pm 0.08$ & 1.8 \\
		\hline
	\end{tabular}
	}
\end{table*}

\begin{figure}
    \hspace{-0.8cm}
    \includegraphics[height=9.5cm,width=9.5cm]{hb_rad_temp.pdf}
    \caption{ Variation of radius of all bright HB stars with their effective temperatures, determined from SEDs. Red filled circles are representing bright HB stars and the black curve representing the function $T_{eff} = \frac{0.5}{\sqrt{R}} + 0.002$, is the fitted curve to the observed distribution.}
    \label{tempradvar}
\end{figure}
\begin{figure}
\hspace{-0.8cm}
\includegraphics[height=9.5cm,width=9.5cm]{Theoretical_evo_track_HB.pdf}
\caption{Evolutionary tracks corresponding to different masses with metallicity close to cluster metallicity. Evolution starting from HB phase to the moment when star has entered to the post-HB phase (\protect\citealp{2019A&A...627A..34M}) are shown. In the plot, magenta, blue and green colours represent the sequences populating the extreme, blue and red parts of the HB. Black dashed and dash-dotted line indicates the ZAHB and TAHB, respectively. Bright HB stars identified with UVIT are shown with red cross symbols.}
\label{evotrack}
\end{figure}

\section{Discussion}
\label{sec:5}
In this study, we used data from UVIT onboard Astrosat satellite to create NUV and FUV CMDs and to investigate the morphology of HB stars. UVIT is not able to detect stars in the MS, RGB and SGB evolutionary phases, since these stars are fainter than 22 mag in UV. To generate UV CMDs, we cross-matched UVIT detected stars with HST data for inner region corresponding to $\sim3.4'$ diameter and with ground based data along with GAIA for outer region as UVIT has a large circular field of view of $28{\arcmin}$ diameter. The optical and UV CMDs are created for the proper motion members of the cluster. In all UV-optical CMDs, we find that the HB stars do not show the horizontal distribution, as found in the  optical CMDs. This is due to the fact that the HB stars have a range in temperature and the flux in UV pass bands are more sensitive to temperature than in optical. The HB feature is therefore found to be slanting and is similar to that found in the GALEX UV CMDs obtained by \cite{2012AJ....143..121S}. They suggest that the slope of HB is mainly a result of bolometric correction effects. \citealp{2013MNRAS.430..459D} investigated the HB morphology of three GCs, namely, M3, M13 and M79, using HST data in optical and UV bandpasses, and they also found the similar HB distribution in Far-UV CMDs (see their Fig. 2). The UV magnitude distribution of the HB stars is found to closely match with the over-plotted isochrone, within error, in all UV-optical CMDs. As we detect the full stretch of the HB including the RHB, RRL and BHB, we find a good match for the UV magnitudes between the observation and the updated BaSTI-IAC models.\\

A large gap is observed between EHBs and BHBs, and that needs to be explored. However, in massive and very dense GCs such as NGC 2808, NGC 6752, $\omega$ cen,  a large number of EHB stars are found with a well defined sequence in the optical CMDs. In less massive clusters, they are less in number and are hard to detect at the faint end of the HB in optical CMDs. In these cases, UV CMDs play an important role since EHB stars are bright in UV CMDs and also follow a separate sequence. The clusters with a few stars bluer than the M-jump are NGC 5466, NGC 6981, NGC 3201, NGC 2298 etc. (see Figure 5 in \citealp{2016ApJ...822...44B}). We have detected full HB in NUV and only hot HB stars in FUV. Two EHB stars are also identified in all UV-optical CMDs, which have bluer colours as compared to other hot HB stars. \cite{2015MNRAS.451..275V} studied 48 galactic GCs to explore the radial distribution of multiple stellar populations known to exist in clusters. They identified 6 EHB stars in NGC 1261, which may or may not be the members of the cluster. They also used the different selection criteria to select different HB sub-populations. The HB stars identified by UVIT are proper motion members of the cluster, including the two EHB stars.\\

The bright HB stars are found to have a range of temperature, from 8000 K to 12750 K, with the hot end very close to G-jump (\citealp{1999ApJ...524..242G}) ($T_{eff}$ = 11,500 K), in the HB distribution . One star is found to have 12750 K along with the two hotter stars. The hot end of the HB distribution therefore, coincides with the G-jump, in this cluster. There is only one star slightly hotter and there is no further extension of the HB to the EHB, beyond the G-jump. Therefore, we do not detect M-jump (\citealp{2002ApJ...576L..65M, 2004A&A...420..605M}), expected at $\sim$ 23,000 K. Instead, we detect two stars with $T_{eff}$ = 31,000 K. These coincide with the $T_{eff}$ expected for the gap between EHB and the blue-hook stars.
The cluster NGC 1261, therefore has an HB distribution truncating at the G-jump at the hot end, and with the presence of two EHB stars.\\

With the estimation of the surface parameters of the BHB stars, we are able to derive a relation between the temperature and radius, as the bolometric luminosity is found to be almost the same for these stars. We find that the temperature and radii of the two EHB stars agree with the relation derived above, suggesting that they can be considered as the extreme end of the HB population.  Thus, in this cluster the EHB stars are normal core He burning stars like other HB stars and are not in any different evolutionary phase. This is supported by their position in the HR diagram overlaid with isochrones. It must be noted that these two EHB stars may be starting to evolve off from the HB.\\

The canonical picture of the EHB stars is established by \cite{1986A&A...155...33H}, in which EHB stars are helium core burning stars with masses close to the core helium flash mass of 0.47M$_{\odot}$, and an extremely thin hydrogen envelope, not more than, 1$\%$ by mass. Our estimation of mass for the detected EHB stars match well with this definition. Although the evolution of EHB stars after the exhaustion of Helium in the core is better understood, the formation pathways that lead to the EHB is much less understood. Many have explored the binary nature of EHB stars and a wide variety of companions are found as the binary fraction among the EHB stars are found to be much higher than normal stars. The way in which the envelope is lost, is mostly attributed to binary interactions, depending on the mass ratio of the system. \cite{2015MNRAS.449.2741L} suggested that tidally enhanced stellar wind in binary evolution is able to provide the enhanced mass loss on the RGB needed for the late hot flash scenario to explain the formation of blue-hook stars. They have adopted different initial orbital periods for binaries to explain the formation of canonical HB and blue-hook stars. The EHB stars detected in this cluster are likely to be single stars, as suggested by the SEDs. The SEDs are well fitted by a single spectrum with minimum residual across the wavelength. It is proposed by \cite{1993ApJ...407..649C} that the delayed helium flash (HEF) is a promosing scenario to explain the existence of EHB stars. Due to high mass loss during RGB evolutionary phase, a star will lose so much envelope mass that it will not be able to ignite the helium flash at the tip of the RGB, thus evolving towards the WD cooling sequence with an electron degenerate core. Depending on the residual mass of the envelope, a star will undergo He flash either at the bright end of the WD cooling sequence, known as early hot flasher (EHF) or along the WD cooling sequence, known as late hot flasher (LHF). After the He flash, these stars will settle on a blue-hook at the hot end of the HB. As these stars have hugely reduced envelope mass, they will be much hotter than their counterparts on the canonical ZAHB. \cite{1996ApJ...466..359D} also suggested hot He-flash scenario to explain the origin of EHB stars in globular clusters. \cite{2003ApJ...582L..43C} computed the models of the population II low mass stars through the Helium flash mixing (HEFM) phase, suggested that the HEFM scenario as a plausible explanation for the existence of blue hook stars
\\

Here we summarise the possible scenarios for the formation of single EHB stars.
\cite{1984ApJ...277..355W} and \cite{2000MNRAS.313..671S} suggested the merger of two He-core WDs as a possible mechanism. Another possibility is the trigger of CE ejection by a giant planet, that evaporates in the process (\citealp{1998AJ....116.1308S}). The fraction of close binaries among the EHB stars in GCs is found to be very small \citep{2006A&A...451..499M, 2009A&A...498..737M, 2011A&A...528A.127M}. It is possible that in old systems like the GCs, WD-WD merger may likely result in the formation of EHB stars, as a consequence they are single stars. If these two stars are the product of the merger of two He-core WDs, then they can be helium rich, which will lead to higher effective temperature compared to estimated using Kurucz stellar atmospheric models (\citealp{1984ApJ...278..702S}). Another model suggested for the EHB star formation is primordial enrichment in helium \citep{2002A&A...395...69D}. In this scenario, the EHB stars are produced via the normal evolution of He-enriched sub-populations in GCs. These sub-populations might have formed out of the material polluted by the ejecta from massive AGB stars. For a given age and metallicity, He enhanced stars have smaller masses than He normal stars, resulting in a bluer HB morphology. A super-solar surface He abundance cause a huge mass loss on RGB phase by increasing the RGB tip luminosity, but the phenomena responsible for a huge He enhancement invoke non-canonical mixing during the RGB stage, and dredge-up induced by H-shell instabilities \citep{1979ApJ...229..624S, 1988ApJ...324..840V, 1997ApJ...474L..23S, 2003ApJ...598.1246D}. The discovery of multiple stellar populations in GCs opened a new frontier, as one of the causes is the variation in Helium enrichment, with the blue HB stars produced by Helium rich stars \citep{2004MSAIS...5..105B,2005ApJ...621..777P,2007ApJ...661L..53P,2012ApJ...760...39P,2015AJ....149...91P}. The models of multiple populations with different helium abundances successfully reproduce both the MS splitting and the multi-modal HB morphology of both $\omega$ Cen (\citealp{2005ApJ...621L..57L}) and NGC 2808 (\citealp{2005ApJ...631..868D}). The He-enhancement thus represents a promising model, and alternative to the binary scenario, for the formation of EHB stars in GCs. We find that both the EHB stars are single stars and estimate their masses to be 0.495 M$_\odot$, which is quite close to the theoretical estimate for EHB stars. The possible formation mechanism for these stars is likely to be the enhanced mass loss in the RGB phase, either due to rotation or enhanced helium (primordial or mixing).\\

 The comprehensive studies of EHB and blue-hook stars by \cite{2001ApJ...562..368B,2010ApJ...718.1332B} suggest that these stars could be produced by delayed helium-flash after the RGB, and are termed as early hot-flashers and late hot-flashers. In the case of early hot-flashers, a star ignites helium during the evolution to the top of the WD cooling curve. Flash mixing will not happen between the envelope and the core of a star as the high entropy of a strong hydrogen-burning shell acts as a barrier that prevents the flash convection from penetrating into the envelope. Hence, the envelope mass or composition of a star will remain unchanged, which leads to more or less the same luminosity compared to the canonical HB stars ( See Figures 1 \& 4 in \citealp{2010ApJ...718.1332B}). Late hot-flash scenario results in the flash mixing between the hydrogen envelope and the helium core, which increases the helium and carbon abundances in the envelope. The flash mixing lowers the luminosity of a star with respect to the canonical EHB star. The EHB stars of this cluster are not sub-luminous with respect to the BHB stars and hence are unlikely to be late hot-flashers. These stars therefore could be early hot-flash objects, which follow the properties of the canonical HB stars.
 Spectroscopic observations are required to explore the nature of these two stars and validate the formation scenarios described above. Since there is not enough mass for the hydrogen shell surrounding the core in EHB stars, these stars are likely to evolve directly to the white dwarf phase.\\

All the detected BSSs occupy a region parallel to HB sequence as well as span a wide range in both colour and magnitude, in the UV-optical CMDs. In general, the temperature  of BSSs ranges from 6000-8000K, but in this cluster, we notice that some of the BSSs have colours similar to the BHB stars, which in turn imply that BSSs may be as hot as the BHB stars. We detected a few BSSs in the outer region of the cluster beyond half-mass radius (r>r$_{h}$). These BSSs might have formed through mass-transfer in close binary systems, which dominates in a low-density environment. Mass-transfer models are required to confirm this.
We also detect two YSSs in the outer part of the cluster. We plan to construct the SEDs of the BSSs as well as the YSSs to understand the properties of these stars.
\section{Summary and Conclusions}
\label{sec:6}
 In this paper, we present the photometric results of NGC 1261 imaged using the UVIT on ASTROSAT. The advantage of using UVIT over HST  is its large field of view covering the full cluster region and over GALEX is its good spatial resolution and multiple filters
 We have characterized the HB member stars for the first time in this cluster using UVIT, HST, ground-based and GAIA data. Below, we summarise the important results from this study
\begin{itemize}
    \item We constructed Optical and UV-optical CMDs of the member stars and overlaid with isochrones generated for respective filters. We detected only BHB and 2 EHB stars in the FUV CMDs whereas the full HB is detected in NUV CMDs. We also detected BSSs which span a wide range in magnitude as well as in colour in NUV CMDs.\\

    \item The effective temperatures, luminosities and radii of 24 BHB and 2 EHB stars are estimated by generating SED using multi-wavelength data. The $T_{eff}$ of BHB stars range from 8000-12750 K whereas EHB stars have $T_{eff}$ more than 30000 K. \\
    \item Keeping the L constant, we fitted a R versus $T_{eff}$ relation for the BHB stars, which is found to fit the EHB stars as well. Their location in HR diagram overlaid with isochrones confirm that they are EHB stars with a mass of $\sim$ 0.5M$_\odot$.\\

\end{itemize}
   Based on the results from the UVIT study of this cluster, we conclude that:
\begin{itemize}
    \item Most of the RHB stars are too faint to be detected by UVIT in FUV filters. HB stars form a tight sequence in the UV-optical CMDs which is largely fitted by the BaSTI-IAC isochrones.\\

    \item The EHB stars in this cluster are likely to be single stars. We constrain the formation pathways of these single EHB stars to extreme mass loss in the RGB phase (either due to rotation or enhanced Helium), OR early hot-flash scenario.\\

\end{itemize}
\section*{Acknowledgements}
We are thankful to the reviewer for the thoughtful comments and suggestions which improved the quality of the manuscript. UVIT project is a result of collaboration between IIA, Bengaluru, IUCAA, Pune, TIFR, Mumbai, several centres of ISRO, and CSA. This publication uses the data from the \textit{ASTROSAT} mission of the Indian Space Research  Organisation  (ISRO),  archived  at  the  Indian  Space  Science  Data Centre (ISSDC). We want to thank Iv\'an H. Bustos Fierro for providing us with GAIA proper motion membership data. This research made use of VOSA, developed under the Spanish Virtual Observatory project supported by the Spanish MINECO through grant AyA2017-84089. This research also made use of Topcat \citep{2005ASPC..347...29T, 2011ascl.soft01010T}, Matplotlib (\citealp{2007CSE.....9...90H}), NumPy (\citealp{2011CSE....13b..22V}), Scipy \citep{2007CSE.....9c..10O, article}, Astropy \citep{2018AJ....156..123A} and Pandas (\citealp{mckinney-proc-scipy-2010}).

\section*{Data availability}
The UVIT data used in this article will be shared on request.

\bibliographystyle{mnras}


\section{Experiments} \label{sec:experiments}

In this section, we conduct extensive experiments to evaluate the Meta-KD framework on two popular text mining tasks across domains.

\subsection{Tasks and Datasets}

We evaluate Meta-KD over natural language inference and sentiment analysis, using the following two datasets. The overall statistics are shown in Table \ref{table:dataset}.
\begin{itemize}
\item \textbf{MNLI} ~\citep{MNLI_wiliam} is a large-scale, multi-domain natural language inference dataset for predicting the entailment relation between two sentences, containing five domains (genres). After filtering samples with no labels available, we use the original development set as our test set and randomly sample 10\% of the training data as development set in our setting.
\item \textbf{Amazon Reviews} ~\citep{DBLP:conf/acl/BlitzerDP07} is a multi-domain sentiment analysis dataset, widely used in multi-domain text classification tasks. The reviews are annotated as positive or negative. For each domain, there are 2,000 labeled reviews. We randomly split the data into train, development and test sets.
\end{itemize}

\begin{table}[!h]
 \centering
 \begin{small}
 \begin{tabular}{lcccc}
  \toprule
  Dataset & Domain  & \#Train & \#Dev & \#Test  \\
  \midrule
  \multirow{5}{*}{MNLI} & Fiction & 69,613 & 7,735 & 1,973 \\
   & Government & 69,615 & 7,735 & 1,945 \\
  & Slate & 69,575 & 7,731 & 1,955 \\
  & Telephone & 75,013 & 8,335 & 1,966 \\
  & Travel & 69,615 & 7,735 & 1,976 \\
  \midrule

  & Book & 1,631 & 170 & 199 \\
Amazon & DVD & 1,621 & 194 & 185 \\
Reviews & Electronics & 1,615 & 172 & 213 \\
  & Kitchen & 1,613 & 184 & 203 \\
 \bottomrule
 \end{tabular}
 \end{small}
 \caption{
    Statistics of the two datasets.
 }
\label{table:dataset}
\end{table}

\begin{table*}[!h]
 \centering
 \begin{small}
 \begin{tabular}{lcccccc}
  \toprule
  Methods  & Fiction & Government & Slate & Telephone & Travel & Average  \\
  \midrule

$\text{BERT}_{\text{B}}\text{-single}$ & 82.2 & 84.2 & 76.7 & 82.4 & 84.2 & 81.9 \\
$\text{BERT}_{\text{B}}\text{-mix}$ & 84.8 & 87.2 & 80.5 & 83.8 & 85.5 & 84.4 \\
$\text{BERT}_{\text{B}}\text{-mtl}$ & 83.7 & 87.1 & 80.6 & 83.9 & 85.8 & 84.2 \\
Meta-teacher & 85.1 & 86.5 & 81.0 & 83.9 & 85.5 & 84.4 \\
  \midrule

$\text{BERT}_{\text{B}}\text{-single}$ $\xrightarrow{\text{TinyBERT-KD}}$  $\text{BERT}_{\text{S}}$ & 78.8 & 83.2 & 73.6 & 78.8 & 81.9 & 79.3 \\

$\text{BERT}_{\text{B}}\text{-mix}$ $\xrightarrow{\text{TinyBERT-KD}}$  $\text{BERT}_{\text{S}}$ & 79.6 & 83.3 & 74.8 & 79.0 & 81.5 & 79.6 \\
$\text{BERT}_{\text{B}}\text{-mtl}$ $\xrightarrow{\text{TinyBERT-KD}}$  $\text{BERT}_{\text{S}}$ & 79.7 & 83.1 & 74.2 & 79.3 & 82.0 & 79.7 \\
$\text{Multi-teachers}$ $\xrightarrow{\text{MTN-KD}}$  $\text{BERT}_{\text{S}}$ & 77.4 & 81.1 & 72.2 & 77.2 & 78.0 & 77.2 \\
  \midrule
$\text{Meta-teacher}$ $\xrightarrow{\text{TinyBERT-KD}}$ $\text{BERT}_{\text{S}}$ & 80.3 & 83.0 & \textbf{75.1} & 80.2 & 81.6 & 80.0 \\
$\text{Meta-teacher}$ $\xrightarrow{\text{Meta-distillation}}$ $\text{BERT}_{\text{S}}$ & \textbf{80.5} & \textbf{83.7} & 75.0 & \textbf{80.5} & \textbf{82.1} & \textbf{80.4} \\

 \bottomrule
 \end{tabular}
 \end{small}
 \caption{
    Results over MNLI (with five domains) in terms of accuracy (\%). Here $X \xrightarrow{\text{A}} Y$ means it uses $X$ as the teacher and $Y$ as the student, with $A$ as the KD method, hereinafter the same.
 }
\label{table:results-mnli}
\end{table*}

\begin{table*}[!h]
 \centering
 \begin{small}
 \begin{tabular}{lccccc}
  \toprule
  Methods  & Books & DVD & Electronics & Kitchen & Average  \\
  \midrule
$\text{BERT}_{\text{B}}\text{-single}$ & 87.9 & 83.8 & 89.2 & 90.6 & 87.9  \\
$\text{BERT}_{\text{B}}\text{-mix}$ & 89.9 & 85.9 & 90.1 & 92.1 & 89.5 \\
$\text{BERT}_{\text{B}}\text{-mtl}$ & 90.5 & 86.5 & 91.1 & 91.1 & 89.8 \\
Meta-teacher & 92.5 & 87.0 & 91.1 & 89.2 & 89.9 \\
 \midrule
$\text{BERT}_{\text{B}}\text{-single}$ $\xrightarrow{\text{TinyBERT-KD}}$  $\text{BERT}_{\text{S}}$ & 83.4 & 83.2 & 89.2 & 91.1 & 86.7 \\
$\text{BERT}_{\text{B}}\text{-mix}$ $\xrightarrow{\text{TinyBERT-KD}}$  $\text{BERT}_{\text{S}}$ & 88.4 & 81.6 & 89.7 & 89.7 & 87.3 \\
$\text{BERT}_{\text{B}}\text{-mtl}$ $\xrightarrow{\text{TinyBERT-KD}}$  $\text{BERT}_{\text{S}}$ & 90.5 & 81.6 & 88.7 & 90.1 & 87.7 \\
$\text{Multi-teachers}$ $\xrightarrow{\text{MTN-KD}}$  $\text{BERT}_{\text{S}}$ & 83.9 & 78.4 & 88.7 & 87.7 & 84.7 \\
 \midrule
$\text{Meta-teacher}$ $\xrightarrow{\text{TinyBERT-KD}}$ $\text{BERT}_{\text{S}}$ & 89.9 & 84.3 & 87.3 & \textbf{91.6} & 88.3 \\
$\text{Meta-teacher}$ $\xrightarrow{\text{Meta Distillation}}$ $\text{BERT}_{\text{S}}$ & \textbf{91.5} & \textbf{86.5} & \textbf{90.1} & 89.7 & \textbf{89.4} \\
 \bottomrule
 \end{tabular}
 \end{small}
 \caption{
    Results over Amazon reviews (with four domains) in terms of accuracy (\%).
 }
\label{table:results-amazon}
\end{table*}

\subsection{Baselines}
For the teacher side, to evaluate the cross-domain distillation power of the meta-teacher model, we consider the following models as baseline teachers:

\noindent \textbf{BERT-single}: Train the BERT teacher model on the target distillation domain only. If we have $K$ domains, then we will have $K$ BERT-single teachers.

\noindent \textbf{BERT-mix}: Train the BERT teacher on a combination of $K$-domain datasets. Hence, we have one BERT-mix model as the teacher model for all domains.

\noindent \textbf{BERT-mtl}: It is similar to the ``one-teacher'' paradigm as in BERT-mix, but the teacher model is generated by multi-task fine-tuning~\citep{DBLP:conf/cncl/SunQXH19}.

\noindent \textbf{Multi-teachers}: It uses all the $K$ domain-specific BERT-single models to supervise $K$ student models, ignoring the domain difference.

For the student side, we follow TinyBERT ~\citep{DBLP:journals/corr/abs-1909-10351} to use smaller BERT models as our student models. In single-teacher baselines (i.e., BERT-single, BERT-mix and BERT-mtl), we use  TinyBERT-KD as our baseline KD approach. In multi-teachers, because TinyBERT does not naturally support distilling from multiple teacher models, we implement a variant of the TinyBERT KD process based on MTN-KD~\citep{DBLP:conf/kdd/YouX0T17}, which uses averaged softened outputs as the incorporation of multiple teacher networks in the output layer. In practice, we first learn the representations of the student models by TinyBERT, then apply MTN-KD for output-layer KD.

\subsection{Implementation Details}

In the implementation, we use the original $\textbf{BERT}_{\textbf{B}}$ model (L=12, H=768, A=12, Total Parameters=110M) as the initialization of all of the teachers, and use the $\textbf{BERT}_{\textbf{S}}$ model (L=4, H=312, A=12, Total Parameters=14.5M) as the initialization of all the students\footnote{https://github.com/huawei-noah/Pretrained-Language-Model/tree/master/TinyBERT}.

The hyper-parameter settings of the meta-teacher model is as follows. We train 3-4 epochs with the learning rate to be 2e-5. The batch size and $\gamma_1$ are chosen from \{16, 32, 48\} and \{0.1, 0.2, 0.5\}, respectively. All the hyper-parameters are tuned on the development sets.

For meta-distillation, we choose the hidden layers in \{3, 6, 9, 12 \} of the teacher models in the baselines and the meta-teacher model in our approach to learn the representations of the student models. Due to domain difference, we train student models in 3-10 epochs, with a learning rate of 5e-5. The batch size and $\gamma_2$ are tuned from \{32, 256\} and \{0.1, 0.2, 0.3, 0.4, 0.5 \} for intermediate-layer distillation, respectively. Following~\citep{DBLP:journals/corr/abs-1909-10351}, for prediction-layer distillation, we run the method for 3 epochs, with the batch size and learning rate to be 32 and 3e-5. The experiments are implemented on PyTorch and run on 8 Tsela V100 GPUs.

\subsection{Experimental Results}

Table~\ref{table:results-mnli} and Table~\ref{table:results-amazon} show the general testing performance over MNLI and Amazon Reviews of baselines and Meta-KD, in terms of accuracy. From the results, we have the following three major insights:
\begin{itemize}
    \item Compared to all the baseline teacher models, using the meta-teacher for KD consistently achieves the highest accuracy in both datasets. Our method can help to significantly reduce model size while preserving similar performance, especially in Amazon review, we reduce the model size to 7.5x smaller with only minor performance drop (from 89.9 to 89.4).
    \item The meta-teacher has similar performance as BERT-mix and BERT-mtl, but shows to be a better teacher for distillation, as ``Meta-teacher $\rightarrow$ TinyBERT/Meta-distillation'' have better performance than other methods. This shows the meta-teacher is capable of learning more transferable knowledge to help the student. The fact that ``Meta-teacher $\rightarrow$ Meta-distillation'' has better performance than other distillation methods confirms the effectiveness of the proposed Meta-KD method.

    \item Generally speaking, Meta-KD gains more improvement on the small datasets than large ones, e.g. it improves from 86.7 to 89.4 in Amazon Reviews while 79.3 to 80.4 in MNLI. This motivates us to explore our model performance on domains with few or no training samples, i.e. few-shot and zero-shot learning in the following section.
\end{itemize}

\subsection{Few/Zero Shot Learning}

We further investigate Meta-KD's capability in the few-shot and zero-shot learning settings.

\noindent \textbf{Few-shot Learning.} We randomly sample a part of the MNLI dataset as the training data in the few-shot experiments. The sample rates that we choose include 0.01, 0.02, 0.05, 0.1 and 0.2. The sampled domain datasets are employed for training student models when learning from the in-domain teacher or the meta-teacher. The experimental results are shown in Figure \ref{fig:few-shot}, with results reported by the improvement rate in averaged accuracy. The experimental results show that when less data is available, the improvement rate is much larger. For example, when we only have 1\% of the original MNLI training data, the accuracy can be increased by approximately 10\% when the student tries to learn from the meta-teacher. It shows Meta-KD can be more beneficial when we have fewer in-domain data.

\noindent \textbf{Zero-shot Learning.}
In this set of experiments, we consider a special case where we assume all the ``fiction'' domain data in MNLI is unavailable. Here, we train a meta-teacher without the ``fiction'' domain dataset and use the the distillation method proposed in ~\citep{DBLP:journals/corr/abs-1909-10351} to produce the student model for the ``fiction'' domain. The results are shown in Table~\ref{table:zero-shot}. We find that KD from the meta-teacher can have large improvement, compared to KD from other out-domain teachers. Additionally, learning from the out-domain meta-teacher has similar performance to KD from the in-domain ``fiction'' teacher model itself. It shows the Meta-KD framework can be applied in applications for emerging domains.

\begin{figure}[t]
\centering
\includegraphics[width=0.375\textwidth]{images/few-shot.eps}
\caption{Improvement rate for few-shot learning. Sample rate means the portion of training data in usage.}\label{fig:few-shot}
\end{figure}

\begin{figure}[t]
\centering
\includegraphics[width=0.375\textwidth]{images/gamme-2-tuning.eps}
\caption{Tuning the hyper-parameter $\gamma_2$.}\label{fig:gamma2-tuning}
\end{figure}

\begin{table}[!h]
 \small
 \centering
 \begin{tabular}{llc}
 \toprule
 \multicolumn{2}{l}{Methods} & Accuracy (\%) \\
 \midrule
 \multicolumn{2}{l}{$\text{BERT}_{\text{B}}$-single (fiction)} & 82.2 \\
 \multicolumn{2}{l}{Meta-teacher (w/o fiction)} & 81.6 \\

 \multicolumn{2}{l}{$\text{BERT}_{\text{B}}$-single (fiction) $\xrightarrow{\text{TinyBERT-KD}}$ $\text{BERT}_{\text{S}}$} & 78.8\\
 \midrule
 \multicolumn{2}{l}{$\text{BERT}_{\text{B}}$-single (govern) $\xrightarrow{\text{TinyBERT-KD}}$ $\text{BERT}_{\text{S}}$} & 75.3 \\
 \multicolumn{2}{l}{$\text{BERT}_{\text{B}}$-single (telephone) $\xrightarrow{\text{TinyBERT-KD}}$ $\text{BERT}_{\text{S}}$} & 75.6 \\
 \multicolumn{2}{l}{$\text{BERT}_{\text{B}}$-single (slate) $\xrightarrow{\text{TinyBERT-KD}}$ $\text{BERT}_{\text{S}}$} & 77.1 \\
 \multicolumn{2}{l}{$\text{BERT}_{\text{B}}$-single (travel) $\xrightarrow{\text{TinyBERT-KD}}$ $\text{BERT}_{\text{S}}$} & 74.1 \\
 \midrule
 \multicolumn{2}{l}{Meta-teacher $\xrightarrow{\text{TinyBERT-KD}}$ $\text{BERT}_{\text{S}}$} & \textbf{78.2} \\
 \bottomrule
 \end{tabular}
\caption{Results of the zero-shot learning setting on MNLI. Here, the distillation is performed on the ``fiction'' domain data. We report accuracy on the domain dataset.}\label{table:zero-shot}
\end{table}

\subsection{Ablation Study}

Here, we explore how $\gamma_2$ affects the distillation performance over the Amazon Reviews dataset. We tune the value of $\gamma_2$ from 0.1 to 1.0, with results are show in Figure \ref{fig:gamma2-tuning}. We find that the optimal value of $\gamma_2$ generally lies in the range of 0.2 ~ 0.5. The trend of accuracy is different in the domain ``DVD'' is different from those of the remaining three domains. This means the benefits from transferable knowledge of the meta-teacher varies across domains.

\section{Introduction} \label{sec:intro}

Pre-trained Language Models (PLM) such as BERT~\citep{conf/naacl/DevlinCLT19} and XLNet~\citep{conf/nips/YangDYCSL19} have achieved significant success with the two-stage ``pre-training and fine-tuning'' process. Despite the performance gain achieved in various NLP tasks, the large number of model parameters and the long inference time have become the bottleneck for PLMs to deploy in real-time applications, especially on mobile devices~\citep{DBLP:journals/corr/abs-1909-10351,DBLP:conf/acl/SunYSLYZ20,DBLP:journals/corr/abs-2006-11316}.
Thus, there are emerging needs for PLMs to reduce the model size and the computational overhead while keeping the prediction accuracy.

Knowledge Distillation (KD)~\cite{hiton_distill} is one of the promising ways to distill the knowledge from a large ``teacher'' model to a small ``student'' model. Recent studies show that KD can be applied to compress PLMs with acceptable performance loss~\citep{DBLP:journals/corr/abs-1910-01108,DBLP:journals/corr/abs-1908-09355,DBLP:journals/corr/abs-1909-10351,DBLP:journals/corr/abs-1908-08962,DBLP:conf/ijcai/ChenLQWLDDHLZ20}.
However, those methods mainly focus on single-domain KD. Hence, student models can only learn from its in-domain teacher, paying little attention to acquiring knowledge from other domains.

\begin{figure}[t]
\centering
\includegraphics[height=0.35\textwidth]{images/intros.eps}
\caption{An academic learning scenario. A physics student can learn physics equations better with a powerful all-purpose science teacher, compared to the other two cases.}\label{fig:intros}
\end{figure}

It has been shown that it is beneficial to consider cross-domain information for KD, by either training a teacher using cross-domain data or multiple teachers from multiple domains~\citep{DBLP:conf/kdd/YouX0T17,DBLP:journals/corr/abs-1911-03588,DBLP:conf/wsdm/YangSGLJ20,DBLP:conf/aaai/PengJLCCZ20}. Consider an academic scenario of learning physics equations in Figure \ref{fig:intros}. A typical way for a physics student to learn is to directly learn from his/her physics teacher. However, if we have a mathematical teacher to teach him/her basic knowledge of equations, the student can obtain a better understanding of physics equations. This ``knowledge transfer'' technique in KD has been proved efficient only when two domains are close to each other~ \citep{DBLP:journals/corr/abs-1908-09122}.
In reality, however, it is highly risky as teachers of other domains may pass non-transferable knowledge to the student model, which is irrelevant to the current domain and hence harms the overall performance~\citep{DBLP:conf/aaai/Tan0P017,DBLP:journals/corr/abs-2003-13003}.  Besides, current studies find multi-task fine-tuning of BERT does not necessarily yield better performance across all the tasks ~\citep{DBLP:conf/cncl/SunQXH19}. In light of this, we seek to build a more powerful teacher that is able to digest knowledge from different domains and then select transferable knowledge to teach the student in a domain-specific KD process.

To achieve the goal, we leverage the idea of meta-learning to capture transferable knowledge across domains, as recent studies have shown that meta-learning can improve the model generalization ability across domains~\citep{DBLP:conf/icml/FinnAL17,DBLP:conf/nips/JavedW19,DBLP:journals/corr/abs-2007-09604} and in many few-shot and zero-shot settings~\citep{DBLP:conf/iclr/BaoWCB20,DBLP:conf/acl/YeGCCXZWZC20,DBLP:conf/acl/CampagnaFML20}. We further argue that the meta-knowledge is also helpful for cross-domain KD.
Re-consider the case in Figure \ref{fig:intros}. If we have an ``all-purpose science teacher'' (i.e.,~the meta-teacher) who has the knowledge of both physics principles and mathematical equations (i.e., the general knowledge of the two courses), the student is able to learn physics equations better with the teacher.

\begin{figure}[t]
\centering
\includegraphics[width=0.5\textwidth]{images/framework.eps}
\caption{A high-level overview of the Meta-KD framework.}\label{fig:framework}
\end{figure}

Our framework is referred to as Meta-Knowledge Distillation (Meta-KD), which facilities cross-domain KD by training
a meta-teacher model across multiple domains to help learning domain-specific student models.
As shown in Figure \ref{fig:framework}, the framework consists of two parts, \textit{meta-teacher learning} and \textit{meta-distillation}. The meta-teacher is firstly trained with multi-domain datasets to
acquire the meta-knowledge across domains by incorporating the domain corruption loss. For each domain, the student model learns to solve the task over a domain-specific dataset with guidance from the meta-teacher. To improve the student's distillation ability, the meta-distillation module minimizes the distillation loss from both intermediate layers, output layers and transferable knowledge, combined with an instance-specific domain-expertise weighting technique.

To verify the effectiveness of Meta-KD, we conduct extensive experiments on two NLP tasks across multiple domains, namely natural language inference~\citep{MNLI_wiliam} and sentiment analysis ~\citep{DBLP:conf/acl/BlitzerDP07}. Experimental results show the effectiveness and superiority of the Meta-KD framework.
Moreover, we find our method is well performed especially when the in-domain dataset is very small while distillation, i.e. the few-shot learning setting, or there is no in-domain dataset, i.e. the zero-shot learning setting.

In summary, the contributions of this study are:
\begin{itemize}
\item To our knowledge, our work is the first to explore meta-learning based algorithms for PLM compression.
\item We propose a novel Meta-KD framework to distill knowledge in PLMs across domains, which consists of two major stages: meta-teacher learning and meta-distillation.
\item We conduct extensive experiments to demonstrate the superiority of Meta-KD and also explore the capability of this framework in both few-shot and zero-shot learning settings. We will release our source code on Github.
\end{itemize}

\section{The Meta-KD Framework} \label{sec:method}

In this section, we formally introduce the Meta-KD framework. We begin with a brief overview on Meta-KD. After that, the techniques are elaborated in details.

\subsection{An Overview of Meta-KD}
Take the task of text classification as an example. Assume there are $K$ training sets, corresponding to $K$ domains. In the $k$-th dataset $\mathcal{D}_k = \{ X_k^{(i)}, y_k^{(i)} \}_{i=1}^{N_k} $, $X_k^{(i)}$ is the $i$-th sample~\footnote{$X_k^{(i)}$ can be a sentence, a sentence pair or any other textual units, depending on the task inputs.} and $y_k^{(i)}$ is the corresponding label of $X_k^{(i)}$. $N_k$ is the total number of samples in $\mathcal{D}_k$. Let $\mathcal{M}_k$ be the large PLM fine-tuned on $\mathcal{D}_k$. Given the $K$ datasets, the goal of Meta-KD is to obtain the corresponding $K$ student models $\mathcal{S}_1,\cdots, \mathcal{S}_K$ that are small in size but has similar performance compared to the $K$ large PLMs, i.e., $\mathcal{M}_1,\cdots, \mathcal{M}_K$.

In general, the Meta-KD framework can be divided into the following two stages:
\begin{enumerate}
    \item\textbf{Meta-teacher Learning}: Learn a meta-teacher model $\mathcal{M}$ over all domains $\bigcup\limits_{k=1}^{K}\mathcal{D}_k$. The model digests transferable knowledge from each domain and has better generalization while supervising domain-specific students.

    \item\textbf{Meta-distillation}: Learn $K$ in-domain students $\mathcal{S}_1,\cdots, \mathcal{S}_K$ that performs well in their respective domains, given only in-domain data $\mathcal{D}_k$ and the meta-teacher $\mathcal{M}$ as input.
\end{enumerate}

During the learning process of the meta-teacher, inspired by prototype-based meta-learning methods~\citep{DBLP:conf/nips/SnellSZ17,DBLP:conf/cvpr/PanYLWNM19}, the meta-teacher model should memorize more information about prototypes. We first compute a~\emph{prototype score} $t_k^{(i)}$ for each sample $X_k^{(i)}$. The classification loss of the meta-teacher is defined as the sum of classification loss across all $K$ domains with~\emph{prototype-based},~\emph{instance-specific weighting} applied. Besides, it also learns~\emph{transferable knowledge} by adding the~\emph{domain-corruption loss} as an auxiliary loss proposed in~\citep{DBLP:journals/corr/abs-2003-13003}.
By these steps, the meta-teacher tends to be more generalized and digests transferable knowledge before supervising the student models.

For meta-distillation, each sample $X_k^{(i)}$ is weighted by a~\emph{domain-expertise score} $\lambda_k^{(i)}$ to address the meta-teacher's capability for this sample in the $k$th domain. Similar to meta-teacher learning, the~\emph{transferable knowledge} are also learned for the students from the meta-teacher. The overall meta-distillation loss is a combination of the Mean Squared Error (MSE) loss from intermediate layers of both models~\citep{DBLP:journals/corr/abs-1908-09355,DBLP:journals/corr/abs-1909-10351}, and the soft cross-entropy loss from output layers~\citep{hiton_distill}, with~\emph{instance-specific domain-expertise weighting} applied.

\noindent \textbf{Addressing Few-shot and Zero-shot Learning.}
When the $k$-th domain dataset $\mathcal{D}_k$ is small (i.e., containing only a few samples), we regard the learning of the $k$-th student model as a~\emph{few-shot learning} problem. We further consider another case where no training data of the $k$-th domain is available while training the teacher. The goal of Meta-KD here is to learn the student model for the $k$-th domain from a teacher purely based on knowledge learned from other domains. We refer to this setting as \emph{zero-shot learning}~\footnote{Note that the task definitions of few-shot and zero-shot learning are sightly different from classical meta-learning research, in order to fit the underlying KD tasks.}.

\subsection{Meta-teacher Learning} \label{subsec:meta_teacher}

In this work, we take the BERT model~\citep{conf/naacl/DevlinCLT19} as our base learner for text classification due to its wide popularity. For each sample $X_k^{(i)}$, the input to BERT is represented as follows: \texttt{[CLS]}, $tok_{k,1}^{(i)}$, $tok_{k,2}^{(i)}$, $\cdots$, \texttt{[SEP]}, where $tok_{k,n}^{(i)}$ is the $n$-th token in $X_k^{(i)}$. After BERT encoding, the last hidden outputs of this sequence is denoted as $h_{[CLS]}, h(tok_{k,1}^{(i)}), h(tok_{k,2}^{(i)}), .., h(tok_{k,N}^{(i)})$, where $h(tok_{k,j}^{(i)})$ represents the last layer embedding of the $j$-th token in $X_k^{(i)}$, and $N$ is the maximum sequence length.
For simplicity, we define $h(X_k^{(i)})$ as the average pooling of the token embeddings, i.e., $h(X_k^{(i)})=\sum_{n=1}^{N}h(tok_{k,n}^{(i)})$.

\noindent \textbf{Prototype-based Instance Weighting.}
To facilitate few-shot/zero-shot learning, we compute a~\emph{prototype score} $t_k^{(i)}$ for each sample $X_k^{(i)}$. Here, we treat the~\emph{prototype representation} for the $m$-th class of the $k$-th domain as:
\begin{equation}
p_k^{(m)}=\frac{1}{\vert\mathcal{D}_k^{(m)}\vert }\sum_{X_k^{(i)}\in\mathcal{D}_k^{(m)}}h(X_k^{(i)})
\end{equation}
where $\mathcal{D}_k^{(m)}$ is the $k$-th training set with the $m$-th class label. The~\emph{prototype score} $t_k^{(i)}$ is then computed as:
\begin{align}
& t_k^{(i)}= \\
& \alpha\cos(p_k^{(m)},h(X_k^{(i)}))
+\frac{1-\alpha}{K-1}\sum_{k'=1}^{K (k'\neq k)}\cos(p_{k'}^{(m)}, h(X_k^{(i)})), \nonumber
\end{align}
where $\cos$ is the cosine similarity function, and $\alpha$ is a pre-defined hyper-parameter. We can see that the definition of the prototype score here is different from previous meta-learning, as we require that an instance $X_k^{(i)}$ should be close to its class prototype representation in the embedding space (i.e., $p_k^{(m)}$), as well as the prototype representations in out-of-domain datasets (i.e., $p_{k'}^{(m)}$ with $k'=1, \cdots, K, k'\neq k$). This is because the meta-teacher should learn more from instances that are~\emph{prototypical across domains} instead of~\emph{in-domain only}. For the text classification task, the cross-entropy loss of the meta-teacher is defined using the cross-entropy loss with the prototype score as a weight assigned to each instance.

\noindent \textbf{Domain Corruption.}
Except for the cross-entropy loss, we leverage a variant of the~\emph{domain-corruption loss}~\citep{DBLP:journals/corr/abs-2003-13003} to increase the meta-teacher's ability for learning transferable knowledge. For each sample $X_k^{(i)}$, we learn an $\vert h(X_k^{(i)})\vert$-dimensional domain embedding of the true domain label $d_k^{(i)}$, denoted as $\mathcal{E}_D (X_k^{(i)})$.

Apart from the original BERT model~\citep{conf/naacl/DevlinCLT19}, a sub-network is constructed by:
\begin{equation}
h_d(X_k^{(i)})) = \tanh((h(X_k^{(i)}) + \mathcal{E}_D(X_k^{(i)}))W + b),
\end{equation}
where $W$ and $b$ are sub-network parameters. The domain-corruption loss for the sample $X_k^{(i)}$ is defined as:
\begin{equation}
\mathcal{L}_{DC}(X_k^{(i)}) = -\sum_{k=1}^K \mathbf{1}_{k=z_k^{(i)}} \cdot\log \sigma(h_d(X_k^{(i)})),
\end{equation}
where $\sigma$ is the $K$-way domain classifier, and $\mathbf{1}$ is the indicator function that returns 1 if $k=z_k^{(i)}$, and 0 otherwise. Here, $z_k^{(i)} \neq d_k^{(i)}$ is a false domain label of $X_k^{(i)}$\footnote{For ease of implementation, we shuffle the domain labels of all instances in a mini-batch.}. Hence, we deliberately maximize the probability that the meta-teacher makes the~\emph{wrong} predictions of domain labels. We call $h_d(X_k^{(i)}))$ as the~\emph{transferable knowledge} for $X_k^{(i)}$, which is more insensitive to domain differences.

Let $\mathcal{L}_{CE}(X_k^{(i)})$ be the normal cross-entropy loss of the text classification task. The total loss of the meta-teacher $\mathcal{L}_{MT}$ is the combination of weighted $\mathcal{L}_{CE}(X_k^{(i)})$ and $\mathcal{L}_{DC}(X_k^{(i)})$, shown as follows:
\begin{align}
& \mathcal{L}_{MT} = \\
& \frac{1}{\sum_{k=1}^K\vert\mathcal{D}_k\vert} \sum_{X_k^{(i)}\in \bigcup\limits_{k=1}^{K}\mathcal{D}_k} (t_k^{(i)}\mathcal{L}_{CE}(X_k^{(i)}) + \gamma_1 \mathcal{L}_{DC}(X_k^{(i)})), \nonumber
\end{align}
where the $\gamma_1$ is the factor to represent how the domain-corruption loss contributes to the overall loss.

\subsection{Meta-distillation} \label{subsec:meta_distillation}

\begin{figure*}[t]
\centering
\includegraphics[width=0.85\textwidth]{images/meta-distillation.eps}
\caption{An overview of meta-distillation and the neural architecture that we use for KD.}\label{fig:meta_distill}
\end{figure*}

As we take BERT as our meta-teacher, for ease of distillation, we again use smaller BERT models as student models. The distillation framework is shown in Figure~\ref{fig:meta_distill}. In our work, we distill the knowledge in the meta-teacher model considering the following five elements: input embeddings, hidden states, attention matrices, output logits and transferable knowledge.
The KD process of input embeddings, hidden states and attention matrices follows the common practice~\citep{DBLP:journals/corr/abs-1908-09355,DBLP:journals/corr/abs-1909-10351}. Recall that $\mathcal{M}$ and $\mathcal{S}_k$ are the meta-teacher and the $k$-th student model. Let $\mathcal{L}_{embd}(\mathcal{M},\mathcal{S}_k, X_k^{(i)})$, $\mathcal{L}_{hidn}(\mathcal{M},\mathcal{S}_k, X_k^{(i)})$ and $\mathcal{L}_{attn}(\mathcal{M},\mathcal{S}_k, X_k^{(i)})$ be the sample-wise MSE loss values of input embeddings, hidden states and attention matrices of the two models, respectively. Here, $\mathcal{L}_{embd}(\mathcal{M},\mathcal{S}_k, X_k^{(i)})$, $\mathcal{L}_{hidn}(\mathcal{M},\mathcal{S}_k, X_k^{(i)})$ and $\mathcal{L}_{attn}(\mathcal{M},\mathcal{S}_k, X_k^{(i)})$ refer to the sum of MSE values among multiple hidden layers. We refer interested readers to~\citep{DBLP:journals/corr/abs-1909-10351} for more details. $\mathcal{L}_{pred}(\mathcal{M},\mathcal{S}_k, X_k^{(i)})$ is the cross-entropy loss of ``softened'' output logists, parameterized by the temperature~\citep{hiton_distill}. A naive approach to formulating the total KD loss $\mathcal{L}_{kd}$ is the sum of all previous loss functions over the dataset $\mathcal{D}_k$, i.e.,
\begin{equation}
\begin{split}
\mathcal{L}_{kd}= & \sum_{X_k^{(i)}\in \mathcal{D}_k} \big( \mathcal{L}_{embd}(\mathcal{M},\mathcal{S}_k, X_k^{(i)})+\mathcal{L}_{hidn}(\mathcal{M},\mathcal{S}_k, X_k^{(i)})\\
& +\mathcal{L}_{attn}(\mathcal{M},\mathcal{S}_k, X_k^{(i)})+\mathcal{L}_{pred}(\mathcal{M},\mathcal{S}_k, X_k^{(i)}) \big).
\end{split}
\end{equation}

However, the above approach does not give special considerations to the transferable knowledge of the meta-teacher. Let $h_d^{\mathcal{M}}(X_k^{(i)})$ and $h_d^{\mathcal{S}}(X_k^{(i)})$ be the transferable knowledge of the meta-teacher and the student model w.r.t. the input $X_k^{(i)}$. We further define the transferable knowledge distillation loss $\mathcal{L}_{TKD}(\mathcal{M},\mathcal{S}_k, X_k^{(i)})$ as follows:
\begin{equation}
\begin{split}
& \mathcal{L}_{tkd}(\mathcal{M},\mathcal{S}_k, X_k^{(i)}) = \\
& \frac{1}{\vert\mathcal{D}_k\vert}\sum_{X_k^{(i)}\in \mathcal{D}_k} MSE \big( h_d^{\mathcal{M}}(X_k^{(i)}) W_{d}^{\mathcal{M}}, h_d^{\mathcal{S}}(X_k^{(i)}) \big),
\end{split}
\end{equation}
where $W_{d}^{\mathcal{M}}$ is a learnable projection matrix to match the dimension between $h_d^{\mathcal{M}}(X_k^{(i)})$ and $h_d^{\mathcal{S}}(X_k^{(i)})$, and $MSE$ is the MSE loss function w.r.t. single element. In this way, we encourage student models to learn more transferable knowledge from the meta-teacher.

We further notice that although the knowledge of the meta-teacher should be highly transferable, there still exists the domain gap between the meta-teacher and domain-specific student models. In this work, for each sample $X_k^{(i)}$, we define the domain expertise weight $\lambda_k^{(i)}$ as follows:
\begin{equation}
\lambda_k^{(i)}= \frac{1 + t_k^{(i)}}{\exp^{(\hat{y}_k^{(i)} - y_k^{(i)})^2} + 1}
\end{equation}
where $\hat{y}_k^{(i)}$ is the predicted result of $X_k^{(i)}$'s class label. Here, the weight $\lambda_k^{(i)}$ is large when the meta-teacher model i) has a large domain weight $t_k^{(i)}$ and ii) makes correct predictions on the target input, i.e., $\hat{y}_k^{(i)}=y_k^{(i)}$. We can see that the weight reflects how well the meta-teacher can supervise the student on a specific input. Finally, we derive the complete formulation of the KD loss $\mathcal{L}_{kd}'$ as follows:
\begin{equation}
\begin{split}
\mathcal{L}_{kd}'= & \sum_{X_k^{(i)}\in\mathcal{D}_k} \lambda_k^{(i)} \big(\mathcal{L}_{embd}(\mathcal{M},\mathcal{S}_k, X_k^{(i)})+\mathcal{L}_{hidn}(\mathcal{M},\mathcal{S}_k, X_k^{(i)})\\
&+\mathcal{L}_{attn}(\mathcal{M},\mathcal{S}_k, X_k^{(i)})   +\mathcal{L}_{pred}(\mathcal{M},\mathcal{S}_k, X_k^{(i)}) \big) \\
&+\gamma_2\mathcal{L}_{tkd}(\mathcal{M},\mathcal{S}_k, X_k^{(i)}),
\end{split}
\end{equation}
where $\gamma_2$ is the factor to represent how the transferable knowledge distillation loss contributes to the overall loss.

\section{Related Work} \label{sec:related_works}

Our study is close to the following three lines of studies.

\subsection{Knowledge Distillation (KD)}
KD was first proposed by~\cite{hiton_distill}, aiming to transfer knowledge from an ensemble or a large model into a smaller, distilled model. Most of the KD methods focus on utilizing either dark knowledge, i.e., predicted outputs ~\citep{hiton_distill,DBLP:conf/aaai/ChenMWF020,DBLP:conf/icml/FurlanelloLTIA18,DBLP:conf/kdd/YouX0T17} or hints, i.e., intermediate representations ~\citep{DBLP:journals/corr/RomeroBKCGB14,DBLP:conf/cvpr/YimJBK17,DBLP:conf/kdd/YouX0T17} or relations between layers ~\citep{DBLP:conf/cvpr/YimJBK17,DBLP:conf/nips/TarvainenV17,DBLP:conf/cvpr/RadosavovicDGGH18} of teacher models.~\citet{DBLP:conf/kdd/YouX0T17} also find multiple teacher networks together can provide comprehensive guidance that is beneficial for training the student network.~\citet{DBLP:journals/corr/RuderGB17} show that multiple expert teachers improve the performances of sentiment analysis on unseen domains. \citet{DBLP:conf/iclr/TanRHQZL19} apply the multiple teachers framework in KD to build state-of-the-art multilingual machine translation system. Our work is one of the first attempts to learn meta-teacher from multiple domains to benefit KD on the target domain.

\subsection{Pre-trained Language Model Compression}
Previous approaches on compressing PLMs such as BERT ~\cite{conf/naacl/DevlinCLT19} include KD ~\citep{hiton_distill}, parameter sharing ~\citep{DBLP:conf/iclr/UllrichMW17}, pruning \citep{DBLP:conf/nips/HanPTD15} and quantization ~\citep{DBLP:journals/corr/GongLYB14}. In this work, we mainly focus on KD for PLMs. In the literature, \citet{DBLP:journals/corr/abs-1903-12136} distill BERT into BiLSTM networks to achieve comparable results with ELMo ~\citep{DBLP:conf/naacl/PetersNIGCLZ18}. \citet{DBLP:journals/corr/abs-1909-11687} use dual distillation to reduce the vocabulary size and the embedding size. DistillBERT ~\citep{DBLP:journals/corr/abs-1910-01108} applies KD loss in the pre-training stage, while BERT-PKD~\citep{DBLP:journals/corr/abs-1908-09355} distill BERT into shallow Transformers in the fine-tuning stage. TinyBERT~\citep{DBLP:journals/corr/abs-1909-10351} further distills BERT with a two-stage KD process for hidden attention matrices and embedding matrices. AdaBERT ~\citep{DBLP:conf/ijcai/ChenLQWLDDHLZ20} uses neural architecture search to adaptively find small architectures.

\subsection{Transfer Learning (TL) and Meta-learning}
TL has been proved to improve the performance on the target domain by leveraging knowledge from related source domains ~\citep{DBLP:journals/tkde/PanY10,DBLP:conf/emnlp/MouMYLX0J16,DBLP:conf/acl/LiuQH17,DBLP:conf/iclr/YangSC17}. In most NLP tasks, the ``shared-private'' architecture is applied to learn domain-specific representations and domain-invariant features ~\citep{DBLP:conf/emnlp/MouMYLX0J16,DBLP:conf/acl/LiuQH17,DBLP:conf/naacl/ChenYZLB18,DBLP:conf/www/ChenQYZHLB19}. Compared to TL, the goal of meta-learning is to train meta-learners that can adapt to a variety of different tasks with little training data ~\citep{DBLP:journals/corr/abs-1810-03548}.
A majority of meta-learning methods for include metric-based~\cite{DBLP:conf/nips/SnellSZ17,DBLP:conf/cvpr/PanYLWNM19}, model-based~\cite{DBLP:conf/icml/SantoroBBWL16,DBLP:conf/iclr/BartunovROL20} and model-agnostic approaches~\cite{DBLP:conf/icml/FinnAL17,DBLP:conf/nips/FinnXL18,DBLP:conf/nips/VuorioSHL19}. Meta-learning can also be applied to KD in some computer vision tasks~\citep{DBLP:journals/corr/abs-1710-07535,DBLP:conf/icml/yunhun19,DBLP:conf/eccv/benliu20,DBLP:conf/aaai/BaiWKL20,DBLP:conf/cvpr/LiL0Z20}. For example, \citet{DBLP:journals/corr/abs-1710-07535} record per-layer meta-data for the teacher model to reconstruct a training set, and then adopts a standard training procedure to obtain the student model. \citet{DBLP:conf/cvpr/LiL0Z20} use a 1 $\times$ 1 convolution layer to each layer of the student model, and fits the block-level outputs of the student model to the teacher model by estimating the parameters of the added layers.

Our work is closely related to meta-learning, as our meta-teacher model learns the transferable knowledge across domains so that it can fit new domains easily. However, we need to claim that our work a meta-learning algorithm but addresses KD across domains by training a powerful meta-learner, instead of addressing the K-way N-shot problems in traditional meta-learning research~\cite{DBLP:journals/corr/abs-1810-03548}.

\section{Conclusion} \label{sec:conlusion}
In this work, we propose the Meta-KD framework to distill PLMs across domains, which consisting of two stages: meta-teacher learning and meta-distillation. We conduct extensive experiments on two widely-used public multi-domain datasets with respect to natural language inference task and text classification task.
Experiments results confirm the effectiveness of Meta-KD and also show the capability of Meta-KD in few-shot and zero-shot learning settings.