text:
\section{Introduction}
\IEEEPARstart{B}enefit from hyperspectral imagery which can capture the local spectral properties of human tissue, hyperspectral face analysis has attracted more and more attention from scholars in the field of face related computer vision tasks because of its satisfactory performance under uncontrolled conditions, such as low-light environment and spoofing attacks. However, the hyperspectral face imaging system is often compromised due to the limitations of the amount of the incident energy. There is always a tradeoff between the spatial and spectral resolution of the real imaging process. With the increase of the spectral bands, all other factors kept constant, to ensure a high signal-to-noise ratio (SNR) the spatial resolution will inevitably become a victim. Therefore, how to obtain a reliable hyperspectral face image with high spatial resolution remains a very challenging problem.

\begin{figure*}
  \centering

  \includegraphics[width= 18.5cm]{Network.pdf}\\
  \caption{The flow chart of our proposed spectral splitting and aggregation network (SSANet) based hyperspectral face super-resolution (HFSR) method. SSANet includes three spectral splitting and aggregation modules followed by a reconstruction module. }\label{fig:Network}
\end{figure*}

Super-Resolution (SR) reconstruction can infer a high-resolution (HR) image from a single low-resolution (LR) image or sequential observed LR images \cite{Park2003}. It is a post-processing technique that does not require hardware modifications, and thus could break through the limitations of the imaging systems. Since the pioneer work of Bake and Kanda \cite{Baker2000}, face SR, \emph{a.k.i}, face hallucination, has received increased attention \cite{Liu2001, Wang2005Eig, Zhuang2007, Park2008}. Especially in recent years, the emergence of deep learning technology has promoted a large number of face SR methods including UR-DGN~\cite{yu2016ultra}, CBN~\cite{zhu2016deep}, LCGE~\cite{song2017learning}, Attention-FH \cite{Cao2017CVPR}, and FSRNet \cite{CT-FSRNet-2018}. However, these methods are mainly for gray/RGB face images and cannot exploit the spectral correlation efficiently. This is mainly due to that the spectral dimensionality of hyperspectral face image is very high and the number of training samples of current hyperspectral face image dataset is extremely small, \emph{e.g.}, around 100. Compared with the general gray/RGB face datasets, the size of existing hyperspectral face databases is smaller. For example, the gray/RGB face database has tens of thousands of samples, while the hyperspectral face database has only a few hundred or even dozens of face images. And thus, there is not enough training samples to support the training of the complex deep network. In addition, these deep learning algorithms require large amounts of memory and are computationally expensive. Please refer to \cite{jiang2021deep} for more details about deep learning-based face SR.
Whether it is a traditional shallow learning-based method or a deep learning-based method, it is difficult to construct a powerful representation model from only dozens of face images to reconstruct the target HR hyperspectral face images.

In summary, when we apply the existing learning-based face SR methods to hyperspectral images, the challenges are twofold: (i) the spectral correlations of hyperspectral face images cannot be fully utilized; (ii) hyperspectral face images are difficult to obtain, and the training samples of existing hyperspectral face image database is very small, thus cannot support these complex learning models.

To this end, in this paper we propose a hyperspectral face super-resolution (HFSR) method with limited training samples based on a spectral splitting and aggregation network (SSANet). Thanks to the amount of spectral bands of hyperspectral data, we split the hyperspectral image into different spectral groups and take each of them as an individual sample (in the sense that each group will be fed into the some network). By this splitting strategy, the training of the complex network can be guaranteed. To exploit the spectral correlations, which are destroyed by the splitting strategy but very important for the hyperspectral image reconstruction, we introduce an aggregation operator to gradually aggregate the neighbor bands at the deeper layers. As shown in Fig. \ref{fig:Network}, at the shallow layers, the complex network (with more blocks or channels) can be well-trained with `adequate' data (each group can be seen as a sample because of the parameter sharing strategy). At deeper layer, we design some light structures (with less blocks or channels) because we have reduce the training samples in order to exploit the spectral correlations. In general, although we only have a small number of training samples, we can expect to alleviate the S3 problem through this carefully designed spectral splitting and aggregation network. Additionally, in order to further cope with the challenge of S3 problem, we propose to expand the training samples by self-representation learning. In particular, based on the assumption that there is a path from the training sample itself to the mean face of all training samples (as shown in Fig. \ref{fig:meanface}), we develop a training sample expanding strategy by self-representation learning. Given one training sample, we leverage the remaining training samples to represent it. According to a predefined smooth parameter, we can obtain a large number of synthetic samples, thus forming a path from the training sample itself to the mean face of the training set. In addition, we also expand the training dataset by the symmetry of the human face, thus we can directly double the size of the training dataset.

To sum up, this paper makes the following contributions: (i) We propose the first SR method for hyperspectral face image. (ii) A novel spectral splitting and aggregation network is introduced to alleviate the S3 problem, which is inevitable when we apply deep learning method to the HFSR task. (iii) We propose some schemes to expand the sample base based on self-representation and symmetry argumentation.

\label{Organization}
The rest parts of this study is organized as follows. In Section \ref{sec:related}, we review some related methods. Section~\ref{sec:proposedmethod} presents the details of the proposed method. In Section~\ref{sec:experiments}, we report experimental evaluations of the proposed framework and compare it with some competitive algorithms. Finally, we conclude this paper in the last section.

\section{Related Work}
\label{sec:related}
Face SR is a very hot topic since the moment it was born. In recent years, a lot of methods have been proposed. In the following, we will revisit these approaches from global face based methods to local patch representation based methods, and then to the deep learning methods that have received much attention most recently. In addition, we also review some general hyperspectral image SR methods.

\subsection{Face Model based Super-Resolution}
Face model based approaches leverage the global face statistical models, such as principal component analysis (PCA)~\cite{Wang2005Eig}, locality preserving projection (LPP) model~\cite{Park2007}, uniform space projection~\cite{Huang2010CCA, Anle2014face} and nonnegative matrix factorization (NMF)~\cite{Yang2010TIP}, to model the facial image and super-resolve the target HR face image globally. They can maintain the main structure of human face well. However, their reconstruction results lack detailed local facial features, and inevitably have ghosting artifacts.

\subsection{Patch Representation based Super-Resolution}
Considering that the human face is a highly structured subject, many face SR approaches try to exploit the prior knowledge by dividing the whole face image into small patches~\cite{Yang2013CVPR, li2014face, hui2017novel}. Among them, position-patch based methods have gained widespread attention in recent years.
However, their solutions for the linear combination problem are unstable or not unique. To this end, our previous work~\cite{Jiang2014LcRTMM} introduces the locality-constrained representation (LcR) technique to simultaneously incorporate sparsity and locality into the patch representation. With the introduced locality constraint, it obtains a stable and reasonable representation. In order to alleviate the inconsistency of the two spaces, LR and HR spaces, some work have been proposed to iteratively obtain the patch representation and perform neighbor embedding or learn the mapping in correlation spaces~\cite{LiaoQM2015DSP, farrugia2015face, gao2020constructing, LiuLC2017Robust, shi2018hallucinating, chen2020robust}.

\begin{figure}
  \centering

  \includegraphics[width= 8.50 cm]{meanface.pdf}\\
  \caption{A vivid illustration of face image ``generation''. We assume that all \textcolor[rgb]{0.00,1.00,0.25}{individual face images} (on the big circle) are ``evolved'' from the \textcolor[rgb]{0.44,0.00,0.94}{mean face} (at the center of the circle). The connection from the center to the circle represents the evolutionary path, and each point on the evolutionary path can be seen as the \textcolor[rgb]{1.00,0.00,0.00}{synthesized face images}.}\label{fig:meanface}
\end{figure}

\subsection{Deep Learning based Face Super-Resolution}
More recently, to reconstruct the latent HR image locally while thinking globally, DNNs, especially CNNs, have been applied to construct the mapping relationship between the LR images and their HR counterparts and shown strong learning capability and accurate prediction of HR images \cite{dong2016image, liu2016robust}.
For example, Dong \emph{et al.}~\cite{dong2016image} developed a general image SR method based on SRCNN. This is the very first attempt to use deep learning tools for image SR reconstruction.
The approach of Liu \emph{et al.} \cite{liu2016robust} proposes to introduce the domain expertise to design a Sparse Coding based Network (SCN). Recently, R-DGN~\cite{yu2016ultra}, CBN~\cite{zhu2016deep}, LCGE~\cite{song2017learning}, Attention-FH \cite{Cao2017CVPR}, FSRNet \cite{CT-FSRNet-2018}, and \cite{Yu2018Imagining} are the most competitive approaches for face hallucination. They unitized very deep networks to model the relationship between the LR images and their HR counterparts, and verified that deeper networks can produce better results due to the large receptive field, which means considering more contextual information, \emph{i.e.}, very large image regions.

\subsection{Deep Learning based Hyperspectral Image Super-Resolution}
According to whether an additional guided image (such as panchromatic, RGB, or multispectral image) is utilized, hyperspectral image SR techniques can be divided into two categories: multi-source information fusion based hyperspectral image SR (sometimes called hyperspectral image pansharpening) and single hyperspectral image SR \cite{loncan2015hyperspectral, yokoya2017hyperspectral}. The former is to leverage high-frequency spatial information from HR auxiliary image, and fuse them to the target HR hyperspectral image \cite{yokoya2011coupled, dong2016hyperspectral, xie2019multispectral, Borsoi2020TIP}. Though achieving good performance, they need a well co-registered auxiliary image which is arduous in real applications.
Without co-registered auxiliary image, the latter single hyperspectral image SR methods have still attracted considerable attention. How to exploit the abundant spectral correlations among successive bands is the essential problem for them. Traditional methods try to  incorporate the low-rank and group-sparse constraints to exploit the spectral-spatial prior \cite{huang2014super, li2016hyperspectral}. Due to its superior performance, deep learning techniques have also been introduced into the single
hyperspectral image SR task. They often adopt a two-step strategy to get an intermediate result through a deep network firstly, and then use spectral decomposition constraint to ensure the accuracy of the reconstructed spectral information \cite{yuan2017hyperspectral, xie2019hyperspectral}. Recently, there are also some approaches to directly learn an end-to-end deep networks to exploit the spatial-spectral prior. For example, Mei \emph{et al.} \cite{mei2017hyperspectral} presented a 3D full convolutional neural network to represent the spectral-spatial information. Wang \emph{et al}. \cite{li2020mixed} proposed to combine the advantages of 2D and 3D convolutions to exploit the spatial and spectral information. In \cite{li2018single}, Li \emph{et al.} developed a grouped deep recursive residual network (GDRRN) based single hyperspectral image SR method. The designed group-wise convolution and recursive structure can guarantee that it could yield very good performance. Inspired by the concept of group convolution, Jiang \emph{et al}. \cite{jiang2020learning} proposed the spatial-spectral prior network based super-resolution network (SSPSR). Most recently, inspired by the work of deep image prior \cite{ulyanov2018deep}, the approach of \cite{sidorov2019deep} presents an effective single hyperspectral image restoration algorithm. In general, these deep methods achieve better results than traditional methods. However, due to the limited hyperspectral training samples and the high dimensionality of spectral bands, it is difficult to fully exploit the spatial information and the correlation among the spectra of the hyperspectral data.

\textbf{\emph{Summary}}: As for the shallow learning based face SR methods, the local patch based models have much more strong representation ability than global face based models, rendering effectively the fine individual details to an input LR face. To obtain an accurate representation of the image patch, the current patch representation based methods all try to exploit the image priors (\emph{e.g}., collaborative, local, sparse, and low-rank constraints) or add some geometric regularizations from the HR space. They can well deal with the noisy input, inconsistency between LR and HR manifold spaces. These hand-designed prior models may not be effective. Deep neural network based methods leverage large-scale training dataset to train the network, and thus they can obtain very good performance. However, they may not suitable for our task, where only about 100 samples are available. Although many SR methods for general hyperspectral images have been proposed in recent years, they all rely on a large-scale training set. Therefore, how to adapt the deep learning techniques for the HFSR task, especially when the training samples are very limited, is an urgent and extremely challenging issue.

\section{The Proposed Method}
\label{sec:proposedmethod}
Different from gray/RGB face SR problem, in which large paired HR and LR face training samples can be collected, HFSR can only leverage a small size of training dataset. Therefore, the very limited data cannot support the representation and modeling of complex hyperspectral data. To this end, in this paper we carefully design a deep network based on gradually spectral splitting and aggregation to alleviate the S3 problem. In addition, we also propose two strategies to expand the training samples by a self-representation model and the symmetry-induced augmentation. In this section, we will first present the details of SSANet and then introduce our training sample expanding strategies.

\begin{figure}
  \centering

  \includegraphics[width= 8.99cm]{enlarge.pdf}\\
  \caption{RMSE values according to the synthesis controlling parameter. Face images marked by red, green, and purple are the \textcolor[rgb]{1.00,0.00,0.00}{synthesized training samples}, the \textcolor[rgb]{0.00,1.00,0.25}{HR ground truth face image}, and the \textcolor[rgb]{0.44,0.00,0.94}{mean face}, respectively. With the increase of the synthesis controlling parameter, the synthesized training sample gradually change from the HR ground truth face image to the mean face.}\label{fig:trainingsamples}
\end{figure}

\subsection{Spectral Splitting and Aggregation Network (SSANet)}
\textbf{\emph{Spectral splitting module}}: As we discussed above, because the dimensionality of hyperspectral images is very high and the training samples are very limited, it is very difficult for existing deep neural network based approaches to effectively represent the hyperspectral data. Thanks to the amount of spectral bands of hyperspectral data, where each spectral band (neighbor spectral bands) can be seen as a training sample, we split the hyperspectral image into different groups (samples), thus largely expanding the size of the original training dataset. Note that we let each spitted group be fed into a branch network, and let different branches share the same parameters. Through this parameter sharing based splitting strategy, it will provide a strong data support for training a deep network.

As shown in Fig. \ref{fig:Network}, at the shallow layer (\emph{e.g.}, the first spectral splitting and aggregation stage) we split the hyperspectral data into as many groups as possible (a band is a group). In this way, we can obtain multiple training samples, which makes it possible to design a complex network (\emph{i.e.}, many spatial-spectral residual blocks (SSRBs)) to extract features of hyperspectral data. In the spectral splitting network, each group will be fed to a branch network to extract the features.

\textbf{\emph{Aggregation module}}: However, these extracted features may overlook the correlations among the spectral bands, which are very import to model and reconstruct the hyperspectral data. Therefore, at the deeper layers, we gradually increase the size (band number) of the group (\emph{i.e.}, the band number of each group, from 1 to 4, and then to 8, and finally to all 33 bands for different spectral splitting and aggregation stages). Meanwhile, we also let the neighbor groups overlap with each other. When we feed these overlapped groups to the splitting branch networks, we obtain the output by an aggregation operator, where the output is averaged according to their spectral indices. It should be noted that before feeding the outputs of different branch networks to the following aggregation module, we apply an additional \emph{Conv} layer to reduce the channel dimension to the number of input bands. Thus, each branch network acts like a ``reconstruction'' network and the input and output have the same channels. As we know, with the increase of the group size, the samples available for training will decrease. Thus, at the deeper layers we design some light structures (with less SSRBs). For example, with the progress of spectral splitting and aggregation reconstruction, the number of SSRBs decreases from 12 to 8, 4, and 2 for different spectral splitting and aggregation stages, respectively.

The proposed SSANet includes three spectral splitting and aggregation modules followed by a reconstruction module. In order to effectively and efficiently represent and reconstruct the hyperspectral face images, we carefully set the numbers of input band as well as the number of SSRB at each spectral splitting network. In addition, to fully exploit the spectral information of hyperspectral data, we also change the feature channels of different spectral splitting networks. At the shallow layer, where the input has only one spectral band, we set the number of feature channel to 64, and then increase it to 128, 128, and then to 192. This is because the input has more spectral bands with the progress of spectral splitting and aggregation reconstruction (from 1 to 4, 8, and then to 33), it calls for more feature channels to represent the spectral information of hyperspectral data.

\textbf{\emph{Upscaling strategy}}: For the proposed SSANet, another design worth mentioning is the upscaling strategy. The most commonly used upscaling strategy is the first upsampling (when the input is fed into the network) or the last upsampling (at the end of the network). In this paper, we are inspired by the pyramid SR network \cite{lai2017deep} and propose an upscaling search strategy. In particular, by inserting upscaling modules at different locations (before each aggregation module) of the network, we search for a set of optimized upscaling schemes for different upsampling factors. As shown in Fig. \ref{fig:Network}, above the input of each splitting network, we give the optimal upscaling scheme (and the size of the input image) under different upsampling factors, \emph{e.g.}, $\times 2$, $\times 4$, and $\times 8$. More details can be found at Section \ref{sec:upsampling}.

\textbf{\emph{Spatial-spectral residual block (SSRB)}}: Different from the commonly used residual module, we introduce a spatial-spectral residual block \cite{he2016deep}. It is a residual module like block, and includes one $3\times3$ \emph{Conv} and one $1\times1$ \emph{Conv} followed by an ReLU layer. We modify the residual module due to the following considerations. First, the added $1\times1$ \emph{Conv} can be leveraged to reorganize and reweight the importance of spectral bands, thus efficiently exploiting the spectral correlations of hyperspectral face image. Second, the added $1\times1$ \emph{Conv} can be seen as the purpose of information distillation and more useful information can be well preserved. Third, it deepens the depth of the network at a small cost in term of parameters.

\begin{figure}
  \centering
  \includegraphics[width= 9cm]{uwasample.pdf}\\
  \caption{The face cube of one example in UWA-HSFD.}\label{fig:uwasample}
\end{figure}

\subsection{Training Sample Expanding}
As we discussed above, the spectral dimensionality of hyperspectral face image is very high, but the training sample number of available hyperspectral face image dataset is extremely small. Therefore, to well represent the high-dimensional hyperspectral face image, one possible solution is to expand the training set by synthesizing new training samples.

In this paper, we develop a self-representation learning method to synthesize the face image. In particular, we assume that all face images are from the same source, \emph{i.e.}, the mean face, and there is a path from the mean face to the individual face. As shown in the Fig. \ref{fig:trainingsamples}, the faces on the red line can be seen as transforming the mean face (top right) to the individual face (bottom left). Therefore, we can use some states on the path to obtain new samples. Mathematically, the self-representation based face synthesis method can be described as following,
\begin{equation}\label{eq:selfrepresentation}
\hat x_i = \sum_{j=1, j\not=i}^N w(x_i,x_j)x_j,
\end{equation}
where $x_i$ is the sample to be reconstructed, $\{x_j|j\not=i, 1\le j\le N\}$ are the training samples except $x_i$ itself, and $w(x_i,x_j)$ represents the reconstruction weight corresponding to $x_j$. From the above definitions, we can learn that the synthesized training sample is a linear combination of the training samples except itself. In particular, the combination weight is proportional to the similarity between current sample $x_i$ and the remaining training samples $x_j$,
\begin{equation}\label{eq:similarity}
w(x_i,x_j) = \frac{sim(x_i,x_j)}{\sum_{j\not=i,j=1}^N sim(x_i,x_j)}.
\end{equation}
Here, $sim(x_i,x_j)$ denotes the similarity between $x_i$ and $x_j$ and is defined as follow,
\begin{equation}\label{eq:similarity}
sim(x_i,x_j) = \exp\left(-\frac{||x_i-x_j||_2^2}{\sigma^2G} \right),
\end{equation}
where $\sigma$ denotes the synthesis controlling parameter, and $G$ is the mean of all elements of $\{||x_i-x_j||_2^2 |j\not=i, 1\le j\le N\}$. Since the entire face image is high-dimensional, the globally reconstruction methods may not accurate or cannot introduce additional information to the training dataset (please refer to the \textbf{\emph{Remark 2}}). Thus, we decompose the global face into small image patches, and apply the above-mentioned approach for each small image patch. Lastly, by integrating all the reconstructed image patches, we can synthesize a new training sample.

In Fig. \ref{fig:trainingsamples}, it plots the reconstruction error according to the synthesis controlling parameter $\sigma$, and some synthesized typical face images are also presented. Through changing the value of $\sigma$, we can obtain different status (synthesis face samples). The smaller the value of $\sigma$, the more similar the synthesized face image is to the image to be reconstructed, and vice versa. When the value of $\sigma$ is very small, these training samples similar to $x_i$ dominate the reconstruction, \emph{i.e.}, their reconstruction weights will be large. In contrast, when the value of $\sigma$ is very large, the reconstruction weights tend to be the same, and thus obtain the mean face. It is worth noting that when the value of $\sigma$ is very small, the reconstruction weights will be sparse (most of the weights that associated with these dissimilar training samples will be 0), and the synthesized face is reconstructed by several similar faces. Due to this sparsity, we have reasons to believe that the synthesized faces may have some detailed features (high-frequency information) that do not exist in $x_i$. We explain this phenomenon as follows: a few face samples cannot accurately represent the original face image $x_i$, and the synthetic error is exactly the high-frequency detail information that does not exist in $x_i$. If the synthesized face image has high frequency detail information that does not exist in the original face image, we will be able to treat the synthesized face image as a meaningful sample.

\setlength{\tabcolsep}{1.5pt}
\begin{table}
\begin{center}
\caption{Average quantitative performance by different upsampling strategies over 10 testing images of UWA-HSFD database with respect to 6 PQIs. }\label{tab:upsampling}
\scriptsize
\begin{tabular}{cccc|cc|cccccc}

  \hline
up1	&	up2	&	up3	&	up4	&	Paras	&	FLOPs	&	CC$\uparrow$	&	SAM$\downarrow$	&	RMSE$\downarrow$	&	ERGAS$\downarrow$	&	PSNR$\uparrow$	&	SSIM$\uparrow$	\\
\hline
\hline
1	&	1	&	1	&	2	&	7.285 	&	269.2 	&	0.9989 	&	0.7659 	&	0.0052 	&	0.4529 	&	47.2386 	&	0.9919 	\\
1	&	1	&	2	&	1	&	6.547 	&	296.0 	&	0.9988 	&	0.7815 	&	0.0052 	&	0.4599 	&	47.0040 	&	0.9919 	\\
\rowcolor[rgb]{0.75,0.75,0.0}
1	&	2	&	1	&	1	&	6.547 	&	396.5 	&	0.9989 	&	0.7555 	&	0.0051 	&	0.4440 	&	47.2672 	&	0.9923 	\\
2	&	1	&	1	&	1	&	6.104 	&	709.9 	&	0.9989 	&	0.7552 	&	0.0051 	&	0.4463 	&	47.2690 	&	0.9922 	\\
\hline
1	&	1	&	1	&	4	&	8.613 	&	74.15 	&	0.9954 	&	1.3583 	&	0.0114 	&	0.9632 	&	41.0984 	&	0.9651 	\\
1	&	1	&	2	&	2	&	7.875 	&	80.84 	&	0.9955 	&	1.3352 	&	0.0112 	&	0.9481 	&	41.2323 	&	0.9661 	\\
1	&	1	&	4	&	1	&	7.137 	&	107.6 	&	0.9954 	&	1.3557 	&	0.0114 	&	0.9611 	&	41.0973 	&	0.9656 	\\
\rowcolor[rgb]{0.75,0.75,0.0}
1	&	2	&	1	&	2	&	7.875 	&	106.0 	&	0.9957 	&	1.2985 	&	0.0110 	&	0.9269 	&	41.4541 	&	0.9673 	\\
1	&	2	&	2	&	1	&	7.137 	&	132.7 	&	0.9957 	&	1.3168 	&	0.0111 	&	0.9356 	&	41.3544 	&	0.9671 	\\
1	&	4	&	1	&	1	&	7.137 	&	233.2 	&	0.9956 	&	1.3176 	&	0.0112 	&	0.9401 	&	41.3151 	&	0.9669 	\\
2	&	1	&	1	&	2	&	7.432 	&	184.3 	&	0.9957 	&	1.2984 	&	0.0110 	&	0.9274 	&	41.4485 	&	0.9673 	\\
2	&	1	&	2	&	1	&	6.695 	&	211.1 	&	0.9958 	&	1.2890 	&	0.0110 	&	0.9250 	&	41.4659 	&	0.9677 	\\
2	&	2	&	1	&	1	&	6.695 	&	311.5 	&	0.9957 	&	1.2942 	&	0.0110 	&	0.9272 	&	41.4051 	&	0.9676 	\\
4	&	1	&	1	&	1	&	6.252 	&	625.0 	&	0.9956 	&	1.2956 	&	0.0111 	&	0.9342 	&	41.3327 	&	0.9670 	\\
\hline
2	&	1	&	1	&	4	&	8.760 	&	52.92 	&	0.9818 	&	2.1048 	&	0.0234 	&	1.9325 	&	35.1264 	&	0.8875 	\\
1	&	2	&	1	&	4	&	9.203 	&	33.33 	&	0.9817 	&	2.1312 	&	0.0235 	&	1.9370 	&	35.0775 	&	0.8856 	\\
1	&	1	&	2	&	4	&	9.203 	&	27.05 	&	0.9821 	&	2.1299 	&	0.0232 	&	1.9146 	&	35.1469 	&	0.8875 	\\
4	&	1	&	1	&	2	&	7.580 	&	163.1 	&	0.9815 	&	2.1160 	&	0.0236 	&	1.9462 	&	35.0292 	&	0.8836 	\\
1	&	4	&	1	&	2	&	8.465 	&	65.14 	&	0.9818 	&	2.1055 	&	0.0233 	&	1.9282 	&	35.0768 	&	0.8868 	\\
1	&	1	&	4	&	2	&	8.465 	&	33.75 	&	0.9815 	&	2.1396 	&	0.0236 	&	1.9461 	&	35.0330 	&	0.8850 	\\
1	&	2	&	2	&	2	&	8.465 	&	40.02 	&	0.9811 	&	2.1280 	&	0.0238 	&	1.9637 	&	34.9534 	&	0.8843 	\\
\rowcolor[rgb]{0.75,0.75,0.0}
2	&	1	&	2	&	2	&	8.023 	&	59.61 	&	0.9825 	&	2.0828 	&	0.0230 	&	1.8961 	&	35.2369 	&	0.8901 	\\
2	&	2	&	1	&	2	&	8.023 	&	84.73 	&	0.9824 	&	2.0698 	&	0.0230 	&	1.9011 	&	35.2179 	&	0.8893 	\\
2	&	2	&	2	&	1	&	7.285 	&	111.5 	&	0.9821 	&	2.1056 	&	0.0231 	&	1.9119 	&	35.1661 	&	0.8871 	\\
8	&	1	&	1	&	1	&	6.400 	&	603.7 	&	0.9811 	&	2.1240 	&	0.0238 	&	1.9660 	&	34.9080 	&	0.8827 	\\
1	&	8	&	1	&	1	&	7.728 	&	192.4 	&	0.9817 	&	2.1230 	&	0.0233 	&	1.9293 	&	35.0427 	&	0.8844 	\\
1	&	1	&	8	&	1	&	7.728 	&	60.52 	&	0.9811 	&	2.1995 	&	0.0239 	&	1.9745 	&	34.9172 	&	0.8813 	\\
1	&	1	&	1	&	8	&	9.940 	&	25.38 	&	0.9812 	&	2.1963 	&	0.0238 	&	1.9653 	&	34.9743 	&	0.8816 	\\
\hline
\end{tabular}
\normalsize
\end{center}
\end{table}

\setlength{\tabcolsep}{1.0pt}
\begin{table}
\begin{center}
\caption{Effectiveness analysis of the proposed spectral splitting and aggregation structure. }\label{tab:SSAS}
\begin{tabular}{c|c|cccccc}
\hline
&Dataset  &   CC$\uparrow$	&	SAM$\downarrow$	&	RMSE$\downarrow$	&	ERGAS$\downarrow$	&	PSNR$\uparrow$	&	SSIM$\uparrow$\\
\hline
\hline
\multirow{4}{*}{w/o SSAS} & O	&	0.9987 	&	0.8100 	&	0.0054 	&	0.4780 	&	46.6650 	&	0.9911 	\\
&O+Self	&	0.9990 	&	0.7088 	&	0.0048 	&	0.4179 	&	47.9166 	&	0.9928 	\\
&O+Sym	&	0.9990 	&	0.7271 	&	0.0049 	&	0.4279 	&	47.6540 	&	0.9925 	\\
&O+Self+Sym	&	0.9991 	&	0.6834 	&	0.0046 	&	0.4041 	&	48.2217 	&	0.9932 	\\
\hline
\multirow{4}{*}{SSAS} & O	&	0.9989 	&	0.7555 	&	0.0051 	&	0.4440 	&	47.2672 	&	0.9923 	\\
&O+Self	&	0.9992 	&	0.6552 	&	0.0044 	&	0.3825 	&	48.6323 	&	0.9938 	\\
&O+Sym	&	0.9992 	&	0.6549 	&	0.0043 	&	0.3746 	&	48.7505 	&	0.9941 	\\
&O+Self+Sym	&	0.9993 	&	0.6086 	&	0.0039 	&	0.3430 	&	49.4840 	&	0.9948 	\\
\hline
\multicolumn{2}{c|}{Average Improvements}           &   0.0002$\uparrow$ 	&	0.0638$\downarrow$ 	&	0.0005$\downarrow$ 	&	0.0460$\downarrow$ 	&	0.9192$\uparrow$ 	&	0.0014$\uparrow$\\
\hline
\end{tabular}
\normalsize
\end{center}
\end{table}

\textbf{\emph{Remark 1}}: Human face, as a highly structured object, has two eyes, a nose, and a mouth. The global structure information of different objects is very similar. The differences between different people are mainly reflected in the detailed features. Therefore, we have reason to believe that through self-representation learning, we can synthesize some feature detail information that is not available in the training dataset. This information can actually be regarded as a kind of noise compared to the represented face image. Therefore, the proposed method based on self-representation learning can bring more information to the training dataset.

\textbf{\emph{Remark 2}}: It seems that putting a reconstructed sample back to the training set will add no extra information to the dataset. If we reconstruct the face image globally, the above point is the truth. But when we reconstruct the face image locally (patch-wisely), this would be not ture. In other words, for those patch-based methods, the reconstructed HR result does not have to be a linear combination of all the original training samples. We can imagine an extreme situation: if we set the patch size to very small level, \emph{i.e}., pixel-wise, we can reconstruct any image (any content) and dose not have to be a linear combination of training samples, \emph{e.g}., a cat or a dog image. Therefore, these patch-based reconstruction method can reconstruct some samples, which dose not have to be a linear combination of the original training samples. Therefore, when we put reconstructed HR samples back to the training set, we can actually introduce some additional information in the sense of that we can add an image (which cannot be linear combination with the original training) to the training dataset.

\section{Experiments}
\label{sec:experiments}
We use Pytorch libraries\footnote{\url{https://pytorch.org}} to implement and train the proposed SSANet method (the code will be available upon acceptance). We train different models to super-resolve the hyperspectral face images for upsampling factors 2, 4 and 8 with random initialization. We use the ADAM optimizer \cite{kingma2014adam} with an initial learning rate of 1e-4 which decays by a factor of 10 at each 30 epochs. In our experiments, we find it will take 60 epochs to achieve the stable performance. The models are trained with a batch size of 4.

All the experiments are conducted on the UWA Hyperspectral Face Database (UWA-HSFD)\footnote{It should be noted that we only use the database of UWA-HSFD due to the following reason. Currently, there are only three widely used hyperspectral face databases, the Hong Kong Polytechnic University Hyperspectral Face Database (PolyU-HSFD) \cite{di2010PolyU}, the CMU Hyperspectral Face Database (CMU-HSFD) \cite{Denes2002}, and UWA-HSFD used in our paper. The face images in PolyU-HSFD are very noisy and not suitable for the supervised learning. As for CMU-HSFD, the related project is unavailable at present, and we cannot obtain the database.} \cite{uzair2015hyperspectral}.
It contains 145 hyperspectral face images of 80 subjects collected in four sessions over time. The face images are acquired by an indoor imaging system using a CRI's VariSpec LCTF filter integrated with a Photon focus camera. In UWA-HSFD \cite{uzair2015hyperspectral}, the spectral range, in which the images have been acquired, is from 400 nm to 720 nm with a step size of 10 nm, resulting 33 spectral bands. Alignment errors are present between individual bands due to subjects movements and eye blinking during image acquisition.
To simulate the situation that the testing face image is not in the training phases, we randomly selecting one cube from one session for each of the 70 subjects (includes 10\% evaluation samples). The remaining 10 subjects are used for testing. An example of a face cube from the UWA-HSFD database can be seen in Fig. \ref{fig:uwasample}.
In our experiments, all the input LR images are obtained by 2$\times$, 4$\times$, or 8$\times$ Bicubic downsampling.

\textbf{Evaluation measures}. Six widely used quantitative picture quality indices (PQIs) are employed to evaluate the performance of our method, including cross correlation (CC) \cite{loncan2015hyperspectral}, spectral angle mapper (SAM) \cite{yuhas1992discrimination}, root mean squared error (RMSE), erreur relative globale adimensionnelle de synthese (ERGAS) \cite{wald2002data}, peak signal-to-noise ratio (PSNR), and structure similarity (SSIM) \cite{wang2004image}. For PSNR and SSIM of the reconstructed hyperspectral images, we report their mean values of all spectral bands. CC, SAM, and ERGAS are three widely adopted quality indices in HS fusion task, while the remaining three indices are commonly used quantitative image restoration quality indices. The best values for these indices are 1, 0, 0, 0, $+\propto$, and 1, respectively.

\begin{figure}[t]
  \centering

  \includegraphics[width=9cm]{spectral.pdf}\\

  \caption{Reconstructed spectral information of our method with and without SSAS for four randomly select pixels (they are (40, 55), (110, 113), (20.20), and (41, 82) from the first testing face image) corresponding to four subfigures when the upsampling factor is $r$ = 8.}\label{fig:spectral}
\end{figure}

\subsection{Investigation to the Proposed SSANet}
\subsubsection{Progressively upsampling}
\label{sec:upsampling}
The simplest upsampler is to do upsampling at the beginning by Bicubic interpolation or perform pixel shuffle at the end. They either increase the parameters and computational complexity of the network or increase the difficulty of neural network training.
Inspired by the Laplacian pyramid SR network \cite{lai2017deep}, we perform upsampling at the intermediate layers to progressively upsample the input image. We insert the upsampling module to each spectral splitting and aggregation module, as shown in Fig. \ref{fig:Network}. Similar to the Neural Architecture Search (NAS) \cite{liu2018progressive}, in our experiments we manually search the optimal structure. Table \ref{tab:upsampling} gives the objective results of some typical upsampling strategies for different upsampling factors (for $\times$2 and $\times$4, we show all the possible cases, while for $\times$8 we only show some typical cases). We use ``up1'', ``up2'', ``up3'', and ``up4'' to denote the parameters of upsampling factors of different upsampling modules. When the upsampling factor is 1, we remove the upsampling module. From these results, we can learn that the upsampling strategy is a very important factor affecting the overall performance. Different upsampling strategies will not only affect the reconstruction performance of the network, but also greatly affect the size and computational complexity of the network. For example, when we insert the upsampling module at the beginning or at the last for the $\times$2 SR, the computational complexity of the network is nearly 2.5 times difference (\emph{e.g.}, 269.2 FLOPs vs. 709.9 FLOPs), despite they achieve the similar reconstruction performance. The differences of reconstruction performance under different upsampling strategies of $\times$2, $\times$4, and $\times$8 SR are as high as 0.3 dB in term of PSNR (\emph{e.g.}, 47.0040 dB vs. 47.2690 for $\times$2 SR, 41.0973 dB vs. 41.4659 for $\times$4 SR, and 34.9080 dB vs. 35.2369 for $\times$2 SR). In order to balance the parameters, model size, and network reconstruction performance, we choose a reasonable set of upsampling parameters, which are marked with yellow background.

\setlength{\tabcolsep}{3.5pt}
\begin{table}
\begin{center}
\caption{Effectiveness analysis of the proposed training sample expanding strategy. }\label{tab:trainingsample}
\begin{tabular}{cccccccc}

  \hline
Dataset & $r$  &   CC$\uparrow$	&	SAM$\downarrow$	&	RMSE$\downarrow$	&	ERGAS$\downarrow$	&	PSNR$\uparrow$	&	SSIM$\uparrow$ \\
\hline
O	&	2	&	0.9989 	&	0.7555 	&	0.0051 	&	0.4440 	&	47.2672 	&	0.9923 	\\
O+Self	&	2	&	0.9992 	&	0.6552 	&	0.0044 	&	0.3825 	&	48.6323 	&	0.9938 	\\
O+Sym	&	2	&	0.9992 	&	0.6549 	&	0.0043 	&	0.3746 	&	48.7505 	&	0.9941 	\\
O+Self+Sym	&	2	&	0.9993 	&	0.6086 	&	0.0039 	&	0.3430 	&	49.4840 	&	0.9948 	\\
\hline
O	&	4	&	0.9957 	&	1.2985 	&	0.0110 	&	0.9269 	&	41.4541 	&	0.9673 	\\
O+Self	&	4	&	0.9960 	&	1.2496 	&	0.0107 	&	0.9002 	&	41.8074 	&	0.9687 	\\
O+Sym	&	4	&	0.9958 	&	1.2734 	&	0.0109 	&	0.9195 	&	41.6118 	&	0.9678 	\\
O+Self+Sym	&	4	&	0.9960 	&	1.2409 	&	0.0107 	&	0.9003 	&	41.8483 	&	0.9689 	\\
\hline
O	&	8	&	0.9825 	&	2.0828 	&	0.0230 	&	1.8961 	&	35.2369 	&	0.8901 	\\
O+Self	&	8	&	0.9838 	&	1.9939 	&	0.0221 	&	1.8281 	&	35.6496 	&	0.8956 	\\
O+Sym	&	8	&	0.9828 	&	2.0371 	&	0.0228 	&	1.8804 	&	35.3614 	&	0.8909 	\\
O+Self+Sym	&	8	&	0.9842 	&	1.9934 	&	0.0219 	&	1.8095 	&	35.6852 	&	0.8973 	\\
\hline
\end{tabular}
\normalsize
\end{center}
\end{table}

\subsubsection{Spectral splitting and aggregation}
To alleviate the S3 problem, we carefully design the spectral splitting and aggregation strategy (SSAS). It splits the hyperspectral face image into as many groups as possible at shallow layers to guarantee the training of a complex network, and gradually aggregates the neighbor bands at deeper layers to fully exploit spectral correlations. To demonstrate the effectiveness of SSAS, we report the comparison results of the proposed deep neural network with and without SSAS at Table \ref{tab:SSAS} when the upsampling factor is 2. To simulate the network without SSAS, we maintain the primary structure of SSANet and set the band size of each branch network to 33 (the number of spectral bands for UWA-HSFD database). In this situation, there is no spectral splitting and aggregation process for SSANet. From the tabulated results, we can clearly learn that SSANet with SSAS can stably improve the performance regardless of the training datasets (the original training dataset and expanded training dataset). In all cases, the average improvement (in term of PSNR) of SSANet with SSAS over SSANet without SSAS is nearly 1.0 dB, and the spectral similarity has also increased by nearly 10\% (please refer to the SAM index). Fig. \ref{fig:spectral} plots the reconstructed 33-dimensional spectral vectors (that are randomly selected from the first testing hyperspectral face image) of the proposed SSANet with and without SSAS. From these reported results, we can learn that SSANet with SSAS can better recover the spectral information. This further verifies that by introducing SSAS, the credibility of the reconstructed spectrum can be improved.

\begin{figure*}[!t]
  \centering

  \includegraphics[width=18cm]{SingleResults.pdf}\\

  \caption{Face SR results of one object of different methods at bands 5, 10, 15, 20, 25, and 30 when the upsampling factor is 8. (a) HR faces, (b) Bicubic interpolation, (c) results of LSR \cite{Ma2010LSR}, (d) results of LcR \cite{Jiang2014LcRTMM}, (e) results of EDSR \cite{lim2017enhanced}, (f) results of SAN \cite{dai2019second}, (g) results of 3DCNN \cite{mei2017hyperspectral}, (h) results of GDRRN \cite{li2018single}, (i) results of SSPSR \cite{jiang2020learning}, (j) results of Our method.}\label{fig:SingleResults}
\end{figure*}

\begin{figure*}[!t]
  \centering

  \includegraphics[width=18cm]{AllResults.pdf}\\

  \caption{Face SR results of all 10 objects of different methods at band 30 when the upsampling factor is 8. (a) HR faces, (b) Bicubic interpolation, (c) results of LSR \cite{Ma2010LSR}, (d) results of LcR \cite{Jiang2014LcRTMM}, (e) results of EDSR \cite{lim2017enhanced}, (f) results of SAN \cite{dai2019second}, (g) results of 3DCNN \cite{mei2017hyperspectral}, (h) results of GDRRN \cite{li2018single}, (i) results of SSPSR \cite{jiang2020learning}, (j) results of Our method.}\label{fig:AllResults}
\end{figure*}

\begin{figure}[!t]
  \centering

  \includegraphics[width=8.90cm]{FailCases.pdf}\\

  \caption{Face SR results of different methods when the upsampling factor is 8. Every three rows give the results of one object of different methods on band 33, 22, and 11. (a) HR faces, (b) results of EDSR \cite{lim2017enhanced}, (c) results of GDRRN \cite{li2018single}, (d) results of SSPSR \cite{jiang2020learning}, (e) results of Our method.}\label{fig:FailCases}
\end{figure}

\setlength{\tabcolsep}{2.5pt}
\begin{table}
\begin{center}
\caption{Quantitative comparisons of different approaches over 10 testing images from UWA-HSFD dataset with respect to six PQIs. The best and second best results are highlighted in boldface and underlined, respectively.}\label{tab:Face}
\begin{tabular}{cccccccc}

\hline
	&	$r$	&	CC$\uparrow$	&	SAM$\downarrow$	&	RMSE$\downarrow$	&	ERGAS$\downarrow$	&	PSNR$\uparrow$	&	SSIM$\uparrow$	\\
\hline
\hline
LSR \cite{Ma2010LSR}	&	2	&	0.9977 	&	1.2394 	&	0.0076 	&	0.6526 	&	43.7839 	&	0.9784 	\\
LcR	\cite{Jiang2014LcRTMM} &	2	&	0.9980 	&	1.0751 	&	0.0073 	&	0.6214 	&	44.4102 	&	0.9808 	\\
EDSR \cite{lim2017enhanced}	&	2	&	\underline{0.9988} 	&	\underline{0.7777} 	&	0.0053 	&	0.4616 	&	\underline{46.9760} 	&	\underline{0.9917} 	\\
SAN	\cite{dai2019second} &	2	&	0.9985 	&	0.8484 	&	0.0055 	&	0.4921 	&	46.3874 	&	0.9906 	\\
3DCNN \cite{mei2017hyperspectral}	&	2	&	0.9986	&	0.8676	&	0.0061	&	0.5218	&	46.2634	&	0.9895	\\
GDRRN \cite{li2018single}	&	2	&	0.9976	&	0.7848	&	0.0054	&	0.4730	&	46.8245	&	0.9911	\\
SSPSR \cite{jiang2020learning}	&	2	&	0.9987 	&	0.7947 	&	\underline{0.0052} 	&	\underline{0.4614} 	&	46.9406 	&	0.9916 	\\
SSANet	&	2	&	\textbf{0.9989} 	&	\textbf{0.7555} 	&	\textbf{0.0051} 	&	\textbf{0.4440} 	&	\textbf{47.2672} 	&	\textbf{0.9923} 	\\
\hline
LSR \cite{Ma2010LSR}	&	4	&	0.9937 	&	1.6073 	&	0.0134 	&	1.1214 	&	39.6160 	&	0.9472 	\\
LcR	\cite{Jiang2014LcRTMM} &	4	&	0.9939 	&	1.5774 	&	0.0134 	&	1.1144 	&	39.7621 	&	0.9480 	\\
EDSR \cite{lim2017enhanced}	&	4	&	0.9949 	&	1.4242 	&	0.0118 	&	0.9995 	&	40.6932 	&	0.9628 	\\
SAN	\cite{dai2019second} &	4	&	0.9948 	&	1.4723 	&	0.0118 	&	1.0048 	&	40.6052 	&	0.9629 	\\
3DCNN \cite{mei2017hyperspectral}	&	4	&	0.9942	&	1.4482	&	0.0132	&	1.0984	&	40.0966	&	0.9556	\\
GDRRN \cite{li2018single}	&	4	&	0.9950	&	1.3897	&	0.0118	&	0.9939	&	40.7684	&	0.9626	\\
SSPSR \cite{jiang2020learning}	&	4	&	\underline{0.9953} 	&	\underline{1.3750} 	&	\underline{0.0114} 	&	\underline{0.9656} 	&	\underline{41.0340} 	&	\underline{0.9651} 	\\
SSANet	&	4	&	\textbf{0.9957} 	&	\textbf{1.2985} 	&	\textbf{0.0110} 	&	\textbf{0.9269} 	&	\textbf{41.4541} 	&	\textbf{0.9673} 	\\
\hline
LSR	\cite{Ma2010LSR} &	8	&	0.9795 	&	2.3873 	&	0.0246 	&	2.0349 	&	34.4513 	&	0.8782 	\\
LcR	\cite{Jiang2014LcRTMM} &	8	&	0.9810 	&	2.3680 	&	0.0240 	&	1.9775 	&	34.8670 	&	0.8835 	\\
EDSR \cite{lim2017enhanced}	&	8	&	\underline{0.9820} 	&	2.2191 	&	\underline{0.0232} 	&	\underline{1.9213} 	&	\underline{35.1152} 	&	0.8849 	\\
SAN	\cite{dai2019second} &	8	&	0.9815 	&	2.3889 	&	0.0234 	&	1.9420 	&	34.9323 	&	0.8855 	\\
3DCNN \cite{mei2017hyperspectral}	&	8	&	0.9741	&	2.2846	&	0.0283	&	2.3150	&	33.5437	&	0.8470	\\
GDRRN \cite{li2018single}	&	8	&	0.9785	&	\underline{2.1964}	&	0.0254	&	2.0910	&	34.3277	&	0.8704	\\
SSPSR \cite{jiang2020learning}	&	8	&	0.9816 	&	2.1979 	&	0.0234 	&	1.9356 	&	34.9966 	&	\underline{0.8856} 	\\
SSANet	&	8	&	\textbf{0.9825} 	&	\textbf{2.0828} 	&	\textbf{0.0230} 	&	\textbf{1.8961} 	&	\textbf{35.2369} 	&	\textbf{0.8901} 	\\
  \hline
\end{tabular}
\normalsize
\end{center}
\end{table}

\subsection{Effectiveness of Training Sample Expanding}
In this paper, we introduce two strategies to expand the size of training dataset. To demonstrate the effectiveness of these strategies, we show the comparison results of our method with different training datasets. As shown in Table \ref{tab:trainingsample}, ``O'' demotes the original training dataset, ``Self'' demotes the expanded training dataset with self-representation based data argumentation, and ``Sym'' is the expanded training dataset using the symmetry nature of humane to flip the face image to double the size of training dataset. From the comparison results, we can clearly learn that the proposed self-representation and symmetry based data argumentation methods can improve the performance of our proposed deep neural network separately, no matter what the upsampling factor $r$ is. The combination of these two strategies can achieve the best performance.

\subsection{Comparison Results}
Since the proposed SSANet is the first hyperspectral image SR method for human faces, there is no direct hyperspectral image SR method for comparison. Therefore, to demonstrate the effectiveness of our method, we adjust two traditional shallow learning based face SR methods, LSR \cite{Ma2010LSR} and LcR \cite{Jiang2014LcRTMM}, two deep learning based general image SR methods, EDSR \cite{lim2017enhanced} and SAN \cite{dai2019second}, and one most recently proposed deep learning based single hyperspectral image SR method, SSPSR \cite{jiang2020learning} for the hyperspectral image SR task. Specifically, as for LSR \cite{Ma2010LSR} and LcR \cite{Jiang2014LcRTMM}, we super-resolve each band of the hyperspectral face image separately. For EDSR \cite{lim2017enhanced} and SAN \cite{dai2019second}, we retrain their networks by changing the number of input channels with the UWA-HSFD database. SSPSR \cite{jiang2020learning} is also retrained by the UWA-HSFD database for fair comparison.

Fig. \ref{fig:SingleResults} shows the super-resolved results of one object of different SR approaches at five specific bands, and in Fig. \ref{fig:AllResults} we also present the results of all objects at band 30. From these comparison results, we learn that Bicubic interpolation lost most of the face detailed features. LSR \cite{Ma2010LSR} and LcR \cite{Jiang2014LcRTMM} require the accurate alignment of face images. However, face images cannot be completely aligned due to non-rigid transformations such as expressions and poses. Therefore, ghosting effects often appear at the regions where the eyes and mouth cannot be completely aligned. EDSR \cite{lim2017enhanced} and SAN \cite{dai2019second} are not specifically designed for hyperspectral faces, and they cannot fully exploit the rich spectral information of hyperspectral faces. Their results are much blur than SSPSR \cite{jiang2020learning}, which is a general hyperspectral image SR method. Results reconstructed by our proposed method are more credible (more similar to the ground truth), and at the same time our method greatly reduces the artificial effects of the reconstructed eyes, mouth, and nose.

It should be noted that almost all these comparison approaches cannot accurately reconstruct the slightly opened eyes. As shown in Fig. \ref{fig:FailCases}, the eyes of the input face image are slightly opened, but the reconstruction result are open (please refer to the first row of the first object and the results of the second object). Actually, this shows a weakness of these learning-based techniques which depend highly upon the consistency (similarity) between the supporting testing image and training dataset. This is mainly because the training data cannot cover the changes caused by all factors (such as posture, expression, lighting, \emph{etc}.), especially when the size of the training dataset is limited.

Table \ref{tab:Face} tabulates the average objective results of seven comparison methods on the 10 testing images from UWA-HSFD dataset when the upsampling factor is set to 2, 4, and 8. Note that SSANet is the model trained by the original training samples, while SSANet+ denotes the model trained by the expanded training samples with ``Self'' and ``Sym''. These objective results also demonstrate the better performance of our method.

\begin{figure*}[t]
  \centering

  \includegraphics[width=17.5cm]{Cave3.pdf}\\

  \caption{Reconstructed images of \emph{paints} at CAVE dataset with upsampling factor $r$ = 4. The first row is the reconstructed results of the comparison methods, while the last row is the corresponding error maps. At the bottom of each reconstructed result, we also report the PSNR (dB) and SSIM indices.}\label{fig:cave3}
\end{figure*}

\begin{figure*}[t]
  \centering

  \includegraphics[width=17.5cm]{Cave6.pdf}\\

  \caption{Reconstructed images of \emph{real\_and\_fake\_apples} at CAVE dataset with upsampling factor $r$ = 4. The first row is the
reconstructed results of the comparison methods, while the last row is the corresponding error maps. At the bottom of each reconstructed result, we also report the PSNR (dB) and SSIM indices.}\label{fig:cave6}
\end{figure*}

\subsection{Results on CAVE dataset}
Since UWA-HSFD is the only released and usable hyperspectral face image database, we just show the face SR comparison results on that database. In this section, to demonstrate the effectiveness of the proposed spectral splitting and aggregation network, we additionally conduct comparison experiments on hyperspectral images of natural scenes.
Specifically, we use the CAVE multispectral image database because it is widely used in many multispectral image recovery tasks.
The database consists of 32 scenes of everyday objects with spatial size of 512$\times$512, including 31 spectral bands ranging from 400nm to 700nm at 10nm steps. To prepare samples for training, we randomly select 20 hyperspectral images from the database (10\% samples are randomly selected for evaluations). When the upsampling factor $r$ is 4, we extract patches with 64$\times$64 pixels (32 pixels overlap) for training; when the upsampling factor $r$ is 8, we let the extracted patches be 128$\times$128 pixels (with 64 pixels overlap). The corresponding LR hyperspectral images are generated by Bicubic downsampling with a factor of 4 or 8. The remaining 12 hyperspectral images of the database are used for testing, where the original images are treated as ground truth HR hyperspectral images, and the LR hyperspectral inputs are generated similarly as the training samples.

We compare the proposed SSANet method with some very competitive approaches, EDSR \cite{lim2017enhanced}, RCAN \cite{zhang2018image}, SAN \cite{dai2019second}, 3DCNN \cite{mei2017hyperspectral}, GDRRN \cite{li2018single}, and two most recently proposed methods, SSPSR \cite{jiang2020learning} and MCNet \cite{li2020mixed}. The average performance of the CC, SAM, RMSE, ERGAS, PSNR, and SSIM results of competing methods for different upsampling factors on the CAVE dataset is reported in Table \ref{tab:Cave}. From these results, we notice that SSPSR \cite{jiang2020learning}, MCNet \cite{li2020mixed}, and the proposed SSANet method outperform other competing methods with large gains. The proposed SSANet method is slightly better than SSPSR \cite{jiang2020learning} and MCNet \cite{li2020mixed}, which are state-of-the-arts in the field of single hyperspectral image SR.

In Fig. \ref{fig:cave3} and Fig. \ref{fig:cave6}, we show the super-resolved visual results (pseudo RGB image) and the corresponding error maps at 480nm, 580nm and 680nm of different methods for testing images \emph{paints} and \emph{real\_and\_fake\_apples} with upsampling factors $r$ = 4, respectively. From these visual results, we learn that all the competing methods can well recover the HR spatial structures of the hyperspectral images. From these error maps, we learn that RCAN, SSPSR, MCNet, and the proposed method achieve better reconstruction fidelity. Below the reconstruction results, we also present the PSNR, and SSIM results of the pseudo RGB image for different comparison methods. Obviously, the proposed SSANet method performs best in most cases, which is consistent with the results in Table \ref{tab:Cave}.

\setlength{\tabcolsep}{2.5pt}
\begin{table}
\begin{center}
\caption{Quantitative comparisons of different approaches over 12 testing images from CAVE dataset with respect to six PQIs. The best and second best results are highlighted in boldface and underlined, respectively.}\label{tab:Cave}
\begin{tabular}{cccccccc}

  \hline
	&	$r$	&	CC$\uparrow$	&	SAM$\downarrow$	&	RMSE$\downarrow$	&	ERGAS$\downarrow$	&	PSNR$\uparrow$	&	SSIM$\uparrow$	\\
    \hline
        \hline
Bicubic	&	4	&	0.9868 	&	4.1759 	&	0.0212 	&	5.2719 	&	34.7214 	&	0.9277 	\\
EDSR \cite{lim2017enhanced}	&	4	&	0.9931 	&	3.5499 	&	0.0149 	&	3.5921 	&	38.1575 	&	0.9522 	\\
RCAN \cite{zhang2018image}	&	4	&	0.9935	&	3.6050 	&	0.0142	&	3.4178	&	38.7585	&	0.9530 	\\
SAN \cite{dai2019second}	&	4	&	0.9935	&	3.5951 	&	0.0143 	&	3.4200 	&	38.7188 	&	0.9531 	\\
3DCNN \cite{mei2017hyperspectral}	&	4	&	0.9928 	&	3.3463	&	0.0154 	&	3.7042 	&	37.9759 	&	0.9522 	\\
GDRRN \cite{li2018single}	&	4	&	0.9934 	&	3.4143 	&	0.0145 	&	3.5086 	&	38.4507 	&	0.9538	\\
SSPSR \cite{jiang2020learning}	&	4	&	\underline{0.9939}	&	\underline{3.1846}	&	0.0138	&	3.3384	&	39.0892	&	0.9553	\\
MCNet \cite{li2020mixed}	&	4	&	\textbf{0.9942} 	&	\textbf{3.1758} 	&	\underline{0.0135} 	&	\underline{3.2767} 	&	\underline{39.1059} 	&	\textbf{0.9605} 	\\
SSANet	&	4	&	\textbf{0.9942} 	&	3.2137 	&	\textbf{0.0134} 	&	\textbf{3.1976} 	&	\textbf{39.3539} 	&	\underline{0.9596} 	\\
    \hline
Bicubic	&	8	&	0.9666 	&	5.8962 	&	0.0346 	&	4.2175 	&	30.2056 	&	0.8526 	\\
EDSR \cite{lim2017enhanced}	&	8	&	0.9778 	&	5.6865 	&	0.0279 	&	3.3903 	&	32.4072 	&	0.8842 	\\
RCAN \cite{zhang2018image}	&	8	&	0.9791 	&	5.9771 	&	0.0268 	&	3.1781 	&	32.9544 	&	0.8884 	\\
SAN \cite{dai2019second}	&	8	&	0.9795	&	5.8683 	&	0.0267	&	3.1437	&	33.0012	&	0.8888 	\\
3DCNN \cite{mei2017hyperspectral}	&	8	&	0.9755 	&	5.0948	&	0.0292 	&	3.5536 	&	31.9691 	&	0.8863 	\\
GDRRN \cite{li2018single}	&	8	&	0.9769 	&	5.3597 	&	0.0280 	&	3.3460 	&	32.5763 	&	0.8890	\\
SSPSR \cite{jiang2020learning}	&	8	&	0.9805	&	\underline{4.4874}	&	0.0257	&	3.0419	&	33.4340	&	0.9010	\\
MCNet \cite{li2020mixed}	&	8	&	\underline{0.9813} 	&	4.6726 	&	\underline{0.0248} 	&	\underline{2.9006} 	&	\underline{33.8149} 	&	\underline{0.9031} 	\\
SSANet	&	8	&	\textbf{0.9825} 	&	\textbf{4.3583} 	&	\textbf{0.0241} 	&	\textbf{2.8988} 	&	\textbf{34.0915} 	&	\textbf{0.9049} 	\\
  \hline
\end{tabular}
\normalsize
\end{center}
\end{table}

\section{Conclusions}
\label{sec:refs}
In this paper, we present a hyperspectral face super-resolution (HFSR) approach based on spectral splitting and aggregation network (SSANet). This is the first face SR work focusing on hyperspectral images. Different from traditional face and general image SR tasks, in which there are enough training samples to support the training of a complex network, HFSR has to face the small training sample size (S3) problem. To this end, on the one hand we carefully design a spectral splitting and aggregation network to fully make use of multiple spectral information. On the other hand we introduce two strategies to expand the training samples by a self-representation and the symmetry augmentation. Experimental results on public hyperspectral face database demonstrate that our proposed SSANet method and the self-representation and symmetry based training sample expanding strategy are effective for the HFSR task. In addition, we also report the comparison results on hyperspectral images of natural scenes, which demonstrate the generalization of our method.

\ifCLASSOPTIONcaptionsoff
  \newpage
\fi

{
\bibliographystyle{IEEEtran}

redpajama_set_name: RedPajamaArXiv


*Please enter through the front of the building.
Pre-registration will close the day before the meeting at 5:00 pm.
Registration will be available onsite the morning of the meeting.
We love seeing you at our monthly meetings! AFP memberships are not transferable; and therefore, only one member ID number should be used to register an individual. Those attending a meeting as a non-member must register as a guest. We love to see our guests become members! Please contact Jenna Jones at jjones@asginfo.net with any membership questions.
Cancellations can be made until 5:00 pm the day before the meeting.
No refunds will be issued the day of the meeting.
Job announcements will be made prior to the meeting.
Job announcers: Please plan to bring printed job descriptions with details and place them on the Job Postings table outside the meeting room. After the meeting, those interested will meet you at the table for more information.
Can't make the meeting? Consider a donation to Be the CAUSE.
A donation of $10, $20, $50 or $100 would help our Chapter support membership in meaningful ways!


text Kiplinger's personal finance [Print and Zinio Online].
Kiplinger's Personal Finance magazine provides advice and reporting on topics regarding personal finance issues. Find information on investments, taxes, insurance, retirement and savings in Kiplinger's Personal Finance magazine.


text:
Imports VBALib.BO
Imports System.ComponentModel.DataAnnotations

Namespace MyModels.Adapters
    <Schema.Table("Orders")>
    Public Class Order
        <Key()>
        Public Property ordId As Integer
        Public Property cstId As Integer

        <Schema.ForeignKey("cstId")>
        Public Property Customer As MyModels.Adapters.Customer

        Dim libOrder As VBALib.BO.Order

        Public Sub New()
            libOrder = New VBALib.BO.Order
            libOrdState = New ModelStateDictionary
        End Sub

        Public Sub New(d As Date, itms As List(Of VBALib.BO.Item))
            libOrder = New VBALib.BO.Order(itms, d)
            libOrdState = New ModelStateDictionary
        End Sub

        Public Property ordItems As List(Of Adapters.Item)

        Public Property ordDate As Date
            Get
                Return libOrder.ordDate
            End Get
            Set(value As Date)
                Try
                    libOrder.ordDate = value
                Catch ex As Exception
                    '* IMPORTANT key value should be same as the property
                    libOrdState.AddModelError("ordDate", ex.Message)
                End Try
            End Set
        End Property

        Dim libOrdState As ModelStateDictionary

        Public ReadOnly Property OrdState() As ModelStateDictionary
            Get
                Return libOrdState
            End Get
        End Property
    End Class
End Namespace

redpajama_set_name: RedPajamaGithub


text:
Eric's Fiction, Short Stories
The Lost Boy
"Kyle! Kyle!" Harriet fought through the woods, her frantic course like something dictated by a crazy pinball machine. She gasped for breath; her body ill-prepared for these demands. She wished the world wasn't so dim, so blurred, so far away. She was usually drunk, so she was used to the experience. But not now. "Please!" she begged.
It had been a kick-back Sunday afternoon in Frank's back yard. Kyle, 3, chasing the bright orange balloon around the yard. The other kids, older, shooting basketballs at the hoop that was lowered to their height. A dozen adults, from two bands, jamming around the fire pit, vanquishing a potluck munificence of booze, brats and baked beans, and creating a parody of their less inebriated performance the night before. Smoke from a half dozen joints making spiritual union with the rich offerings of the pine logs.
As one of their lengthy improvs came to an end, Harriet had rested her fiddle on her knee. Looking around, she slowly grasped that Kyle wasn't there.
"Where's Kyle?" she screamed.
One of the kids thought he chased his balloon into the woods.
She became a lot more sober, fast. At least, she tried to. As she raced into the woods, she screamed to the others, "Kyle's in the woods. We gotta find him!" But where? Frank's house was surrounded on three sides by woods darkening in the twilight.
First one way, then another—no idea where she should run.
She almost missed it. The little, orange scrap of rubber, the remains of a popped balloon.
"This way! This way!" she shouted. No one followed her. "Come on!" she cried as she ran. Why don't they come?
She broke into a small clearing, through brush that scratched and tore to hold her back. There was no sun; a tiny cabin, painted with elaborate patterns that twisted and swirled in slow, uncanny motion, seemed to glow by absorbing the surrounding light. A woman sat on the porch. A cape whose patterns matched the cabin partially hid a white blouse and navy blue straight skirt. Weird bangles cluttered her wrists and arms. As Harriet registered the woman's black, sunken eyes, hooked nose, protruding chin and shriveled skin, childhood terrors exploded in her soul. More terrifying: Kyle sat on the woman's lap. He held a green balloon.
"Kyle!" she called, rushing toward him.
Something was wrong. As hard as she ran, she didn't get any closer. The cabin seemed to float in wavering, blurring mist.
"Mommy! Mommy! Come see my new balloon."
"I'm coming, Kyle."
Her legs were in slow motion. The cabin taunted her.
The woman disappeared into the cabin with Kyle.
Harriet woke up at the kitchen table, an empty glass in her hand, her head lying on a letter from Child Protective Services. She got up to refill her glass, stopped, picked up the letter, reread it. Her scream as she hurled the glass at the ceramic backsplash sent the cat racing to cower under the sofa. Harriet sat down and cried.
An orange balloon, shrunken and tired, oscillated vaguely in the corner.
Harriet made her way across the dark lawn, uncoiling the extension cord as she went. She poked the end through the hedge separating her house from Jack and Glenda's, then walked to the sidewalk and back up the other side of the hedge. She found the end of the cord in the newly fallen snow and pulled enough through to reach the neighbors' house, a mirror-image of her 1950s ranch. On her knees, her hands getting numb, she found the outlet behind the shrubs and plugged in the cord.
The power and gas had been cut off four days ago. She didn't fight it. Then they took Kyle. After a day of rage and self-abuse, she had given up. The case worker said she'd have to stop drinking, get a job, get the utilities turned back on, and convince her that she was a responsible mother.
Hell, she thought. Why don't they demand that I grow wings and fly too?
Two drunken days later, the cold spell came, and she knew she had to do something.
Her small, thin body was thoroughly chilled when she got back to the house. She had extension cords running to a heater in the basement, to keep the pipes from freezing, to the kitchen for a hot plate and to a heater in the hall between the bathroom and the kitchen. She'd have to heat water on the hot plate for coffee, dishes and sponge bath. She had closed off the living room and dining room with blankets tacked to the wall and hauled a twin mattress, sheets and more blankets to the hall. She hoped she had enough blankets to stay warm through the night.
With a glass of whiskey, she settled into a folding sling-back chair in front of the electric heater. This will work till I find a job, she told herself. How she had come to this state she didn't totally understand. Sure, she drank a bit. But lots of musicians drank heavily. So why did her gigs dry up? When they did, she simply stopped paying bills. Fortunately the house was paid for, an inheritance from her parents. She could thank them for that, at least.
She finished her drink, changed quickly into sweat pants and hooded sweat shirt, and climbed into her chilly bed. She lay awake with images of Kyle filling her mind. Suddenly a huge sob escaped and she cried till she fell asleep.
In the morning she splashed her face with icy tap water and boiled water for a cup of instant coffee. Then she poured herself another double shot and settled down with her stack of romances from the library.
About eleven o'clock she walked to McDonald's, got herself an Egg McMuffin and logged onto their WiFi. She spent five minutes searching for jobs but found nothing that looked interesting. She walked back home.
From two houses away she saw Jack and Glenda standing on their porch. Jack had the extension cord in his hand.
Embarrassment. Shame. Turn around? Cut through a yard and get to her house from another direction? Too dumb. Just face the music. Even if her hair hadn't been washed in a week. What the fuck did it matter, anyway? She could live without heat and electricity. She could drain all the pipes and traps and sleep in the bus station. As she approached, she compared their house with hers. The contrast mirrored the contrast between their propriety and her impropriety: Freshly painted taupe with dark green trim versus 20-year old faded blue; neat shrubs versus scraggly lawn poking through the thin layer of snow and bare cinder block foundation.
Jack and Glenda met her on the sidewalk. Jack, about 55, was bald, his fringe shaved, with a growing pot belly bulging over his belt, his suit coat open, his tie several inches of bulge short of his belt. Glenda was petite, with granny glasses and dull house-wifey clothes. They looked concerned, not angry.
"This isn't a solution to your problem," Jack said. They knew about her lack of gigs and her drinking. She had told them everything was fine and she had a job lined up. Now they knew different. She knew they had somehow found out about Kyle.
"We want to help you," Glenda said. "But you have to help yourself as well. Come on in and let's talk."
"I'm sorry," she said. "That was stupid of me." She just wanted to get into her house and have a drink. "I have to do something in the house first."
"Nope," Jack said. "You don't need more 'something.' You need to break that habit."
"Don't say it that way," she said. "It's not like I'm an alcoholic or anything. Just having a hard stretch."
"We're not going to argue. We don't begrudge you some electricity. But on the condition that you quit drinking." She knew Jack was a recovering alcoholic and had found Jesus when he stopped drinking.
She didn't argue. What was the point? After Jack reconnected the extension cord, they sat around the dining room table. Glenda served coffee and almond coffee cake.
"We should begin by praying together," Jack said.
She thought that was nonsense but again didn't argue.
"Dear God," Jack prayed, "we come to you this morning to thank you for your many blessings. We lift up Harriet to you and just ask that you come into her heart and give her the strength to fight the demon alcohol. We just beg you to fill her with your Spirit so she might find your love and just repent of her ways and turn back to you. We ask all this in the name of your Son, Jesus Christ. Amen."
"Now," he continued, "how are we going to get your life straightened out?"
"I don't know what you mean. I'm fine. I just need some luck finding a new job."
"Come on, Harriet. You know that's not true. You're almost totally dysfunctional right now," Jack said. "To begin with," — why was he doing all the talking, she wondered since Glenda was unquestionably smarter than he — "you're moving in with us."
Harriet didn't like this. The memory of sleeping on the floor and the cold water in her face in the morning stalled any resistance, however. She was feeling dependent on them in a way that went beyond the extension cord linking their two houses.
Glenda settled her in the guest bedroom. It was still a boy's bedroom, with blue walls, sports posters on the wall. On the dresser were photos of their grown son, Jack Junior, his wife and the grandkids.
Seeing the look on her face when she saw the pictures, Glenda gathered them up.
"You don't need these."
Not only had Kyle been taken from her, but her husband Bobby had left a year or so back because of her drinking.
Harriet sat on the bed. She fought off tears. She told herself she didn't need the drink she craved. This is so stupid, she told herself. They're treating me like a derelict, like a child who can't take care of herself. Stuck in a child's bedroom. I'll show them. When they see I can do without alcohol they'll let me go back to my house.
At dinner that night, Jack and Glenda seemed to follow their usual routine, with Jack telling something about his day and Glenda updating him with what happened on the soaps. Harriet was a bit startled when they began the meal with prayer and ended with a bit of Bible reading and a prayer. At least they didn't pry or lecture.
She didn't sleep that night, between her body's demand for alcohol and her heart's mourning Kyle.
They settled into a routine. Jack went off to work — he was a salesman of some kind. Glenda stayed with Harriet continuously. They ate together, shopped together, went to the library together, did housework together. While Glenda watched her soaps, Harriet read romances and mysteries. Glenda checked out some books on alcoholism recovery, but Harriet wasn't interested. Gradually her craving for alcohol diminished as she rested in the care of her hosts. She felt plugged into a source of love and energy. When she thought about Kyle, she fought the heartache with promises to herself about recovering him soon. She didn't spend any time thinking about when or how that would happen.
Glenda always asked Harriet what she'd like to do, but somehow it was always Glenda's ideas and plans they ended up following. Harriet simply had no plans or ideas beyond going through the motions of looking for work each day.
A week and a half on, some lines in the romance she was reading caused her to stop and stare at the wall. What am I doing here, she wondered. I should be out working, not hanging around Glenda like I'm a little child. I don't need to be here. I've shown I can do without the alcohol.
But the thought of actually engaging the world without alcohol was enough to send her back to the romance.
With all the time they spent together and Glenda's quiet support, Harriet started opening up, talking about things she hadn't talked about to anyone before. It started with a letter from Child Protective Services setting a date for a hearing.
"What about Bobby? Can't he come and take Kyle?"
"Bobby's not his father."
"Couldn't he adopt him? He's still your husband, after all."
"He wanted to at one time, but not any more. He gave up on me when my drinking got so bad."
Glenda tried to soothe her. "Maybe it's not too late."
Harriet started to cry. "It is too late. Bobby's not coming back . . . I've ruined everything with my drinking."
She rushed to her room. After crying, she felt a bit better for having talked about it with Glenda.
Over the next days she talked more with Glenda — about Bobby, about her ex, about how he had raped her during the divorce proceedings, about her parents. Gradually, about her drinking. She had started to lose heart during the late stages of the marriage. After the rape she had given up. Drinking was easier than struggling. She met Bobby; he loved her; she married him. But it wasn't enough to give her strength to stop drinking.
On a quiet afternoon something prompted her to reflect more on what was happening. She was proud of not having a drink in three weeks. She knew that, comfortable as the arrangement with Jack and Glenda was, it wasn't right. She knew she had to do something. She was too old to be their child. Well, actually she wasn't. Biologically they were old enough to be her parents. Whatever. It was time she grew up.
She took out her violin and started with her warm-up exercises and a favorite etude. Irritated that her fingers were slow and awkward and her bow clumsy, she quit. She wanted to go for a walk, but knew Glenda would insist on coming along. This too irritated her. But she routed Glenda from her soap opera and they went out into the December cold. As usual, she averted her eyes from the extension cord connecting the two houses.
"I loved hearing your violin," Glenda said.
"I was playing terribly."
"Well, it's been several weeks since you played anything."
"This is worse than several weeks off," Harriet replied. "This is years of neglect." Thinking about the neglect of her art brought a wave of self-loathing and regret. She loved the violin. She loved the way her music spoke to people. But after the rape she had practiced less and less and eventually had stopped altogether. Then gigs became scarce, meaning she played even less.
"Let's go back," she said, turning as she spoke. She hurried to her room and practiced for an hour, till her uncalloused fingers were sore and her shoulders ached.
The household routine continued, except now Harriet spent two, and then three hours a day practicing. Over two weeks, the fingers toughened and the shoulders stopped aching.
A friend called. "I hear Lambert needs a couple of more violins for his Community Theater production of The Unsinkable Molly Brown. Why don't you give him a call?"
She called, not without anxiety. Lambert was an old friend who had watched her slide and tried to help.
"I need violins," he said. "But . . . this is embarrassing, Harriet, but I have to ask you to play for me first."
She didn't know what to say. She wanted to just hang up.
"Harriet . . . ?"
She knew he was right.
"I'll come over."
After she played for him and looked in his eyes, he just shook his head.
"I'm sorry, Harriet. It's just not there."
She had told him about living with Jack and Glenda and about how hard she had been practicing. She hadn't expected this.
Glenda was waiting for her in the car. She got in without a word, and Glenda knew not to say anything. Three blocks later, at a red light, Harriet opened the door, climbed out, and headed down the side street.
She turned in at The Speakeasy. Standing inside the entrance she soaked in the old familiar atmosphere, the thirty years' worth of embedded tobacco smoke in the walls and ceiling, the mixed bouquet of beer, whiskey and other alcohols, the chatter of voices in the nearly dark room. All the old need returned. She nodded at the bartender, a woman she had known here for years, and headed to her old table in the corner.
A waitress brought her an Old Fashioned glass with a double shot of Jack Daniels and one ice cube. She sat and stared at it. For ten minutes.
Jack walked in the door and slid into the booth, across the table from her.
She cursed herself silently for making that happen. She knew he hadn't been in a bar in twenty years, and she knew how hard it would be for him in this atmosphere.
"I didn't drink it," she said.
He nodded. "Let's go."
He found a parking space in front of her house. Walking to his door, she turned and stared at the extension cord. When she got in the house, she called the utility company and paid her bill with her credit card, which she had somehow managed not to max out. The next day the gas and electricity were turned back on.
She packed up her stuff, and she and Glenda carried it back to her house. The house was warm; the lights worked; she felt good.
Glenda gave her a hug. "Remember, it's not gonna be easy. You can't afford to get lonely. Our door's always open. The bedroom's yours. Come to work or come to sleep. There's always a place for you at the table, too. And pray."
Harriet thought AA would be more useful than a lot of prayer. She could already feel the demands on her resilience as the tentacles of reality approached, promising struggle.
After Glenda left, she went outside and disconnected the extension cord. As she coiled it up, she felt the beginning of a new life, like she had been born again. Indoors, she tuned her violin and started to practice.
Please share. Support an indie author.
Originally published on medium.com.
Tags: addiction parenting Passions redemption
← Review: Girl Waits with Gun by Amy Stewart
Too Old to Scream →

redpajama_set_name: RedPajamaCommonCrawl


text:
(function() {
  var out$ = typeof exports != 'undefined' && exports || typeof define != 'undefined' && {} || this;

  var doctype = '<?xml version="1.0" standalone="no"?><!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">';

  function isElement(obj) {
    return obj instanceof HTMLElement || obj instanceof SVGElement;
  }

  function requireDomNode(el) {
    if (!isElement(el)) {
      throw new Error('an HTMLElement or SVGElement is required; got ' + el);
    }
  }

  function isExternal(url) {
    return url && url.lastIndexOf('http',0) == 0 && url.lastIndexOf(window.location.host) == -1;
  }

  function inlineImages(el, callback) {
    requireDomNode(el);

    var images = el.querySelectorAll('image'),
        left = images.length,
        checkDone = function() {
          if (left === 0) {
            callback();
          }
        };

    checkDone();
    for (var i = 0; i < images.length; i++) {
      (function(image) {
        var href = image.getAttributeNS("http://www.w3.org/1999/xlink", "href");
        if (href) {
          if (isExternal(href.value)) {
            console.warn("Cannot render embedded images linking to external hosts: "+href.value);
            return;
          }
        }
        var canvas = document.createElement('canvas');
        var ctx = canvas.getContext('2d');
        var img = new Image();
        href = href || image.getAttribute('href');
        if (href) {
          img.src = href;
          img.onload = function() {
            canvas.width = img.width;
            canvas.height = img.height;
            ctx.drawImage(img, 0, 0);
            image.setAttributeNS("http://www.w3.org/1999/xlink", "href", canvas.toDataURL('image/png'));
            left--;
            checkDone();
          }
          img.onerror = function() {
            console.log("Could not load "+href);
            left--;
            checkDone();
          }
        } else {
          left--;
          checkDone();
        }
      })(images[i]);
    }
  }

  function styles(el, selectorRemap) {
    var css = "";
    var sheets = document.styleSheets;
    for (var i = 0; i < sheets.length; i++) {
      try {
        var rules = sheets[i].cssRules;
      } catch (e) {
        console.warn("Stylesheet could not be loaded: "+sheets[i].href);
        continue;
      }

      if (rules != null) {
        for (var j = 0; j < rules.length; j++) {
          var rule = rules[j];
          if (typeof(rule.style) != "undefined") {
            var match, selectorText;

            try {
              selectorText = rule.selectorText;
            } catch(err) {
              console.warn('The following CSS rule has an invalid selector: "' + rule + '"', err);
            }

            try {
              if (selectorText) {
                match = el.querySelector(selectorText);
              }
            } catch(err) {
              console.warn('Invalid CSS selector "' + selectorText + '"', err);
            }

            if (match) {
              var selector = selectorRemap ? selectorRemap(rule.selectorText) : rule.selectorText;
              css += selector + " { " + rule.style.cssText + " }\n";
            } else if(rule.cssText.match(/^@font-face/)) {
              css += rule.cssText + '\n';
            }
          }
        }
      }
    }
    return css;
  }

  function getDimension(el, clone, dim) {
    var v = (el.viewBox && el.viewBox.baseVal && el.viewBox.baseVal[dim]) ||
      (clone.getAttribute(dim) !== null && !clone.getAttribute(dim).match(/%$/) && parseInt(clone.getAttribute(dim))) ||
      el.getBoundingClientRect()[dim] ||
      parseInt(clone.style[dim]) ||
      parseInt(window.getComputedStyle(el).getPropertyValue(dim));
    return (typeof v === 'undefined' || v === null || isNaN(parseFloat(v))) ? 0 : v;
  }

  function reEncode(data) {
    data = encodeURIComponent(data);
    data = data.replace(/%([0-9A-F]{2})/g, function(match, p1) {
      var c = String.fromCharCode('0x'+p1);
      return c === '%' ? '%25' : c;
    });
    return decodeURIComponent(data);
  }

  out$.svgAsDataUri = function(el, options, cb) {
    requireDomNode(el);

    options = options || {};
    options.scale = options.scale || 1;
    options.responsive = options.responsive || false;
    var xmlns = "http://www.w3.org/2000/xmlns/";

    inlineImages(el, function() {
      var outer = document.createElement("div");
      var clone = el.cloneNode(true);
      var width, height;
      if(el.tagName == 'svg') {
        width = options.width || getDimension(el, clone, 'width');
        height = options.height || getDimension(el, clone, 'height');
      } else if(el.getBBox) {
        var box = el.getBBox();
        width = box.x + box.width;
        height = box.y + box.height;
        clone.setAttribute('transform', clone.getAttribute('transform').replace(/translate\(.*?\)/, ''));

        var svg = document.createElementNS('http://www.w3.org/2000/svg','svg')
        svg.appendChild(clone)
        clone = svg;
      } else {
        console.error('Attempted to render non-SVG element', el);
        return;
      }

      clone.setAttribute("version", "1.1");
      if (!clone.getAttribute('xmlns')) {
        clone.setAttributeNS(xmlns, "xmlns", "http://www.w3.org/2000/svg");
      }
      if (!clone.getAttribute('xmlns:xlink')) {
        clone.setAttributeNS(xmlns, "xmlns:xlink", "http://www.w3.org/1999/xlink");
      }

      if (options.responsive) {
        clone.removeAttribute('width');
        clone.removeAttribute('height');
        clone.setAttribute('preserveAspectRatio', 'xMinYMin meet');
      } else {
        clone.setAttribute("width", width * options.scale);
        clone.setAttribute("height", height * options.scale);
      }

      clone.setAttribute("viewBox", [
        options.left || 0,
        options.top || 0,
        width,
        height
      ].join(" "));

      var fos = clone.querySelectorAll('foreignObject > *');
      for (var i = 0; i < fos.length; i++) {
        fos[i].setAttributeNS(xmlns, "xmlns", "http://www.w3.org/1999/xhtml");
      }

      outer.appendChild(clone);

      var css = styles(el, options.selectorRemap);
      var s = document.createElement('style');
      s.setAttribute('type', 'text/css');
      s.innerHTML = "<![CDATA[\n" + css + "\n]]>";
      var defs = document.createElement('defs');
      defs.appendChild(s);
      clone.insertBefore(defs, clone.firstChild);

      var svg = doctype + outer.innerHTML;
      var uri = 'data:image/svg+xml;base64,' + window.btoa(reEncode(svg));
      if (cb) {
        cb(uri);
      }
    });
  }

  out$.svgAsPngUri = function(el, options, cb) {
    requireDomNode(el);

    out$.svgAsDataUri(el, options, function(uri) {
      var image = new Image();
      image.onload = function() {
        var canvas = document.createElement('canvas');
        canvas.width = image.width;
        canvas.height = image.height;
        var context = canvas.getContext('2d');
        if(options && options.backgroundColor){
          context.fillStyle = options.backgroundColor;
          context.fillRect(0, 0, canvas.width, canvas.height);
        }
        context.drawImage(image, 0, 0);
        var a = document.createElement('a'), png;
        try {
          png = canvas.toDataURL('image/png');
        } catch (e) {
          if ((typeof SecurityError !== 'undefined' && e instanceof SecurityError) || e.name == "SecurityError") {
            console.error("Rendered SVG images cannot be downloaded in this browser.");
            return;
          } else {
            throw e;
          }
        }
        cb(png);
      }
      image.onerror = function(error) {
        console.error('There was an error loading the data URI as an image', error);
      }
      image.src = uri;
    });
  }

  function download(name, uri) {
    var a = document.createElement('a');
    a.download = name;
    a.href = uri;
    document.body.appendChild(a);
    a.addEventListener("click", function(e) {
      a.parentNode.removeChild(a);
    });
    a.click();
  }

  out$.saveSvg = function(el, name, options) {
    requireDomNode(el);

    options = options || {};
    out$.svgAsDataUri(el, options, function(uri) {
      download(name, uri);
    });
  }

  out$.saveSvgAsPng = function(el, name, options) {
    requireDomNode(el);

    options = options || {};
    out$.svgAsPngUri(el, options, function(uri) {
      download(name, uri);
    });
  }

  // if define is defined create as an AMD module
  if (typeof define !== 'undefined') {
    define(function() {
      return out$;
    });
  }
})();

function genuri(istyp,svgid)
{
   svgAsPngUri(document.getElementById(svgid), {}, function(uri)
    {
        document.querySelector('#png-container').innerHTML = '';

       var sampleImage = document.getElementById("getimg"),
          canvas = convertImageToCanvas(sampleImage);

      // Actions
       document.getElementById("canvasHolder").appendChild(canvas);

       var imag = canvas.toDataURL("image/jpeg");
      var imagpng = canvas.toDataURL("image/png");

       switch(istyp) {
          case 'png':
                var a = document.createElement("a");
                a.href=imagpng;
                a.download="vla.png";
                document.body.appendChild(a);
                a.click();

          break;
          case 'pdf':
                var pdf = new jsPDF();
                pdf.addImage(imag, 'JPEG', 0, 0);
                var download = document.getElementById('download');
                pdf.save("download.pdf");

          break;
          case 'jpg':
                var a = document.createElement("a");
                a.href=imag;
                a.download="file.jpg";
                document.body.appendChild(a);
                a.click();

          break;
          case 'svg':
              svgAsDataUri(document.getElementById("svg"), {}, function(uri) {
                console.log(uri);
              var a = document.createElement("a");
                a.href=uri;
                a.download="fil.svg";
                document.body.appendChild(a);
                a.click();

              });

          break;
          case 'print':
          printCanvas();
          break;
          $("#mycanvas").remove();

        }

        function printCanvas()
{
    var dataUrl = document.getElementById('mycanvas').toDataURL("image/jpeg"); //attempt to save base64 string to server using this var
    var windowContent = '<!DOCTYPE html>';
    windowContent += ''
    windowContent += '<title></title>';
    windowContent += ''
    windowContent += '';
    windowContent += '';
    windowContent += '';
    var printWin = window.open('','','width=1000,height=760');
    printWin.document.open();
    printWin.document.write(windowContent);
    printWin.document.close();
    printWin.focus();
    printWin.print();
    printWin.close();
}
      function convertImageToCanvas(image)
       {
          var canvas = document.createElement("canvas");
          canvas.width = image.width;
          canvas.height = image.height;
          canvas.style="display:none";
          canvas.id="mycanvas";
          canvas.getContext("2d").drawImage(image, 0, 0);

         return canvas;
      }

      // Converts canvas to an image
      function convertCanvasToImage(canvas)
       {
          var images = new Image();
          images.src = canvas.toDataURL("image/png");
          return images;
      }
});
}


text:
LaBarbera: Tammy Baldwin 'Hardly a Role Model,' Lacks 'Healthy, Wholesome Behaviors'
By Brian Tashman | November 14, 2012 12:20 pm
Peter LaBarbera of Americans for Truth about Homosexuality was indignant about the "shocking" election of Wisconsin Democrat Tammy Baldwin to the U.S. Senate, making her the first openly gay Senator. He told Jim Schneider of VCY America on Crosstalk that Baldwin is "hardly a role model for young women," whom he says should uphold "healthy, wholesome behaviors, not lesbianism."
LaBarbera: All of the sudden I wake up to the news and look on the TV and see that Tammy Baldwin, a radical homosexual and pro-abortion feminist, everything's on the left in her policies, is now elected Senator in Wisconsin. It was quite shocking.
Schneider: The Human Rights Campaign, they call themselves 'the largest civil rights organization working to achieve lesbian, gay, bisexual and transgender equality,' they celebrated this, they attribute part of this, to ensure her victory they said 'they raised and contributed well over $500,000 for her campaign' and they said Tammy Baldwin is 'a role model for LGBT youth and young women across the country.'
LaBarbera: Human Rights Campaign should be called the Human Wrongs Campaign; it's a homosexual pressure group and we would expect them to be celebrating Tammy Baldwin's victory. But she's hardly a role model for young women, we want women to embrace marriage and healthy, wholesome behaviors, not lesbianism. Her policies are to the far-left, she's pro-abortion, in so many ways she represents what's wrong with the left and now she's going to be in the Senate for six years.
Recounting the other defeats gay equality opponents suffered in the election, LaBarbera warned that more Americans are embracing the "complete blasphemy" of same-sex marriage and that we are "seeing a civilization's moral decline" with the rise of the "side of sin, of evil, the pro-homosexual side."
The idea of homosexual marriage is a complete blasphemy and you would think that with most Christians the vast majority would get it and not affirm such positions, but sadly I think our society is de-Christianizing, it is becoming more secular, a recent study showed that a shocking number of Americans are unchurched. You see in such an environment, basically you are seeing a civilization's moral decline, the secular forces, the sin movement which is homosexual rights, has more momentum. Unless we see revival in this country, we're going to see more of this where the side of sin, of evil, the pro-homosexual side, has more energy than the side of good, which is the defense of marriage.
LaBarbera also attacked UPS for ending its financing of the Boy Scouts because the organization conflicts from the company's anti-discrimination policy, and said that "I think most Americans agree that we don't want homosexual men out camping in tents in forest with young boys, I certainly wouldn't want a homosexual man, an open homosexual, with my young son."
Look who the real discriminators are, it's UPS, they are saying if you adhere to a certain policy, which happens to be the Boy Scouts' moral creed, their morally straight policy which precludes homosexual Scout Masters, and I think most Americans agree that we don't want homosexual men out camping in tents in forest with young boys, I certainly wouldn't want a homosexual man, an open homosexual, with my young son. Yet that is called discrimination and they lose the funding while the Human Rights Campaign, which has multimillions of dollars, this hugely funded homosexual group, will continue to get UPS's funding.
Tags: Peter LaBarbera Anti-Gay Marriage Equality Americans for Truth Voice of Christian Youth America (VCY America)
LaBarbera: Government Cannot Be Trusted To Stop Ebola Because Of Gay Rights
LaBarbera: LGBT Rights Will Cause America's Downfall
Peter LaBarbera: Gay Rights Movement Is 'A Liberal Cult' And 'Anti-God'
Peter LaBarbera: LGBT Rights Advocates Want People To Be 'Ignorant'