text:
\section{Introduction}
Given a set of observable data, Latent Variable Models (LVMs) \citep{loehlin1987latent} aim to extract a reduced set of hidden variables able to summarise the information into a low dimensional space.
These models have become crucial in multi-view problems \citep{atrey2010multimodal, sharma2012generalized, li2019generative}, where data is represented by different modalities or views, since LVMs are able to explain the common information among all the modalities.
One of the most well-known types of LVMs are the
classical MultiVariate Analysis (MVA) methods, such as Principal Component Analysis (PCA) and Canonical Correlation Analysis (CCA) \citep{pearson1901liii,hotelling1936relations}, which aim to exploit the data correlation to obtain a low dimensional latent representation of the data. Its usage has been generalised due to its easy non-linear extension by means of kernel methods \citep{scholkopf1998nonlinear, zhang2016multi}. The fact of supporting a kernel formulation allows these methods to learn arbitrarily complex non-linear models with a complexity determined by the number of training points \citep{yu2013kernel} and make them highly convenient in scenarios with high dimensional data.

Probabilistic formulations of LVMs methods  are known as Factor Analysis (FA) \citep{harman1976modern}, which emerge as a linear Bayesian framework where one can obtain the desired latent representation together with a measure of the uncertainty. Among their many variants, such as Probabilistic PCA \citep{tipping1999probabilistic}, Supervised PCA \citep{yu2006supervised}, Bayesian Factor Regression \citep{bernardo2003Bayesian} or Bayesian CCA \citep{klami2013bayesian}, the Inter-Battery FA model \citep{klami2013bayesian} stands out for its capability of handling not only latent variables associated to the common information among all the views, but also for being able to model the intra-view information. This model has been recently extended in \citep{sevilla2020sparse}, named as Sparse Semi-supervised Inter-Battery Bayesian Analysis (SSHIBA),  to incorporate missing attributes, feature selection and the ability to handle heterogeneous data such as categorical or multi-dimensional binary data.

The use of kernel methods in Bayesian approaches has been mostly developed with Gaussian Processes (GP) \citep{williams2006Gaussian} and their non-supervised version to perform dimensionality reduction explored in GPLVMs \citep{lawrence2005probabilistic}. These approaches combine the advantages of the kernels methods, exploiting the non-linear relationships among the data, with those of a probabilistic framework. In \citep{damianou2012manifold}, the authors propose a shared GPLVMs approach, called Manifold Relevance Determination (MRD), to provide a non-linear latent representation for multi-view learning problems. This model is extended in \citep{damianou2016multi}, including an Automatic Relevance Determination (ARD) prior \citep{neal2012Bayesian} over the kernel formulation, to endow it with feature relevance analysis.

GPLVMs come with practical scalability drawbacks that need to be addressed. The cubic complexity with the number of training points requires the use of inducing points and variational approaches \citep{pmlr-v5-titsias09a}. Selecting the number of inducing points to use, and where to place them in the latent space, is still a challenging problem, being a common solution to place them in a regular basis along the latent space and only optimize the pseudo-observation at those points \citep{pmlr-v37-wilson15}. Furthermore, up to our knowledge, there is no versatile implementation in the state-of-the-art of a multi-view GPLVM able to handle heterogeneous observations (integer, categorical, real and positive observations) and missing values.

In this paper we propose a novel method to implement non-linear probabilistic LVMs that still builds upon a linear generative model, hence inheriting their computational and scalability properties. Instead of implementing a kernel method, i.e. a GP, to move from the latent representation to the observed data, we propose to reformulate probabilistic FA so that it generates kernel relationships instead of data observations. In the same way that Kernelized PCA (KPCA) or Kernelized CCA (KCCA) are able to generate non-linear latent variables by linearly combining element of a kernel vector; here, from a Bayesian generative point of view, we first i.i.d. sample latent representations and
project on an $N$-dimensional space (being $N$ the number of points) using a weight matrix representing the dual parameters.
We apply this trick over the SSHIBA formulation \citep{sevilla2020sparse}
to exploit their functionalities over this kernelized formulation. Thanks to that, we can efficiently face semi-supervised heterogeneous multi-view problems combining linear and non-linear data representations; in this way,
one can combine kernelized views to deal with non-linear relationships with linearly kernelized to work with high dimensional problems.
Although \citep{lian2015integrating} presents a first attempt of using kernelized data representations for FA models, here we propose a complete framework including the  following novelties/functionalities: (1) we can work with any type of kernel and are not limited to linear kernels; (2) we can force the automatic selection of Relevance Vectors (RVs) of the kernel to obtain scalable solutions  (equivalent to the support vectors of the Support Vector Machines (SVMs)); (3) we can include an ARD kernel to obtain a per-variable relevance that can be exploited for feature selection; (4) we can adapt the multiview model to work as multiple kernel learning algorithm for FA models.

The article is organised as follows: Section \ref{sec:KSSHIBA} presents the kernelized formulation of SSHIBA as well as the proposed formulations for the RVs and feature selection. Section \ref{sec:Results} defines the implementation details and the baselines and databases used for the experiments in different scenarios, where we show the performance of each proposed extension to the model. Finally, Section \ref{sec:conclusion} gives some final conclusions and highlights the main results.

\section{Bayesian Sparse Factor Analysis with Kernelized Observations}
\label{sec:KSSHIBA}

Let's consider a multi-view problem where we have $N$ data samples represented in $M$ different modalities, $\llav{\Xm}_{m=1}^M$, and our goal is to find an inter and intra-view non-linear latent representation, $\Z$. That is, given that $\Xnm \in \R^{D_m}$ is the $m$-th view   of  the $n$-th data, $\Zn$ has to compress, in a low dimensional space of size $K_c <<( D_1, \ldots, D_M)$ both the intra-view and inter-view information
of $\Xnm$ over all the views exploiting the correlations among the data.\footnote{Given a matrix $\mathbf{B}$, we denote the $i$-th row by $\mathbf{b}_{i,:}$ and the $j$-th column by $\mathbf{b}_{:,j}$.}
Whereas kernel MVA obtains the latent representation of the $n$-th data as a linear combination, by some dual variables, of its kernel representation $\Knm$, here we propose to reformulate this idea from a generative point of view.
In particular, we start from the SSHIBA algorithm formulation \citep{sevilla2020sparse} (see \ref{sec:AppendixSSHIBA} for a brief introduction) and consider that there exist some latent variables $\Zn \sim \N(\bf{0},\bf{I}_{K_c})$ which are linearly combined with a set of dual variables $\Am \in \R^{N \times K_c}$ to generate a kernel vector, $\Knm$, as:
\begin{align}
\Knm \enskip=&\enskip \Zn \AmT + \taum \label{eq:KernelDefinition}
\end{align}
where $\taum$ is zero-mean Gaussian noise, with noise power following a Gamma distribution of parameters $a^{\taum}$ and $b^{\taum}$,
and, given a mapping function $\phi(\cdot)$ and its associated kernel function $K(\bf x, \bf x') = \phi(\bf x)^\top  \phi(\bf x') $, $\Knm$ is the kernel between $\Xnm$ and all the training data, i.e.,  $\Knm = \cor{K\p{\Xnm,\Xonem}, \dots, \\ K\p{\Xnm,\XNm}}$. The dual variable matrix $\Am$ plays the role of the linear projection matrix and it is defined using the same structured ARD prior considered in both \citep{klami2013bayesian} and \citep{sevilla2020sparse}. Namely, an ARD prior that promotes that full rows of this matrix are cancelled, i.e.
\begin{align}
    \Akm \enskip\sim &\enskip \N\p*{0,\p*{\akm}^{-1} I_{K_c}} \label{eq:Adist}\\
    \alpha_{k}^{(m)} \enskip\sim &\enskip \Gamma\p*{a^{\alpha^{(m)}}, b^{\alpha^{(m)}}},
\end{align}
so that the product in \eqref{eq:KernelDefinition} induces sparsity in the latent factors, leading to a selection of the appropriate set of them \citep{tipping2001sparse}.

\begin{figure}[t]
 \centering
 \includegraphics[page=1,width=\linewidth]{Images/Graphic_Model.pdf}\\
 \captionof{figure}{Graphic model of SSHIBA with kernelized views (KSSHIBA).}
  \label{fig:Scheme_kernel}
\end{figure}

Figure \ref{fig:Scheme_kernel} shows the graphical model of KSSHIBA.
Following \citep{klami2013bayesian}, for the data views that are directly explained given the latent projection
we have $\Xnm = \Zn\WmT+\taum$, where the weight matrix $\Wm$ follows the same structured ARD prior mentioned above. We can refer to these as primal observations. For some other views, we might be interested in explaining them indirectly through a kernelized observation following \eqref{eq:KernelDefinition}. This conversion can be of interest when either we want to exploit intra and inter-view non-linear relationships, or when the view's dimensionality is much larger than the number of data points $N$ and we prefer to work in the dual space, to reduce the number of parameters to be learnt. When both primal and kernelized observations are used, the learned latent projection $\Zn$  attempts to faithfully reconstruct the original data representation in the primal views, as well as their kernel representation, i.e., their (non)-linear relationships with the other data points.

Note that sampling from the model in \eqref{eq:KernelDefinition} does not ensure a valid kernel positive semi-definite matrix. The kernel matrix is simply treated as an observation (a kernelized observation) and, as such, the model parameters will be chosen to minimize the reconstruction error. In Figure \ref{fig:kernel} we include a graphical representation of both a kernelized observation and the map reconstruction through \eqref{eq:KernelDefinition} using the  mean of the posterior distribution of $\Zn$, where we can observe that the kernelized observations (or kernel matrix) are almost perfectly reconstructed. Certainly, more appropriate models could be used to adapt the observation model (given $\Zn$) to the properties of a kernelized observation. To address this issue, we have explored alternative formulations based in non-independent noise; for example, defining the noise distribution as an inverse-wishart to have a full rank covariance noise or modelling its covariance as the product of two low rank matrices \citep{murphy2012machine}. However, these schemes led to considerably more complicated (less flexible) formulation which limited the rest of the properties of this proposal (as the ones proposed in the following sections). Henceforth, in this work, we restrict to the model in \eqref{eq:KernelDefinition}, and leave this line of work open for future research.

\begin{figure}[t]
  \centering
  \rotatebox{90}{\hspace{0.5cm} \small Linear Kernel}
  \begin{subfigure}[t]{0.28\linewidth}
    \centering
    \includegraphics[width=\linewidth]{Images/Original.png}\\

  \end{subfigure}
  ~
  \begin{subfigure}[t]{0.28\linewidth}
    \centering
    \includegraphics[width=\linewidth]{Images/Learnt.png}

  \end{subfigure}
  ~
    \begin{subfigure}[t]{0.34\linewidth}
    \centering
    \includegraphics[width=\linewidth]{Images/Difference_bar.png}

  \end{subfigure}
  \\
  \rotatebox{90}{\hspace{0.8cm} \small RBF Kernel}
  \begin{subfigure}[t]{0.28\linewidth}
    \centering
    \includegraphics[width=\linewidth]{Images/Original_rbf.png}\\
    \caption{Original kernel matrix}
    \label{fig:original_rbf}
  \end{subfigure}
  ~
  \begin{subfigure}[t]{0.28\linewidth}
    \centering
    \includegraphics[width=\linewidth]{Images/Learnt_rbf.png}
    \caption{Reconstructed matrix}
    \label{fig:learnt_rbf}
  \end{subfigure}
    ~
  \begin{subfigure}[t]{0.34\linewidth}
    \centering
    \includegraphics[width=\linewidth]{Images/Difference_rbf_bar.png}
    \caption{Reconstruction error}
    \label{fig:difference_rbf}
  \end{subfigure}
    \caption{Example of the generative properties of KSSHIBA to reconstruct a complete kernel matrix.}
  \label{fig:kernel}
\end{figure}

With this model we can evaluate the posterior distribution of the variables given the observed data
\begin{align}\label{eq:post}
p(\Theta|\mathbf{k}_{1,:},\dots, \mathbf{k}_{N,:}) &=\frac{\prod_{n=1}^{N}p(\mathbf{k}_{n,:}|\Theta)p(\Theta)}{p\p*{\mathbf{k}_{1,:},\dots, \mathbf{k}_{N,:}}},\\\label{eq:marglik}
p\p*{\mathbf{k}_{1,:},\dots, \mathbf{k}_{N,:}} &= \int p\p*{\Theta, \mathbf{k}_{1,:},\dots, \mathbf{k}_{N,:}} d(\Theta),
\end{align}
where $\Theta$ comprises all random variables (rv) in the model. As presented in \citep{sevilla2020sparse}, we rely on an approximate inference approach through mean-field variational inference \citep{Blei17}, where a lower bound to \eqref{eq:marglik} of the form
\begin{align}\label{elbo}
\log p\p*{\mathbf{k}_{1,:},\dots, \mathbf{k}_{N,:}} \geq \int q(\Theta) \log\left(\frac{\prod_{n=1}^{N}p(\mathbf{k}_{n,:}|\Theta)p(\Theta)}{q(\Theta)}\right)d(\Theta)
\end{align}
is maximised, and a fully factorised variational family is chosen to approximate the posterior distribution in \eqref{eq:post}
\begin{align}
p(\Theta|\KM)
\enskip\approx&\enskip \prod_{m=1}^{\M}\p*{q\p*{\Am} q\p*{\taum} \prodk q\p*{\akm}} \prod_{n=1}^{N}q\p*{\Zn}  \label{eq:qModel}
\end{align}
where $\KM$ represents a stacked version of the kernel for each sample, $\KnM$, of dimension $N\times D_m$.

The mean-field posterior structure along with the lowerbound in \eqref{elbo} results into a feasible coordinate-ascent-like optimization algoritm in which the optimal maximization of  each of the factors in \eqref{eq:qModel} can be computed if the rest remain fixed using the following expression
\begin{align}\label{eq:meanfield}
q^*(\theta_i) \propto \mathbb{E}_{\Theta_{-i}}\left[\log p(\Theta,\mathbf{k}_{1,:},\dots, \mathbf{k}_{N,:})\right],
\end{align}
where $\Theta_{-i}$ comprises all rv but $\theta_i$. This new formulation is in general feasible since it does not require to completely marginalize $\Theta$ from the joint distribution.

Redefining the input matrix as a kernel matrix leads to a formulation equivalent to SSHIBA, which, accordingly, permits to keep the previous model functionalities. Table \ref{tab:KSSHIBA} shows the KSSHIBA mean-field factor update rules. For a compact notation, we stuck in matrix $\mathbf{Z}$, of dimension $N\times K_c$, the latent projection of all data points and $<>$  represents the mean value of the rv.
The posterior distribution of all model parameters and latent projections is approximated using variational inference with a fully factorized posterior where it can be noted that each update has a computational cost of $O(N^2K_c+K_c^3)$.

\begin{table}[h!b]
\caption{Updated rules, obtained by a mean field approximation, of $q$ distribution for the different variables of KSSHIBA model.}
\begin{adjustbox}{max width=\textwidth}
\renewcommand{\arraystretch}{1.}
\centering
\begin{tabular}{ccc}
\toprule
{\textbf{Variable}} & { $\bm{q}^*$ \textbf{distribution}} & \textbf{Parameters} \\\midrule
\multirow{3}{*}{$\Zn$}
& \multirow{3}{*}{$\N\Zn | \mu_{\Zn},\Sigma_{\Z}$}
& $\mu_{\Zn} = \summ\p*{\ang{\taum} \Km \ang{\Am} \Sigma_{\Z}}$ \\
& & $\Sigma_{\Z}^{-1} = \p*{I + \summ\p*{\ang{\taum} \ang{\AmT \Am}}}$ \\&&\\
\midrule
\multirow{3}{*}{$\Am$}
& \multirow{2}{*}{$\prodn \p*{\N \p*{\Anm | \mu_{\Anm}, \Sigma_{\Am}}}$}
& $\mu_{\Anm} = \ang{\taum} \KmT \ang{\Z} \Sigma_{\Am}$ \\
& & $\Sigma_{\Am}^{-1} = \p*{\text{diag}(\ang{\am}) + \ang{\taum}\ang{\ZT \Z}}$  \\&&\\
\midrule
\multirow{3}{*}{$\akm$}
& \multirow{2}{*}{$\Gamma\p*{\akm | a_{\akm},b_{\akm}}$}
& $a_{\akm} = \frac{D_m}{2} + a^{\am}$ \\
& & $b_{\akm} = b^{\am} + \frac{1}{2} \ang{\AmT\Am}_{k,k}$ \\&&\\
\midrule
\multirow{5}{*}{{$\taum$}}
& \multirow{5}{*}{$\Gamma\p*{\taum | a_{\taum},b_{\taum}}$}
& $a_{\taum} = \frac{D_m N}{2} + a^{\taum}$ \\
& & $b_{\taum} = b^{\taum} + \frac{1}{2} \left(\sumn\sumns \Knnsm^2 \right.$ \\
& & $\left. - 2 \Tr\llav*{\ang{\Am}\ang{\ZT}\Km}\right.$  \\
& & $\left. + \Tr\llav*{\ang{\AmT\Am} \ang{\ZT \Z}}\right)$  \\
\bottomrule
\end{tabular}
\end{adjustbox}
\label{tab:KSSHIBA}
\end{table}

\subsection{Multiple kernel learning}

By using different kernels in different views, the model adapts to a multiple kernel learning scenario (MKL SSHIBA). To further analyse the implications of this combination, let's analyse the update rule for the mean of latent variables $\Z$
\begin{align}\label{eq:meanZ}
\mu_{\Zn} = \summ \ang{\taum} \Km \ang{\Am} \Sigma_{\Z}.
\end{align}
Here we can see that the latent factors are obtained as a linear combination of the kernels on each view. This entails the additional advantage of automatically learning these mixing coefficients of this combination (depending on parameters $\taum$, $\Am$ and $\Sigma_{\Z}$) through variational inference. Furthermore, if we define one view as an output view, the values that need to be predicted are calculated using this learnt variable Z, therefore conserving the MKL structure for the prediction.

Most MKL methods are based on heuristics \citep{de2010methods, qiu2008framework} or two step optimization process \citep{fung2004fast}. However, the proposed model determines the kernel parameters trhough the mean-field update. Furthermore, this formulation also benefits from the advantages of the SSHIBA formulation, namely semi-supervised learning, feature selection and RV selection.

\subsection{Automatic Bayesian Relevance Vector Selection}

On the basis of a full $N\times N$ kernel, with a more structured ARD prior we can achieve not only the shrinkage of the number of effective latent factors, but also a more compact representation of the data by means of a reduced kernel matrix in which only a reduced set of RVs are kept.

For this purpose, the proposed formulation can introduce a double ARD prior over the dual variables $\Am$,
\begin{align}
    \Ankm \enskip\sim &\enskip \N\p*{0,\p*{\gamnm\akm}^{-1}} \label{eq:Aspdist}
\end{align}
This way, $\akm$ continues forcing row-wise sparsity to automatically select the number of latent factors and, additionally,
\begin{align}
    \gamnm \enskip\sim &\enskip \Gamma\p*{a^{\gamm}, b^{\gamm}} \label{eq:Gamdist}
\end{align}
induces column-wise sparsity, which automatically forces many of the RVs to not influence in the final model, consequently achieving a more compact solution.
Table \ref{tab:KSSHIBA_sparse} shows the KSSHIBA mean-field factor update rules for the double ARD case.

\begin{table}[h!b]
\caption{Updated $q$ distribution for the automatic RV selection.}
\begin{adjustbox}{max width=\textwidth}
\renewcommand{\arraystretch}{1.}
\centering
\begin{tabular}{ccc}
\toprule
\textbf{Variable} & {$\bm{q}^*$ \textbf{distribution}} & \textbf{Parameters} \\\midrule
{\multirow{2}{*}{$\Am$}}
& \multirow{2}{*}{$\prodn\N \p*{\Anm | \mu_{\Anm}, \Sigma_{\Anm}}$}
& $\mu_{\Am} = \ang{\taum} \XmT \ang{\Z} \Sigma_{\Wm}$ \\
& & $\Sigma_{\Anm}^{-1} = \text{diag}(\ang{\am})\ang{\gamnm} + \ang{\taum}\ang{\ZT \Z}$ \\
\midrule
\multirow{3}{*}{$\gamm$}
& \multirow{3}{*}{$\prodn \Gamma\p*{\gamnm | a_{\gamnm},b_{\gamnm}}$}
& $a_{\gamnm} = \frac{K_c}{2} + a^{\gamm}$ \\
& & $b_{\gamnm} = b^{\gamm} + \frac{1}{2} \sumk\ang{\akm}\ang{\Ankm\Ankm}$ \\
\bottomrule
\end{tabular}
\end{adjustbox}
\label{tab:KSSHIBA_sparse}
\end{table}

\subsection{Automatic feature-relevance determination}\label{sec:fl}

Furthermore, we can additionally endow the proposed kernelized data representation with automatic feature relevance. If by using the double ARD structure we can cancel full rows or columns, equivalently, by using an ARD kernel we can perform this feature relevance determination. In the ARD kernel, each feature of the original observations is multiplied by a variable $\ldm$ in the kernel definition. For example, for a Radial Basis Function (RBF) kernel,
\begin{align}
    \Knnsm \enskip=&\enskip  \exp\p*{-\sumd\p*{\Xndm - \Xndm}^2 \ldm}, \label{eq:sparseKrbf}
\end{align}
we can optimise $\lm = \cor{\lonem, \dots, \lDm}$ by maximising the lower bound of our mean field approach given by direct optimisation over the variational lower bound. In our model, if the $m$-th view is kernelized then the only term in the lower bound affected by the ARD is the $\E_{q}\cor*{\lnp{p\p*{\Km|\Theta}}}$, having that (see \citep{sevilla2020sparse} for further details on the lower bound):
\begin{align}
    LB \enskip=&\enskip  -\frac{\ang{\taum}}{2} \sumn\sumu\p*{\Knum^2 - 2\Knum \ang{\Aum} \ang{\ZnT} + \ang{ \AumT,\Aum} \ang{ \ZnT,\Zn}} \label{eq:LBderivative}
\end{align}
We alternate between mean-field updates over the variational bound and direct maximization of \eqref{eq:LBderivative} w.r.t. $\lm$ using any gradient ascend method (we use Pytorch and Adam for such updates). Finally, by setting a threshold for $\lm$, the model is capable of automatically selecting the most relevant features while training.

\section{Results}
\label{sec:Results}

Throughout this section we define the different baselines as well as the implementation details for the analysis of the methods in terms of performance and interpretability of the inferred model parameters and latent projections.
Furthermore, an exemplary notebook with the \textit{Python} code of the proposed method is openly available in this github repository: \href{https://github.com/sevisal/KSSHIBA.git}{KSSHIBA}.

\subsection{Experimental setup}
\label{sec:Baselines}
In order to truthfully analyse the performance of the proposed framework, we decided to compare it with some state-of-the-art algorithms with similar capabilities. In particular we wanted to focus on some of the most relevant algorithms for FA. For the multi-dimensional regression datasets, we compare ourselves with KPCA, KCCA, MRD, Support Vector Regression (SVR) and MultiLayer Perceptron (MLP). Conversely, for the classification datasets, we compare with CCA and Support Vector Machine (SVM). We will now explain the specific parameters and formulations we used for these baselines.

As we proposed a kernel version of a FA model, we decided to compare to most common non-linear LVM methods, such as, KPCA and KCCA with a RBF kernel. For each of these models we explored 20 values of $\gamma$ kernel hyperparameter in log-scale from $[10^{-8}$ to $10^{0.5}]$ divided by the number of tasks ($C$). In order to carry out predictions KPCA was combined with Linear Regression (KPCA+LR), as KCCA can work in a supervised manner, we decided to use it on its own and with LR (KCCA+LR) for a complete comparative analysis. The number of latent factors has been fixed to the maximum possible, $C$, in KCCA and to those which explain $95\%$ of the variance in KPCA.

Due to the equivalency to some of our functionalities, namely working with multiview data, carrying out RV selection and latent representations,
we compared our model with the MRD \citep{damianou2012manifold} including an ARD for RVs selection. We used the available library in \textit{Matlab} \citep{damianou2012manifold}, setting the
number of latents to twice the number of tasks ($2*C$). We used the RBF kernel with ARD and set the number of model optimisation iterations to 100 due to the long computational time required to train.

Another kernel model used as baseline is SVR, where we also used a RBF kernel, exploring the same values of $\gamma$ as on KCCA/KPCA. The regularization parameter, $\lambda$, has also been validated, exploring 11 values in a logarithmic scale from $[10^{-4}$ to $10^{4}]$.

Finally, we also wanted to compare our model with a MLP neural network in two different scenarios: (1) a MLP to force a bottleneck of dimension $C$ in the hidden layers to have the same dimensionality reduction as KCCA. For this scenario, we validated two different configurations: (i) one hidden layer with $C$ neurons and (ii) two hidden layer with $C$ neurons and number of features ($D_m$) neurons, respectively. (2) A second scenario without bottleneck is presented, validating between three configurations: (i) one hidden layer with 100 neurons, (ii) two hidden layers with 100 and 50 neurons, and (iii) three hidden layers with 100, 50 and 100 neurons.

Regarding KSSHIBA, we have used its semi-supervised capability to predict the output, using the test samples (without their targets) during the training and, later, predicting their labels with the mean of the posterior distribution. To determine the number of iterations of the inference process of KSSHIBA, we used a convergence criteria based on the evolution of the lower bound. In particular, we stop the algorithm either when $mean(LB[-101:-2]) > LB[-1](1 - 10^{-4})$, where $LB[-1]$ is the lower bound at the last iteration and $mean(LB[-101:-2]$ the mean value of the previous values of the lower bound, or when it reaches $10^{4}$ iterations. The KSSHIBA  models were randomly initialized 10 times, keeping the one with the best lower bound. KSSHIBA automatically prunes the latent factors using the ARD prior included in the projection matrix $\Wm$.

We calculated the hyperparameters of each model with a nested 10-folds Cross-Validation (CV). The outer CV is used to divide the dataset into training and test partitions, while the inner CV is in charge of validation and, therefore, it divides the training partition into a second training set and a validation set. This way we were able to estimate the performance of the whole framework and, additionally, validate the model parameters.
We used the coefficient of determination (R2) to compare the performance of the different variations of the methods and to adjust the method hyperparameters, which were CV.

\subsection{Performance evaluation of KSSHIBA for multi-dimensional regression}\label{sec:mulan_performance}

This section aims to analyse the performance of KSSHIBA for semi-supervised multi-dimensional regression in comparison with some state-of-the-art baselines. To do so, we used 9 multi-dimensional regression datasets from the \textit{Mulan} repository \citep{spyromitros2016multi, karalivc1997first, dvzeroski2000predicting}, whose main properties are summarized in Table \ref{MTR-datasets}.
\begin{table}[h!b]
\caption{Characteristic of the multi-task databases used in this work.}
\label{MTR-datasets}
\centering
\begin{adjustbox}{max width=\textwidth}
\begin{tabular}{cccc}
\toprule
Database & Samples & Features & Tasks \\ \midrule
\textit{at1pd}        & 337        & 411        & 6          \\
\textit{at7pd}        & 296        & 411        & 6          \\
\textit{oes97}        & 334        & 263        & 16         \\
\textit{oes10}        & 403        & 298        & 16         \\
\textit{edm}          & 154        & 16         & 2          \\
\textit{jura}         & 359        & 15         & 3          \\
\textit{wq}           & 1,060      & 16         & 14         \\
\textit{enb}          & 768        & 8          & 2          \\
\textit{slump}        & 103        & 7          & 3          \\
\bottomrule
\end{tabular}
\end{adjustbox}
\end{table}

Table \ref{tab:MultiTask} shows the results obtained by the proposed model in two different scenarios (one in which the number of latent factors $K_c$ is automatically learnt with the ARD prior and another in which we set $K_c$ to $ C$) and we compare their results with those of the baselines, namely, KCCA, KPCA, MRD, SVR and MLP.

\begin{table*}[h!b]
\caption{Results on multitask databases of KSSHIBA and the baselines. The white subrow represents the mean and standard deviation of R2 score and the gray subrow the number of effective latent factors found.}
\label{tab:MultiTask}
\centering
\begin{adjustbox}{max width=\linewidth}
\begin{tabular}{lcccccccccc}
\toprule
 & \multirow{2}{*}{KSSHIBA} & KSSHIBA & \multirow{2}{*}{MRD} & \multirow{2}{*}{KPCA + LR} & \multirow{2}{*}{KCCA + LR} & \multirow{2}{*}{SVR-RBF} & \multirow{2}{*}{MLP} \\
 &  &\small $K_c = C$ &  & &  &  &  \\\midrule
\multirow{2}{*}{\textit{at1pd}}
& $\mathbf{0.79 \pm 0.09}$ & $0.78 \pm 0.09$ & $0.67 \pm 0.07$ & $0.67 \pm 0.12$ & $0.75 \pm 0.11$ & $0.01 $ & $0.77$\\
& \cellcolor{gray!10} $46 \pm 6$ & \cellcolor{gray!10} $6 $ & \cellcolor{gray!10} $12 $ & \cellcolor{gray!10} $22 \pm 10$ & \cellcolor{gray!10} $6 $ & $\pm 0.05$ & $\pm 0.11$\\ \midrule

\multirow{2}{*}{\textit{at7pd}} & $0.50 \pm 0.18$ & $0.52 \pm 0.13$ & $0.48 \pm 0.12$ & $0.39 \pm 0.19$ &  $\mathbf{0.57 \pm 0.16}$ & $0.01 $ & $0.35 $ \\
& \cellcolor{gray!10} $14 \pm 6$ & \cellcolor{gray!10} $6 $ & \cellcolor{gray!10} $12 $ & \cellcolor{gray!10} $21 \pm 1$ & \cellcolor{gray!10} $6 $ & $\pm 0.03$ & $\pm 0.69$  \\ \midrule

 \multirow{2}{*}{\textit{oes97}} & $\mathbf{0.71 \pm 0.10}$ & $0.69 \pm 0.10$ & $0.34 \pm 0.07$ & $0.45 \pm 0.20$ & $0.36 \pm 0.09$ & $0.39 $ & $0.58$ \\
& \cellcolor{gray!10} $17 \pm 6$ & \cellcolor{gray!10} $16 $ & \cellcolor{gray!10} $32 $ & \cellcolor{gray!10} $12 \pm 7 $ & \cellcolor{gray!10} $16 $ & $\pm 0.10$ & $\pm 0.21$\\ \midrule

 \multirow{2}{*}{\textit{oes10}} & $\mathbf{0.82 \pm 0.05}$ & $0.80 \pm 0.07$ & $0.38 \pm 0.07$  & $0.59 \pm 0.15$ & $0.43 \pm 0.12$ & $0.48 $ & $0.76 $ \\
 & \cellcolor{gray!10} $16 \pm 5$ & \cellcolor{gray!10} $16 $ & \cellcolor{gray!10} $32 $ & \cellcolor{gray!10} $14 \pm 7$ & \cellcolor{gray!10} $16 $ & $\pm 0.12$ & $\pm 0.08$\\ \midrule

 \multirow{2}{*}{\textit{edm}} & $\mathbf{0.51 \pm 0.18}$ & $0.21 \pm 0.09$ & $-0.17 \pm 0.45$ & $0.38 \pm 0.19$ & $0.18 \pm 0.26$ & $0.35 $ &  $0.26 $ \\
 & \cellcolor{gray!10} $30 \pm 11$ & \cellcolor{gray!10} $2 $ & \cellcolor{gray!10} $ $ & \cellcolor{gray!10} $16 \pm 5$ & \cellcolor{gray!10} $2 $ & $\pm 0.19$ &  $\pm 0.21$\\ \midrule

 \multirow{2}{*}{\textit{jura}} & $\mathbf{0.62 \pm 0.08}$ & $0.30 \pm 0.10$ & $0.57 \pm 0.06$ & $0.38 \pm 0.11$ & $0.18 \pm 0.15$ & $0.60 $ & $0.61$ \\
 & \cellcolor{gray!10} $21 \pm 3$ & \cellcolor{gray!10} $3 $ & \cellcolor{gray!10} $6 $ & \cellcolor{gray!10} $23 \pm 1$ & \cellcolor{gray!10} $3 $ & $\pm 0.05$ & $\pm 0.06$\\ \midrule

 \multirow{2}{*}{\textit{wq}} & $\mathbf{0.14 \pm 0.01}$ & $0.12 \pm 0.01$ & $-0.35 \pm 0.08$ & $0.09 \pm 0.02$ & $-0.01 \pm 0.01$ & $0.08 $ & $0.13$ \\
 & \cellcolor{gray!10} $76 \pm 9$ & \cellcolor{gray!10} $14 $ & \cellcolor{gray!10} $ 28 $ & \cellcolor{gray!10} $29 \pm 1$ & \cellcolor{gray!10} $14 $ & $\pm 0.02$ & $\pm 0.03$\\ \midrule

 \multirow{2}{*}{\textit{enb}} & $\mathbf{0.99 \pm 0.01}$ & $0.86 \pm 0.02$ & $0.91 \pm 0.01$ & $0.86 \pm 0.01$ & $0.98 \pm 0.01$ & $\mathbf{0.99}$ & $\mathbf{0.99 }$ \\
 & \cellcolor{gray!10} $118 \pm 4$ & \cellcolor{gray!10} $2 $ & \cellcolor{gray!10} $4 $ & \cellcolor{gray!10} $13 \pm 1$ & \cellcolor{gray!10} $2 $ & $\mathbf{\pm 0.01}$ & $\mathbf{\pm 0.08}$\\
\bottomrule
\end{tabular}
\end{adjustbox}
\end{table*}

In particular, we can see that KSSHIBA consistently outperforms most reference methods in every database, pointing out the performance advantages obtained in \textit{edm} and \textit{oes97}. Additionally, it is remarkable that this performance improvement is accomplished with an effective dimensional reduction, since KSSHIBA, applying a feature extraction, is able to outperform both a SVR and a MLP that use all the original features. At the same time, the results obtained by KSSHIBA with $K_c = C$ reveal that standard KSSHIBA is being too conservative in the number of extracted features and we could force a more restrictive pruning without degrading the final performance (note that  KSSHIBA with $K_c = C$ only deteriorates in the problems with only $2$ or $3$ (\textit{edm, jura} and \textit{enb}) since in this cases the number of latents is extremely reduced.

\subsection{Evaluation of the solution in terms of RVs}\label{sec:RV_exp}

Now, we want to test the capabilities of the KSSHIBA approach to automatically construct compact solutions by selecting a subset of training points, in other words, using RVs. For this purpose, we use the same databases and experimental setup as Section \ref{sec:mulan_performance}, but we compare with KPCA+LR and KCCA+LR using a Nyström \citep{nystrom} subsampling technique. In this way, KPCA+LR and KCCA+LR models will use this subsampling to select RV subsets; in these cases, the optimum percentage of RVs (respect to the total number of training data) has been selected by CV exploring their values in the set $[1,2,3,4,5,10,\dots,100] \%$.

Table \ref{tab:SupportVectors} shows
 that the inclusion of the automatic RV selection on KSSHIBA keeps the original model performance for most databases, even improving it for \textit{oes97} and \textit{edm}. This is done while the model complexity is drastically reduced; in fact, analysing these results in detail, it is observed that the fact of reducing the number of RVs favours an additional reduction in the final number of latent factors. When comparing to KPCA+LR and KCCA+LR, we can observe that KSSHIBA tends to show a lower percentage of RVs to describe the kernel. This is due to the fact that KSSHIBA learns the relevance of each element and eliminates them accordingly, whereas KPCA and KCCA obtain this compact solutions with a random selection of RVs.

\begin{table*}[h!b]
\centering
\caption{Results on the multitask databases for the automatic SV selection. The first subcolumn shows on the white subrow the mean and standard deviation of the R2 score and on the gray subrow the number of effective latent factors ($K_c$), the second subcolumn includes the percentage of RVs selected($\%RVs$).}
\label{tab:SupportVectors}
\begin{adjustbox}{max width=\linewidth}
\begin{tabular}{lccccccc}
\toprule
& \multicolumn{2}{c}{Sparse KSSHIBA} & \multicolumn{2}{c}{KPCA + LR} & \multicolumn{2}{c}{KCCA + LR} \\
& R2 - $K_c$ & $\%RVs$ & R2 - $K_c$ & $\%RVs$ & R2 - $K_c$ & $\%RVs$ \\
\midrule

\multirow{2}{*}{\textit{at1pd}} & $0.77 \pm 0.09$ & \multirow{2}{*}{$18.4 \pm 24.1$} & $0.78 \pm 0.09$ & \multirow{2}{*}{$69.7 \pm 32.9$} & $\mathbf{0.80 \pm 0.09}$ & \multirow{2}{*}{$84.8 \pm 27.5 $}\\
& \cellcolor{gray!10} $41 \pm 11$  & & \cellcolor{gray!10} $87 \pm 35$ & & \cellcolor{gray!10} $6$\\\midrule

\multirow{2}{*}{\textit{at7pd}} & $0.55 \pm 0.15$ & \multirow{2}{*}{$18.5 \pm 26.3$} & $0.56 \pm 0.18$ & \multirow{2}{*}{$79.7 \pm 31.7$} & $\mathbf{0.60 \pm 0.12}$ & \multirow{2}{*}{$73.9 \pm 34.1$}\\
& \cellcolor{gray!10} $70 \pm 27$ & & \cellcolor{gray!10} $90 \pm 37$  & & \cellcolor{gray!10} $6$\\ \midrule

 \multirow{2}{*}{\textit{oes97}} & $\mathbf{0.58 \pm 0.15}$ & \multirow{2}{*}{$38.6 \pm 24.5$} & $0.52 \pm 0.24$ & \multirow{2}{*}{$81.7 \pm 27.8$} & $0.42 \pm 0.30$ & \multirow{2}{*}{$23.9 \pm 27.8$}\\
& \cellcolor{gray!10} $61 \pm 7$ &  & \cellcolor{gray!10} $124 \pm 34$  &  & \cellcolor{gray!10} $16$ \\   \midrule

 \multirow{2}{*}{\textit{oes10}} & $\mathbf{0.77 \pm 0.11}$ & \multirow{2}{*}{$44.4 \pm 38.4$} & $0.71 \pm 0.12$ & \multirow{2}{*}{$71.9 \pm 11.6$} & $0.66 \pm 0.10$ & \multirow{2}{*}{$57.8 \pm 35.2$}\\
& \cellcolor{gray!10} $74 \pm 6$ &   & \cellcolor{gray!10} $132 \pm 53$ &  & \cellcolor{gray!10} $16$ \\ \midrule

 \multirow{2}{*}{\textit{edm}} & $\mathbf{0.42 \pm 0.21}$ & \multirow{2}{*}{$53.8 \pm 28.5$} & $0.41 \pm 0.26$ & \multirow{2}{*}{$52.5 \pm 30.5$} &  $0.20 \pm 0.14$ & \multirow{2}{*}{$22.7 \pm 13.6$}\\
& \cellcolor{gray!10} $13 \pm 4$ &  & \cellcolor{gray!10} $29 \pm 14$ &  & \cellcolor{gray!10} $2$ \\ \midrule

 \multirow{2}{*}{\textit{jura}} & $\mathbf{0.58 \pm 0.14}$ & \multirow{2}{*}{$48.7 \pm 38.4$} & $0.57 \pm 0.10$ & \multirow{2}{*}{$60.7 \pm 28.9$} & $0.36 \pm 0.09$ & \multirow{2}{*}{$18.9 \pm 7.5 $} \\
& \cellcolor{gray!10} $30 \pm 4$  &  & \cellcolor{gray!10} $59 \pm 14$ &  & \cellcolor{gray!10} $3$\\ \midrule

 \multirow{2}{*}{\textit{wq}} & $\mathbf{0.12 \pm 0.01}$ & \multirow{2}{*}{$58.1 \pm 33.2$} & $0.12 \pm 0.02$ & \multirow{2}{*}{$22.9 \pm 15.9$} & $0.10 \pm 0.01$ & \multirow{2}{*}{$5.9 \pm 3.1$}\\
& \cellcolor{gray!10} $21 \pm 2$ &  & \cellcolor{gray!10} $96 \pm 49$ &  & \cellcolor{gray!10} $14$ \\ \midrule

 \multirow{2}{*}{\textit{enb}} & $\mathbf{0.99 \pm 0.01}$ & \multirow{2}{*}{$19.5 \pm 12.8$} & $0.91 \pm 0.01$ & \multirow{2}{*}{$48.9 \pm 32.9$} & $0.97 \pm 0.01$ & \multirow{2}{*}{$41.9 \pm 12.2 $}\\
& \cellcolor{gray!10} $78 \pm 8$  &  & \cellcolor{gray!10} $28 \pm 1$ &  & \cellcolor{gray!10} $2$\\
\bottomrule
\end{tabular}
\end{adjustbox}
\end{table*}

To complete this analysis, Figure \ref{fig:sparse} includes the mean R2 over 10 folds varying the percentage of RVs. In this case, we show the results with the two databases for which SSHIBA is outperformed, the results with the other databases are available in \ref{sec:AppendixResults}. For the sake of comparison, we also included the MRD results when its percentage of inducing points is varied.
On the one hand, we can observe that the MRD behavior w.r.t. to the number of inducing points is unstable and quite dependent on the database. Further, the position of inducing points uses a regular grid, since this implementation does not allow their optimization, causing high fluctuations in its performance.
 On the other hand, while KPCA+LR and KCCA+LR present fluctuations in their performance requiring to adjust the number of RVs to obtain an accurate performance, KSSHIBA has a relatively constant R2 value.  This phenomenon occurs because KSSHIBA learns the relevance of each SV and weight their influence on the update of the parameters during all the model inference.

\begin{figure}[h!t]
  \centering
  \begin{subfigure}[t]{0.48\linewidth}
    \centering
    \includegraphics[width=\linewidth]{Images/Sparse_SV_atp1d.png}
    \caption{\textit{at1pd} database}
    \label{fig:sparse_atp1d}
  \end{subfigure}
  ~
  \begin{subfigure}[t]{0.48\linewidth}
    \centering
    \includegraphics[width=\linewidth]{Images/Sparse_SV_atp7d.png}
    \caption{\textit{at7pd} database}
    \label{fig:sparse_oes97}
   \end{subfigure}
  \caption{R2 results with different percentages of RVs in KSSHIBA, KCCA+LR and KPCA+LR or inducing points in MRD.}
  \label{fig:sparse}
\end{figure}

\subsection{Analysis of the feature relevance}

In order to test the feature relevance functionality (see Section \ref{sec:fl}),  we now study KSSHIBA on classification images database, using the image as input view and the category label as the output view, as this scheme provides an illustrative mechanism to analyze the feature (pixel) relevances. In particular, we used an aligned version of the Labeled Faces in the Wild (LFW) dataset \cite{LFWTech} obtained by \cite{wolf2010effective} and the  \textit{warpAR10P} ($60 \times 40$ pixels), \textit{Yale} ($32 \times 32$ pixels) and \textit{Olivetti} ($32 \times 32$ pixels) databases, which can be found in the Feature Selection Repository\footnote{\url{http://featureselection.asu.edu/datasets.php}}. Whereas the later databases were preprocessed, in LFW we had to crop the images to eliminate undesirable information and resize them to $60 \times 40$ pixels to reduce the computational cost of training the models; besides, to limit the size of the database we only used the images of the 7 people with most images in the database. The characteristics of these databases are  described in Table \ref{tab:Img-databases}.

\begin{table}[hb]
\renewcommand{\arraystretch}{1.2}
\caption{Characteristic of the faces databases used in this work.}
\label{tab:Img-databases}
\vspace{0.2cm}
\centering
\begin{adjustbox}{max width=\textwidth}
\begin{tabular}{cccc}
\toprule
Database & Samples & Features & Classes \\ \midrule
\textit{LFW}          & 1,277      & 2,400       & 7          \\
\textit{warpAR10P}    & 130        & 2,400       & 10          \\
\textit{Yale}         & 165        & 1,024       & 15         \\
\textit{Olivetti}     & 400        & 1,024       & 40         \\
\bottomrule
\end{tabular}
\end{adjustbox}
\end{table}

The KSSHIBA scheme followed in this extension consist in including an ARD kernel in the input images to obtain a feature relevance analysis. This way,  Figure \ref{fig:Masks} shows the relevance mask  learnt by the model, over echa dataset, having a lighter colour for relevant pixels a darker colour when the pixel is not relevant. With these databases we can easily recognise the learnt face shape, which determine which pixels the model have to center on, paying less attention to the background and, in some cases, clearly defining the nose, cheek or chin, i.e., defining a mask of the face.

\begin{figure}[h!t]
  \centering
  \begin{subfigure}[t]{.3\linewidth}
    \centering
    \includegraphics[width=\linewidth]{Images/mask.png}
    \caption{\textit{LFW}}
    \label{fig:mask_LFWA}
  \end{subfigure}
  ~
  \begin{subfigure}[t]{.381\linewidth}
    \centering
    \includegraphics[width=\linewidth]{Images/mask_warpAR10P_bar.png}
    \caption{\textit{warpAR10P}}
    \label{fig:mask_warpAR10P}
  \end{subfigure}
  \\

    \begin{subfigure}[t]{.3\linewidth}
    \centering
    \includegraphics[width=\linewidth]{Images/mask_Yale.png}
    \caption{\textit{Yale}}
    \label{fig:mask_Yale}
  \end{subfigure}
    ~

    \begin{subfigure}[t]{.36\linewidth}
    \centering
    \includegraphics[width=\linewidth]{Images/mask_ORL_bar.png}
    \caption{\textit{Olivetti}}
    \label{fig:mask_Olivetti}
  \end{subfigure}
  \caption{Feature masks learnt by the feature selection extension of KSSHIBA for different face recognition problems. The mask represent the importance of each pixel: lighter colours imply the pixel is more relevant while darker ones represent the pixel is less relevant.}
  \label{fig:Masks}
\end{figure}

Analysing these masks we can see that the relevances learnt for \textit{LFW} is more informative and have a higher definition than the other, mainly because there is a considerably higher number of samples for this database. Both the \textit{warpAR10P} and the \textit{Olivetti} databases agree to focus on the area related to the glasses as there is representative number of images with glasses in both databases. Besides, we have that if one subject wears glasses in one image, he will also wear glasses in the rest, therefore having a relevant area to classify the different subjects in these datasets. Another fact that stands out for us is that the mouths and eyes are generally considered as non-relevant for the classification task, while mainly focusing on the hair, cheeks and the face shape. This is not so accentuated in the \textit{warpAR10P} dataset because there are a considerable number of images with cloths covering the face under the nose.

\subsection{Analysis of the extracted latent factors}

In this section we want to evaluate the interpretability of the extracted latent factors obtained by the proposed model in comparison to the MRD approach based on shared GPLVMs. In particular, we will analyse both of them on the \textit{Oil} classification database \citep{bishop1993analysis} (which has 2,000 samples, 12 features and 3 output classes). For this purpose we have trained both models with $15$ latent factors (number of features plus output classes) combined with ARD latent factor selection. KSSHIBA uses a RBF kernel only for the input view meanwhile MRD uses it for both their input and output views. Under these conditions, the accuracy in the prediction of the labels for the MRD was of $99.0\%$ and KSSHIBA achieved a $99.4 \%$. Furthermore, with the available MRD implementation (\textit{Matlab}), the computational time is not scalable for the number of data.

\begin{figure}[h!t]
  \centering
  \begin{subfigure}[t]{0.4\linewidth}
    \centering
    \includegraphics[width=\linewidth]{Images/GPLVM_Latents.png}
    \caption{MRD - common}
    \label{fig:GPLVM_Latents}
  \end{subfigure}
  \\
  \begin{subfigure}[t]{0.4\linewidth}
    \centering
  \includegraphics[width=\linewidth]{Images/KSSHIBA_InputLatent.png}
    \caption{KSSHIBA - input}
   \label{fig:KSSHIBA_input}
   \end{subfigure}
     ~
  \begin{subfigure}[t]{0.4\linewidth}
    \centering
  \includegraphics[width=\linewidth]{Images/KSSHIBA_OutputLatent.png}
    \caption{KSSHIBA - output}
   \label{fig:KSSHIBA_output}
   \end{subfigure}
  \caption{Measure of relevance for each learnt latent factor on the \textit{Oil} database. Figure \ref{fig:GPLVM_Latents} shows the relevance of the commons for MRD model (all latents have resulted to be shared by both views). Figures \ref{fig:KSSHIBA_input} and \ref{fig:KSSHIBA_output} show, respectively, the relevance for the input view and the output view for KSSHIBA.}
 \label{fig:Oil_Latents}
\end{figure}
Figure \ref{fig:Oil_Latents} shows the learnt relevance for each latent factor for MRD ($W$) and KSSHIBA ($\gamm$). In the case of MRD, the formulation limits the model to specify the number of latent factors that are going to be used for the input and for the output data (Figure \ref{fig:GPLVM_Latents} shows the relevance for all these common factors). This means that the first 12 latent factors are related to the input view and the last 3 to the output, seeing that the model mainly focuses on the latter.
On the other hand, KSSHIBA presents independent weights for each view (see Figures \ref{fig:KSSHIBA_input} and \ref{fig:KSSHIBA_output}). This implies that the latent factors might be not relevant and could be pruned (latent 7), are only relevant for one view (latents 5 and 14) or are simultaneously relevant for more than one view (highlighting latents 0, 2 and 8). This latent information obtained by the inclusion of an ARD prior over the projection matrices $\Wm$, provides a more interpretable model.

Finally, to analyse the interpretability of the results, Figure \ref{fig:Oil_Latents3D} shows the most relevant extracted latent features with both models. For this three classes classification problem, we can see that the KSSHIBA algorithm is capable of finding a subspace where, with the three most relevant common latent factors, the classification problem is easily solved. Meanwhile, MRD with three latent factors projects most samples into a single point, needing more latent factors to discriminate the different classes.

We think that MRD needs for a large number of common latent to obtain a discriminative space due to the inclusion of a non-linearity in its output view. In fact, this suggests that using a simple linear output view in KSSHIBA facilities the achievement of not only a better performance, but also a more informative and discriminative latent space.

\begin{figure}[h!t]
  \centering
  \begin{subfigure}[t]{0.4\linewidth}
    \centering
    \includegraphics[width=\linewidth]{Images/GPLVM_K15.png}
    \caption{Latent space of MRD}
    \label{fig:Oil_GP}
  \end{subfigure}
  ~
  \begin{subfigure}[t]{0.4\linewidth}
    \centering
  \includegraphics[width=\linewidth]{Images/KSSHIBA_K15.png}
    \caption{Latent space of KSSHIBA}
  \label{fig:Oil_KSSHIBA}
  \end{subfigure}
  \caption{Learnt projections for the Oil database. Each figure shows the projections over the three most relevant factors: latents 12, 13 and 14 for MRD and latents 0, 2 and 8 for KSSHIBA.}
  \label{fig:Oil_Latents3D}
\end{figure}

\subsection{Multiple kernel learning for factor analysis}

One of the main functionalities of KSSHIBA is its capacity to combine multiple views into a single model. We can take advantage of this property when reconstructing kernel representations to combine different types of kernels (one per view) and  obtain a latent representation resulting from the linear combination of different kernels, known as MKL.
Furthermore, the nature of the algorithm allows us to automatically learn the relevance of each kernel while combining different kernels in one model.

To analyse this functionality, we decided to test it with three different databases: \textit{Arrhythmia} and \textit{Landsat} from the UCI repository \citep{UCI:2019} and \textit{Fashion MNIST} \citep{FMNIST}. These databases are described in Table \ref{tab:MKL_database}.
\begin{table}[hb]
\renewcommand{\arraystretch}{1.2}
\caption{Characteristic of the databases used for MKL.}
\label{tab:MKL_database}
\vspace{0.2cm}
\centering
\begin{adjustbox}{max width=\textwidth}
\begin{tabular}{ccccc}
\toprule
Database & Samples & \multicolumn{2}{c}{Features} & Classes \\
 & & Kernel 1 & Kernel 2 \\
\midrule
\textit{Arrhythmia}          & 452      & 15  & 264   & 2          \\
\textit{Landsat}             & 6,435    & 4   & 4     & 6        \\
\textit{Fashion MNIST}       & 1,000    & 784 & 784   & 10         \\
\bottomrule
\end{tabular}
\end{adjustbox}
\end{table}

To analyze the MKL KSSHIBA performance, we have compare it with a SVM classifier and a CCA to extract the latent factors and then a SVM to classify (CCA+SVM), using as input a linear combination of two different kernels:
\begin{align}
    K\p*{X_{i,:}, X_{j,:}} = \mu K^1\p*{X_{i,S}, X_{j,S}} + (1-\mu) K^2\p*{X_{i,R}, X_{j,R}} \label{eq:Kernel}
\end{align}
where $K^1$ is the first kernel, $K^2$ is the second kernel and $\mu \in [0,1]$ is the combination coefficient.
Using these equations for the \textit{Arrhythmia} database, we defined $K^1$ as a RBF kernel with the demographics and general ECG information (15 features) and $K^2$ as a RBF kernel with the channel parameters like width and amplitude (263 features). For the \textit{Landsat} database we defined $K^1$ as a RBF kernel with the spectral information (4 features) while $K^2$ was a RBF kernel uses the contextual information (4 features), as proposed in \citep{amoros2011multitemporal}. Finally, for the \textit{Fashion MNIST} database we used only 1,000 samples for the experiment, with 784 features, using the same data for each kernel, setting $K^1$ to be RBF and $K^2$ to be polynomial.
We validated the $C$ and $\gamma$ parameters for the SVM as well as the combination coefficient $\mu$. For CCA we set the number of latent factors to $\#classes-1$.

\begin{table}[hb]
\renewcommand{\arraystretch}{1.2}
\caption{Multiple Kernel Learning analysis. The performance measure used is the multiclass AUC, which we calculated in a 10-fold cross-validation.}
\label{tab:MKL}
\vspace{0.2cm}
\centering
\begin{adjustbox}{max width=\textwidth}
\begin{tabular}{ccccccccc}
\toprule
Database & MKL SSHIBA & MKL SSHIBA + RVS & SVM & CCA+SVM \\\midrule
\textit{Arrhythmia} & $0.809 \pm 0.057$ & $\bm{0.818 \pm 0.046}$ & $0.774 \pm 0.045$ & $0.737 \pm 0.059$         \\
\textit{Landsat}    &  $\bm{0.983 \pm 0.002}$ & $\bm{0.983 \pm 0.002}$ & $0.963 \pm 0.042$ & $0.980 \pm 0.002$         \\
\textit{Fashion MNIST}    &  $0.951 \pm 0.011$ & $0.952 \pm 0.013$ & $\bm{0.970 \pm 0.010}$ & $0.844 \pm 0.012$         \\
\bottomrule
\end{tabular}
\end{adjustbox}
\end{table}

The performance obtained with these configurations are included in Table \ref{tab:MKL}, where we used two versions of KSSHIBA, with and without RVs Selection (RVS), and multiclass AUC as the performance measure. The results prove that the combination of the different kernels carried out by MKL-SSHIBA is capable of providing good results with respect to the baselines. In particular, in the \textit{Arrhythmia} dataset we obtained an improvement of up to $0.044$ with respect to the SVM. Furthermore, we can see that the inclusion of the RV selection provides a more robust model, as it automatically selects the RVs that are relevant for the problem. Although in the \textit{Fashion MNIST} database the SVM provides a better result than the proposed model, we consider that the improvement is not enough to justify the lack of feature extraction; in fact, the results obtained by SVM with feature extraction are considerably lower than the obtained by the proposed model.

To further analyse the capabilities of the presented model, we decided to use the \textit{Fashion MNIST} database with three different kernels to analyse the learnt latent factors. In particular, we decided to combine the RBF and polynomial with a linear kernel. We included the labels in another view and used them as an output view. The performance obtained with this framework is equivalent to the one previously explored, probing that the combination of different kernels do not deteriorate the performance of the model.

\begin{figure}[ht]
 \centering
 \includegraphics[page=1,width=\linewidth]{Images/MultiKernel_noLin.png}\\
 \captionof{figure}{Relevance of the learnt latent factors. Measure of relevance on \textit{Fashion MNIST} database combining kernels and labels on different views.}
  \label{fig:multikernel}
\end{figure}

Figure \ref{fig:multikernel} shows the relevance of each latent factor in the joint scenario (all kernels used).  From the original $100$ latent factors, the RBF and the polynomial kernels share $3$ and the output view only uses $17$, being most of them private. In fact, we can observe that the improved performance of the model is obtained using only four common factors to all views for the prediction of the output label. Note that, only the private output and the shared factors are used for the prediction, while the rest capture relations between views.

\section{Conclusions}
\label{sec:conclusion}

We propose a novel probabilistic latent variable model to generate kernel relationships, instead of data observations, based on a linear generative model. We introduce this model using the Bayesian Inter-Battery Factor Analysis (BIBFA) approach proposed in \citep{sevilla2020sparse} to show its capabilities to efficiently face semi-supervised heterogeneous multi-view problems combining linear and non-linear data representations. Besides, we extend the model formulation to provide the automatic selection of RVs, obtaining scalable solutions, as well as include an ARD prior over the kernel to obtain the feature relevance functionality. The model performance is evaluated in multi-dimensional regression, feature relevance over images and multiple-kernel learning problems demonstrating that the inclusion of kernelized observations provide fruitful results.

The results prove the relevance of the proposed formulation, achieving a not only competitive performance, but also transforming the data to a reduced set of interpretable latent variables and a compact model consisting in a reduced subset of RVs. Furthermore, the feature relevance criteria is able to learn relevant masks which provide insight knowledge of the input space for the goal task. Finally, we also proved that taking advantage of the multiview nature of KSSHIBA we can easily combine different kernels to enhance, even more, the final performance of the model. This model contributes to the field of Artificial Intelligence by providing a model capable of adapting to a wide range of scenarios. In particular, the ability of combining feature selection, with heterogeneous data, imputation of missing values and combination of multiple kernels can be very useful for some biomedic or face recognition problems.  In these, there usually are a considerable number of missing values, the interpretability given by the feature relevance and the learnt latent factors might improve the analysis of the results and MKL allows the model to combine linear and non-linear data.

\section*{Acknowledgments}
\label{sec:acknowledgments}
The work of Pablo M. Olmos is supported by   Spanish  government   MINECO   under   grant  RTI2018-099655-B-10,  by  Comunidad de   Madrid   under   grants   IND2017/TIC-7618, IND2018/TIC-9649, and Y2018/TCS-4705,  by BBVA Foundation under the Deep-DARWiN project,  and   by   the European  Union  (FEDER and the European Research Council (ERC) through the European Unions Horizon 2020 research and innovation program under Grant 714161). C. Sevilla-Salcedo and V. G{\'o}mez-Verdejo's work has been partly funded by the Spanish MINECO grant TEC2017-83838-R. The work of Alejandro Guerrero is supported by MINECO and EU FEDER.

redpajama_set_name: RedPajamaArXiv


text:
MainAll NewsDefense/SecurityProtesters, incitement return to Temple Mount
Protesters, incitement return to Temple Mount
Friday's prayers saw the return of protestors affiliated with the outlawed Islamic Movement to the Temple Mount.
Shimon Cohen, 24/01/16 08:03
Arabs on Temple Mount (illustration)
Muath Al Khatib/Flash 90
As the number of violent terror attacks in Israel have begun to decline in recent weeks, protests inciting Palestinians to further violence returned to the Temple Mount on Friday.
According to reports by the Temple Institute, loud protests including inciteful remarks, ISIS flags and covered faces returned to the Temple Mount plaza following Friday's prayers on the Mount.
The Institute revealed video footage from Palestinian media sources that depicted the protests and the incitement. Speeches were made in an attempt to encourage Palestinians to commit terror acts against Jews and Israelis. Large posters bearing the faces of terrorists who have been killed and imprisoned by Israeli security forces after carrying out attacks were set up near the Dome of the Rock mosque.
In one frame, a large poster bearing the image of Nashat Milhem - the Israeli Arab terrorist who murdered three people in a Tel Aviv shooting spree earlier this year - can be seen.
The Temple Institute warned that this may be a case of "signaling the return of Temple Mount incited violence."
"This is a bitter sign that terrorism is still plaguing Israel. It seems that the police have forgotten that the seeds for all of the terror attacks were sown right under their noses, for months by the Northern Branch of the Islamic Movement, right here on the Temple Mount."
"The busses of the Northern Branch of the Islamic Movement are still working and they are bringing busloads of followers of Ra'ad Salah to the Temple Mount. It is as if the organization were never made illegal.
"The continuing incitement to terrorism on the Temple Mount proves there is no truth in what police are calling equality on the mount, especially with regard to searching the worshippers who visit," the organization blasted. "Many of the Islamic protesters also brought firecrackers and firebombs, and were still allowed access to the Temple Mount."
"Under Internal Security Minister Gilad Erdan, Jews are more limited to the Temple Mount than ever, but terrorists are allowed to freely celebrate on the holy site of of our former Temple. It seems that the mount has returned to the reign of terrorists from the days of MK Yitzhak Aharonovich."
Tags:Temple Mount, Gilad Erdan, Northern Islamic Movement

redpajama_set_name: RedPajamaCommonCrawl


Hilit Karsh, owner of a successful interior design office, has years of experience designing apartments, villas, penthouses, accommodations, stores, offices and lobbies in high-rise residential buildings. She's also experienced in working with leading entrepreneurs in Israel as well as real estate groups.
I design with the heart and soul, planning, creating and learning incessantly. I'm a great believer in design as a force of change in the soul. I dare to dream and don't hesitate to realize those dreams.
Studied Furniture Design at Shenkar College of Engineering and Design (2014).
Studied Interior Design at WIZO Haifa Academy of Design and Education (1999).
B.A. in Statistics from the University of Haifa (1996).
With every project I use a personal touch, based on meticulous attention to detail and the desire to create something new. The work process is based on listening, internalizing and understanding the client's wishes and tailoring the design to their needs.
May 2013 – Participated in a hotel design concept exhibition sponsored by Bait Venoy Magazine.
December 2012 – Won second place in the international competition hosted by Architecture of Israel Quarterly Magazine, Interior Design category.
June 2012 – Chosen by BVD Magazine to be in the Promising 10 Architects 10 Designers of 2012, presented in the Young and Promising Generation of Architects and Designers Exhibition of 2012 at the Tel Aviv port.
May 2011 – Participated in the sectional exhibition sponsored by Domos Magazine at the Tel Aviv Convention Center with the top 50 design and architecture offices in Israel.
March 2010 – Participated in the Israeli Home in Color exhibition sponsored by Tambour.
We begin by listening to the client, looking at what they want and examining their needs to tailor the right design for them. We will then accompany our client professionally throughout the design process. With us, you will get personalized treatment from the initial concept, to work plans all the way through to execution and eventually, styling your home.
We find solutions outside the box. We turn lemons into lemonade and find creative solutions to any design conundrum to make the end result even better.
With every project Hilit provides a personal touch, based on meticulous attention to detail and a desire to create something new.
The meticulous planning emphasizes close attention to detail and compliance with regulations and schedules.
Your personal touch is the main focus in each of our projects. We "take your hand" and accompany you throughout the process, being attentive to your needs to achieve a result that would be mutually satisfying. We give each of our clients personalized attention to fit their needs and personality.
We will combine the "ingredients" of the design – Practicality and functionality with aesthetics and beauty, while taking into account both your needs and budget. We can lower your costs with smart planning, recommending what could be ruled out without hurting the end result – As well as what you should never compromise on!
Inspiration that accentuates more for less. Architecture with clean lines and form, using minimal decorativeness. Added to them are visual perceptions from the performing arts as well as the influence of multiple artists, whose language and artistic approach Hilit integrates into her work.
Another source of inspiration is Eastern culture, particularly Japanese culture, asserting the connection between form and material further and accentuating the beauty of simplicity.


text:
import {Component, ViewEncapsulation} from 'angular2/angular2';
import {
  RouteConfig,
  ROUTER_DIRECTIVES,
  Router
} from 'angular2/router';
// import {HTTP_PROVIDERS} from 'http/http';

import {HomeCmp} from '../home/home';
import {RoomCmp} from '../rooms/rooms';
import {LoginComponent} from '../login/login'
import {NameList} from '../../services/name_list';
import {StreamList} from '../../services/StreamList';
import {VideoStream} from '../../services/VideoStream';

@Component({
  selector: 'app',
  viewProviders: [NameList, VideoStream, StreamList],
  templateUrl: './components/app/app.html',
  styleUrls: ['./components/app/app.css'],
  encapsulation: ViewEncapsulation.None,
  directives: [ROUTER_DIRECTIVES]
})

@RouteConfig([
  { path: '/', component: LoginComponent, as: 'Login' },
  { path: '/home', component: HomeCmp, as: 'Home' },
  { path: '/rooms', component: RoomCmp, as: 'Room' }
])

export class AppCmp {

  constructor(private router : Router) {
    // This route is the default, no matter where you load on the page
    // to prevent crashes and the like
    this.router.navigate(['Login']);
  }

}

redpajama_set_name: RedPajamaGithub


Shop at Adidas via Shopcred and you can save money with our Adidas deals, coupon codes and offers. Plus earn rewards on eligible purchases which are redeemable as cash and join our conversations. Shopcred is the modern place to Shop.


'Bubbles of belief will propel you to achieve'. English on the one side and arabic on the o..
'Surround yourself with love, inspiration & laughter'. English on the one side and arabic on ..
925 Sterling Silver heart pendant on 55cm adjustable leather. ..


text:
Tag: David Roper
TV Review: The Crown, Series 2
A quick whip through Series 2 of The Crown
"History is not made by those who did nothing"
Do I still love The Crown? Yes. Do I still find it a little hard to muster enthusiasm about it until I'm watching it. Absolutely. It remains lavish prestige drama that carries little excitement about it and that's perhaps inevitable as it trundles through the decades of the second half of the twentieth century, little dramatic surprise can really be sprung.
Instead, the thrills come from the script of Peter Morgan's fantasia into the emotional life of our monarch, and a production that looks like the multi-millions of dollars that have been spent on it. Oh, and the cream of British acting talent popping in for a scene or two at an astonishingly high rate. Continue reading "TV Review: The Crown, Series 2"
Review: An Inspector Calls, Garrick
"We don't live alone. We are members of one body. We are responsible for each other."
Despite being a much-lauded and much-travelled production, and a mainstay of many a GCSE English Lit exam, An Inspector Calls has completely passed me by until now, my first engagement with this play. Time and the Conways at the National was my first Priestley play earlier this year, so I was interested to see another of his plays, especially one so well known. Representing the other side of the coin was my companion for the evening, Aunty Jean a former English teacher who knew the play inside out, so we had the makings of an intriguing night at the theatre.
JB Priestley's period thriller, adapted here by Stephen Daldry, opens in 1912 with the self-satisfied Birling family celebrating the engagement of daughter Sheila to Gerald Croft. Oozing wealth and pomposity, Arthur Birling takes the opportunity to share his theories on money and success along with the glories of being on the right side of the social divide. Interrupting this cozy evening strides Inspector Goole, who informs them a young local girl has killed herself just hours before. As he quizzes them about her sacking, pregnancy and suicide, the previously composed family gradually falls apart as various revelations about their involvement with the girl come to the surface and how each of them contributed to her downfall. Continue reading "Review: An Inspector Calls, Garrick"


text:
Latino Book & Family Festivals
Virtual Festival
Partners/Exhibitors
2020 Book & Family Festival
2018 Inland Empire Festival
2018 North San Diego Festival
2017 Los Angeles Festival
2017 San Diego Festival
Great Summer Book Giveaway
Latino Book & Family Festivals July 26, 2021
SUMMER BOOK GIVEAWAY
July 31st, 2021 from 10am-4pm
Education Being in the Home welcomes the community of North County San Diego to attend their first Book Giveaway. More events are expected to come, but this event will mark the beginning of the 69th Latino Book & Family Festivals. The events are free and seek to promote awareness of college, career paths, and community resources.
For this Saturday event, teachers, students, and others in the community are welcome to come and pick up books. Up to 30 FREE books to teachers from 10 – 12 noon and up to 15 FREE books to the general public from 12 – 4 pm. Boxes will be provided, but you are invited to bring your own as well.
Education Begins in the Home
523 Edgewater Ave
Oceanside, CA 92057
Education Begins in the Home has distributed over 150,000 FREE books at more than 425 events. The organization seeks to help increase literacy in the Latino community by providing free access to books. It has helped empower over 54,000 children by giving them books and encouraging reading. With the help of volunteers, the books get sorted into categories for distribution at events or to be ready for pick up. The program director, Edward Becerra, personally volunteers to coordinate this effort because he believes that in every home, there should be a home library.
Visit www.OpenABook.org | Find us on Facebook >
Edward Becerra, EducationBeginsInTheHome@gmail.com
← 1998 – The 2nd Latino Book & Family Festival
What are the Latino Book & Family Festivals? and what to expect this 2021 →
Sign up to receive our latest Festival newsletter.
Select list(s) to subscribe to Q Latino Book & Family Festival
Yes, I would like to receive emails from Latino Book & Family Festivals. (You can unsubscribe anytime)
By submitting this form, you are consenting to receive marketing emails from: Latino 247 Media Group, 3445 Catalina Dr., Carlsbad, CA, 92010-2856, http://www.empoweringstudents.com. You can revoke your consent to receive emails at any time by using the SafeUnsubscribe® link, found at the bottom of every email. Emails are serviced by Constant Contact
Empowering Latino Futures, formerly Latino Literacy Now
A 501c3 nonprofit headquartered in Fallbrook, California and founded in 1997. Programs include The Latino Book & Family Festivals, The Int'l Latino Book Awards, The Latino Books into Movies Awards, and the Int'l Society of Latino Authors.
Empowering Latino Futures
624 Hillcrest Ln, Fallbrook CA 92028
festivals@lbff.us
© 2021	Latino Book & Family Festivals | Designed by: Theme Freesia | Powered by: WordPress


Wow! What a busy start to the month!
The children found the recent snowfall fascinating and ventured outside to make patterns in the fresh snow. 'It's snowing!" was the cry of delight from many of them.
Our yoga classes continue to take place, helping the children to build their strength and coordination, as well as reinforcing messages about the importance of exercise.
The big celebration of the month was Chinese New Year. As 2019 is the 'Year of the Pig' in the Chinese calendar, we spent time finding out about pigs and other animals. Some of the children made greetings cards. They concentrated really hard as they applied paint using lollipop sticks.


On average, most people don't know how to recognize the first signs of hearing loss because it most often occurs gradually and is an invisible issue that can be disregarded initially without realizing there is a problem.
Many times it is friends and family that notice the loss first because the person with the hearing loss mistakenly thinks other people are mumbling or not speaking clearly and doesn't realize the difficulty is with their ability to hear.
If you suspect you or a loved one may have a hearing problem, it is recommended you have your hearing evaluated by an audiologist. An audiologist is a hearing care professional specializing in identifying, diagnosing, treating and monitoring disorders of the auditory and vestibular system. They are trained to diagnose and treat hearing loss, tinnitus and/or balance problem.
If you are ready to schedule an appointment with our caring doctors and staff, call us at 423-641-0956.


text:
g-clef
Past Jury Members
Sunghoon Simon Hwang (South Korea)
Pietro Borgonovo (Italy)
Onpavee Nitisingkarin (Thailand)
Jeong Yoo-jeong (South Korea)
Jeongmin Wee (South Korea)
Hye Woon Cho (South Korea)
Ham Jae-ryeong (South Korea)
Artistic Director & President of Jury (Piano)
The concert pianist Sunghoon Simon Hwang is one of the most prominent pianists of his generation. He has appeared internationally with major orchestras such as the Prague Radio Symphony Orchestra, the Orchestre de Montbeliard, the Janáček Philharmonic, the San Remo Symphony Orchestra, the Duisburg Symphony, the Orquestra Granada, the Orquestra Nacional do Porto, the Filarmonica Marchigiana, the Filarmonica Banatul, the Orchestra Sinfonica Sicilliana, the Vina del Mar Orchestra, the Bacau Symphony, the Taiwan National Symphony Orchestra, the Orquestra Sinfonica Perfecto Garcia Chornet, the Seoul Academy Symphony Orchestra and the Busan Metropolitan Orchestra.
His performances as soloist with orchestra, as a chamber musician and piano recitals received worldwide recognition from the audience and the music press, specially after his Debut at the Carnegie Hall in New York (USA) Mr. Hwang received a great critic at the New York Concert Review Inc.
Sunghoon Simon won more than 40 international awards and 9 special interpretation prizes at prestigious piano competitions like Epinal, Jaen, Porto, Senigallia, Maria Canals, Luis Sigall, Glenn Gould, Serge Prokofiev, Sergei Rachmaninoff, Anton Rubinstein, Johannes Brahms and Bellini. He got numerous concert invitations from European and Asian music festivals like the Ferrucio Busoni festival, the Alfredo Speranza Festival, the Grosseto international piano festival, the "Klassik in der Altstadt" festival, the Chateau de Lourmarin festival, Niestetal international piano festival, Busan Maru International Music Festival(BMIMF), the Wennigsen music festival and many others.
Sunghoon Simon Hwang studied at the Korean National University of Arts in Seoul with Prof. J.P. Lim, at the Santa Cecilia Academia in Roma with Prof. Sergio Perticaroli, at the École Normal de Musique in Paris with Prof. Youngshin An, at the Hannover Musikhoschule in the classes of Prof. Einar Steen-Nökleberg and Prof. Mi-Kyung Kim.
Sunghoon Simon Hwang has taught worldwide master classes for piano and got many invitations from universities, conservatories and music summer schools, as for example at the Otto-von-Guericke University in Magdeburg, at the Kloster Michaelstein in Blankenburg with an invitation of the "Landes musik akademie Sachsen-Anhalt" in Germany. In 2010, he was invited as a Professor by the "Tenerife international concert academy festival"in Spain. From 2009 to 2013 he taught piano at the Hildesheim University in Germany.
Mr. Hwang is currently the Artistic Director of the Sviatoslav Richter international piano online competition, the Classic On international piano competition and the Glenn Gould in Memoriam piano competition in South Korea. Furthermore, he was invited as a member of Jury by numerous piano competitions such as the Maria Canals in Barcelona, the Claude Bonneton in Sete, the Rina Sala Gallo in Monza, the Tokyo international music competition, the Roma international music competition, the G.Raciti international music competition, the Terra degli Imperiali international music competition, the Euterpe international music competition.
Jury (Piano)
Born in Milan, Pietro Borgonovo distinguishes himself in conducting symphonic and operatic productions for prestigious festivals and theatres. Noteworthy is his presence at Salzburger Festpiele in the Progetto Pollini as the conductor of Klangforum Wien and the Arnold Schoenberg Chor as well as his work in the conducting the Orchestra del Maggio Musicale Fiorentinoat Maggio Musicale Fiorentino, at Ravenna Festival with the Teatro dell'Opera di Roma, at Venice Biennale. Moreover he has conducted the Orchestra del Teatro San Carlo di Napoli, the Orchestra dell'Arena di Verona, the Orchestra del Teatro La Fenice in Venice, the Orchestra del Teatro Lirico in Cagliari,the Orchestra del Teatro Verdi in Trieste, the Orchestra del Teatro Carlo Felice in Genova, the Semperoper in Dresden, theOrchestra della Toscana, the Haydn Orchestra in Bolzano, the Orchestra Sinfonica Siciliana in Palermo, the OrchestraCherubini, the Orchestra Verdi in Milano, the Nordwestdeutsche Philharmonie. and more importants Orchestras of Italy, Europe and USA.
In October 2003 he conducted the world première of "Medea" by Adriano Guarnieri at Teatro La Fenice in Venice. The opera was very successful with both audience and critics, some newspapers saying it was a "triumph". The Italian Critics Association awarded the "Abbiati 2003" prize for the "refined music and stage performance which rendered the original preciosity of the composing and multimedia concept."
A versatile musician, he is the Musical and Artistic Director of one of the oldest and most prestigious Italian concert societies, GOG (Giovine Orchestra Genovese) and the Artistic Director of the International Competition G.B. Viotti in Vercelli.
From a very early age he has excelled in the international scene as one of the best known oboe soloists. A pupil of Heinz Holliger, he plays at the major festivals and for international institutions: Salzburger Festpiele, Maggio Musicale Fiorentino, Venice Biennale, Musica Bayreuth, Festival d'Automne in Paris, Montreux Festival, Donaueschinger Musiktage, New York, Berlin, London, Vienna, Amsterdam, Monte Carlo, Prague, Bonn, Luxembourg, Moscow, Saint Peterbourg, Madrid, Lisbone.
Profoundly interested in contemporary music he also cooperates with many contemporary composers such as George Benjamin, Luciano Berio, Bruno Maderna, Azio Corghi, Franco Donatoni, Ivan Fedele, Adriano Guarnieri, Salvatore Sciarrino, Fabio Vacchi, Carlo Galante, Iannis Xenakis … He has played and conducted their works, also premières, which have often been dedicated to him.
Both as an oboe soloist and as a conductor he has many recordings produced by important international labels such as Denon, Erato, Ricordi, RCA, Col legno, BMG to his credit. He received the Grand Prix du Disque and a classification among the five most appreciated records of symphonic music from Italian critics.
Onpavee Nitisingkarin, is a native of Thailand, where she received her undergraduate degree in piano performance at College of Music, Mahidol University.
After completed her Bachelor Degree, she attended WVU for her Master's and Doctorate in piano performance under Prof.
Dr.Peter Amstutz. Her dissertation focuses on the topic of "Composers as Parents: Selected Twentieth-century Piano Works Written by Fathers for Their Children.
Dr.Nitisingkarin is a certified Orff Schulwerk teacher from Goerge Mason University and also in the DalcrozeEurhythmics Certification program at Carnegie Mellon University, Pittsburgh.
She w as awarded scholarships from the Valerie Canady Charitable Trust Foundation in 2011.
Onpavee also appears in numerous concerts as well as several piano competition venue.
She was WVU Young Artist concerto winner in 2012 and was selected to perform with WVU Symphony Orchestra.
Moreover, she was also semi- finalist in the 3rd Intersection of Classical and Jazz Piano Festival at WVU.
In February 2017, Onpavee was invited to perform the King Composition in "Concert in Remembrance of H.M.the late King Bhumibol Adulyadej" at Mater Dei School, Bangkok.
Besides being an active learner and performer, Dr.Onpavee also an experienced piano teacher.
She starts teaching piano since 2000 in Bangkok, Thailand.
Before relocated back in Bangkok, Onpavee was a senior piano instructor in WVU CMP program teaching piano, Eurhythmics (music and movement 1 and 2), Explore and Enjoy classes for toddlers.
She was also a piano faculty at Washington and Jefferson College as well as at Lincoln Park Performing Arts Center in Pennsylvania.
Onpavee also conducted Eurhythmics workshop in the 3rd Intersection of Classical and Jazz Piano Festival at WVU as well as Toddlers class to the WV Early Head Start program in 2014.
Additionally, she also served as a Vice President of West Virginia Collegiate Chapter of Music Teacher National Association in 2014 and also held the Trophy Chair position of West Virginia Music Teacher Association from 2014-2016.
In November 2017, Onpavee was invited to lead a workshop presentation on the topic of "Nurturing Musical Abilities: A Creative Movement in Piano Lessons" at the 2017 Piano Pedagogy Conference, UCSI University, Kuala Lumpur, Malaysia as well as at the SWU Piano Festival II.
Onpavee has also served as a jury member in several piano competitions and festivals.
Recently, she was a jury member for SET Youth Music Competition, Osaka International Music Competition (Bangkok regional round), the 3rd and 4th (2019) Singapore International Youth Piano Competition, and the 2nd Hong Kong International Music Competition for Young Pianists held July 2019.
Additionally, she also joined as an artist faculty at MusicFest Perugia 2018 in Italy.
Currently, she serves as an Asst.
Dean for Educational Services, Chair of Piano Department, Director of the Thailand International Piano Competition (TIPC), and a piano instructor at College of Music, Mahidol University in Bangok, Thailand.
Jury (Soprano)
(currently an Adjunct Professor at the Department of Vocal Music of Suwon University and served as an Adjunct Professor at Seoul National University)
- Graduated from Seoul National University and successfully completed the Master of Music program at Manhattan School of Music
- Currently completed the Ph.D. program in Art Management at Suwon University
- Made a debut in New York while performing for Orfeo ed Euridice, invited to the Aspen Music Festival and School, and stared at Lucia di Lammermoor, Carmen
- Starred for Uijeongbu Opera, Seoul Opera, Siheung Opera, La Pace Opera, etc.
- Performed with Seoul Philharmonic Orchestra, Seoul Philharmonic, Prime Philharmonic, Gyeonggi Provincial Orchestra, Mostly Philharmonic Orchestra, New York Christian Orchestra, and MCM Orchestra
- Won the Grand Prize in the International Music Competition and many other awards from Monthly Music, Music Journal and Baejae University Music Competition, etc.
Jury (Tenor)
* received his bachelor of Music in Vocal Arts from Seoul National University, Master of Music in Vocal Arts at University of Southern California and Doctorate of Musical Arts in Opera Performance from University of Texas at Austin
* made Disney Hall debut as a tenor soloist for Beethoven's Choral Fantasy and performed various roles in opera including Rodolfo in opera <La bohéme>, Tanzmeister in <Ariadne auf Naxos> with the Butler Opera Center and worked with Austin Lyric opera for Pagliacci Project as Beppe
* recent performances include Frantz in opera <Les contes d'Hoffmann> with Korea National Opera, Vogelsang in <Der Schauspieldirektor> with Seoul Metropolitan Opera, Baron, Inquisitor, Don Issachar, Cacambo, and Ragotski in <Candide> with Seoul Philharmonic Orchestra, Tybalt in <Roméo et Juliette>, Basilio in <Le nozze di Figaro>.
* Currently teaching at Sejong University as a Assistant Professor of Voice Department
Jury (Violin)
Hye Woon Cho, a violinist known for her expressive emotional depth and technique, graduated from Busan Arts High and completed Pre-School of the Korea National University of Arts, and graduated with distinction from the Korean National University of Arts. She then went to the United States and completed her master's degree at Mannes College, and received her D.M.A degree from Boston University.
Having been recognized for her excellent musicianship and musical interpretation through her performances with the Busan National Symphony Orchestra and the Alioshio Orchestra in Korea, she has appeared on Italian radio with Italian Radio Sinfonia and made her New York debut at Carnegie Hall by winning the Special Presentation Winner audition. The Clara Piano Quartet, of whom she is the leader, won the first prize at the Boston International Chamber Music Competition (ICMEC) and made their chamber music debut at Carnegie Hall. She performed at the 20th Century Chamber Music Festival (New School Univ.), Brahms Festival chamber music (Boston Univ.), served as guest principal of the New Jersey Philharmonic, and actively performed in New York and Boston in the U.S.A.
She has also been invited to numerous international music programs and festivals as professor including Fall Foliage Chamber Music Program (2008, U.S.), International Children's Arts and Culture Festival hosted by the Malaysian government (2011, Malaysia), International Academy of Arts, Cremona (2012 & 2014, Italy), Pyeongchang Special Music Festival (2013, Pyeongchang), Master Class, Malaga (2016, Spain), Jirisan International Music Festival (2017), International Academy of Arts Summer Festival in Saas-Fee (2017, Switzerland), Alps Music Festival (2017~2018, France),International Academy of Arts Summer Festival in Antwerp(2018, Belgium), where she has been further widening her musical spectrum through performances and masterclasses.
Violinist Hye Woon Cho continues to pursue her musical passion as current principal of Busan Chamber Music Society, and is nurturing future musicians in Busan National University, KyungSung University, Korean National Institute for the Gifted in Arts, and Art Education in Busan National University.
Jury (Clarinet)
Graduated from Sunhwa Arts School
Successfully completed the Gifted Artist Program affiliated with the St. Petersburg National Conservatory of Music, Russia
Graduated summa cum laude from Bachelor of Arts, Master of Arts and Artist Diploma Program at Mozarteum University Salzburg, Austria
Performed with Central Symphony Orchestra for 'Young Artist Concert' and Seoul Philharmonic Orchestra for 'Youth Concert'
Performed solo at Seoul Arts Center Recital Hall in 2011 and 2013
Invited to many musical performances, including Ensemble Tonkün at Sejong Art Center and Kumho Art Hall
Invited to perform for Saturday Concert at Baum Art Center,
Performed at various concerts at home and abroad, including Korea International Wind Band Festival, Peace Concert at Seoul Arts Center Concert Hall, Charity Concert for East Japan Great Earthquake in Tokyo, and the Invitation Concert at Seocho Cultural Center
Current) serving as an Associate Professor of the Department of Music and Arts at Seoul Arts College and lecturer at Seoul National University of Education and Seonhwa Arts Middle/High School, and a director of the Korean Society for Early Childhood and Elementary Music Education
Clarinettiste for Ensemble Luminasia and Ensemble MOZ


text:
BY MUNEEB YOUSUF
Publish Date: Sep 10 2018 3:19AM
Updated Date: Sep 10 2018 3:21AM
GK Photo
A beaming picture of Showkat Ahmad Dar, sitting cross-legged with a volleyball resting in his right palm, is pasted on the gate of his hometown ground at Bogund in south Kashmir's Kulgam district. The ground, locally known as nebar-bagh (out-garden), would never see Showkat absent no matter what. Such was his devotion to the game of volleyball that he would leave the court only on the occasions of lunch and tea. Sometimes, he would skip his meals––a dedication to the game that even moved the spectators. In pursuit of his dream––a dream that he both lived and died for––Showkat rose to higher standards of excellence, pushing all odds that came his way.
Showkat was no ordinary player. A strong physique, chiseled face with a height of about six feet and three inches, Showkat's big hands would practically reach anywhere in the context of the game. He was referred to as Maxim Mikhaylov of Kashmir. Popularly known as Jessi, Showkat would run three steps diagonally forward, lifting his body to a height of three feet in the air and smash the ball hard with crowd cheering "Jessi maar!" "He was an unstoppable smasher, who would hit the ball two and a half feet above the net, making the option of block literally impossible and all the opposition team could do was watch the ball come down like a huge mass,'' remarked Rayees Ahmed Dar, a longtime opposite player of Jessi.
From the sixth standard, Showkat took great interest in volleyball. He straightaway impressed physical teacher of Maroofia Public English Medium School SrandooKulgam, who happily selected him for inter-school competition. From there on Jessi earned his place for inter-district, state-level, and national level competitions. In March 2018, Jessi was selected for Indian Volleyball team selection trials. Earning a selection trial for international championship just at the age of 19 was no small feat. "Showkat was an extremely talented player, and didn't play under-16, instead he was directly promoted to play under-20 given his height and skills," said MuneerAlam Mir, the Volleyball Coach of Kashmir University. In 2016, Jammu and Kashmir team defeated Tripura in 64th Senior National Volleyball Championship with Jessi showing a class performance in that match. Jessi had huge respect towards senior players and, importantly, coaches. "He always kept his head down during our conversations and has never ever looked into my eyes," Jessi's coach recalled. "Jessi was a cool player, who always entered the court with a winning attitude, and never teased any player of opposition team," a senior player mentioned. "Before beginning the match, Jessi would thrice touch his open hands to the surface of the court," Zubair Ahmed Ganie said, who was part of champions of JK Police Volleyball tournament 2016. Jessi had a hallmark of celebrating victory by lifting his left arm in air.
On 5th of August 2018, Showkat left his home for a volleyball match against Punjab Club at HarnagAnantnag. Before leaving for Harnag, Jessi had a light lunch with couple of spoons of curd. Given to a call for a shutdown Jessi waited for about hour and a half, finally getting lift by a motorcyclist. During the match, Showkat single-handedly gave a tough competition to Punjab team. In the pen-ultimate set, Showkat smashed the winning shot and the crowd went ecstatic lifting Jessi on their shoulders. According to many spectators, while Jessi was celebrating the victory of a set that brought both teams on equal score, he felt an ache in his chest. With no medical facilities available on the court, Jessi collapsed and died on the way to hospital. There was no C.P.R technique applied to Jessi, while he was indicating pain in his heart, perhaps, this could have saved his life. It seems that no one in the ground was known to this emergency procedure.
An avid listener of Punjabi singer, Jassi Gill, Showkat got the title Jessi from him. He had a fascination for Asics brand, and in most matches, Jessi was seen wearing shorts of the same brand. "Showkat had little interest towards material things be it a car, property, house," mentioned his elder brother, Aijaz Dar. He had no interest in family issues and always kept himself busy with sports. After returning in the evening, Showkat would again busy himself by playing volleyball with kids in a rectangle shaped lawn of their home. Before going to sleep, Showkat would play volleyball against the walls of his room. "Though we accompanied Jessi on many places where he was playing, but we never watched him playing, because we thought doing so would make him nervous," his brother mentioned.
Despite being an ace volleyball player, Showkat never felt proud of his position and always happily greeted people and especially shared love with children. In any corner of the valley, where Showkat went for a volleyball match, huge crowds would follow up to watch Jessi's powerful smashes. Apart from being an ace volleyball player, Jessi was kind at heart. Jessi's friend recounted one of the incidents. Last year in December, he and Jessi went for a dinner in Jammu. According to him, Jessi had just one hundred and thirty-seven rupees in his pocket. Before entering the hotel, a lady having couple of children asked for money. Jessi graciously gave a hundred rupee note to her. The friend asked, "Jessi what are you doing? And how would we eat now?" "Where would that lady go, she had nothing, we can at least lend from hotel owner and pay him tomorrow," Jessi replied to him.
Ghulam Mohammad Dar, Showkat's father, sitting lonely in the second storey of an old house where Jessi used to sleep, recalls how he accompanied his son who was 13 then, to Jammu for a volleyball match. Though deeply painful about the separation of his young child, Dar is contented with the output that his son returned to him in terms of how common people expressed several good qualities of Showkat. Taking a deep breath, Dar said, "It was a sacred mission, and we fully supported him in this endeavor." "The only barrier that came in this mission was Allah's decision of taking Showkat back to him and his death again reinforced that no matter how much young you are you could be taken back anytime," Dar said with moist eyes.
Unflinching support from family and vital guidance by coaches made Jessi a star who brought laurels to himself, his family and the place to which he belonged. A proud son of the soil, who died on the volleyball court, for the cause that he was deeply passionate about, and we must salute him. The Volleyball Federation of India issued condolence letter on the death of this young star. Terming Jessi's death as a major setback for the federation, the letter read that VFI had lost a great volleyball player from Kashmir Valley. Surprisingly, no reward has been granted to the family of Jessi. The unavailability of medical and other facilities again stands reminder to the fact of how sports authorities take least concern about the safety of sportsmen.
A strange silence has gripped the entire village. Jessi's fans and other volleyball players don't believe that their hero and a player that they looked up to has departed from this world. The nebar-bagh looks deserted, mourning the death of a star who religiously visited it. Since then, no one has mustered the courage to come inside the premises of this ground and bring back the action.
(Muneeb Yousuf is a Ph.D. Candidate at MMAJ Academy of International Studies, Jamia Millia Islamia New Delhi)


text:
Wednesday, Oct 12 2016
Detention Facility Looks To Overcome History Of Medical Negligence
KPCC takes a look at both the troubled history of the Adelanto Detention Facility and the changes its made in the past few years.
KPCC: Have Changes At Adelanto Immigrant Detention Center Led To Better Health Care?
ICE has relied increasingly on private contractors to hold a detainee population that has exploded over the past two decades as immigration rules have tightened. ICE officials say they don't have the capacity to handle so many detainees themselves. In recent months, private facilities have drawn increasing attention, first from California state lawmakers, and more recently the federal Department of Homeland Security, which holds roughly 33,000 immigrants in detention on a daily basis. There have been a series of problems over the years in ICE detention centers, including lawsuits tied to issues like overcrowding and medical negligence. (Berenstein Rojas, 10/12)
KPCC: Inside The Adelanto Detention Facility: Troubled History, Vows For Reform
California's largest privately run immigrant detention facility – and a key holding site for immigrants from Southern California – has recorded lapses in care during a four-year period when at least three detainees have died, according to documents obtained by KPCC and interviews with former detainees and their families... Upon arriving in Adelanto, [Carlos] Hidalgo said he faced multiple challenges, including access to legal materials, long wait times for medical care and the psychological stress of being far away from his children and parents. He also said the food was of poor quality and left him constantly hungry. (Merina, 10/11)


Web design means two things in practice: how your site works and the personality it conveys. Is it easy for your visitors to find what they're looking for? Do they have easy access to everything you want them to do, such as buy a product or register for a newsletter? Moreover, is it easy for your team to manage? The easier it is for them and you to use, the better the outcome. That's what makes design effective, not just pretty.
Our award-winning team asks all these questions and more when approaching the visual style that best reflects your organization's message and personality. Each design starts with our Creative Brief, refined over many years, which outlines whom you admire online, ranks your core audiences, details your major conversion points, and describes the overall tone. For many projects, wireframe sketches also help us to identify the important information that needs to be on each page ahead of time so that we have a solid foundation from which our team will design to bring your vision to life.
Any designer must be familiar with the medium: screens are not just electronic versions of paper sheets. Since our designers all have experience as web developers, it enables them not only to anticipate how their designs will function on the web — and consider how special effects or techniques will improve user delight — but also how the experience should scale to different device sizes. Design that works as well for the viewer on a mobile phone as on a laptop is a smart way to show your visitor you care about her experience.
Effective design is more than great typography, engaging photos, subtle textures & drop shadows; it's about how well your site helps a visitor find the information she's looking for. That's what makes NMC designs so effective: check out our portfolio for more examples or get in touch today and see for yourself.


Last day to file a protest form in the clerk's office on your property valuation. Last day to file a homestead exemption application.
Courthouse Closing at Noon for our former Veteran's Service Officer Alfred Schipporeit's Funeral.
Driver's Examiner 8:30 A.M. - 1:00 P.M.