text:
\section*{Supplementary materials}
The supplementary code is available at \url{https://github.com/huogy/CaEGCN}.

\section{Introduction}
\IEEEPARstart{C}lustering is an essential topic in data mining area, which divides a collection of objects into multiple clusters of similar objects.
Inspired by the powerful feature extraction capability of deep convolutional network, many deep learning based clustering methods have been proposed in recent years, demonstrating much significant progress in clustering research \cite{hinton2006reducing,xie2016unsupervised,ji2017deep,guo2017improved}. The two-step spectral clustering method is usually employed here: A `good' data representation or similarity matrix which is learned  from these deep learning methods can be pipelined to the downstream models/algorithms such as K-means \cite{macqueen1967some} or Normalized Cut \cite{tabatabaei2012ganc} to obtain the final clustering result.

However, existing deep clustering methods only focus on the data content, and usually ignore the relationship between the data, i.e., the structural information.
With the development of the data collection and analysis technologies, people not only collect the data but also obtain or build the relationship between the data in the form of graphs, such as social networks \cite{girvan2002community}, biochemical structure networks \cite{velickovic2018graph} and railway networks \cite{zhang2020multi}.
These graphs can help people make better data-driven decisions.
Therefore, how to embed the relationship between the data into deep clustering becomes a thorny problem.

Furthermore, based on these raw graphs, one wishes to mine the latent relationship between the data effectively.
As we know, the edges in a graph represent the explicit relationship, which is also regarded as the first-order structural relationship.
Many graph embedding methods exploit such relationship, including DeepWalk \cite{perozzi2014deepwalk}, node2vec \cite{grover2016node2vec}, and LINE \cite{tang2015line}.
But, the data relationship in the real-world is complicated.
There still exist many implicit and complicated relationships.
For example, two nodes may  not be directly connected in a graph, while they have many identical neighbors. However it is natural belief that these two nodes have a high-order structural relationship.

In order to improve the clustering effect of deep clustering methods, utilizing the high-order relationships is necessary.
As an important approach in deep learning methods, Graph Convolutional Network (GCN) \cite{defferrard2016convolutional,bruna2014spectral,velickovic2018graph} can mine such potential high-order relationship between the data. GCN transfers the graph structured data to a low-dimensional, compact, and continuous feature space. While GCN simultaneously has very successful application in encoding and exploring graph structure and node content, it seems little attention has been given to applying GCNs to deep clustering tasks.

For the purpose of clustering, we can naturally construct an auto-encoder module based on a GCN, the so-called GAE module. GCN can lead the signals to be smoother, which is the inherent advantage of GCN.  However, such signal smoothing operation makes the signals more similar, losing the diversity of signals. It has been proven that GCN is prone to over-smoothing when the number of layers becomes large \cite{kipf2017semi}, which results in a poor performance in related tasks.
So, GCN cannot be stacked as deeply as the CNN model in visual tasks.

To overcome this drawback of GCN, we introduce a common auto-encoder network to supplement the data content information to GAE, like the effect of the residual network.
Multiple layers are usually stacked in the deep network, and each layer captures different latent features of the data.
To combine the high-order relationship of the data (in GAE) with the potential details of the corresponding content information (in auto-encoder) layer-by-layer, we propose a cross-attention fusion mechanism, which highlights the discriminative information for clustering tasks.
Different from the traditional attention mechanism, our cross-attention fusion mechanism fuses two kinds of heterogeneous representations, i.e., the regular data and the irregular graph.

In this paper, we propose a novel clustering framework, named Cross-Attention Fusion based Enhanced Graph Convolutional Network (CaEGCN).
In CaEGCN, we can extract the high-order relationship between the data through the Graph Convolutional Auto-encoder module (GAE).
To alleviate the over-smoothing problem of GAE and supplement the content information to GAE, we build a Content Auto-encoder module (CAE) composed of a common auto-encoder, which extracts the content information of the data.
Besides, we propose a cross-attention fusion mechanism to encode above two modules to output a complete representation.
In order to guide the optimal clustering direction of the entire model in an end-to-end manner, we introduce a self-supervised module.

The contributions of this paper are listed as follows as a summary,
\begin{itemize}
    \item We propose an end-to-end cross-attention fusion based deep clustering framework, in which the cross-attention fusion module creatively concatenates the graph convolutional auto-encoder module and content auto-encoder module in multiple layers;

    \item We propose a cross-attention fusion module to
    assign the attention weights to the fused heterogeneous representation;
    \item In the graph convolutional auto-encoder module, we propose simultaneously reconstructing the content and relationship between the data, which effectively strengthens the clustering performance of CaEGCN;

    \item We test CaEGCN on the natural language, human behavior and image datasets to prove the robustness of CaEGCN.
\end{itemize}

The rest of the paper is organized as follows. In Section \ref{Sec:2}, we briefly review the graph convolutional network, deep clustering and attention mechanism, respectively. In Section \ref{Sec:3}, we detail the cross-attention fusion based enhanced graph convolutional network for clustering by presenting the four main modules. In Section \ref{Sec:4}, the proposed method is evaluated on clustering problems with several public datasets. Finally, the conclusion and the future work are discussed in Section \ref{Sec:5}.

\begin{figure*}[htbp]
    \centering
    \includegraphics[width=0.8\textwidth]{overall6.png}
    \caption{The conceptual framework of CaEGCN, which includes four modules: content auto-encoder module, graph convolutional auto-encoder module, cross-attention fusion module and self-supervised module.
    $X$ is the original data, $\hat{X}$ is the reconstructed data, and $A$ is the original graph. $H_l$ and $Z_l$ represent the $l$-th layer output of CAE and GAE, respectively.
    $R_l$ is the cross-attention fused representation of $H_l$ and $Z_l$.
    $\mathcal L_{{CAE}^{content}}$ is the content reconstruction loss of CAE. $\mathcal L_{{GAE}^{content}}$ and $\mathcal L_{{GAE}^{graph}}$ are the content reconstruction loss and graph reconstruction loss of GAE, respectively.
    $\mathcal L_{cae}$ and $\mathcal L_{gae}$ form the self-supervised module losses.}
    \label{overall}
\end{figure*}

\section{Related Work}\label{Sec:2}

In this section, we review the necessary knowledge related to the research of this paper, which are \emph {graph convolutional network, deep clustering and attention mechanism.}

\subsection{Graph Convolutional Network (GCN)}
Many research fields consider certain natural graph structures, such as the traffic road network \cite{zhao2020t}, the human skeleton points \cite{yan2018spatial} and molecules structures in biology \cite{sanyal2020proteingcn}.
Graph is a kind of irregular structural data, which is dispersive and disorderly. To cope with such irregular data, a lot of GCN based methods have been proposed.
These methods can be divided into two main categories: spectral-based GCN methods \cite{defferrard2016convolutional,kipf2017semi,bruna2014spectral} and spatial-based GCN methods \cite{velickovic2018graph,hamilton2017inductive}.
In a GCN, nodes can be assigned to the features of data, and the edge weight information describes the similarity between nodes, which shows that graph has a strong information organization ability.

The spectral-based GCN methods exploit the spectrum representation of a graph.
Kipf \emph{et al.} \cite{kipf2017semi} initially proposed the graph convolutional networks for prediction tasks, which simulates the graph convolutional operation through the local first-order approximation of spectral convolutions.
Wang \emph{et al.} \cite{wang2018graphgan} introduced the generative adversarial mechanism into the learning of graph representation, and developed a new graph softmax function utilizing the latent structure information of the data.

The spatial-based GCN methods directly define the operations on the graph and extract the information from the spatial neighbor groups.
Velickovic \emph{et al.} \cite{velickovic2018graph} proposed a graph attention network, which computes the corresponding hidden information for each node and
uses the attention mechanism to weight the importance of each node compared with its neighbors.
More comprehensive reviews about GCN can be found in \cite{wu2020a}.

\subsection{Deep Clustering}
The current deep learning researches mainly concentrate on the supervised learning tasks.
How to extend it onto a framework for unsupervised clustering is a meaningful problem.
Fortunately, some researchers have conducted the related works.
Xie \emph{et al.} \cite{xie2016unsupervised} proposed a deep embedded clustering method, which exploits the deep learning to learn the feature representations and the  cluster assignments of the data.
Ji \emph{et al.} \cite{ji2017deep} constructed a self-expression layer between the encoder and decoder of the auto-encoder.

With the development of multi-view clustering, more and more researchers introduce the relationship between the data to enhance the clustering performance.
Kipf \emph{et al.} \cite{kipf2016variational} used the graph convolutional encoder and an inner product decoder to build a Variational Graph Auto-encoder (VGAE), which learns the latent features of undirected graphs for clustering.
Pan \emph{et al.} \cite{pan2020learning} improved the VGAE framework and introduced an adversarial regularization rule to optimize the learned representation for clustering.
Wang \emph{et al.} \cite{wang2019attributed} employed the graph attention network to weight the importance of neighboring nodes, and obtained a more accurate representation of each node.
Li \emph{et al.} \cite{li2020embedding} jointed the advantages of K-means and spectral clustering, and embedded it into the graph auto-encoder to generate the better data representations.
Bo \emph{et al.} \cite{bo2020structural} transferred the representation learned by the auto-encoder to the corresponding GCN, and proposed a dual self-supervision mechanism to unify these two different deep neural architectures, which is an important baseline in this paper.

In above, GCN-based clustering methods update network parameters by reconstructing the adjacency matrix and sufficiently exploit the structure information, but they ignore the node information and the over-smoothing problem.

\subsection{Attention Mechanism}
Recently, in the fields of machine translation \cite{vaswani2017attention,tang2018why}, semantic segmentation \cite{fu2019dual} and image generation \cite{xu2015show}, the attention mechanism has become a trick module that improves the effectiveness of most models.
Self-attention is a variant of the traditional attention mechanism.
Vaswani \emph{et al.} \cite{vaswani2017attention} proposed the self-attention mechanism in machine translation applications, which obtains the satisfactory experimental results.
Besides, self-attention mechanism is robust and easily embedded into recurrent neural networks \cite{cheng2016long}, generative adversarial networks \cite{zhang2019self} and other neural networks, which also achieves the excellent experimental results.

Many scholars continuously optimize the self-attention mechanism.
Wang \emph{et al.} \cite{wang2019self} introduced a dependency tree into the self-attention mechanism to represent the relationship between words. Yu \emph{et al.}
\cite{yu2018qanet} extracted the local information through the convolution model to complement the global interaction of the self-attention mechanism. Xue \emph{et al.} \cite{xue2019danet} used the self-attention mechanism in image segmentation, which better achieves the accurate segmentation through the long-range context relations

To simultaneously handle the heterogeneous data in the proposed model, i.e., the regular data and the irregular graph, we propose the cross-attention fusion module in this paper.

\section{Cross-Attention Fusion based Enhanced Graph Convolutional Network}\label{Sec:3}

In this section, we present a novel cross-attention fusion based enhanced graph convolutional network model, which sufficiently integrates the content information and the relationship between the data in a multi-level adaptive manner to improve the clustering performance.

The overall network architecture is shown in Figure \ref{overall}, consisting of four main modules: an auto-encoder module for extracting the content information; a GCN based auto-encoder module for exploiting the relationship between the data; a cross-attention module for concatenating above two modules
 where the multi-level adaptive fusing strategy supplements the effective content information as much as possible during the transmission process; and a self-supervised module used to constrain the consistency of the distributions of middle layer representations.

\subsection{Constructing the Graph}
Before presenting the proposed CaEGCN model, we first construct the necessary graph of raw data.
Given a set of data $X\in\mathbb{R}^{D \times N}$ containing $N$ samples and the $i$-th sample $x_i\in \mathbb R^D$,
we employ the commonly-used $K$-nearest neighbor (KNN) to construct the corresponding graph to exhibit its structure information.

For the image data, we calculate the similarity between samples using the heat kernel method as  \cite{grigoryan2012heat},
\begin{equation}\label{Heat Kernel}
\begin{aligned}
    S_{ij}=e^{-\frac{{\|x_i- x_j\|_2^2}}{t}},
\end{aligned}
\end{equation}
where $t$ represents the variance scale parameter.

As for the natural language data, the inner-product method is chosen to measure the similarity between samples as follow,
\begin{equation}
    \label{Dot-product}
    S_{ij}=  x_j^T  x_i.
\end{equation}

Then, with the above calculated similarities among all samples, we pick up the $K$ highest correlation neighbors of each sample and connect them; so, a graph $A$ is obtained. In many applications, the graph information actually comes with the given dataset $X$.

\subsection{Content Auto-encoder Module (CAE)}
As we know, deep convolutional network can effectively extract the critical features from the complex data. Auto-encoder can reconstruct the samples and reduce the missing information during the learning procedure, which is naturally proper for unsupervised learning.
To extract the content information in the data, we first train a deep convolutional network based auto-encoder module, which is named as Content Auto-encoder Module (CAE).

We represent the input of the $l$-th layer as $H_{l-1}$, then its output $H_l$ can be obtained by,
\begin{equation}
    \label{hidden_layer}
    \begin{aligned}
    &H_l=a_l(U_lH_{l-1}+b_l),\\
    &l=1,2,\cdots, L,
    \end{aligned}
\end{equation}
where the activation function of the $l$-th layer, $a_l$, can be chosen according to the practical applications, such as ReLU or Sigmoid.
$U_l$ and $b_l$ denote the weight and bias of the $l$-th layer of CAE, respectively.
In addition, the input in the first layer of CAE is the raw data $X$, i.e., $H_0= X$.
The  output in the final layer reconstructs the raw data, i.e., $\hat{X}=H_{L}$, and the final loss function of CAE can be defined as,
\begin{equation}
    \label{L_content}
    \begin{aligned}
    \mathcal L_{CAE^{content}}=\frac{1}{2}||X-\hat{X}||_F^2,
    \end{aligned}
\end{equation}
where $\|\cdot\|_F^2$ denotes the Frobenius norm.

\subsection{Cross-Attention Fusion Module}
As shown in Figure \ref{overall}, CAE extracts the content information in the data, and GAE exploits the corresponding relationship between the data. How to fuse these two kinds of information for clustering tasks is a key problem.

Cross-attention fusion mechanism has the global learning ability and good parallelism, which can further highlight the critical information in the fusion representations while suppressing the useless noise.
Therefore, we use the cross-attention fusion mechanism to integrate the content information learned by CAE and the data relationship learned by GAE in a multi-level adaptive manner, which is the so-called Cross-Attention Fusion Module.

We define the cross-attention fusion mechanism as,
\begin{equation}\label{CrossAttention}
    \begin{aligned}
     R = F_{att}(Q,K,V),
    \end{aligned}
\end{equation}
where the query $Q=W^qY$, the key $K=W^kY$ and the value $V=W^vY$. The raw fusion representation $Y$ is the input of the cross-attention fusion module, which is defined as,
\begin{equation}\label{Y}
    \begin{aligned}
     Y=\gamma Z_l+(1-\gamma)H_l
    \end{aligned}
\end{equation}
where $H_l$ is the output of $l$-th layer in CAE and $Z_l$ is the output of corresponding layer in GAE. $\gamma$ is a trade-off parameter, which is set to $0.5$ in our experiments.

To discover the latent relationship between data and generalize the cross-attention fusion mechanism \eqref{CrossAttention}, we firstly calculate the similarity $s_{ab}$ between the fusion query $q_a$ and the fusion key $k_b$,
\begin{equation}
    \label{s}
    \begin{aligned}
       s_{ab}=q_a*k_b,
    \end{aligned}
\end{equation}
where $q_a$ and $k_b$ denote the $a$-th and $b$-th vectors in $Q$ and $K$, respectively.

Then, we execute the softmax normalization on above $s_{ab}$ to obtain the relevance weight $\alpha_{ab}$ as follows,
\begin{equation}
    \label{alpha}
    \begin{aligned}
    \alpha_{ab}= \text{softmax}(s_{ab}) =\frac{\exp(s_{ab})}{\sum^{D_{att}}_{a=0}\exp(s_{ab})}.
    \end{aligned}
\end{equation}

Finally, the output of the cross-attention fusion mechanism $R=(r_1, r_2, r_3, \cdots, r_N)$, i.e., the fusion representation of the data content information and the relationship between data, can be written as,
\begin{equation}
    \label{C}
    \begin{aligned}
    r_a=\sum_{b=0}^N\alpha_{ab}v_b.
    \end{aligned}
\end{equation}

To further perceive different aspects of the data, multi-head mechanism is also introduced, which contains multiple parallel cross-attention fusion modules. Specifically, we repeatedly project the query $Q$, key $K$ and value $V$ to obtain $M$ parallel cross-attention modules.
Each cross-attention fusion module is regarded as one head, and each head has the different weight matrices $\{W^q_m \in \mathbb{R}^{N\times D_l}$, $W^k_m\in \mathbb{R}^{N\times D_l}$, $W^v_u\in \mathbb{R}^{N\times D_l}\}$ to linearly transform the fusion features $Q_m=W^q_mQ$, $K_m=W^k_mK$, $V_m=W^v_mV$ where $D_l$ is the dimensionality of the $l$-th layer. The $m$-th head is,
\begin{equation}
    \label{head}
    \begin{aligned}
    R^m = F_{att}(Q_m,K_m,V_m), m=1,2,3,\cdots, M.
    \end{aligned}
\end{equation}

We concatenate the outputs of all $M$ heads, and multiply the weight matrix $W\in  \mathbb{R}^{N \times(M\times D_l)} $ to get the final cross-attention fusion representation,
\begin{equation}
    \label{multihead}
    \begin{aligned}
    R = W \cdot Concat(R^1, \cdots, R^M),
    \end{aligned}
\end{equation}
where $Concat(\cdot)$ denotes the matrix concatenate operation. This is the so-called multi-head mechanism and cross-attention fusion module.

\subsection{Graph Convolutional Auto-Encoder Module (GAE)}
As we mentioned before, the relationship between the data can effectively improve the clustering performance. Most deep clustering methods only consider the content information of data, while ignoring the important data relationship \cite{xie2016unsupervised}.
Fortunately, Graph Convolutional Network (GCN) \cite{hamilton2017inductive} is able to handle such relationships and the content information of data collaboratively.
To exploit GCN in unsupervised clustering tasks, we propose a GCN based Auto-Encoder module (GAE), which creatively reconstructs both graph and content information.

The previous cross-attention mechanism module combines the content representation $H_l$ from CAE with the relationship representation $Z_l$
from GAE to output a fusion representation $R_l$ in different layers. Then, the GAE executes the spectral graph convolution on such $R_l$ to learn the high-order discriminative information based on the adjacency matrix $A$. Finally, the middle layer $Z_{\frac{L}{2}}$ is used for clustering.

The convolution operation in each GAE layer can be expressed as follow,
\begin{equation}
    \label{GCN}
    Z_l=GAE(R_{l-1},A)=a_l(\hat{D}^{-\frac{1}{2}}\hat{A}\hat{D}^{-\frac{1}{2}}R_{l-1}U_{l}),
\end{equation}
where $\hat{D}^{-\frac{1}{2}}\hat{A}\hat{D}^{-\frac{1}{2}}$ is the approximated graph convolutional filter and $\hat{D}$ is the degree matrix of $\hat{A}$, where $\hat{D}_{ii}=\sum_{j}\hat{A}_{ij}$. With the identity matrix $I$ and the adjacency matrix $A$, we use $\hat{A} = A+I$ to ensure the self-loop in each node. Additionally, $U_{l}$ denotes the weight of the $l$-th layer, and
$Z_l$ is the output of the $l$-th GAE layer.

It should be noted that the input of the first layer in GAE is slightly different. The first layer just uses the raw data $X$ instead of $R_0$ as input,
\begin{equation}
    \label{fristGCN}
    Z_1=GAE(X,A)=a_l(\hat{D}^{-\frac{1}{2}}\hat{A}\hat{D}^{-\frac{1}{2}}X U_{1}).
\end{equation}

After this multi-layer learning, the GAE encoder encodes both the raw relationship $A$ and the content $X$ into a useful representation $Z_{\frac{L}{2}}$.
In order to preserve more information, we set the graph reconstruction and content reconstruction errors as the loss functions of GAE.

\vspace{1ex}
\textbf{i) Graph Reconstruction Loss.} We choose a simple inner product operation to reconstruct the relationship between samples as \cite{kipf2016variational},
\begin{equation}
    \label{inner_product}
    \tilde{A}=Sigmoid(Z_L^TZ_L),
\end{equation}
where $Z_L$ is the output of the last GAE layer, and $\tilde{A}$ is the reconstructed adjacency matrix.
The loss of graph reconstruction can be defined as,
\begin{equation}
    \label{graphloss}
    \mathcal L_{GAE^{graph}}=||A-\tilde{A}||_F^2.
\end{equation}

By minimizing the error between $A$ and $\tilde{A}$, the GAE module may preserve more data relationship in the latent representation $Z_{\frac{L}{2}}$ to improve the clustering performance.

\vspace{1ex}
\textbf{ii) Content Reconstruction Loss.}
Except for the relationship between the data, we also constrain the GAE module to preserve enough content information, which has much difference with formula \eqref{L_content};
so, we creatively define its loss function as,
\begin{equation}
    \label{contentloss}
   \mathcal L_{GAE^{content}}=||X-Z_L||_F^2,
\end{equation}
where $Z_L$ is the output of the last layer in GAE, which has the same size with the raw data $X$.
In this way, the GAE encodes both the relationship and the content of samples into a discriminative representation for clustering.

\subsection{Self-Supervised Module}
It is difficult to judge whether the learned representation $Z_{\frac{L}{2}}$ is optimally for clustering during the optimization procedure. We need to give an optimization target about clustering.

To solve this problem, we firstly get a set of initial cluster centers $\{\beta_c\}_{c=1}^C$ by performing K-means on $H_{\frac{L}{2}}$, where $C$ is the number of clusters. These cluster centers guide approximately the optimization direction for $Z_{\frac{L}{2}}$, which is the so-called Self-Supervised Module.

We use the Student's $t$-distribution \cite{maaten2008visualizing} to calculate the similarity between the middle layer representation $H_{\frac{L}{2}}$ and the cluster centers $\beta_c$ as follow,
\begin{equation}
    \label{T}
    t_{ic}=\frac{(1+||h_i-\beta_c||^2)^{-1}}{\sum_{c=1}^{C}(1+||h_i-\beta_{c}||^2)^{-1}}
\end{equation}
where $h_i$ is the $i$-th sample representation of $H_{\frac{L}{2}}$.
And $t_{ic}$ measures the probability that the $i$-th sample is assigned to the $c$-th cluster,
so $T=[t_{ic}]$ is the overall soft assignment distribution.

Furthermore, the choice of target distribution directly determines the clustering quality. We believe that the high-confidence assignments in $T$ is reliable and can be used as the target distribution.
We raise $p_{ic}$ to highlight the role of high-confidence distribution,
\begin{equation}
    \label{P}
    p_{i c}=\frac{{t_{i c}^2}/f_c}{\sum_{c=1}^C(t_{ i c}^2/f_{c})}
\end{equation}
where $f_c=\sum_i t_{i c}$ is the soft cluster frequency.
The distribution of $T$ and $P$ should be close to each other as follow,
\begin{equation}
    \label{KL_cae}
   \mathcal L_{cae} = \text{KL}(P||T) = \sum_i\sum_c p_{i c}\log\frac{p_{i c}}{t_{i c}}.
\end{equation}

Similarly, it is easy to construct a soft assignment distribution $Z$ for the representation $Z_{\frac{L}{2}}$, then we can use the target distribution $P$ to supervise the distribution $Z$ as,
\begin{equation}
    \label{KL_gae}
   \mathcal L_{gae} = \text{KL}(P||Z) = \sum_i\sum_c p_{i c}\log\frac{p_{i c}}{z_{i c}}.
\end{equation}
Now, the optimization goals of GAE and CAE are unified into a distribution $P$, which makes the learned representation more suitable for clustering tasks.

\subsection{Overall Loss Function}
The overall objective loss function of Cross-Attention Fusion based Enhanced Graph Convolutional Network (CaEGCN) can be summarized as,
\begin{equation}
\begin{aligned}
    \mathcal{L}_{overall} &=\mathcal L_{GAE^{graph}}+\mathcal L_{GAE^{content}}\\
    &+ \mathcal L_{CAE^{content}}+\mathcal L_{cae}+\mathcal L_{gae}.
\end{aligned}
\end{equation}

There are five items in the above objective function, including three reconstruction losses and two self-supervision losses,
which optimizes the data representations for clustering tasks from different perspectives.

After optimizing the above objective function, the local optimal representation $G_{\frac{L}{2}}$ is obtained. Then, we perform the softmax operation on $G_{\frac{L}{2}}$ to get the final clustering results, i.e.,
$\text{max}(\text{softmax}(G_{\frac{L}{2}}))$.

\section{Experimental}\label{Sec:4}
In this section, CaEGAN is evaluated on various type of public datasets, including natural language, human behavior and image datasets. We present the experimental settings and analysis below.

\begin{table*}
\centering
\resizebox{!}{55mm}{
\begin{tabular}{c|c|cccccccc}
\hline
Dataset  & Metric                                                       & \multicolumn{1}{l}{K-means}                                               & AE                                                                                & IEDC                                                                                & VGAE                                                                      & ARGA                                                                      & DAEGC                                                                     & SDCN                                                                                & CaEGCN                                                                             \\ \hline
    ACM      & \begin{tabular}[c]{@{}c@{}}ACC\\ NMI\\ ARI\\ F1\end{tabular} & \begin{tabular}[c]{@{}c@{}}0.6820\\ 0.3263\\ 0.3119\\ 0.6846\end{tabular} & \begin{tabular}[c]{@{}c@{}}0.8278\\ 0.5020\\ 0.5553\\ 0.8295\end{tabular}                     & \begin{tabular}[c]{@{}c@{}}0.8645\\ 0.5824\\ 0.6421\\ 0.8632\end{tabular}                     & \begin{tabular}[c]{@{}l@{}}0.8294\\ 0.5285\\ 0.5618\\ 0.8286\end{tabular} & \begin{tabular}[c]{@{}l@{}}0.8327\\ 0.5039\\ 0.5646\\ 0.8335\end{tabular} & \begin{tabular}[c]{@{}l@{}}0.8694\\ 0.5618\\ 0.5935\\ 0.8707\end{tabular} & \begin{tabular}[c]{@{}c@{}}\underline{0.8860}\\ \underline{0.6326}\\ \underline{0.6931}\\ \underline{0.8857}\end{tabular}                     & \textbf{\begin{tabular}[c]{@{}c@{}}0.9012\\ 0.6703\\ 0.7300\\ 0.9009\end{tabular}} \\ \hline
DBLP     & \begin{tabular}[c]{@{}c@{}}ACC\\ NMI\\ ARI\\ F1\end{tabular} & \begin{tabular}[c]{@{}c@{}}0.3646\\ 0.0886\\ 0.0657\\ 0.2637\end{tabular} & \begin{tabular}[c]{@{}c@{}}0.5435\\ 0.2220\\ 0.1651\\ 0.5325\end{tabular}                     & \begin{tabular}[c]{@{}c@{}}0.6571\\ 0.3080\\ 0.3210\\ 0.6439\end{tabular}                     & \begin{tabular}[c]{@{}l@{}}0.5763\\ 0.2189\\ 0.2348\\ 0.5456\end{tabular} & \begin{tabular}[c]{@{}l@{}}0.5450\\ 0.2019\\ 0.1949\\ 0.5343\end{tabular} & \begin{tabular}[c]{@{}l@{}}0.6205\\ 0.3249\\ 0.2103\\ 0.6175\end{tabular} &
\begin{tabular}[c]{@{}c@{}} \underline{0.6613}\\ \underline{0.3249}\\ \underline{0.3338}\\ \underline{0.6556}\end{tabular}                     & \textbf{\begin{tabular}[c]{@{}c@{}}0.6823\\ 0.3388\\ 0.3617\\ 0.6669\end{tabular}} \\ \hline
Citeseer & \begin{tabular}[c]{@{}c@{}}ACC\\ NMI\\ ARI\\ F1\end{tabular} & \begin{tabular}[c]{@{}c@{}}0.3384\\ 0.1502\\ 0.0893\\ 0.2246\end{tabular} & \begin{tabular}[c]{@{}c@{}}0.5909\\ 0.3066\\ 0.3134\\ 0.5483\end{tabular}                     & \begin{tabular}[c]{@{}c@{}}0.6023\\ 0.3074\\ 0.2924\\ 0.5230\end{tabular}                     & \begin{tabular}[c]{@{}l@{}}0.5161\\ 0.2572\\ 0.2405\\ 0.4184\end{tabular} & \begin{tabular}[c]{@{}l@{}}0.5912\\ 0.3069\\ 0.3138\\ 0.5485\end{tabular} & \begin{tabular}[c]{@{}l@{}}\underline{0.6454}\\ \underline{0.3641}\\ \underline{0.3778}\\ \textbf{0.6220}\end{tabular} &
\begin{tabular}[c]{@{}c@{}} 0.6222\\ 0.3601\\0.3623\\ 0.5893\end{tabular}                     & \begin{tabular}[c]{@{}c@{}}\textbf{0.6802}\\ \textbf{0.4000}\\ \textbf{0.4240}\\ \underline{0.6138}\end{tabular} \\ \hline
HHAR     & \begin{tabular}[c]{@{}c@{}}ACC\\ NMI\\ ARI\\ F1\end{tabular} & \begin{tabular}[c]{@{}c@{}}0.5998\\ 0.5887\\ 0.4609\\ 0.5833\end{tabular} & \multicolumn{1}{l}{\begin{tabular}[c]{@{}l@{}}0.4621\\ 0.3610\\ 0.2257\\ 0.4182\end{tabular}} & \multicolumn{1}{l}{\begin{tabular}[c]{@{}l@{}}0.7920\\ 0.7960\\ 0.7033\\ 0.7333\end{tabular}} & \begin{tabular}[c]{@{}l@{}}0.6252\\ 0.6059\\ 0.4601\\ 0.5696\end{tabular} & \begin{tabular}[c]{@{}l@{}}0.7040\\ 0.7154\\ 0.6114\\ 0.6667\end{tabular} & \begin{tabular}[c]{@{}l@{}}0.7651\\ 0.6910\\ 0.6038\\ 0.7689\end{tabular} & \multicolumn{1}{l}{\begin{tabular}[c]{@{}l@{}} \underline{0.8449}\\ \underline{0.8021}\\ \underline{0.7292}\\ \underline{0.8297}\end{tabular}} & \textbf{\begin{tabular}[c]{@{}c@{}}0.8742\\ 0.8256\\ 0.7627\\ 0.8724\end{tabular}} \\ \hline
USPS     & \begin{tabular}[c]{@{}c@{}}ACC\\ NMI\\ ARI\\ F1\end{tabular} & \begin{tabular}[c]{@{}c@{}}0.6682\\ 0.6272\\ 0.5464\\ 0.6494\end{tabular} & \multicolumn{1}{l}{\begin{tabular}[c]{@{}l@{}}0.4402\\ 0.4850\\ 0.3082\\ 0.3665\end{tabular}} & \multicolumn{1}{l}{\begin{tabular}[c]{@{}l@{}}0.7684\\ 0.7795\\ 0.7011\\ 0.7565\end{tabular}} & \begin{tabular}[c]{@{}l@{}}0.6381\\ 0.7004\\ 0.5636\\ 0.5861\end{tabular}  & \begin{tabular}[c]{@{}l@{}}0.7196\\ 0.6859\\ 0.6081\\ 0.7093\end{tabular} & \begin{tabular}[c]{@{}l@{}}0.7355\\ 0.7112\\ 0.6333\\ 0.7245\end{tabular} & \multicolumn{1}{l}{\begin{tabular}[c]{@{}l@{}}\underline{0.7722}\\ \underline{0.7907}\\ \textbf{0.7110}\\ \underline{0.7626}\end{tabular}} & \begin{tabular}[c]{@{}c@{}}\textbf{0.7755}\\ \textbf{0.7923}\\ \underline{0.7107}\\\textbf{ 0.7634}\end{tabular} \\ \hline
\end{tabular}}\\[2mm]
\caption{Clustering results on all five datasets. We mark the best-performing and the second-best-performing results by bolded and underlined.}\label{results}
\end{table*}

\subsection{Datesets}
\begin{itemize}
    \item ACM \cite{bo2020structural} contains $3025$ papers of $3$ major categories (i.e., database, wireless communication and data mining). The keywords of each paper are chosen as its feature. Different papers of the same author should have the relative strong correlation, so we can construct the structure graph for GCN.
    \item DBLP \cite{ley2009dblp} is an author network dataset collected from the DBLP website, which includes $4057$ authors of $4$ categories. The research fields of each author are treated as the feature.
    \item Citeseer\footnote{https://csxstatic.ist.psu.edu/downloads/data} is a citation network dataset which is composed of paper features and citation connections between papers. This dataset has $3327$ papers of $6$ categories.
    \item HHAR \cite{stisen2015smart}
    consists of $10299$ sensor records collected from smart phones and smart watches, which is divided into $6$ categories, including biking, sitting, standing, walking, stair up and stair down.
    \item USPS \cite{lecun2015deep} contains $9298$ gray images of $10$ hand-written digits, and the size of each images is $16 \times 16$.
\end{itemize}

\subsection{Compared Methods}
To verify the effectiveness of CaEGCN, we compare it with several state-of-the-art clustering methods, including,
\begin{itemize}
    \item \textbf{K-means} \cite{lecun2015deep} is a basic clustering algorithm based on the content of data only.
    \item \textbf{Auto-Encoder (AE)} \cite{hinton2006reducing}
    performs K-means on the low-dimensional representations learned from the deep auto-encoder network.
    \item \textbf{Improved deep embedded  clustering (IEDC)} \cite{guo2017improved} adds the clustering oriented loss and the reconstruction loss to the deep auto-encoder network, which realizes the one-step clustering of low-dimensional representations.
    \item \textbf{Variational Graph Auto-Encoders (VGAE)} \cite{kipf2016variational} is a variational graph auto-encoder with both topology and content information, which introduces the GCN architecture and the graph reconstruction loss to build a graph convolutional auto-encoder network.
    \item \textbf{Adversarially Regularized Graph Auto-encoder (ARGA)} \cite{pan2020learning} is a GAN architecture deep clustering model. They firstly construct a graph convolutional auto-encoder network, then the adversarial training principle is applied to enforce the latent codes to match a prior Gaussian or uniform distribution.
    \item \textbf{Deep Attentional Embedded Graph Clustering (DAEGC)} \cite{wang2019attributed} uses the graph attention network to build the encoder and trains the internal product decoder to reconstruct the graph structure. In addition, soft labels are generated according to the graph embedding to monitor the self-training graph clustering process.
    \item \textbf{Structural Deep Clustering Network (SDCN)} \cite{bo2020structural} uses the structure information learned by the GCN module to strengthen the data representation learned by the auto-encoder. Furthermore, it constructs a dual self-supervised loss to combine two networks and supervise clustering, which is an important baseline.
\end{itemize}

The parameter settings of compared methods are listed below.
For K-means, we repeatly run K-means $20$ times and report the best result.
For AE and IEDC, following the work in \cite{bo2020structural}, the network dimension of each dataset is set as $500-500-2000-10-2000-500-500$.
VGAE is a two-layer network, so we set network dimension of its encoder as $500-10$, \cite{kipf2016variational}.
For ARGA, following the work in \cite{pan2020learning}, we set the dimension of the encoder as $32-16$, in addition, the discriminator's dimension is set as $16-64$.
For DAEGC, the dimension of the graph attention encoder is set as $256-16$, \cite{wang2019attributed}.
For SDCN, we set the dimensions of the encoder and GCN module for $500-500-2000-10$, \cite{bo2020structural}.

To evaluate all methods from multiple aspects, we choose four popular clustering evaluation metrics, including Accuracy (ACC), Normalized Mutual Information (NMI), Average Rand Index (ARI) and macro F1-score (F1). For all metrics, a higher score indicates a better clustering quality.

\subsection{Parameter Settings}
When a dataset does not come with graph information, to construct the initial graph of data, we select the popular K-Nearest-Neighbor algorithm (KNN), and the value of $K$ is positively correlated with the number of samples and categories.
Following the strategy in SDCN \cite{bo2020structural}, we tune different $K$ in the range \{$3,5,10$\} to get the best performance. Generally, USPS and HHAR employ $K=10$ and $K=5$ to construct the corresponding graphs, respectively. As for ACM, DBLP, and Citeseer, we directly exploit the existing graphs in the datasets.

In the proposed CaEGCN, we set the dimensions of both CAE and GAE modules as $input-500-10-cluster-500-500-output$, where $input$ and $output$ denote the dimension of the raw data, and $cluster$ represents the number of cluster categories.
The purpose of the last layer in the decoder is to reconstruct the raw data, so the dimension of the last layer equals to the first layer, i.e., $output=input$.

In the cross-attention fusion module, we set the number of heads as $8$.
In the self-supervised module, the iteration number of K-means is set as $1000$ to initialize the cluster centers.
At last, we employ the Xavier method to initialize our model parameters \cite{glorot2010understanding}, and the initial learning rate is set as $0.001$.

\begin{figure*}
\centering
\subfigure[ ACM-raw data.]{ \label{ACMR}

\includegraphics[width=0.22\linewidth]{acm.png}}
\subfigure[ACM-CaEGCN.]{ \label{ACMC}

\includegraphics[width=0.22\linewidth]{acm_pred.png}}
\subfigure[DBLP-raw data.]{ \label{DBLPR}
\includegraphics[width=0.22\linewidth]{dblp.png}}
\subfigure[DBLP-CaEGCN.]{ \label{DBLPC}
\includegraphics[width=0.22\linewidth]{dblp_pred.png}}
\subfigure[Citeseer-raw data.]{\label{CitesseerR
\includegraphics[width=0.22\linewidth]{cite.png}}
\subfigure[Citeseer-CaEGCN.]{\label{CitesseerC
\includegraphics[width=0.22\linewidth]{cite_pred.png}}
\subfigure[HHAR-raw data.]{ \label{HHARR}
\includegraphics[width=0.22\linewidth]{hhar.png}}
\subfigure[HHAR-CaEGCN.]{ \label{HHARC}

\includegraphics[width=0.22\linewidth]{hhar_pred.png}}
\subfigure[USPS-raw data.]{ \label{USPSR}

\includegraphics[width=0.22\linewidth]{usps.png}}
\subfigure[USPS-CaEGCN.]{ \label{USPSC}

\includegraphics[width=0.22\linewidth]{usps_pred.png}}
\caption{2D visualization. The comparison of the raw data and the clustering results of CaEGCN on ACM, DBLP, Citeseer, HAR and USPS datasets.} \label{Samplefig}
\end{figure*}

\subsection{Experiment Results Analysis}
Table \ref{results} exhibits the whole experimental results compared with other clustering methods. Obviously, the proposed CaEGCN model achieves the best performance in most cases.

We can see that the content information based deep clustering methods (such as AE and IEDC) work better than the graph convolutional auto-encoder method VGAE. The reason is that VGAE exists the over-smoothing problem. In other words, the information received by the nodes has a low signal-to-noise ratio.
SDCN and the proposed CaEGCN supplement the content information into the GCN module in each layer, which effectively relieves the over-smoothing problem, so SDCN and CaEGCN receive the satisfactory performance.

The experimental results show that SDCN and the CaEGCN are superior to other methods on the whole datasets. Compared with directly using GCN, supplementing the content information into the structure representation layer by layer can help the clustering work better, and it also illustrates the significance of the interaction between the heterogeneous information.
In addition, the proposed CaEGCN performs better than SDCN in most cases, which proves that the cross-attention fusion module in CaEGCN can promote the learned data representation containing more prominent information for clustering tasks.

\begin{table*}
\centering
\resizebox{!}{55mm}{
\begin{tabular}{r|c|cccc}
\hline
Dataset  & Metric                                                       & CaEGCN                                                                             & \begin{tabular}[c]{@{}c@{}}CaEGCN\\ w/o attention\end{tabular}            & \begin{tabular}[c]{@{}c@{}}CaEGCN\\ w/o graph\end{tabular}                & \begin{tabular}[c]{@{}c@{}}CaEGCN\\ w/o content\end{tabular}              \\ \hline
ACM      & \begin{tabular}[c]{@{}c@{}}ACC\\ NMI\\ ARI\\ F1\end{tabular} & \textbf{\begin{tabular}[c]{@{}c@{}}0.9012\\ 0.6703\\ 0.7300\\ 0.9009\end{tabular}} & \begin{tabular}[c]{@{}c@{}}0.8883\\ 0.6415\\ 0.6994\\ 0.8879\end{tabular} & \begin{tabular}[c]{@{}c@{}}0.8860\\ 0.6391\\ 0.6989\\ 0.8877\end{tabular} & \begin{tabular}[c]{@{}c@{}}0.8869\\ 0.6394\\ 0.6961\\ 0.8866\end{tabular} \\ \hline
DBLP     & \begin{tabular}[c]{@{}c@{}}ACC\\ NMI\\ ARI\\ F1\end{tabular} & \textbf{\begin{tabular}[c]{@{}c@{}}0.6823\\ 0.3388\\ 0.3617\\ 0.6669\end{tabular}} & \begin{tabular}[c]{@{}c@{}}0.6766\\ 0.3261\\ 0.3515\\ 0.6581\end{tabular} & \begin{tabular}[c]{@{}c@{}}0.6628\\ 0.3127\\ 0.3257\\ 0.6530\end{tabular} & \begin{tabular}[c]{@{}c@{}}0.6732\\ 0.3249\\ 0.3422\\ 0.6620\end{tabular} \\ \hline
Citeseer & \begin{tabular}[c]{@{}c@{}}ACC\\ NMI\\ ARI\\ F1\end{tabular} & \textbf{\begin{tabular}[c]{@{}c@{}}0.6802\\ 0.4000\\ 0.4240\\ 0.6138\end{tabular}} & \begin{tabular}[c]{@{}c@{}}0.6411\\ 0.3550\\ 0.3706\\ 0.5975\end{tabular} & \begin{tabular}[c]{@{}c@{}}0.6616\\ 0.3905\\ 0.3898\\ 0.5744\end{tabular} & \begin{tabular}[c]{@{}c@{}}0.6267\\ 0.3533\\ 0.3463\\ 0.5406\end{tabular} \\ \hline
HHAR     & \begin{tabular}[c]{@{}c@{}}ACC\\ NMI\\ ARI\\ F1\end{tabular} & \textbf{\begin{tabular}[c]{@{}c@{}}0.8742\\ 0.8256\\ 0.7627\\ 0.8724\end{tabular}} & \begin{tabular}[c]{@{}c@{}}0.8637\\ 0.8090\\ 0.7320\\ 0.8595\end{tabular} & \begin{tabular}[c]{@{}c@{}}0.8693\\ 0.8231\\ 0.7599\\ 0.8624\end{tabular} & \begin{tabular}[c]{@{}c@{}}0.8650\\ 0.8224\\ 0.7559\\ 0.8568\end{tabular} \\ \hline
USPS     & \begin{tabular}[c]{@{}c@{}}ACC\\ NMI\\ ARI\\ F1\end{tabular} & \textbf{\begin{tabular}[c]{@{}c@{}}0.7755\\ 0.7871\\ 0.7107\\ 0.7634\end{tabular}} & \begin{tabular}[c]{@{}c@{}}0.7623\\ 0.7712\\ 0.6902\\ 0.7552\end{tabular} & \begin{tabular}[c]{@{}c@{}}0.7698\\ 0.7746\\ 0.6984\\ 0.7566\end{tabular} & \begin{tabular}[c]{@{}c@{}}0.7704\\ 0.7794\\ 0.7019\\ 0.7578\end{tabular} \\ \hline
\end{tabular}}\\[2mm]
\caption{The results of ablation experiments on all five datasets. We mark the best-performing result by bolded.}\label{AblationResult}
\end{table*}

For the academic papers datasets, various factors interfere the clustering or recognition tasks, such as the cross-domain applications of popular algorithms, different research topics in the same field and different research fields of the same author, and so on.
As for the ACM dataset,
the CaEGCN achieves the significant improvements in all four evaluation metrics. The accuracy of CaEGCN increases by from $1.7\%$ compared with SDCN to $24.1\%$ compared with K-means; The NMI of CaEGCN increases from $5.6\%$ compared with SDCN to $51.9\%$ compared with K-means; The ARI of CaEGCN increases by from $5\%$ compared with SDCN to $57.3\%$ compared with K-means; The F1 score of CaEGCN increases by from $1.7\%$ compared with SDCN to $24\%$ compared with K-means.

We also note that VGAE has achieved good clustering results, which simultaneously considers the graph topology information and node content of the graph.
ARGA uses the adversarial training to optimize the method based on graph convolution, and achieves some improvements. DAEDC uses the graph attention module, and also achieves the better experimental results. The huge gap between the CaEGCN and SDCN (and others) further proves the superiority of the CaEGCN. Similar to the ACM dataset, DBLP and Citeseer are datasets related to academic papers, and their experimental results show the same pattern and trend.

Large scale dataset is an important challenge for clustering methods. When the scale of samples increases, the performance of many state-of-the-art clustering methods drops dramatically. The scale of HHAR and USPS datasets are $3$ times larger than the previous datasets.

For the HHAR dataset, it is difficult to distinguish some human daily behaviors,
e.g., walking and biking. This imposes a challenge for clustering tasks. Compared with VGAE which integrates the structural features into the content information, the accuracy of the CaEGCN increases by $28.48\%$; compared with ARGA, the accuracy increases by $19.47\%$; and compared with DAEGC, the accuracy increases by $12.48\%$.
It is our belief that the poor performance of these methods is due to
the over-smoothing problem of GCN.  The experimental results prove the effectiveness of our cross-attention module.
Compared with the best baseline SDCN, the accuracy of CaEGCN still increases by $3.4\%$.

For the USPS dataset, its background is simple, so we regard it as a baseline data to test the robustness of the CaEGCN.
The experimental results of the CaEGCN, SDCN, and IDEC are similar. This may be caused by the fact that
the content information of the some handwritten digit images is difficult to distinct.
We emphasize here that each node in the initial graph we constructed can only connect $10$ closest nodes when considering efficiency, so the initial graph fails to contain enough valuable relationship between the data. Such a limited graph restricts the learning ability of the convolutional network.

In summary, these performance improvements of the CaEGCN can be attributed to two aspects: first, the cross-attention fusion mechanism integrates the content information and the data relationship, and highlights the critical information in the fusion representations;
second, the self-supervised module further optimizes the distributions of the middle layer representations to strengthen the performance of deep clustering.

\subsection{Clustering Result Visualization}
We visualize the clustering results of five datasets in a two-dimensional space with the t-SNE algorithm \cite{maaten2008visualizing}. The location distribution of the raw data is overlapping, while the CaeGCN can obviously drive the raw data into different groups.

\subsection{Ablation Experiment Analysis}
To prove the effectiveness of each critical module in our model, we designed a set of ablation experiments. Specifically, we repeatedly remove one module from the CaEGCN model and test the clustering performance of these incomplete models on five datasets.
The designed incomplete models are displayed as follow,
\begin{itemize}
    \item \textbf{CaEGCN w/o attention:} The proposed CaEGCN without the cross-attention fusion module.
    \item \textbf{CaEGCN w/o graph:} The proposed CaEGCN without the graph reconstruction loss $\mathcal L_{GAE^{graph}}$ in the GAE module,
    \begin{equation}
      \begin{aligned}
    \mathcal{L}_{overall} &=\mathcal L_{GAE^{content}} + \mathcal L_{CAE^{content}}+\mathcal L_{cae}+\mathcal L_{gae}.
      \end{aligned}
    \end{equation}
    \item \textbf{CaEGCN w/o content:} The proposed CaEGCN without the content reconstruction loss $\mathcal L_{GAE^{content}}$ in the GAE module,
    \begin{equation}
      \begin{aligned}
    \mathcal{L}_{overall} &=\mathcal L_{GAE^{graph}} + \mathcal L_{CAE^{content}}+\mathcal L_{cae}+\mathcal L_{gae}.
      \end{aligned}
    \end{equation}
\end{itemize}

From Table \ref{AblationResult}, we observe that the CaEGCN still achieves the best results on all datasets. The clustering results of the above three incomplete models decline, which verifies the importance of each critical module.

Among them, the experimental results of \textbf{CaEGCN w/o attention} drop sharply, which proves that the lack of the content and data relationship fusion can decrease the learning ability of the GAE module.

Without the graph reconstruction loss $\mathcal L_{GAE^{graph}}$, the clustering performance of \textbf{CaEGCN w/o graph} still decrease obviously. This reconstruction loss ensures the learned middle layer representations have abundant structure information, which improves the clustering performance.
Meanwhile, the experimental results of the \textbf{CaEGCN w/o graph} on the datasets using the original graph (e.g., ACM, DBLP) significantly decrese, which reflects that the graph reconstruction loss can effectively improve the quality of data representations with the accurate graph structure.

As for \textbf{CaEGCN w/o content}, its experimental results are not bad.
Compared with other two modules, the impact of content reconstruction loss $\mathcal L_{GAE^{content}}$ is relatively small.
However, we believe that content reconstruction loss is also indispensable.
The CAE module likes a residual network to supplement the high-quality content information to the GAE module layer-by-layer. Then, the content reconstruction loss ensures the middle layer representation learned by the GAE module contains more content information of the raw data.

Throughout these three ablation experiments, it turns out that each module improves the clustering performance from different aspects and is meaningful.

\section{Conclusion} \label{Sec:5}
We propose a cross-attention fusion based enhanced graph convolutional network for subspace clustering (CaEGCN), which connects the CAE and GAE modules layer-by-layer through the cross-attention fusion module, and strengthens the essential information.
The fusion representation is used as the input of the GAE module.
The novel graph reconstruction loss and content reconstruction loss in the GAE module further ensure the middle layer representation more appropriate for clustering.
Finally, we build the self-supervised module to train the entire end-to-end model.
The excellent experimental results on various datasets prove the superiority of the proposed methods.

\section*{Acknowledgements}
The research project is partially supported by National Natural Science Foundation of China under Grant No. U19B2039, 61906011, 61632006, 61772048, 61672071, U1811463, 61806014, Beijing Natural Science Foundation No. 4204086, Beijing Municipal Science and Technology Project
No. KM202010005014, KM201910005028, Beijing Talents Project (2017A24), Beijing Outstanding Young Scientists Projects (BJJWZYJH01201910005018).

\begin{comment}
\GaoC{Referece [6]:   Ganc -- GANC}

\GaoC{Referece [10]:   Conference name:  First letter of each word to be in Capital}

\GaoC{Referece [12]:   Line -- LINE}

\GaoC{Referece [16]:   T-gcn -- T-GCN}

\GaoC{Referece [18]:   Proteingcn -- ProteinGCN}

\GaoC{Referece [20]:   Graphgan -- GraphGAN}

\GaoC{Referece [21]:   Get full bib information such volume, pages etc, otherwise to appear}
\textcolor{red}{Referece [21] is a early access article.}

\GaoC{Referece [22]:   Paper was published at NIPS Workshop on Bayesian Deep Learning.  Please update it}

\GaoC{Referece [29]:   Add "International Conference on"}

\GaoC{Referece [31, 33]:   Add "Conference on"}

\GaoC{Referece [34]:   Qanet -- QANet}

\GaoC{Referece [35]:   Danet -- DANet}

\GaoC{Referece [37]:   t-sne -- t-SNE}

\GaoC{Referece [38]:   Dblp -- DBLP}

\GaoC{Referece [39]:   Add "ACM Conference on"}
\end{comment}

\bibliographystyle{IEEEtran}

redpajama_set_name: RedPajamaArXiv


A lot of charm only in Narita including the athletic to enjoy in a ranch, the Nature which View Point that can sense force bodily close and pretty animals meet of the airplane which a detached room arrives in Narita International Airport. Camping and the barbecue to enjoy in families are recommended. It is active and will play exhaustively in Narita.
Around Narita International Airport, there is the view spot that can enjoy the arrival and departure of the airplane close.
Even if it is taken even if not only I get on, but also I look at the airplane, it is fun. Let's go out for visiting airplane view spots to camera one hand!
The Sakataga-ike Park is next to the Chiba Prefectural Boso-no-Mura (Boso Village) in a very large park of approximately 17ha to surround around Sakata ヶ pond blessed with rich Nature. It is an oasis to be able to enjoy from an adult to a child in the park which is full of water and green.
In the park, there are playground equipment and barbecue ground and Campsites, and there is turnout in many people in the season for outings.


text:
7004 Magic decks were found matching your criteria.
<<Prev - - Next>>
Deck Finish Player Event Format Date Location
Abzan Aggro 6th Ian pantagis Star City Games Invitational Qualifier Standard 2015-03-14 Newport News
Mardu Aggro 6th Jonathan Mckinney Star City Games Invitational Qualifier Standard 2015-03-01 Blacksburg
U/R Twin 6th Stephen J Nagy Star City Games Invitational Qualifier Modern 2015-03-15 Falls Church
Jeskai Tokens 6th Todd Anderson Star City Games Invitational Standard 2015-03-27 Richmond
Infect 6th Todd Anderson Star City Games Invitational Legacy 2015-03-27 Richmond
Jeskai Tokens 6th Alex Bianchi Star City Games Standard Open Standard 2015-03-28 Richmond
Elves 6th Ryan Hare Star City Games Premier IQ Legacy 2015-03-29 RIchmond
Merfolk 6th John Taylor Star City Games Premier IQ Modern 2015-03-29 Richmond
Naya Aggro 6th Patrick Markholm Star City Games Invitational Qualifier Standard 2015-04-04 Springfield
Grixis Twin 6th Harlan Firer States Modern 2015-04-12 Hopewell
G/W Devotion 6th David Fulk States Standard 2015-04-11 Hopewell
Abzan Aggro 6th Michael Woolwine Star City Games Invitational Qualifier Standard 2015-04-04 Roanoke
Jund 6th Graham Taylor Star City Games Super IQ Modern 2015-04-19 Falls Church
Abzan Megamorph 6th Adam McDougall Star City Games Invitational Qualifier Standard 2015-04-18 Staunton
Esper Dragons 6th Nicholas Arestad Star City Games Invitational Qualifier Standard 2015-04-18 Chesapeake
Jeskai Delver 6th Damon Howell Star City Games Invitational Qualifier Legacy 2015-04-19 Roanoke
Jeskai Dragons 6th Jacob Shepard Star City Games Super IQ Standard 2015-04-25 Roanoke
U/R Twin 6th David Fulk Star City Games Invitational Qualifier Modern 2015-04-25 Portsmouth
Abzan 6th Jonathan Wright Star City Games Invitational Qualifier Modern 2015-05-02 Roanoke
Esper Control 6th Matthew Mains Star City Games Premier IQ Standard 2015-05-03 Richmond
Jeskai Pyro 6th Adam McAlister Star City Games Invitational Qualifier Modern 2015-05-16 Staunton
Atarka Red 6th Sean Park Star City Games Invitational Qualifier Standard 2015-05-16 Springfield
Mono-Red Aggro 6th John Dale Beety Star City Games Super IQ Standard 2015-05-23 Roanoke
Infect 6th Nick Miller Star City Games Invitational Qualifier Legacy 2015-05-17 Roanoke
Mono-Black Tron 6th Charles Boyer Star City Games Invitational Qualifier Modern 2015-05-24 Winchester
<<Prev - [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17] [18] [19] [20] [21] [22] [23] [24] [25] [26] [27] [28] [29] [30] [31] [32] [33] [34] [35] [36] [37] [38] [39] [40] [41] [42] [43] [44] [45] [46] [47] [48] [49] [50] [51] [52] [53] [54] [55] [56] [57] [58] [59] [60] [61] [62] [63] [64] [65] [66] [67] [68] [69] [70] [71] [72] [73] [74] [75] [76] [77] [78] [79] [80] [81] [82] [83] [84] [85] [86] [87] [88] [89] [90] [91] [92] [93] [94] [95] [96] [97] [98] [99] [100] [101] [102] [103] [104] [105] [106] [107] [108] [109] [110] [111] [112] [113] [114] [115] [116] [117] [118] [119] [120] [121] [122] [123] [124] [125] [126] [127] [128] [129] [130] [131] [132] [133] [134] [135] [136] [137] [138] [139] [140] [141] [142] [143] [144] [145] [146] [147] [148] [149] [150] [151] [152] [153] [154] [155] [156] [157] [158] [159] [160] [161] [162] [163] [164] [165] [166] [167] [168] [169] [170] [171] [172] [173] [174] [175] [176] [177] [178] [179] [180] [181] [182] [183] [184] [185] [186] [187] [188] [189] [190] [191] [192] [193] [194] [195] [196] [197] [198] [199] [200] [201] [202] [203] [204] [205] [206] [207] [208] [209] [210] [211] [212] [213] [214] [215] [216] [217] [218] [219] [220] [221] [222] [223] [224] [225] [226] [227] [228] [229] [230] [231] [232] [233] [234] [235] [236] [237] [238] [239] [240] [241] [242] [243] [244] [245] [246] [247] [248] [249] [250] [251] [252] [253] [254] [255] [256] [257] [258] [259] [260] [261] [262] [263] [264] [265] [266] [267] [268] [269] [270] [271] [272] [273] [274] [275] [276] [277] [278] [279] [280] [281] - Next>>

redpajama_set_name: RedPajamaCommonCrawl


I highly recommend this store! The staff and dress selection is amazing. You can try on dresses that fit and make you feel amazing!! I got my wedding dress from here I'm not a true plus size but walked out with an amazing dress off the rack that fit perfectly.
We also got my mom's dress from here the staff were super helpful in helping her get a dress that makes her feel beautiful without breaking the bank!
Thank you Bella Mia for all of your help and support!!!!
I was looking for a dress that was sold out at another location. Unfortunately, not many stores carried the dress. I had to get the dress urgently. I was able to find out online that Bella Mia Bridal was one of few stores in Canada that carried the same dress/could order it for me. Anna from the store did a great job not only ordering the dress for me on short notice, but also got it to me well in advance of when I needed it by. All of this without me even having to visit their store. She was pleasant to deal with and kept me up to date every step of the way. The service was prompt, friendly and professional.
Amazing customer service, like THE BEST I got at any dress shop (and I went to a lot of them). The only reason I'm giving a 4/5 is that the staff insisted they put the gowns on you, and I was not comfortable with that AT ALL and they won't let you take pictures. All the other shops would let you get into dresses and then do it up for you, so if you're shy like me this may be an issue as well. Also, I found it hard to see what I liked about myself without taking a picture! They had a great selection and I LOVED that the dresses ACTUALLY fit, unfortunately, I ended up going with another store as a gown there caught my eye more than the ones I tried at Bella Mia.
I bought my dress at Bella Mia Bridal!!! My bridal party and I got great attention and felt taken care of through the whole appointment. After I bought my dress I wanted some unique alterations. Anna went above and beyond to make them work. So thankful for such a great experience!! I would recommend this place to any bride looking for great service.
What a fabulous experience! We stopped by on a Sunday afternoon looking to just browse and ended up buying a gown for me and for my fiancée. Anna, her daughter and the seamstress are a great team and make sure you feel comfortable with your dress. I highly recommend this shop for brides. Thanks for the experience!
Excellent service at Bella Mia Bridal a few weeks ago. My bridesmaids found and purchased their dresses here, and the service was outstanding. We booked the Bridal Party "Party", so we were able to shop outside of store hours and got to enjoy delicious treats and sparkling juice. The ladies made us feel very welcome, and were incredibly helpful in finding us the perfect dress for my bridal party! Thank you so much, Bella Mia Bridal!
Thank you to Bella Mia for all the help with buying my wedding dress! Personable, helpful and affordable and with a great selection. I had lots of compliments on my dress and will definitely recommend you to anyone looking for a wedding dress! Thanks!
What a wonderful experience to choose my wedding dress. More than 100 dresses to select from. A nice change when the dresses really do fit and you can see what you look like in them.
Really didn't enjoy going to other stores. While they can source your size you will never really know what it looks like unless you can try it on.
The drive or the flight is worth it to have that magical bride experience.
It's no wonder some of her clients traveled as far away as Europe!!
They were so accommodating with the fact that some of my bridal party had to pay over the phone or by e-mail because none of us live in the city.
They helped me find THE dress! They offered suggestions on how to add some colour to it and where to get it altered. And they hand-crafted the birdcage I wore in my hair!
In addition to that, they helped me to feel great about the whole process. The environment is friendly and welcoming. All shapes and sizes are welcome, accepted and made to feel beautiful.
I couldn't be happier about being a Bella Mia Bride!
Today I went to Bella Mia's to look for a wedding dress. The service was amazing, Anna really out did herself, so friendly and helpful. Lots of selection of plus size gowns and prices were very reasonable. She made me feel beautiful no matter what size I am. I would definitely recommend shopping there if you are a plus size bride. I ended up getting my dream dress and can't wait to go back for my accessories. Thank-you Bella Mia from the bottom of my heart!
Lots of Beautiful dresses and Jewelry to complete your look. Anna is so helpful! Great place to shop!
Bella Mia fills a real need in the bridal market! It is lovely as a plus size bride to go somewhere you can actually try stuff on in a private, welcome, lovely environment. Would recommend it for a plus sized bride or even for a bride with bridesmaids of all sizes…would make the experience fun for all…not just fun for small!
Thank you Anna for helping me find my perfect dress, veil, and crown. My day was so special and I looked amazing.
Very friendly. Very knowledgeable. Very patient. Great experience. Would HIGHLY recommend!
Had an Amazing experience, both Anna and Olivia were extremely supportive and helpful. Very comfortable environment and they helped work within my budget. A huge plus is that they have everything you need in one spot for great prices- was extremely thankful that they even do custom garters as it is almost impossible to find plus size garters.
Would highly recommend to any plus size brides and or bridesmaids needing dresses.


December Viz Arts preview: Jared Haug, "Vertical Limit Under the Volcano," 2012, faded construction paper. Photo courtesy of the artist.
The Pacific Northwest has a very long history of artists, from Native Americans to the painters of the Northwest School, interpreting the natural beauty of the land as an innately spiritual realm. Few contemporary practitioners have upheld -- and humorously subverted -- this legacy as long or consistently as Portlander Bruce Conkle.
As part of the collective Eco Baroque with the photographer Marne Lucas, he has rolled the symbols of clichéd New Age mysticism and cold consumerism into strangely primitive combinations. In one memorable work, the duo suspended the heat lamps of a tanning bed like a chandelier, festooned it with flora and strung it with glinting crystals in a perverse marriage of the organic and the artificial.
Following Eco Baroque's site-specific installation in the Gobi Desert as part of Land Art Mongolia 360º last summer, Conkle presents "Tree Clouds," a solo exhibition of new sculpture and drawing at Portland State's Autzen Gallery. The title is an allusion to the smoke produced when aromatic tree resin is burned. For the show, Conkle has harvested resins from local species including Sitka spruce, Douglas fir, Ponderosa pine and shore pine. Several of the sculptures in the show -- including a bronze snowman similar to those permanently installed downtown on the Fifth Avenue bus line -- are functional incense burners, which he will use to burn the resins on Thursdays at 4 p.m. throughout the run of the exhibition.
Rocksbox Fine Art at the Colony One of TBA 2011's highlights was Portland artist Patrick Rock's "Oscar's Delirium Tremens," a gigantic inflatable sculpture of a pink elephant lying supine on an equally large circular bed. While "Oscar" was designed for viewers to climb inside and jump around the elephant's stomach to the point of dizziness, it was damaged in the early days of the festival, sidelining it as a work simply to be looked at. Rock, who operates Rocksbox Fine Art, is resurrecting the work for two nights, fully repaired and ready for action.
Recess Gallery This weekend is your last chance to catch "Window Smokers," a collaborative exhibition by Jared Haug and Brooks Dierdorff, both co-directors of Springfield's Ditch Projects. While the work in the show is never attributed to a single artist, it's easy to suss out who made what. Dierdorff continues to explore the culture of game hunting with tightly cropped images of elk pelts and a video projected onto a murky monolith of ballistics gel. Haug, on the other hand, turns in a rapturous suite of sun-faded construction paper works, whose overlaid imagery is produced from stenciled film stills.
FalseFront Following shows by Zachary Davis and David Knowles, FalseFront has emerged this fall as a proving ground for Portland's young artists. Case in point: Daniel J. Glendening presents "Conjurer," a meditation on the shamanistic enterprise of art-making, which echoes his ritual-as-performance contribution to Disjecta's current group show, "The Lathe of Heaven."
Fourteen30 Contemporary The sleek, sparse group show "Romancing the Stone" gathers Adam Marnie's blindingly white wall pieces, in which the artist appears to have punched through the plane with his fist; Monique Mouton's painted staffs, which casually lean against the gallery's walls; Davida Nemeroff's glacially cool sculptural photography; and Bobbi Woods' signature modified movie posters. Like the 1984 film from which it takes its name, these works function like treasure maps, inviting a hunt for discovery, but insisting that the hunt itself is the real treasure.
Pulliam Gallery This month, Pulliam presents a show featuring two local painters: G. Lewis Clevenger, whose coolly geometric abstractions call to mind architectural blueprints, and Brenden Clenaghen, whose new body of paintings marks a sizable stylistic shift. Clenaghen's paintings are at once gestural, flaunting the artist's hand, and more precisely composed, carving up the picture plane with squares, triangles and diamonds. Despite their formalism, they hold an intriguing aura of runic mystery.
PNCA For "Binary Lore," PNCA has partnered with Chicago art nonprofit Threewalls for an exhibition exchange, in which PNCA curator Mack McFarland and Threewalls director Shannon Stratton each selected an artist from one another's city. Their choices were inspired: PNCA alum Brenna Murphy, a member of the Oregon Painting Society, creates bizarre and disorienting digital worlds, while Edie Fake's drawings imagine the bygone landmarks of Chicago's queer culture, including bars and clubs like Mama Peaches, Sappho and the Virgo Out.


Next time you order a smartphone with online credit, cautiously order it in the morning usual hours and not midnight. Chances are that your loan just might be rejected because its midnight. Queer it may sound, but this is done to keep a check on the fraudsters who prefer odd hours to make online purchases.
Online credit and deferral payments have been a growing trend among young online shoppers. The range of these credits varies from Rs 500-1.5 lakh. Startups like ePayLater, Simpl, PayU, ZestMoney offer these services which allow the online buyers to make purchases instantly and pay their bills later with zero or a minimal interest. The age group usually varies from 22-35.
"I love the pay later option. It lets me check out very quickly and I don't have to wait long for mobile wallets or enter debit card details," says 22-year old Shailaja Kabra, an MBA student of Christ University. Estimates say that cart abandonment rate on e-commerce portals in India can be a high 60-70%, who intend to buy the product but fail to transact due to multiple reasons.
ePayLater, Simpl, PayU which allows one to purchase and pay later sees an average ticket size of around Rs 700-900 and allows users to pay it within 15 days with zero interest. On the other hand, ZestMoney which works like an online credit card offers larger credits with a minimum interest. These loans that range between Rs 3000-1.5 lakh are approved within few hours depending on the customer profile. The average ticket size for ZestMoney is around Rs 20,000.
Simpl has partnered with companies like BookMyShow, Faaso's, FreshMenu, Box8 and Zipgo. Similarly, ePayLater has partnered with PVR, Crownit, Indiamart, IndustryBuying and vareorganic while ZestMoney has partnered with Overcart, Credr, Velvetcase and Cuemath.
Notably, PayU also acquired a minority stake in Bengaluru-based ZestMoney earlier this year.
"We started the pilot program on deferral payments in March and are planning to invest $50 million in it over the next couple of years," says Jitendra Gupta, MD, PayU India.
These companies heavily rely on analytics and machine learning to understand a customer's online behaviour and purchase patterns so as to avoid fraudsters.
He further adds that ePayLater witnessed a steep growth after demonetisation. "In the last 3 months we have grown around 200% on a monthly basis." He also notes that non-metro customers who are usually accustomed with cash on delivery, are keenly adapting online credit these days.
Simpl handles around 11-12k transactions in a day. Moreover, Nitya adds that these transactions spike up from Wednesday to Saturdays as well on occasions like Diwali and Christmas when big movies hit the screens or family outings are planned.
Both Simpl and ePaylater have partnered with NBFCs who manages the cost of capital or interest that accrues on the micro credit.
"Not every buyer can use Simpl and it depends on the customer's online profile. In addition if a customer delays or fails to pay they are ousted from the system," adds Nitya.


text:
Иван Минов (Минев) е български революционер, деец на "Охрана" по време на Втората световна война.

Биография
Иван Минов е роден в костурското село Сетома, тогава в Османската империя, днес Кефаларион, Гърция. Деец е на гръцката комунистическа партия. В началото на 1943 година се присъединява към "Охрана" и е определен за ръководител на отряд от около 100 души в Желево. През август 1943 година е определен за началник (войвода) на Кономладския център. Негов подвойвода тогава е Александър Панджаров.

Бележки

Български революционери от Македония
Дейци на Охрана
Родени в Сетома
Българи в Гърция
Български комунисти

redpajama_set_name: RedPajamaWikipedia


text:
Q: Web3.js-- getBalance to web3.utils.fromWei, store the value in a variable to be used in a table So ive tried figuring this out for a few hours and I can console.log the balance, but I cannot seems to store the balance in a var to be used later. The goal is the get the users account balance (eth) and display it in the table. Anyway heres what I got.

 function App(){

 ///.....////

 const Web3 = require("web3");
  const web3 = new Web3(
    Web3.givenProvider ||
      "https://ropsten.infura.io/v3/9aa3d95b3bc440fa88ea12eaa4456161"
  );

  async function checkBalance() {
    try {
      await web3.eth.getBalance(account).then(web3.utils.fromWei());
    } catch (error) {
      console.log(error);
    }
  }

  ////.....//// {checkBalance} to be displayed in a table

            <TableCaption>Token balances of {account}</TableCaption>

                Token
                Contract Address
                Balance

                ETH
                0x0000000000000000000000000000000000000000
                {checkBalance}

                Token
                Contract Address
                Balance

The error I get in the console is "Warning: Functions are not valid as a React child. This may happen if you return a Component instead of  from render. Or maybe you meant to call this function rather than return it."

A: The function web3.eth.getBalance(address).then() requires a function to be passed into the then() clause.
In the snippet below I'm passing a function that accepts balanceInWei as a parameter. Only then you can pass balanceInWei variable to the web3.utils.fromWei(balanceInWei) function for it to return the ETH balance of the address
const Web3 = require("web3");
const web3 = new Web3("https://ropsten.infura.io/v3/9aa3d95b3bc440fa88ea12eaa4456161");
let address = "0x0000000000000000000000000000000000000000";
let balance;

async function checkBalance() {
    try {
        web3.eth.getBalance(address).then((balanceInWei) => {
            balance = web3.utils.fromWei(balanceInWei);
            console.log("Balance in wei:", balanceInWei);
            console.log("Balance in ETH:", balance);
        });
    } catch (error) {
        console.log(error);
    }
}

checkBalance();

Additionally you shouldn't post your infura links. You made the credentials of your infura project public by posting it here. Next time replace it with eq. https://ropsten.infura.io/v3/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

redpajama_set_name: RedPajamaStackExchange


text:
Coretta Scott King's Daughter Says Her Mother Should Be Honored On Martin Luther King Jr. Day
Diddy Settles Sexual Harassment Lawsuit With Former Chef
Diddy has reportedly reached a settlement with his former personal chef in a sexual misconduct lawsuit that was filed in 2017, NBC Los Angeles reports. The settlement stems from a lawsuit…
By Jessica McKinney
Jessica McKinney
Plus Icon @jigga_jess
More Stories by Jessica
Take Five: DJ Khaled Talks 'Father of Asahd' And #Summergram Partnership
How CJ Wallace Turned His Connection To Notorious B.I.G. Into A Cannabis Brand
The Long Road: The Triumph And Truth Of B2K
John Lamparski
Diddy has reportedly reached a settlement with his former personal chef in a sexual misconduct lawsuit that was filed in 2017, NBC Los Angeles reports.
The settlement stems from a lawsuit Cindy Rueda filed nearly two years ago. Rueda was reportedly Diddy's personal chef from Apr. 2015 to May 2016. She accused the Bad Boy mogul of sexual harassment and unfair wages. She claimed that she never received additional pay for extra work hours during her time of employment.
Diddy's Revolt TV Responds To Reverse Discrimination Lawsuit
Additionally, she alleged that Diddy requested that Rueda serve breakfast to him and a guest in his bedroom. During those incidents, she said she was forced to see "Mr. Combs and his guest indisposed and engaged in sexual activity." Upon entering, she alleged that he would then yell and curse at her.
In Aug. 2017, a Los Angeles Superior Court Judge sent Rueda's case to mediation outside of the court due to the terms of an agreement she previously signed as an employee for Diddy.
The terms or value of the settlement has not been disclosed at this time.


To write a list about the ten best restaurants in New York City is to call upon yourself a certain sense of doom and gloom. For one, there are so many to wade through, and then to choose from - think for a moment of every small patisserie hidden down an alleyway to the sprawling dining rooms that stand guard over Central Park. And then there's the personal opinion of every New Yorker and visitor - each with their own favorite fare and neighborhood gem.
Now that we've all agreed it's fully impossible to write a list of the ten best restaurants in New York City, let's get down to business presenting you with a list of contenders for best restaurant. By no means is this list comprehensive, but it does offer a glimpse into New York's culinary greatness. From obvious front runners like Eleven Madison Park and Jean-Georges, to wine bars, such as Wildair, serving insanely tasty bites we're willing to bet that these restaurants will leave you supremely satisfied no matter their price point, location or crowd.
Below, we take a look at ten of the best New York City restaurants and break down exactly why you should add them to your list of musts on your visit to the Big Apple.
Ever since Missy Robbins opened, restaurant reviewers have been raving that finally she's finally cooking pasta again. Born into a family of hosts, travelers and cooks, Missy Robbins brings the best of Italy to Williamsburg where wood-fired seafood, hand-crafted pastas, classic Italian cocktails and warm hospitality come together to create a casual dining experience. Robbins says she found her Italian soul in her five years as Executive Chef at Spiaggia in Chicago. While focusing her skills on fine dining Italian she found her true passion in cooking a deeper understanding of regional Italian cooking. If you come here and don't get the pasta, you're doing it all wrong.
There's something to be said for staying power, and Le Bernardin certainly delivers. Celebrity chef Eric Ripert has created a New York space that's romantic, refined and seductive, while still managing to infuse his French-Spanish influence into every corner. Le Bernardin, regarded by many as the best restaurant in NYC, has received more James Beard Awards than any other restaurant in the city, and holds four New York Times stars, in addition to three Michelin stars. The food, as you might imagine, is impeccable. Simple preparations and respect for the ingredients trump any type of frou-frou experiments, though the cuisine still manages to shine with creativity that can only come from years of intense love and attention-to-detail. If you love seafood, put Le Bernardin at the very top of your culinary bucket list.
When a past love re-enters your life, it can either go horribly wrong or beautifully right. At Union Square Cafe, the iconic Danny Meyer spot recently re-opened after a two-year hiatus, we're reminded exactly of why the good old days were so fantastic. The chef's reinstated classics including the burger and the ricotta gnocchi, pure and simple comfort food done exceptionally well. Unlike your most loved comfort food joint, however, you can't just stroll in and plop yourself down at a table. Reservations are tough to nail down, so it's best to plan ahead. Still, it's worth the extra leg work. Union Square Cafe is back – and dare we say, better than ever.
Unless it's freakishly low, or ridiculously lofty, a restaurant's ceiling probably isn't the first thing that draws your attention. At Keen's, it'll be all you can look at - that is, until they bring your meal. Keens Steakhouse - established in 1885 - is one of New York's oldest and most revered eateries. It also happens to have the largest collection of churchwarden pipes in the world literally hanging from its rafters. The tradition stems from the days when travelers left their delicate pipes at their favorite inns until they returned. Today, Keen's is well known for its mutton chop - a deep cut of meat so succulent you'll wonder why its popularity ever waned.
One of New York's most recognized restaurants, Gramercy Tavern is an old standby that functions like a delicious utility knife. You can dress it up or dress it down, visit on a random weekday or save it for a special occasion and it will mold itself to your exact state of mind. The NY Times Pete Wells awarded Gramercy Tavern three stars in 2016, nearly 22 years after it first opened. Why? For one, things here simply work. You know the service is going to be on par and the food irresistible. You also know that the kitchen will continue to innovate on everything from their $110 veggie tasting menu to their decadent desserts. An old dog can learn new tricks, and you, the diner, should reward (and relish) them.
Helmed by Executive Chef, Daniela Soto-Innes, a 27-year-old force to be reckoned with, Enrique Olvera's Cosme burst onto the scene leaving New Yorkers clamoring for a seat. Olvera's Chef's Table episode featuring his Mexico City joint, Pujol, certainly didn't hurt his appeal, but the food at Cosme has proven to stand on its own. Take the duck carnitas, for example. Salted and hung for three days before being roasted and then slow-cooked overnight in duck fat and Mexican Coke, these aren't your typical tacos. This is modern Mexican food at its best - cool and comforting, while still keeping a keen eye on what makes New York different from Mexico City. Is this the best restaurant in New York right now? It may be.
Celebrity chef Jean-Georges Vongrichten has a culinary empire that spans the globe, but his eponymous restaurant connected to the Upper West Side's Trump Hotel remains a gold standard for the chef's subtly Asian-inflected, French haute cuisine. His now-legendary tuna tartare and young garlic soup remain enormously popular with Harry Winston-wearing hotel guests, well-dressed ladies who lunch, and top-tier executives from the neighboring high-rise offices. Expect stalwart service and expense account-worthy cuisine while taking in the views of Central Park West and Columbus Circle from the dining room's floor-to-ceiling windows. Without interruption, Jean-Georges has held three Michelin stars since it opened its doors in 1997.
Upon entering Sushi Yasuda, one is immediately aware of the absence of art, decorations, or anything that might attract the eye. It is minimalist to the extreme with bamboo-plank walls and open spaces. This simplicity is reflected in Chef Naomichi Yasuda's cuisine as well. The fillets of fish are custom-cut for each order, as pre-cut fish begins to break down almost immediately, losing some of its delicate textures and flavors. The pure and clean dishes are built out of respect for tradition have helped this restaurant to earn a three star review in the New York Times, and numerous accolades on lists like these ones the city over.
In 2017, Eleven Madison Park underwent a major renovation, which still kept the building's original Art Deco details and expanded the bar to accommodate more regulars. On the food front, acclaimed chef, Daniel Humm, continues to shine in the wake of his place on the 50 Best Restaurants list. While the menu changes frequently, guests can expect innovative food, but without the pomp and pretension that's rampant amongst fine dining restaurants in the city. Grubstreet described the food as a move toward "hyperminimal" with Humm himself saying, "Today, it's (the food) much more about feeling – it's harder to explain how to make something look organic and beautiful."
In the cluster of craziness that's become the new Lower East Side, a few spots rise above the noise. One, in particular, is Wildair, the laid-back sister restaurant of neighboring Contra. Helmed by two of the most exciting chefs in NYC, Wildair is mostly a wine bar...albeit one that serves really, really good food and unique, never-before-tasted pours. Most everything is made in-house, including the freshly baked bread that's served with salted olive oil. The wagyu steak for two is also a star, but if you're not looking to drop a cool Benjamin on grub, stick to the steak tartare topped with cheddar, horseradish and brazil nuts.


Dr. One-Zero won't stop until the whole world embraces his poisonous potion, Green Pop. Hopper, Eni and Josh must open a new portal to stop him.
Comics (Graphic works) Graphic novels. Detective and mystery comics. Graphic novels. Comic books, strips, etc. Comics (Graphic works) Detective and mystery comics. Graphic novels. Graphic novels, Juvenile. Mystery comic books, strips, etc. Graphic novels.


Vichy lies in the center of France in the volcanic region of Auvergne, a land rich in natural resources.
This is the starting point for its pH balancing water, crammed with minerals and incredibly pure. From its source to your skin cream, how does it retain its precious properties?
Flowing freely up from 4000 meters below ground, the spring gushes with a rate of 7 liters per minute. Vichy harvests this water with all its benefits intact at a depth of 15 meters, since this meticulous task protects its 15 minerals!
The first security checks on the water are performed instantly at its extraction from the source. An analysis is then performed not only on different bacteriological criteria, but also on the physiochemical properties of the water.
Next, Vichy pH Balancing Water undergoes a first filtration to remove impurities. It is then transported in stainless steel containers to the Vichy production site, located just 4 km from the source.
Upon arrival, the sealed containers are opened and the water is sampled. It is then examined a second time to guarantee optimal safety for the epidermis.
During this second step, each sample is analyzed on 5 key factors: the appearance, scent, clarity, pH value and conductivity. This last criterion is for measuring the mineral concentration of Vichy pH Balancing Water.
To guarantee its traceability, a sample is taken from each container and then stored for a maximum of 4 years.
The purpose of all these measures is to preserve the exceptionally high mineral content in the Vichy pH Balancing Water, which is at the heart of all Vichy's products. A natural source of skin health, Vichy Vichy pH Balancing Water strengthens, resulting in skin that is fortified against harmful elements.
Vichy pH balancing water has a volcanic history of over 2000 years. Focus on the path traveled by this unique thermal water. A secret that only nature truly knows.
Sourced in the heart of the French volcanoes, during its journey through the depths, Vichy pH Balancing Thermal Water draws in many minerals and trace elements. What are its other secrets?
Sourced from the heart of French volcanoes, Vichy pH Balancing Thermal Water contains 15 different minerals, with properties that help treat sensitive skin and signs of aging. Here are 4 reasons why pH Balancing Thermal Water should be a part of your anti-wrinkle skincare routine.
How can Vichy pH balancing water combat skin aggressors ?
Misting your face with Vichy pH balancing water helps combat external aggressors that damage your skin every day. Here's the proof.
100% natural, Vichy pH Balancing Water is the ideal skincare product to soothe, moisturize and refresh the skin of both children and adults.


consumerism or something, who knows.
stuff I've broken over the years and how it held up.
the TV or the console. That's a much more expensive tantrum.
Total Cost: Nothing. Nobody ever made me pay.
things and whatnot. And it usually doesn't hurt.
Also, if you don't fix it the hole, a cute girl might ask about it.
DOLLARS!" and began crying and punched the wall.
case: you have broken your hand.
few hours or months. So probably everything was alright.
constant reminder of how stupid you are.
"Nonononono!" as if he didn't realize what was going to happen.
LCD screen, complete with rainbow spiderweb cracks.
nerd would do a thing like that.
stories of interesting things you've punched and I'll probably post them to make myself feel better.


Commissioned by the Dutch Ministry of Security and Justice through the Repatriation and Departure Service, this project examines the concept of diaspora work placements, the recruitment of migrants and their descendants by multinational companies in the country of residence for placement in company offices in a migrant or diaspora member's country of (ancestral) origin. While much of the literature on the role of return migrants and diaspora members in the private sector in their countries of origin or ancestry has focused on the role of such mobile populations as investors or entrepreneurs, their role as employees has been less well explored. This research explicitly focuses on the advantages companies perceive in hiring migrants/diaspora members for international work placements, the current mechanisms by which diaspora work placements are made, and the factors that would encourage or facilitate large, multinational companies to engage with migrant/diaspora populations in international work placements. The project has approached this topic through desk review of the global academic and grey literature on international and diaspora work placements complemented by in-depth interviews with a select group of large, multinational companies based in the Netherlands.
This research has resulted in a diagnostic report that addresses current global experiences related to diaspora work placements, the practises of large Dutch multinational companies related to such international placements, and recommendations of how the private sector can be better facilitated to engage in such practises.


There are in the National Library of Portugal (BNP) about fifty paintings from former convents, mainly portraits executed between the sixteenth and nineteenth centuries, which are hung in rooms and along corridors, far from the eyes of the public and simultaneously protected by recent conservation and restoration interventions. These paintings, of little artistic interest, are, however, individual cases for the study of conservation and restoration interventions made in the nineteenth century. The data obtained will contribute to the history of paintings restoration, in terms of its practical aspects (about materials and techniques used), that is still to be written in Portugal. Our research, which began in January 2011 and is predicted to end in December 2013, has been oriented by three complementary lines of action, with an interdisciplinary methodological base: the exploitation of the BNP's precious and unpublished documental archive; the assessment to the conservation status of the paintings and identification of restoration works; and the material study of the works, from physical and chemical methods of examination and analysis, in order to deepen our technical knowledge about the restoration of easel painting in the nineteenth century. Through historical documents we could identify periods of restoration, restorers and materials used. The observation of paintings allowed selecting a set of twenty-seven pieces that showed old restorations, made in the nineteenth or early twentieth century. In addition it's very important to use examination methods and laboratory analysis to characterise the restoration works and the materials used.


text:
Students STEAM into Wakanda
LEAD summer camp bases curriculum, life lessons on 'Black Panther'
ERYN DION
edion@pcnh.com
PANAMA CITY — Action, intrigue, futuristic technologies and the struggle between good and evil.
No, it's not just the latest Marvel superhero blockbuster to hit movie theaters — it's also a middle school summer camp hosted by the LEAD Coalition and sponsored by Gulf Coast State College at Everitt and Jinks middle schools.
Heading into their third year of hosting a STEAM (Science, Technology, Engineering, Arts and Math) summer camp for sixth-, seventh- and eighth-graders, the camp this year is taking a more cinematic turn, basing its curriculum around the Marvel hit "Black Panther" and the fictional African civilization of Wakanda.
In the movie, the Wakandans used advanced technology to keep themselves hidden from the rest of the world. While much of it is light years ahead of what is available today, LEAD Coalition Executive Director Janice Lucas said there are real world lessons that can be drawn from the film. Throughout the summer, students will learn to make holograms, pilot underwater robots and even watch drone demonstrations at the local Navy base. They will hear about the geography of real-life ancient African civilizations and use literary skills to make connections between what they see in the film and the world around them.
On Wednesday, the 100-plus students from both schools got their first taste of the new lessons, piling into the Regency theater for a special screening of the film.
"It's such a positive movie, and the kids are already excited about the Marvel comics," Lucas said. "And then we're going to sneak some science in there as a result of that."
But beyond the academic lessons, Lucas said the movie models some of the conflicts raging inside every one of the students. Middle school can be a tumultuous time, mirrored by king of the Wakandans struggling to be kind but remain a leader, or deciding whether to keep Wakanda secluded or help the descendants of Africans all around the world who are suffering.
"They're wrestling with this as middle-schoolers," Lucas said. "How are they going to behave? How do you handle conflict? Do you just beat people up or do you work it out?"
The abundance of strong female characters, paired with African and African American cast members in prominent, hero-type roles, resonates with the summer camp's students, Lucas said, giving them examples they can look up to, something very different from how they are normally portrayed in films.
For Lucas, this summer's focus is creativity and showing students who they are and who they can become in the world, giving them tools to take care of themselves — babysitting certifications, cooking classes, martial arts and Girl Scouts — and exposing them to the many opportunities available to them in the local community. At the end of the camp, they also will get a chance to show off what they have learned in an expo at the A.D. Harris Learning Center.
"If we can help kids in sixth, seventh and eighth grade make up their mind that they're aware of and can get the better-paying jobs in our community, can create jobs in our community, then we will have helped them not only stay out of trouble for the summer, but get a better direction for their lives," Lucas said.


Summer with Girlstuff Bloggers' Elite Summer Collection | Vanity and Everything in Between.
gorgeous shades of nail polish. Girlstuff has tons of shades to go gaga about!
Two weeks ago, Girlstuff had their launch for their Bloggers Elite Summer Collection. I couldn't let this even pass because I love Girlsuff nail polish! I was sold right away the moment I saw the unique shape of its bottle!
Isn't this candy bar so cute?
On the side is their nail bar!
I like the pink and beige nail polish in the middle!
Merienda is served! I love Stacey's food because it's so tasty.
Too bad I had late lunch.
Everything nail related can be found on a Girlstuff kiosk. Aside from their wide range of shades of nail polish, they also have the trinkets for nail art. I don't know what it's called exactly but its those thingamajig that you put on your nail like a sticker.
I'll have a separate about Girlstuff nail polish per se. I'm sure females of all ages like nail polishes and if you haven't tried Girlstuff yet, you're missing out! It doesn't damage our nails like other nail polishes do (more of this on my review). Plus, it's very affordable. Their nail polish ranges from PhP50-PhP150 only with the fact that this is made in France!
Ms. Janina was kind enough to give all of us a token of appreciation for attending Girlstuff's launch. All the girls went home with Girlstuff Bloggers' elite summer collection nail polish and a little drawer that we can put on our vanity. Happy that it's white!
nice shades youve got here ms gem! perfect sa summer! i want to see a nail swatch from your posts soon =) pink.. blue.. yellow and green all for the win!
Looks like you had a lovely day! Can't get over how pretty everything looks in your pictures :) Love the prezzie you got at the end as well - gorgeous!


text:
Bulletin #7 : Access to Telecommunications
By United States Access Board
File Size: 215.45 KB.
Title: Bulletin #7 : Access to Telecommunications
Author: United States Access Board
Subject: Government publications, Access Board, Architectural and Transportation Barriers Compliance Board
Collections: U.S. Access Board Collection
Publisher: United States Access Board
Access Board, U. S. (n.d.). Bulletin #7 : Access to Telecommunications. Retrieved from http://www.nationalpubliclibrary.com/
Government Reference Publication
Excerpt: The Law Section 255. As technology continues to improve our means of telecommunication, it can pose challenges to accessibility on one hand, while on the other hold the key to innovative access solutions. Section 255 of the Telecommunications Act requires telecommunications products and services to be accessible to people with disabilities. This is required to the extent access is readily achievable, meaning easily accomplishable, without much difficulty or expense. If manufacturers cannot make their products accessible then they must design products to be compatible with adaptive equipment used by people with disabilities, where readily achievable. What is readily achievable will be different for each manufacturer based on the costs of making products accessible or compatible and their resources. What?s Covered Telecommunications products covered include...
Accessible Pedestrian Signals (by Bentzen, Billie Louise)
Accessible Swimming Pools and Spas (by United States Access Board)
Accessible Sports Facilities (by United States Access Board)
Accessible Miniature Golf (by United States Access Board)
Accessible Golf Courses (by United States Access Board)
High-Speed Rail Cars, Monorails and Syst... (by United States Access Board)
Rapid Rail Vehicles & Systems Technical ... (by United States Access Board)
Intercity Rail Cars and Systems Technica... (by United States Access Board)


Because artistry is not limited to professional artists.
(Spot.ph) Those cool graphics on the Google home page? You know, the ones that change everyday depending on the weather or the holiday? You can have your artwork on display, too. Google is offering the Filipino youth to showcase their creativity on their page with Doodle 4 Google.
Before and after. Photos from WeeWillDoodle's Facebook page.
With a "What can I do for the Philippines?" theme, Doodle 4 Google is coming to the Philippines with the goal of igniting the spirit of selfessness, service, and national pride among the Filipino youth.
The competition is open to students, aged five to 17, currently enrolled in public or private schools. Marking another first in its history, Google has also opened the competition to those enlisted in the Department of Education's alternative learning system.
Google doodles for Independence Day in 2012 and 2014.
Doodles will be judged in the following age groups: 1) five to eight years old, 2) nine to 11 years old, 3) 12 to 14 years old, and 4) 15 to 17 years old. The winners per age group will each receive P5,000 worth of gift certificates from National Book Store, a 3D doodling pen, and a Nexus 7 tablet.
The overall winning Doodle will be featured on Google Philippines' homepage for 24 hours, apart from receiving a Google trophy, P400,000 educational grant from BPI Foundation at any school or college in the Philippines, an art kit from National Book Store, an Acer C720 Chromebook, and a P350,000 connectivity grant from PLDT-Smart Foundation for the school or learning center of the winner. Deadline is September 19.
For more details about the Doodle 4 Google competition, visit google.com.ph/googlefordoodle.