text:
\section*{Appendices}
\section{Topological Features Calculation}
\label{app:top_algorithm}
\begin{algorithm}
	\caption{Topological Features Calculation}
    \label{our_alg}
	\begin{algorithmic}[2]
    	\smallskip
	    \Require {Text sample $s$}
	    \Require {Set of chosen attention heads $H_M$ of model $M$}
	    \Require {Thresholds array $T$}
	    \Require{Topological feature $f$ of unweighted graph}
	    \smallskip
	    \Ensure {Features array $Features$}
		\medskip
		\Procedure{features\_calc}{$s, H_M, T$}
	        \ForAll {$h \in H_M, t \in T$}
		        \State Calculate attention graph $\mathsf{\Gamma^{h}_s} = (V, E, W^{att}_{h, s})$ on sample $s$ on head $h$
		            \State $E^h_s(t) \leftarrow \{e \in E(\mathsf{\Gamma^{h}_s}):$ $ W^{att}_{h, s}(e) \geq t\}$
    				\State $\Gamma^{h}_s(t) \leftarrow (V, E^h_s(t))$

    				\State $\overline{E^h_s(t)} \leftarrow \left\{ \{i, j\} : (i, j) \in E^h_s(t) \right\} $
    				\State $\overline{\Gamma^h_s(t)} \leftarrow (V, \overline{E^h_s(t))}$
    				\State Calculate
    				$f(\overline{\Gamma^h_s(t)})$

    			\EndFor
			\State $Features \leftarrow \left[ f(\overline{\Gamma^h_s(t)})\right]_{t \in T}^{h \in H_M}$
    		\State \textbf{return} $Features$
    	\EndProcedure
	\end{algorithmic}
\end{algorithm}

\section{Persistent Homology and Betti Numbers}
\label{app:betti_numbers}
Recall that a simplicial complex $K$ is a collection of subsets of a finite set called {\em simplices} such that each subset of any element of $K$ also is an element of $K$; such subsets of a simplex are called {\em faces}.
In particular, an undirected graph is a simplicial complex where the simplices correspond to the edges and vertices of the graph.
The set of all formal $\mathbb{Z}$-linear combinations of the $p$-dimensional  simplices (that is, $(p+1)$-element finite sets from the collection) of $K$  is denoted $\mathsf{C}_{p}(K)$.  These linear combinations $c =\sum_{j} \gamma_{j}\sigma_{j}$ are called {\em $p$-chains}, where the $\gamma_j \in \mathbb{Z}$ and the $\sigma_{j}$ are $p$-simplices in $K$.

The boundary, $\partial(\sigma_j)$, is the formal sum of the $(p-1)$-dimensional faces of $\sigma_j$ and the boundary of the chain is obtained by extending $\partial$ linearly,
\begin{equation}\nonumber
\partial(c) = \sum_{j} \gamma_j\partial(\sigma_j)
\end{equation}
for $c$ as above.

The $p$-chains that have boundary $0$ are called $p$-{\em cycles}, they form a subgroup
$\mathsf{Z}_{p}(K)$ of $\mathsf{C}_{p}(K)$.
The $p$-chains that are the boundary of $(p + 1)$-chains are called $p$-boundaries and form a subgroup $\mathsf{B}_{p}(K)$ of $\mathsf{C}_{p}(K)$.
The quotient group $\mathsf{H}_p(K) = \mathsf{C}_{p}(K) / \mathsf{B}_{p}(K)$ is called the $p$-th {\em homology} of $K$.
Their ranks $\beta_p = rk (\mathsf{H}_p(K))$ of these abelian groups are called {\em Betti numbers}. The homology and the Betti numbers
are classical topological invariants of $K$.

In particular, a graph $G = (E,V)$ contains  0-dimensional and 1-dimensional faces. It follows that its topological form is essentially described by the numbers $\beta_0$ and $\beta_1$, which are the only nonzero Betti numbers. Here $\beta_0$ is the number of connected components of $G$, and $\beta_1$ is the number of independent cycles of the graph (which is equal to $|E|-|V|+\beta_0$).

A {\em subcomplex} of $K$ is a subset of simplices that is closed under the face relation. A {\em filtration} of $K$ is a nested sequence of subcomplexes that starts with the empty complex and ends with the complete complex,
\begin{equation}\nonumber
\emptyset  \subset K_{t_1} \subset K_{t_2}   \subset \cdots \subset K_{t_m} = K.
\end{equation}
In particular, to any weighted undirected graph
$G = (V,E)$
one can associate naturally the filtration
\begin{equation}
\label{eq:graph_filtration}
\emptyset   \subset G_{t_1} \subset \dots \subset G_{t_m} = G,
\end{equation}
where $\{t_i\}$ is the set of all weights of the edges, $G_{t_i} = (V,E_{t_i})$ and $E_{t_i}$ consists of all edges of $E$ with weight less or equal to $t_i$.

In our calculations, the increasing filtration is obtained by reversing the attention matrix weights: $w\mapsto 1-w$.

The $p$-th persistent
homology of $K$ is the pair of sets of vector spaces $\{
\mathsf{H}_p(K_{t_i}) | 0\le i\le l \}$
and maps
$\{
f_{i,j}: \mathsf{H}_p(K_{t_i}) \to
\mathsf{H}_p(K_{t_j})| 1\le i < j \le l
\}$, where the maps are
induced by the inclusion maps $K_{t_i}\to K_{t_j}$.

Each persistent homology class $\alpha$ in this sequence is ``born'' at some $K_{t_i}$ and ``dies'' at some $K_{t_j}$ or never dies \citep{barannikov2021canonical}. One can visualize this as an interval $[t_i,t_j]$ or $[t_i,+\infty[$. The collection of all such intervals is called the {\em barcode} of the filtration. It is the most useful invariant of the filtration. Note that the information about the persistent homology classes is generally essential to calculate the barcode, whereas the information about the Betti numbers only is insufficient.

In the case of the filtration associated to a weighted graph~(\ref{eq:graph_filtration}),
the $H_0-$th barcode (respectively, $H_1-$th) consists of the intervals of the form $[0, d_i]$ (resp., $[s_i, \infty[$) only.
Given a number $l$, the number of intervals of length at most $l$ for $H_0$
(respectively, the number of intervals with the left endpoint at most $l$) is therefore equal to the
the Betti number $\beta_0 (K_l)$ (resp., $\beta_1 (K_l)$). We see that in this case, we can recover the barcode from the collection of all Betti numbers $\beta_i (K_{t_j})$  where the thresholds set $\{t_j\}$ is the set of all weights of the edges in the graph. On the other hand, a calculation of the $H_0-$barcode gives all Betti numbers $\beta_i (K_{t_j})$ simultaneously for all thresholds.

In TDA, the following filtered simplicial complex is also commonly associated with a graph. The $n$-dimensional simplices of
the {\em clique complex} (or Whitney complex) $X = X (G)$ of a graph $G = (E,V)$  are the $(n+1)$-cliques of $G$, that is, complete subgraphs with $n+1$ vertices. For example, its $0$-simplices are the vertices, the 1-simplices are the edges, and the 2-simplices are the triangles. The clique complex has a richer topological structure than the graph itself since it may have nonzero Betti numbers $\beta_n (X)$ for $n\ge 2$.
Note that the $H_0-$barcode of the filtered clique complex coincides with the $H_0-$barcode of ~(\ref{eq:graph_filtration}).
Several software packages are aimed at calculating
the persistent homology of graph clique complex. For this purpose, we have used  Ripser++~\cite{bauer2019ripser,zhang2020gpu}.

The entropy of the barcode is a measure of the entropy of the points in a persistence diagram. Precisely, if we have a barcode as a list of pairs of ``birth'', ``death'': $D = \{(b_i, d_i)\}_{i \in I}$, then the entropy is defined as: $$E(D) = - \sum_{i \in I} p_i \log (p_i)$$ where $p_i = \frac{d_i - b_i}{L_D}$, and $L_D = \sum\limits_{i \in I} (d_i - b_i)$.

Each bar in $H_0-$barcode has the form $[0, d_i]$ where $1-d_i$ is the attention weight of the edge which ``kills'' the  connected component corresponding to this bar.
The sum of lengths of bars in the $H_0-$barcode coincides  with $128-M$ where $M$ is the sum of edge weights of the attention graph maximal spanning tree.

\begin{figure}[t!]
  \includesvg[width=\linewidth]{images/appendix_dist.svg}
  \caption{The distribution shift of the sum of the bars in $H_0$ between the generated and natural texts for different layers and heads. Top row: (Layer: 7; Head: 5), (Layer: 8; Head: 10); middle row: (Layer: 9; Head: 11), (Layer: 9; Head: 7); down row: (Layer: 0; Head: 3), (Layer: 5; Head: 6)}
  \label{dist_plot}
\end{figure}

\section{Training Details}
\label{app:train_details}
\vspace{0.5em}\noindent \textbf{Topological Features and TF-IDF} Similar to \citep{solaiman2019release}, the training of the linear classifiers over the topological and count-based features is run with the regularization parameter $L^2$ $\in$ $[1\emph{e}^{-5}, 5\emph{e}^{-5}, \dots, 0.1, 0.5, 1]$ and the maximum number of iterations $max_{iter} \in [1, 2, 3, 5, 10, 100]$ tuned on the validation set.

\vspace{0.5em}\noindent \textbf{BERT-based} classifiers are trained  with the linear scheduler with the initial learning rate $lr \in [1e^{-5}, 5e^{-5}, 1e^{-4}, \dots , 1e^{-1}, 1]$ and epochs number $e \in [2, 3, 5, 10, 15, 20]$.

\section{Reproducibility checklist}
For the task of generated texts detection, we used logistic regression as the classification model for all of our features. We took the features calculated for each head and concatenated them, thus the total number of features for each method was equal to $12 \times 12 \times \text{number of features per method}$.

To calculate the features itself, we used 12 Gb GeForce GTX 1080 GPU and 40 CPUs. We also used scaling to increase the speed of fitting the model.

For methods, we searched for the best hyperparameters of logistic regression on validation set. The best hyperparameters we found were $C=0.001$ with $L_2$ regularization, trained with for maximum iterations of 250. The best validation score was 87.9\% accuracy.

For BERT baseline training, \textbf{BERT [CLS]} was trained for 20 epochs, and \textbf{BERT [Fully trained]} was trained for 2 epochs. We used early stopping controlled by the accuracy on validation set.

\section{Introduction}
Recent text generation models (TGMs) based on the transformer architecture \citep{vaswani2017attention} have demonstrated impressive capabilities of creating texts which are very close to human in terms of fluency, coherence, grammatical and factual correctness \citep{keskar2019ctrl,zellers2019defending,yang2019xlnet}. Extensive GPT-style TGMs~\citep{gpt2paper} have achieved outstanding results over a great scope of NLP tasks employing zero-shot, one-shot, and few-shot techniques, even outperforming state-of-the-art fine-tuning approaches \citep{brown2020language}. However, such models can be misused for generating fake news \citep{zellers2019defending,uchendu-etal-2020-authorship}, product reviews \citep{adelani2020generating}, and even extremist and abusive content \citep{mcguffie2020radicalization}.

Many attempts have been made to develop artificial text detectors \citep{jawahar2020automatic}, ranging from classical ML methods over count-based features \citep{uchendu2019characterizing} to advanced transformer-based models \citep{adelani2020generating} and unsupervised approaches \citep{solaiman2019release}. Despite the prominent performance of these methods across various domains, they still lack interpretability and robustness towards unseen models.

This paper introduces a novel method for artificial text detection based on Topological Data Analysis (TDA) which has been understudied in the field of NLP. The motivation behind this approach relies on the fact that (i) the attention maps generated by the transformer model can be represented as weighted bipartite graphs and thus can be efficiently investigated with TDA, (ii) TDA methods are known to capture well surface and structural patterns in data which, we believe, are crucial to the task.

The contributions are summarized as follows. (i) To the best of our knowledge, this work is the first attempt to apply TDA methods over the transformer model's attention maps and interpret topological features for the NLP field. (ii) We propose three types of interpretable topological features derived from the attention graphs for the task of artificial text detection. We empirically show that a simple linear classifier trained on the TDA features produced over BERT attentions~\citep{devlin2019bert} outperforms count- and neural-based baselines up to 10\%, and can perform on par with the fully fine-tuned BERT model across three domains: social media, news articles and product reviews. (iii) Testing the robustness towards unseen TGMs, we find that the TDA-based classifiers tend to be more robust as opposed to the existing detectors. (iv) The probing analysis of the features demonstrates their sensitivity to surface and syntactic properties. (v) Finally, we are publicly releasing the code\footnote{\url{https://github.com/danchern97/tda4atd}}, hoping to facilitate the applicability of the TDA methods to other NLP tasks, specifically the ones that incorporate structural information.

\section{Related Work}
\paragraph {Applications of Topological Data Analysis}
TDA has been applied in NLP to study textual structural properties, independent of their surface and semantic peculiarities. These applications include detection of children and adolescent writing \citep{zhu2013persistent}, discourse and entailment in law documents \citep{savle-etal-2019-topological}, and exploring discourse properties of the plot summary to identify the movie genre \citep{doshi2018movie}. \citet{guan2016collapse} apply the topologically motivated transformation of the document's semantic graph to summarize it further. However, these studies neither incorporate neural data representations nor explore the properties of neural language models.

The research in the emerging scope of TDA applications to neural networks and neural data representations has mainly focused on artificial datasets or common problems in computer vision. The desired topological properties of the data representation can be incorporated into the objective function during the training of a neural network, improving its robustness and performance on the downstream tasks such as human action recognition and image classification \citep{som2020pinet}, image simplification \citep{solomon2021fast}, image segmentation \citep{clough2020topological} or generation \citep{bruel2019topology}. Another line aims to develop the topological criteria of the network's generalization properties \cite{rieck2018neural,corneanu2020computing, naitzat2020topology, barannikov2020topological} or its robustness to adversarial attacks \citep{corneanu2019does}.

\paragraph{Exploring Attention Maps}
Several studies have shown that attention maps of pre-trained language models (LMs) capture linguistic information. For the sake of space, we will discuss only a few well-known recent works. \citet{clark2019does} attempt to categorize the types of attention patterns observed in the BERT model. In particular, they discover certain attention heads in which prepositions attend to their objects or coreferent mentions attend to their antecedents. Further, they explore the typical behavior of the attention heads and introduce five patterns, e.g. attending to the next token or previous token, which the vast majority of the attention heads follow.
\citet{htut2019attention} explore the syntactic information encoded in intra-word relation in the attention maps. A maximum spanning tree (MST) is constructed from the computed attention weights and mapped to the corresponding dependency tree for a given sentence. This method achieves a prominent Undirected Unlabeled Attachment Score (UUAS), indicating that the attention graphs indeed can capture the dependency-based relations.
\citet{NEURIPS2019_2c601ad9} explore the importance of the attention heads with respect to a downstream task. They show that a large proportion of the attention heads can be pruned without harming the model downstream performance. Beneficially, the pruned model speeds up at the inference time.
Finally, visualization of the attention maps \cite{exbert} allows introspecting the model's inner workings interactively.

\paragraph{Supervised Artificial Text  Detectors} Several well-established classical ML methods have been applied to the task of artificial text detection combined with topic modeling and linguistic features \citep{manjavacas2017assessing,uchendu2019characterizing,uchendu-etal-2020-authorship}. The rise of pre-trained LMs has stimulated various improvements of the detectors. The RoBERTa model \citep{liu2019roberta} has demonstrated an outstanding performance with respect to many TGMs and domains \citep{adelani2020generating,fagni2020tweepfake}. The capabilities of generative models such as GROVER \citep{zellers2019defending} and GPT-2 \citep{radford2019language} have been also evaluated on the task \citep{bahri2020generative}. Last but not least, \citet{bakhtin2019real} discriminate artificial texts by training a ranking energy-based model over the outputs of a pre-trained LM.

\paragraph{Unsupervised Artificial Text Detectors} Another line of methods incorporates probability-based measures combined with a set of pre-defined thresholds \citep{solaiman2019release}. Such methods open up a possibility of the \emph{human in the loop} approach where a human makes decisions with the help of pre-trained LMs \citep{ippolito-etal-2020-automatic}. The GLTR tool \citep{gehrmann-etal-2019-gltr} supports human-model interaction by visualizing the properties of a text inferred by the model, which improves the human detection rate of artificial texts. A promising direction is involving acceptability and pseudo-perplexity metrics \citep{lau-etal-2020-furiously,salazar-etal-2020-masked} that can be used to evaluate text plausibility.

\section{Background}
\subsection{BERT Model}
BERT is a transformer-based LM that has pushed state-of-the-art results in many NLP tasks. The BERT architecture comprises $L$ encoder layers with $H$ attention heads in each layer. The input of each attention head is a matrix $X$ consisting of the $d$-dimensional representations (row-wise) of $m$ tokens, so that $X$ is of shape $m \times d$. The head outputs an updated representation matrix  $X^\mathrm{out}$:

\begin{equation}
\begin{split}
X^\mathrm{out} & = W^\mathrm{attn}(XW^{\mathrm V}) \\
 \mbox{ with }
 W^\mathrm{attn} & = \mathrm{softmax}\left(\frac{(XW^{\mathrm Q})(XW^\mathrm{K})^\mathrm{T}}{\sqrt{d}}\right),
\label{eqn:attention}
\end{split}
\end{equation}
where $W^\mathrm{Q}$, $W^\mathrm{K}$, $W^\mathrm{V}$ are trained projection matrices of shape $d \times d$ and $W^\mathrm{attn}$ is of shape $m \times m$  matrix of attention weights. Each element $w^\mathrm{attn}_{ij}$ can be interpreted as a weight of the $j$-th input's \emph{relation} to the $i$-th output: larger weights mean stronger connection between the two tokens.

\subsection{Attention Map and Attention Graph}
An {\it attention map} displays an attention matrix $W^{attn}$ (Equation \ref{eqn:attention}) in form of a heat map, where  the color of the cell $(i,j)$ represents the \emph{relation} of the $i$-th token to the output representation of the $j$-th token. We use a {\it graph representation} of the attention matrix. The attention matrix is considered to be a weighted graph with the vertices representing tokens and the edges connecting pairs of tokens with strong enough mutual relation (the higher the weight, the stronger the relation). The construction of such graph appears to be quite problematic: a threshold needs to be set to distinguish between weak and strong relations. This leads to instability of the graph's structure: changing the threshold affects the graph properties such as the number of edges, connected components, cycles. The choice of the optimal thresholds is essential to define which edges remain in the graph. TDA methods allow extracting the overall graph's properties which describe the development of the graph with respect to changes in the threshold.

\subsection{Topological Data Analysis}
TDA instruments permit tracking the changes of a topological structure across varying thresholds for different objects: scalar functions, point clouds, and weighted graph~\cite{chazal2017introduction}. Given a set of tokens $V$ and an attention matrix of pair-wise weights $W$, we build a family of graphs termed as {\it filtration}: an ordered set of graphs for the sequence of increasing thresholds. Figure \ref{fig:M1} depicts the filtration for a toy example. First, we build a graph for a small threshold, using which we filter out the edges with the weights lower than this threshold. Next, we increase the threshold and construct the next graph. Then we compute the core topological features of different dimensions: for $d=0$ these are connected components, for $d=1$ -- ``loops'' (loosely speaking, they corresponds to basic cycles in a graph), and $d$-dimensional ``holes'' for higher dimensions. The amounts of these features at each dimension $\beta_0, \beta_1,...,\beta_d$  are referred to as {\it Betti numbers} and serve as the main invariants of the objects in topology (see Appendix~\ref{app:betti_numbers} for formal definitions).  While the threshold is increasing and the edges are being filtered, new features may arise. For example, the graph can decay into several connected components. At the same time, the features can also disappear when a cycle is broken. For each feature, we check the moment in the filtration when it appears (i.e., its ``birth'') and when it disappears (i.e., its ``death''). These moments are depicted on a diagram called {\it barcode} (see Figure \ref{fig:M1}). The barcode's horizontal axis corresponds to the sequence of thresholds. Each horizontal line (``bar'') corresponds to a single feature (``hole''): the line lasts from the feature's ``birth'' to the feature's ``death''. Barcodes characterize the ``persistent'' topological properties of the graph, showing how stable topological features are.

\input{./images/graph_example.tex}

We now detail building the attention graphs, the filtration procedure, and the proposed features which are derived from the attention graphs.

\section{Persistent Features of the Attention Graphs} \label{features}

\paragraph{Informal Definition and Interpretation}
We extract three groups of features from the attention graphs.
{\bf Topological features} (Section \ref{topological_features}) include a set of standard graph properties: the number of connected components, the number of edges, and the number of cycles. These features are calculated for each pre-defined threshold separately and then concatenated. We consider two following variants of the feature calculation: for a directed and an undirected attention graph. {\bf Barcode features} (Section \ref{barcode_features}) are extracted from barcodes.
{\bf Distance to patterns} (Section \ref{distance_features}) is the group of features derived from the attention maps by computing the distance to the attention patterns \citep{clark2019does}.

\vspace{-2pt}
To give the linguistic interpretation of our features, recall that the graph structures are used in lexicology for describing semantic change laws \citep{hamilton2016diachronic, lipka1990outline, arnold1973english}. The evolution of the meaning of a word with time can be represented as a graph, in which edges represent a semantic shift to different word meanings. Two typical patterns are distinguished in the graph structure: {\it radiation} -- the ``star'' structure, where the primary meaning is connected to other connotations independently; {\it concatenation}, or {\it chaining shift} -- the ``chain'' structure when the connotations are integrated one-by-one. Note that the typical attention patterns \citep{clark2019does} have the same ``radiation'' and ``concatenation'' structure. In pre-trained LMs, the evolution goes through the layers of the model, changing the representation of each token, ending up with highly contextualized token representations, and the aggregated representation of the whole sentence (in the form of the [CLS]-token).

We consider persistent features as the numerical characteristic of the semantic evolution processes in the attention heads. Topological features deal with clusters of mutual influence of the tokens in the sentence and the local structures like chains and cycles. The barcode features characterize the severity and robustness of the semantic changes. The features with long persistence (large distance between ``birth'' and ``death'') correspond to the stable processes which dominate the others, while short segments in the barcode define processes highly influenced by noise.
Pattern features provide a straightforward measure of the presence of typical processes over the whole sentence. The so-called ``vertical'' pattern corresponds to the ``radiation'' around the single token when the meaning of the sentence or a part of the sentence is aggregated from all words equally. ``Diagonal'' pattern represents consequent ``concatenation'' structure, going through all the sentence and thus reflecting the dependence of each token's meaning on its left context.

\subsection{Topological Features}
\label{topological_features}

First, we fix a set of thresholds $T = \{t_i\}_{i=1}^{k}, 0 < t_1 < ... < t_k < 1$. Consider an attention head $h$ and corresponding weights $W^\mathrm{attn} = (w^{attn}_{i, j})$. Given a text sample $s$, for each threshold level $t \in T$ we define the weighted directed graph $\Gamma^{h}_s(t)$ with edges $\{j \to i \mid w^\mathrm{attn}_{ij} \ge t\}$ and its undirected variant  $\overline{\Gamma^{h}_s(t)}$ by setting an undirected edge $v_iv_j$ for each pair of vertices $v_i$ and $v_j$ which are connected by an edge in at least one direction in the graph $\Gamma^{h}_s(t)$.

We consider the following features of the graphs:
\begin{itemize}[topsep=0pt,itemsep=-1ex,partopsep=1ex,parsep=1ex]
\item the first two Betti numbers of the undirected graph $\overline{\Gamma^{h}_s(t)}$. The feature calculation procedure is described in Appendix \ref{app:top_algorithm};
\item  the number of edges ($\mathbf{e}$), the number of strongly connected components ($\mathbf{s}$) and the amount of simple directed cycles ($\mathbf{c}$) in the directed graph $\Gamma^{h}_s(t)$.
\end{itemize}

To get the whole set of topological features for the given text sample $s$ and the attention head $h$, we concatenate the features for all the thresholds, starting from~$T$.

\subsection{Features Derived from Barcodes}
\label{barcode_features}
For each text sample we calculate barcodes of the first two persistent homology groups (denoted as $H_0$ and $H_1$) on each attention head of the BERT model (see Appendix~\ref{app:betti_numbers} for further details). We compute the following characteristics of these barcodes:
\begin{itemize}[topsep=0pt,itemsep=-1ex,partopsep=1ex,parsep=1ex]
    \item
    The sum of lengths of bars;
    \item

    The mean of lengths of bars;
    \item
    The variance of lengths of bars;
    \item
    The number of bars with time of birth/death greater/lower than threshold;
    \item
    The time of birth/death of the longest bar (excluding infinite);
    \item
    The overall number of bars;
    \item
    The entropy of the barcode.
\end{itemize}

\begin{table*}[!ht]
\centering
\begin{threeparttable}
\resizebox{\textwidth}{!}{
\begin{tabular}{cccccccccccc}
\toprule
\multicolumn{2}{c}{\textbf{Text Source}} & \multicolumn{2}{c}{\textbf{Train}} & \multicolumn{2}{c}{\textbf{Validation}} & \multicolumn{2}{c}{\textbf{Test}} &
\multicolumn{2}{c}{\textbf{$|$Vocab$|$}} &
\multicolumn{2}{c}{\textbf{Length}}
\\ \cmidrule{3-12}

\multicolumn{2}{l}{} & \textbf{H} & \textbf{M} & \textbf{H} & \textbf{M} & \textbf{H} & \textbf{M} &
\textbf{H} & \textbf{M} & \textbf{H} & \textbf{M}
\\ \midrule
WebText & \makecell{GPT-2 Small;\\ pure sampling} & 20K & 20K & 2.5K & 2.5K & 2.5K & 2.5K  & 220K & 532K & 593 \tiny{$\pm$ 177} & 515 \tiny{$\pm$ 322} \\
\makecell{Amazon \\ Review} & \makecell{GPT-2 XL\\ pure sampling} & 5K & 5K & 1K & 1K & 4K & 4K & 47K & 49K & 179 \tiny{$\pm$ 170} & 177 \tiny{$\pm$ 171} \\
RealNews &  \makecell{GROVER \\ top-$p$ sampling} & 5K & 5K & 1K & 1K & 4K & 4K & 98K & 75K & 721 \tiny{$\pm$ 636} & 519 \tiny{$\pm$ 203} \\
\bottomrule
\end{tabular}
}
\caption{Statistics for the datasets used in the experiments on the artificial text detection task. \textbf{H}=Human; \textbf{M}=Machine.}
\label{tab:dataset_stata}
\end{threeparttable}
\end{table*}

\subsection{Features Based on Distance to Patterns}
\label{distance_features}
The shape of attention graphs in distinct attention heads can be divided into several patterns \cite{clark2019does}. We hypothesize that appearance of such patterns in a particular head or ``intensity'' of the pattern (i.e., the threshold $t$ on which the pattern appears) may carry essential linguistic information. Thus, we formalize these attention patterns and calculate the distances to them as follows.

Let $A = (a_{ij})$ be an incidence matrix of the graph $\Gamma$ with $n$ vertices, where $a_{ij} = 1$ for all edges $(ij) \in E$ and $0$ for all other $i, j$.
Let $\Gamma = (V, E)$ and $\Gamma' = (V, E')$ be two graphs with the same set of vertices, and let $A$, $A'$ be their incidence matrices.
As a distance $d$ between such graphs we use Frobenius norm of the difference
$|| A - A'||_F = \sqrt{\sum_{i,j} (a_{ij}-a'_{ij})^2},$ normalized by the norms of the matrices of compared graphs:
$$
    d(\Gamma, \Gamma') = \frac{|| A - A'||_F}{\sqrt{||A||_F^2 + ||A'||_F^2}} $$
$$
    = \sqrt{\frac{\sum_{i,j} {(a_{ij}-a'_{ij})}^2}{\sum_{i,j} (a_{ij}^2 + {a'_{ij}}^2)}}.
$$
Such distance takes values between 0 and 1. For the unweighted graphs we have:
$$
d(\Gamma, \Gamma') = \sqrt{\frac{|E \triangle E'|}{|E|+|E'|}},
$$
where $E \triangle E' = (E \backslash E') \bigcup (E' \backslash E)$ is the symmetric difference of sets $E$ and $E'$.

We consider distances from the given graph $\Gamma$ to attention patterns $\Gamma_i$ as the graph features $ d_i (\Gamma ) = d(\Gamma, \Gamma_i)$, and the patterns posed by \citep{clark2019does}:

\begin{itemize}[topsep=0pt,itemsep=-1ex,partopsep=1ex,parsep=1ex]
    \item Attention to the previous token.    $\Gamma_{feature}: E = (i+1,i)$, $i=\overline{1, n-1}$.

    \item Attention to the next token.
    $\Gamma_{feature}: E = (i,i+1)$, $i=\overline{1, n-1}$.

    \item Attention to [CLS]-token.
    [CLS]-token corresponds to the vertex 1 of the set $V = [1,n]$ as it denotes the beginning of the text.
    $ \Gamma_{feature}: E = (i,1)$, $i=\overline{1, n}$.

    \item  Attention to [SEP]-token.
    Suppose
    $i_1,  \dots, i_k$ are the indices of [SEP]-tokens. Then $\Gamma_{feature}: E = (i,i_t)$, $i= \overline{1,n}$, $t = \overline{1,k}$.

    \item Attention to punctuation marks.
    Let $i_1, \dots, i_k$ be the indices of the tokens which correspond to commas and periods. $\Gamma_{feature}: E = (i,i_t)$,
    $i= \overline{1,n}$, $t = \overline{1,k}$. Note that this pattern can be potentially divided into Attention to commas and Attention to periods.

\end{itemize}

\section{Experiments}
\label{experimentssection}
\subsection{Artificial Text Detection}
\label{detection-task}
\paragraph{Data}
We prepare three datasets from different domains to conduct the experiments on the task of artificial text detection. Table \ref{tab:dataset_stata} outlines statistics for the datasets. Each split is balanced by the number of samples\footnote{Each sample is truncated to 128 BertTokenizer tokens (\texttt{bert-base-uncased}).
} per each target class.

\vspace{0.5em}\noindent \textbf{WebText \& GPT-2} comprises a subset of natural and generated texts from the datasets proposed by \citet{gpt2paper}. (i) \textbf{WebText} contains filtered and de-duplicated natural texts from Reddit; (ii) \textbf{GPT-2 Output Dataset}\footnote{\url{https://github.com/openai/gpt-2-output-dataset}} includes texts generated by various versions of the GPT-2 model fine-tuned on \textbf{WebText}. We use texts generated by GPT-2 Small (117M) with pure sampling.

\vspace{0.5em}\noindent \textbf{Amazon Reviews \& GPT-2} consists of a subset of Amazon product reviews \citep{amazon_review} and texts generated by GPT-2 XL (1542M) with pure sampling, fine-tuned on this dataset \citep{solaiman2019release}.

\vspace{0.5em}\noindent \textbf{RealNews \& GROVER} ~\citep{zellers2019defending} includes a subset of the news articles from RealNews (that are not present in the GROVER training data) and news articles generated by GROVER with top-$p$ sampling.

\begin{table*}[t!]
\centering
\small
\begin{tabular}{lccc}
 \toprule
  \textbf{Model} & \makecell{\textbf{WebText \&} \\ \textbf{GPT-2 Small}} & \makecell{\textbf{Amazon Reviews \&} \\ \textbf{GPT-2 XL}} & \makecell{\textbf{RealNews} \& \\ \textbf{GROVER}} \\ [0ex]
 \midrule

 \textbf{TF-IDF, N-grams} & 68.1 & 54.2 & 56.9 \\

 \textbf{BERT [CLS trained]} & 77.4 & 54.4 & 53.8 \\

 \textbf{BERT [Fully trained]} & \textbf{88.7} & \textbf{60.1} & \textbf{62.9} \\

 \textbf{BERT [SLOR]} & 78.8 & 59.3 & 53.0 \\

 \midrule

 \textbf{Topological features} & 86.9 & 59.6 & 63.0 \\

 \textbf{Barcode features} & 84.2 & 60.3 & 61.5 \\

 \textbf{Distance to patterns} & 85.4 & 61.0 & 62.3 \\
 \midrule
 \textbf{All features} & \textbf{87.7} & \textbf{61.1} & \textbf{63.6} \\
\bottomrule
\end{tabular}
\smallskip
\caption{The results of the artificial text detection experiments. The performance is measured by the accuracy score~(\%).
}
\label{table_results}
\end{table*}

\paragraph{Baselines}
We use \texttt{bert-base-uncased}\footnote{\url{https://huggingface.co/bert-base-uncased}} model from the HuggingFace library \citep{wolf2019huggingface} for the BERT-based baselines described below. (i)~\textbf{BERT [CLS trained]} is a linear layer trained over [CLS]-pooled text representations. Note that the weights of the BERT model remain frozen. (ii) \textbf{BERT [Fully trained]} is a fully fine-tuned BERT model. We also train Logistic Regression classifier from scikit-learn library \citep{pedregosa2011scikit} over (iii) \textbf{TF-IDF, N-grams} with the N-gram range $\in [1, 2]$ and (iv) \textbf{BERT [SLOR]} \cite{pauls2012large}, an pseudo-perplexity-based acceptability measure inferred with the BERT model under the implementation by \citet{lau-etal-2020-furiously}\footnote{\url{https://github.com/jhlau/acceptability-prediction-in-context}}.

\paragraph{Models} We train Logistic Regression classifier over the persistent graph features derived from the attention matrices from the BERT model: (i) \textbf{Topological features} (Section \ref{topological_features}), (ii) \textbf{Barcode features} (Section \ref{barcode_features}) and (iii) \textbf{Distance to patterns} (Section \ref{distance_features}). (iv) \textbf{All features} is the concatenation of the features mentioned above. The training details for the baselines and models are outlined in Appendix \ref{app:train_details}.

\paragraph{Results} Table \ref{table_results} outlines the results of the artificial text detection experiments on the three datasets. Note the diversity of the experiment setting where the methods are tested with respect to the TGM, TGM's size, the decoding method, domain, and stylistic properties (texts from the \textbf{Amazon Reviews \& GPT-2} are shorter as compared to those of \textbf{WebText \& GPT-2} and \textbf{RealNews \& GROVER}). The overall tendency is that the proposed TDA-based classifiers outperform the count-based (\textbf{TF-IDF, N-grams}) and two BERT-based baselines (\textbf{BERT [CLS trained]}, \textbf{BERT [SLOR]}) up to 10\%. The concatenation of the features achieves the performance on par with the fully trained BERT model on all datasets.

\subsection{Robustness towards Unseen Models}
\label{robustness}
This setting tests the robustness of the artificial text detection methods towards unseen TGMs on the \textbf{WebText \& GPT-2} dataset. The baselines and models are trained on texts from the GPT-2 small model and further used to detect texts generated by unseen GPT-style models with pure sampling: GPT-2 Medium (345M), GPT-2 Large (762M) and GPT-2 XL (1542M). Note that such a setting is the most challenging as it requires the transfer from the smallest model to that of the higher number of parameters ~\citep{jawahar2020automatic}.

\paragraph{Results}
Figure \ref{quality-drop-plots} demonstrates the results on the robustness experimental setup. The simple linear classifier trained over the \textbf{Topological features} demonstrates the minor performance drop on the task of detecting artificial texts by the larger GPT-style models as opposed to the considered methods. However, the TDA-based classifier performs slightly worse than \textbf{BERT [Fully trained]} on the test subset by GPT-2 Small.

\begin{figure}[t!]
  \includegraphics[width=\linewidth]{images/quality-drop-plots-small.png}
  \caption{The results of the robustness experiments. X-axis=GPT-2 model size. Y-axis=Accuracy score.}
  \label{quality-drop-plots}
\end{figure}

\subsection{Attention Head-wise Probing}
\label{attention-probing}

\begin{figure*}[htp!]
    \centering
    \begin{subfigure}[b]{0.47\linewidth}
    \centering
    \includegraphics[width=\linewidth]{images/probe_length.png}
    \caption{\textsc{Length}}
    \label{fig:heatmap-length}
    \end{subfigure}
    \begin{subfigure}[b]{0.47\linewidth}
    \centering
    \includegraphics[width=\linewidth]{images/probe_depth.png}
    \caption{\textsc{Depth}}
    \label{fig:heatmap-depth}
    \end{subfigure}
    \caption{Heat maps of attention head-wise probing on \textsc{Length} (Left) and \textsc{Depth} (Right) tasks. Attentions=Frozen attention weights.
    X-axis=Head index number. Y-axis=Layer index number.
    The brighter the color, the higher the accuracy score for the attention head.
    }
    \label{fig:probe_heatmaps}
\end{figure*}

\paragraph{Data}
SentEval \citep{conneau2018you} is a common probing suite for exploring how various linguistic properties are encoded in the model representations. The probing tasks are organized by the type of the property: \emph{surface}, \emph{syntactic} and \emph{semantic}. We use the undersampled tasks\footnote{Each probing task is split into 25K/5K/5K train/validation/test sets. The sets are balanced by the number of instances per each target class.} to analyze what properties are stored in the topological features.

\paragraph{Method} Attention head-wise probing \citep{jo2020roles} allows investigating the patterns of how attention heads from each layer of the model contribute most to a probing task. Logistic Regression is trained over the intermediate outputs of the model $h_{i,j}$, where \emph{i} and \emph{j} denote the indices of the layer and the attention head. We use the publicly available code\footnote{\url{https://github.com/heartcored98/transformer_anatomy}} to train the classifier over two groups of the input features: (i) the intermediate outputs $h_{i,j}$ produced by the frozen BERT model and (ii) the topological features derived from $h_{i,j}$ as outlined in Sections \ref{topological_features}, \ref{barcode_features}. The performance is evaluated by the accuracy score, and the heat maps of the probing scores are constructed to introspect how a certain linguistic property is distributed across different layers and attention heads. Refer to \citet{jo2020roles} for more details.

\paragraph{Results} The results demonstrate that the topological features tend to be sensitive to the surface and syntactic properties as opposed to the semantic ones.
Figure \ref{fig:probe_heatmaps} shows heat maps of the attention head-wise evaluation on \textsc{Length} (Figure \ref{fig:heatmap-length}, \emph{surface} property) and \textsc{Depth} (Figure \ref{fig:heatmap-depth}, \emph{syntactic} property) tasks\footnote{\textsc{Length} is a 6-way classification task and \textsc{Depth} comprises 7 classes denoting the depth of a syntax tree.}. While the sentence length is distributed across the majority of the frozen attention heads, specifically at the lower-to-middle layers~$[1-8]$, the topological features capture the property at layer~$[1]$ and by fewer heads at layers~$[2, 4-5, 9-11]$. The depth of the syntax tree is encoded in the frozen heads at the lower-to-middle layers~$[1-5]$, whereas the barcode features predominantly localize the property at the middle-to-higher layers~$[5-9]$.

The overall pattern for the surface and syntactic tasks is that the persistent graph features can \emph{lose} some information on the linguistic properties during the derivation of the features from the attention matrices. The localization of the properties after the derivation gets changed, and the head-wise probe performance may significantly decrease. Notably, the majority of the semantic tasks receive rapid decreases in the probe performance on the persistent graph features as compared to the frozen heads. The reason is that the features operate purely on the surface and structural information of the attention graph, leaving semantics unattended.

\section{Discussion}

\begin{figure}[t!]
    \includegraphics[width=0.5\textwidth]{images/h0.png}
    \caption{The distribution shift of the sum of the bars in $H_0$ between the natural and generated texts on the \textbf{WebText \& GPT-2 dataset} (Layer: 9; Head: 7). TGM: GPT-2 Small with pure sampling.}
    \label{fig:h0_dist}
\end{figure}

\paragraph{Structural Differences between Natural and Generated texts} The TDA-based classifiers rely on the  structural differences in the topology of the attention maps to distinguish between natural and generated texts. Figure~\ref{fig:h0_dist} shows that the distributions of the sum of bars in $H_0$ differ for natural and generated texts. For the former, it is shifted to the left. We provide more examples of the distribution shift for different heads and layers in Figure~\ref{fig:dist}, Appendix~\ref{app:betti_numbers}. The weights for natural texts are concentrated more on the edges of the maximum spanning tree (MST), so that the model focuses on the sentence structure, or on the  ``skeleton'' of the MST. The weights for the artificially generated texts are distributed more evenly among all edges. As the TDA-based classifiers appear to be robust towards unseen TGMs, we may conclude that such structural properties are inherent to the models of different sizes, so that shifts in the distribution of the sum of bars in $H_0$ hold for texts generated by different TGMs. This feature appears to be the key one as utilizing it alone for the prediction provides us with the 82\% accuracy score on the \textbf{WebText \& GPT-2} dataset.

\paragraph{Semantics is Limited}
The TDA-based methods do not take the semantic word similarity into account, as they only capture inter-word relations derived from the attention graphs. The probing analysis supports the fact that the features do not encode the semantic properties, carrying only surface and structural information. However, this information appears to be sufficient for the considered task.

\paragraph{Time Complexity}
The attention matrices are computed each time when an input sample is fed to BERT. It follows that the computational complexity of our methods can not be lower than the one for BERT's complexity itself, which makes asymptotically $O(n^2d + nd^2)$ per one attention head \cite{vaswani2017attention}, where $n$ is the sequence length, and $d$ is the words embedding dimension. On the other hand, the calculation of the topological features by thresholds (given that the number of thresholds is constant), aside of the number of simple cycles, features of $0$-dimensional barcodes, and features based on the distance to patterns are linear by the number of edges of the attention graphs. This means that for at least these features we do not go beyond the asymptotic complexity of the BERT model inference, even for sparse attention variants.

The number of simple cycles and the features of $1$-dimensional barcodes are more computationally expensive. Note that omitting these features provides a significant speed up with minor performance drops.

\section{Conclusion}
This paper introduces a novel method for the task of artificial text detection based on TDA. We propose three types of interpretable topological features that can be derived from the attention maps of any transformer-based LM. The experiments demonstrate that simple linear classifiers trained on these features can outperform count- and neural-based baselines, and perform on par with a fully fine-tuned BERT model on three common datasets across various domains. The experimental setup also highlights the applicability of the features towards the TGM architecture, TGM's size and the decoding method. Notably, the TDA-based classifiers tend to be more robust towards unseen GPT-style TGMs as opposed to the considered baseline detectors. The probing analysis shows that the features capture surface and structural properties, lacking the semantic information. A fruitful direction for future work is to combine the topological features with those that encode the semantics of the input texts, and test the methods on a more diverse set of the TGM architectures, decoding methods and transformer LMs to infer the attention graphs from. We are publicly releasing the code, hoping to stimulate the research on the TDA-based investigation of the inner workings of the transformer-based models and the applicability of TDA methods to other NLP tasks.

\section*{Acknowledgement}
The work of Serguei Barannikov and Evgeny Burnaev, related to topological data analysis and machine learning in Sections \ref{features} and \ref{experimentssection}, is supported by Ministry of Science and Higher Education grant No. 075-10-2021-068. Ekaterina Artemova is supported by the framework of the HSE University Basic Research Program.

redpajama_set_name: RedPajamaArXiv


text:
Pete Peterson, Legendary Waterman
Surfing's OG legends
This morning I was cruising through posts on my Facebook page and saw one that showed my friend Lisa Peterson with some of her father's old boards on display behind her. BAM, perfect idea for todays destruction of the English language also known as my column. A look back at one of the greatest legends in the surfing world, Preston "Pete" Peterson, also known as Lisas dad. Unless you are an older and very hard-core surfer you might not know of Pete Peterson, but let me assure you that he was one of the BIGGEST surf stars of all time, and most of it was in the time before there were actual surf stars.
Let's go back to the beginning. In the 1930's there was an annual competition held known at that time as the PACIFIC COAST SURFING CHAMPIONSHIP, this was obviously way before the West Coast Surfing Championship began in Huntington Beach in 1959. Pete Peterson won the PCSC in 1932, 1934, 1936 and 1941. He was the pretty much undisputed top dog on the West Coast back then and thru the early and mid 1950's. When the contests started in '59 and spread to a whole circuit of events here in California through the 1960's Pete was still winning trophies as the number one Tandem surfing team (along with Barrie Boehne, wife of Infinity Surfboards Steve Boehne) and in the Masters division.
We're making surfing really fun again for older surfers.
But Pete Peterson was far more than just a surfer. He was the all time all around "waterman." He did everything known to man that you can do in the ocean and was great at all of it. My pal Craig Lockwood, the well known paddle board racer, wrote a killer piece in SURFERS JOURNAL back in 2005 that describes Pete really well. "Peterson defined what would become the paradigm of the West Coast's classic waterman. Peterson's life and livelihood were the sea. There was no ocean skill he didn't possess. He swam competitively, surfed, bodysurfed, rowed, sailed, lifeguarded, tandem surfed, paddleboarded, dove hardhat, SCUBA, and free, water skied, did movie stunt work, served as a boat master and marine coordinator for numerous films, designed and shaped surfboards, rescue and racing paddleboards, helped test and later manufactured his friend and co-lifeguard Lieutenant Wally Burton's pioneering flexible lifeguard rescue tube, designed and built the West Coast's finest surf dories (molds of which are still in use today), fished commercially, ran a marine salvage business as a licensed skipper and contractor, and designed for this business one of the most sophisticated salvage craft on the West Coast."
I loved Pete, he was absolutely one of the coolest and nicest people you ever could meet. My first experience with him came at San Onofre in 1958. A couple of pals of mines mom, from down the street in Surfside, took all of us on a "surf safari" down the coast one day in her Buick station wagon. First we surfed at Doheny and then went to San Onofre. The surf was really good, fairly big for us as we were about 10 at the time. I got outside and was sitting for quite a while waiting for a wave, maybe a bit nervous. A guy had been sitting way farther out than everybody else, I thought he was just hanging out and maybe fishing or something. Then a huge set came in and he was the only one far out enough to catch it. He did and went screaming past the rest of us at mock speed. One of the other guys said, "That's Pete Peterson, one of the best who ever lived."
A few years later when I was entering all the surf meets up and down the coast, I started seeing him and got to know him very well. He was always nice to me, even during my intolerable loud mouthed brat stage. One of my favorite Pete Peterson memories was one day in the mid-sixties when I was out surfing on a big day at Sunset Beach, on the North Shore of Oahu, Pete came paddling out on his big tandem board with his partner (not sure if it was Barrie but I think it was). Everybody is kinda looking at each other going, "wow, what's this?" It was freaking SUNSET BEACH on a big day. People didn't go tandem out there; most people didn't even go out on a big day. But there they were. To the amazement of everyone they took off and rode a couple of the biggest waves of the day. Impressed the heck outta me, I can tell you that. Even Mike Doyle, the best big wave rider of the time and also known as "Iron Mike" for his all-around water skills, was blown away.
Pete pulled out in 1983. For those of you who never knew about him but are interested in surfing history I encourage you to read up on him. There is a great account of his life at http://www.legendarysurfers.com/2011/01/1930s-pete-peterson-1.html
Stay safe and see ya next time…same Bat time, same Bat channel. (wow, dating myself here).
I knew Barry and her little sister Esther, who both son Championships with Pete. He also took my sister's husband under his wing and got him a Captains license. He later became Harbor Master of Santa Monica. This was a guy who never finished high school. He always said he owed Pete a lot.
randall rosenthal link
thanks...as an OG i love this stuff.

redpajama_set_name: RedPajamaCommonCrawl


text:
The Maker Of The AK-47 Is Now Making Electric Motorcycles
Erik Schelzig
Kalashnikov, the Russian conglomerate best known for the AK-47 assault rifle, is getting into the electric motorcycle game. The details are sparse, but it's my sad duty to report that it doesn't look like the bikes will come with banana-magazine range extender batteries.
Kalashnikov Group presented its new bikes at the big Army 2017 International Military-Technical Forum in Russia last month, saying the 50 of the police versions will be used by the Moscow police at next year's World Cup.
All we know for now is that the police version of the bikes are supposed to have range of about 150 kilometers, or nearly 100 miles. We don't know the differences between the military versions, other than the camouflage paint scheme and gun rack on the handlebars.
Kalashnikov isn't the only electric motorcycle maker with dreams of military or law enforcement contracts. Virginia-based Logos Technologies announced last month that it has been awarded a second contract by the the Defense Advanced Research Project Agency to keep developing its hybrid-electric SilentHawk motorcycle.
The SilentHawk is basically the Alta Redshift MX (that I rode earlier this year) with a multi-fuel generator to recharge the battery on long missions. They're also developing an all-wheel drive system, for reasons I guess I'm not off-roady enough to understand.
Anyway, back to the Kalashnikov (how often have you said that in casual conversation?).
It is made by subsidiary IZH, which has a history of motorcycle production dating back to 1928. Production hit its high point after Soviet troops packed up and moved a DKW motorcycle plant (along with its top executives and engineers) from eastern Germany to the Ural mountains as reparations for World War II.
A video on the Kalashnikov site gives little detail in terms of specs for the military version of the bike, but it does present some footage of a camo-wearing guy with a chunky watch riding the bike through some mild terrain. It also shows some not-so-dramatic scenes of the guy stuffing the charging cable into a side-case, which appears suspiciously like it might be filled with a huge battery.
The bike comes with a gun rack mounted on the handlebars that, while presumably handy in certain situations, could certainly do some damage if you're not wearing proper head and eye protection ...
... such as sunglasses and a bandana. Sigh.
On balance it's hard to say whether the Kalashnikov electric bikes are going to be any good. We don't know what they will cost, weigh or how far they will go under real world conditions.
But for a company responsible for the creation of millions of AK-47s, it'd probably be foolhardy to discount them entirely.
This post has been updated.


This Fall, our club will present Dictionary4Life books to ten schools in both the public and Catholic school boards.
To encourage donations for our local literacy project, Dictionary4Life, the Rotary Club of West Ottawa (Service Fund) Inc. now has a profile on Canada Helps (www.canadahelps.org). Donating to this project has never been simpler! Simply click to make a safe & secure donation. A $20 donation will purchase at least 2-3 dictionaries! A tax receipt will be issued by Canada Helps for donations.
Our order for Fall 2018 presentations of books to almost 1,800 Grade 3s in Ottawa, Prescott (Brockville club supported), Cornwall, Napanee, Smiths Falls, Kingston, Eganville, Montreal-Lakeshore and Arnprior will be on its way soon from Dubai. How exciting!
Seven other Ottawa Rotary clubs participate. They are Nepean-Kanata, Orleans, Ottawa, Ottawa Bytown, Ottawa South, Ottawa-Stittsville and South Nepean.
Rotary Clubs of Brockville, Kingston-Waterfront, Napanee, Cornwall-Sunrise, Smiths Falls, Eganville and Arnprior and Montreal-Lakeshore joined the District D4L project over the past few years.
And the project expands again in Canada. Port Coquitlam, BC has been delivering the D4L project for many years and their 2018 order is for 2,610 books for 4 clubs. Outstanding! Further, former West Ottawa Rotarian Susan Fisher has introduced the project to District 7810 and will have 4 Rotaract clubs present 110 books to Sackville, NB students. Great work Rotary Club of Sackville and Susan in particular.
Our literacy project continues to be augmented with the Ottawa Reads initiative wherein a team of volunteers read to students at Agincourt Road and Regina Street Public Schools.
And NEW on our website is the Free Reading Program -- check it out in the right hand column of this website's home page.


Na internet browser sung ah chrome://settings/content hih URL copy inla, pai in (Google Chrome ah). Tua a ban pen a nuai a lim om bang in zom in. Manglai in akicing zaw in Google website tung pan lamlahna hong ki guang sawn hi.
Raise or lower windows: This option allows sites to place windows below or above existing windows.


text:
Q: How to convert different file into .doc file using php If i click the button convert and file should be converted into .doc file through php. if i have file of pdf in database , so how can it be display in word file.

A: Your question may be more precise but here some hint if your try to convert a PDF to a Doc file.
Depending of the file type, but using an API you can convert file format to another one. Eg : Cloudconvert has a PDF to Doc converter by eg ( https://cloudconvert.com/pdf-to-doc ).
It has a cost but if you do it on you own (possible with a Windows server) it will cost a lot more.
I use Cloud Convert on one of my project : it works well, but it depends a lot of the source file.

redpajama_set_name: RedPajamaStackExchange


As reported by Steven Ahle| Paul Ryan's amnesty bill has leaked out and it should make Ryan the biggest traitor since Judas Iscariot. The original DACA amnesty bill covered 800k illegal aliens. Ryan's bill extends that to 3.5 million not counting the mad rush to our borders for more to sign up.
Not only that but he has added a new amnesty program for kids whose parents came here on temporary work visas. They would then make their parents also eligible for amnesty just like parents of dreamers. The initial amnesty could cover 5 million illegals just to start and quickly top 10 million.
The conditions for this amnesty is ridiculously low.
These amnesties create 3 winners. Dreamers, Democrats who give up practically nothing and businesses that want cheap labor and the big losers are Americans whose jobs will be taken and wages depressed from this bill.
Paul Ryan told the press that President Trump will sign this bill. I don\'t believe it. But if Republicans pass this bill, there will not be a blue wave, there will be a blue hurricane in November.
Unknown are the numbers of illegal aliens who will surge the United States-Mexico border at the enactment of such an amnesty plan.
Previously, Kansas Secretary of State and gubernatorial candidate Kris Kobach told Breitbart News that a DACA amnesty — even one smaller than this amnesty plan — would potentially cause a surge of one million illegal immigrants rushing the southern border.
I will immediately seize your rss feed as I can't in finding your e-mail subscription link or e-newsletter service. Do you have any? Please allow me realize in order that I could subscribe. Thanks.
Hello there I am so excited I found your blog, I really found you by accident, while I was looking on Yahoo for something else, Anyways I am here now and would just like to say thank you for a tremendous post and a all round thrilling blog (I also love the theme/design), I don't have time to browse it all at the minute but I have book-marked it and also included your RSS feeds, so when I have time I will be back to read a great deal more, Please do keep up the great job.
I not to mention my guys ended up analyzing the nice things found on the website and then at once I got an awful feeling I never thanked the web site owner for those techniques. The men are actually certainly joyful to study all of them and now have absolutely been taking advantage of these things. Thank you for simply being indeed thoughtful and then for selecting this form of remarkable tips millions of individuals are really desperate to learn about. My personal sincere apologies for not saying thanks to earlier.


Michael has appeared in numerous shows and the movie Six Weeks with Mary Tyler Moore, as well as playing for Juliet Prowse, who has endorsed his five dance record albums. Other compositions include two New Age Albums, three Theatre Musicals, as well as dance compositions.
Excerpts of DANCING SCORES are in MP3 format.
DANCING SCORES is $15.00 $12.00 ON SALE for the CD.


Rubies, Always do research prior to purchasing a ring or a setting that you are going to add Rubies to. Like Diamonds, Ruby has high reputation in the gemstone industry and is amongst most sought after gemstones in the world are rubies and one of the Oldest Known gems in the World. Rubies make exquisite or modern style in rings, pins, earrings and ruby bracelets.
There are many vendors on this site that can help you find the piece of jewelry you want to purchase. If you don't see the piece you want or know how to select a piece of jewelry with ease ask several of the vendors for their opinion, Then look over the responses you get and work with the vendor you feel most comfortable with, Look at the feedback of your vendor as well as the stock in the stock in their store.
Ruby gemstones can be treated fissure filled or just simple heat process.
Ruby Gemstones are one of the worlds most popular gemstone and is ideal to set in jewelry, The Ruby can be in a cabochon form, that is domed shape and makes ideal ring stone or ruby gemstone can be faceted.
There are many different grades of faceting rubies, basically there are broad pattern cuts to fine close cuts, As we stated above Most rubies with little inclusion will be in a smaller size. The better quality the ruby is the more facets are cut to bring out the best possible color display. Clarity normally determines the cut of a gem. Gems are normally treated or enhanced in the rough 'uncut' form. If a ruby has a Very good clarity You will normally see a Round or Emerald cut on a ruby.
When a ruby has more inclusion or is opaque 'meaning you cannot see thru the gem' the ruby will be cut in a Cabochon. Cabochon rubies are mostly set in rings or pendants as claw set or bezel set and some people prefer diamonds to enhance the setting, which can make the ruby jewelry look traditional or modern.
Today most rubies are heated and most have treatments, The only time You can be certain of a gem with NO treatments is if the ruby is certified, and even then you must look to see if the gem was tested for treatment, Many certified gems will just offer verification of authenticity not treatment.
Testing for treatment will cost extra. Many buyers think they can just buy a gem and take the gem to a local jeweler for appraisal and testing for authenticity but unless your Jeweler is a Certified Gemologist they cannot offer you this information and be 100% correct. But gemstone Certification will allow you be sure of a natural ruby with treatment as there are so many treatments now done to rough ruby.
Rubies have been set in large cluster ruby rings and ruby pendants for centuries. The ruby can be set in groups of rubies and / or mixed with other gemstones such as sapphires and emeralds. It is very common to see a piece of jewelry that includes rubies, emeralds and sapphires.
Today lab created rubies are very popular due to the clean red fire colors which are now hard to find naturally. Lab created rubies shares the the same chemical composition as a real earth mined ruby, they are both made from chromium and titanium , The lab created ruby is less expensive and color is always very accurate. A better choice for those who do not want to spend thousands of dollars on a Inclusion free top color ruby. Lab created rubies can offer a more affordable gemstone. Either way 'a lab created ruby or a Earth mined ruby will make a good selection for jewelry, they both share the same chemical make up and are very hard, a 9 on the Mohs' scale, Making this gem a great choice for your piece of jewelry.
Only recently has white gold been used in ruby settings as traditionally yellow gold compliments the red ruby color.


text:
New obligations for solar retailers taking part in Solar Homes program
Thursday 21 March 2019 5:43am
Solar Victoria has welcomed the State Government's decision to require retailers taking part in the Solar Homes program to be signatories to the Clean Energy Council's Solar Retailer Code of Conduct or be excluded from the program.
Requiring participating solar retailers to commit to behave ethically, provide quality service and reasonable warranties bolsters existing consumer protections and an existing requirement for solar installers to be accredited by the Clean Energy Council.
"While existing consumer protections already go a long way to protect purchasers, we have identified a gap, which this Code of Conduct addresses," Sustainability Victoria's Chief Executive Officer, Stan Krpan, said.
"Having sales staff, their employers, and installers committing to high standards will ensure consumers get a better solar outcome."
From 1 July 2019, the most active solar retailers taking part in the Government's Solar Homes program must be signatories to the Clean Energy Council's Solar Retailer Code of Conduct if they wish to continue to take part in the $1.42 billion program
Lower volume solar retailers will be required to be signatories to the Code of Conduct from 1 November 2019.
"Our primary interest is to protect consumers and ensure integrity in an industry where, unfortunately, some operators still seek to take advantage of people who are making a substantial financial outlay," Mr Krpan said.
"Solar Victoria is committed to delivering a quality program. To do that we must ensure solar retailers understand what is required of them."
"Lifting standards will help to establish a level playing field for all companies and add to the work already being done with Energy Safe, WorkSafe, the Victorian Building Authority and the Essential Services Commission and Consumer Affairs Victoria," Mr Krpan said.
All solar retailers that are not yet signatories to the Clean Energy Council's Retailers Code of Conduct are encouraged to apply for approval now.
More information on the Retailers Code of Conduct can be found on our page about Quality and safety standards or the Clean Energy Council's website.


This form is for general communication with our site. Please give us your feedback on what we've done, and what you'd like us to do!
Oh… and thank you for making us awesome!
*Your submission of a book, short story, manuscript, woolly mammoth or Tesla does not automatically guarantee inclusion on the site (though sending a woolly mammoth or Tesla may admittedly have a higher appeal 😉 ). All reviews are at the sole discretion of the Site Owner and Site Staff. Thank you in advance.


Emmanuelle Colboc has been working in Paris as an architect for over 30 years. A few years ago, she founded her own agency, Emmanuelle Colboc et Associés, with three colleagues. Today, the agency employs about a dozen employees, focusing mainly on the design of hospitals, schools, public facilities and housing. Regardless of how diverse these projects may be, they are all centred around people, and they have a soul and a touch of character to ensure the occupants and users feel right at home. The architects always try to bring in as much natural light as possible, because they see it as the ultimate source of well-being.
"I love working on projects with a great focus on people, in which the technical aspects must adapt to the well-being of the location's users", says Emmanuelle Colboc.
This project focuses on the extension of the ophthalmology centre of the Cochin hospital, which is currently housed in a XIXth-century wing on one of the historical axes of the building. The new building, which is the same size as the existing wing and will be built symmetrically, is compact and surrounded by tall, large blocks. To allow as much natural light as possible into the new extension, the architects are placing a glass element on the façade between the two buildings, creating a patio reminiscent of the old patio. They are also installing skylights on the roof, which let light shine into each patio and illuminate the gable of the old building, currently the interior façade of the entrance hall. The new building is articulated around the patio which is used as the reception hall for the consultation rooms and leads to the emergency ward. The operating theatres are located on the first floor, which resembles a balcony onto this patio, that lets in natural light and creates a sense of warmth. As the operating theatres themselves are on the side of the façade, they benefit from natural light. Above the entrance hall, there is a spacious, double-height waiting room with large windows between the first and the second floor. This hall for day-hospital patients bathes in pleasant natural light and boasts framed views of the old scenery.
The bricks were chosen for several reasons. Firstly, since the new brick extension is a nod to the old XIXth-century wing, the architects deemed it fit to use the same material. Secondly, as the building is surrounded by two large blocks, they needed to make sure its outline would not be swamped by its surroundings. They therefore opted for a sober look. The architects designed a breezy glass ground floor and contrasting solid brick façades. With the exception of a few large corner windows above the entrance, the first floor is entirely covered with bricks, placed at an angle, creating a striking, unassuming look. This playful brickwork on the façade ensures a strong, solid contrast with the neighbouring buildings. The bricks used are Nelissen Rosina, which perfectly met the architects' colour demands. Their rusty hue works well with the terracotta roof tiles of the old building and gives the façades of the new building a sturdier look. The architects set out to find solid, elegant materials that would blend in well with the neighbouring building and the roof, rather than with the pink ochre shades of the existing façade. Another red façade down the road boasts the same style as the new building.


text:
Trains from Lye (West Midlands) to Small Heath
Lye (West Midlands)
Home Destinations Small Heath Trains from Lye (West Midlands) to Small Heath
From Lye (West Midlands), our trains run up to 2 times per hour to Small Heath.
Train journey information from Lye (West Midlands) to Small Heath
How much is a train ticket from Lye (West Midlands) to Small Heath by train?
Catch the train from Lye (West Midlands) to Small Heath from just £2.50 one-way when you book in advance.
How long does it take to travel from Lye (West Midlands) to Small Heath by train?
Our fastest train to Small Heath takes 33 minutes. Journey times may be slightly longer on weekends or public holidays. Use our Journey Planner to access downloadable timetables, live departures and arrivals. Stay connected while on the move with our free on-board WiFi and entertainment service. Access a range of entertainment, including TV programmes, magazines, short films, news and more!
How many trains a day are there from Lye (West Midlands) to Small Heath?
There are up to 34 trains per day from Lye (West Midlands) to Small Heath.
Are there any direct trains from Lye (West Midlands) to Small Heath?
Yes, you can travel from Lye (West Midlands) to Small Heath without having to change trains. Find direct trains to Small Heath using our Journey Planner.
What time is the first train from Lye (West Midlands) to Small Heath?
The first train departs from Lye (West Midlands) to Small Heath at 06:06. Book your tickets in advance and enjoy a few more minutes of hitting snooze.
What time is the last train from Lye (West Midlands) to Small Heath?
The last train from Lye (West Midlands) to Small Heath departs at 23:07.
More trains from Lye (West Midlands)
Lye (West Midlands) to Hall Green
Lye (West Midlands) to Old Hill
Lye (West Midlands) to Wembley Central
Lye (West Midlands) to Bilbrook
Lye (West Midlands) to Runcorn
More trains from Small Heath
Small Heath to Rowley Regis
Small Heath to Claverdon
Small Heath to Apsley
Small Heath to Worcester Shrub Hill
Small Heath to Rugeley Trent Valley
Lichfield to Smethwick
Albrighton to Butlers Lane
Adderley Park to Bow Brickhill
Stechford to Cosford
Gravelly Hill to Malvern Link
More tickets types between Lye (West Midlands) to Small Heath


This entry was posted on lunedì, novembre 5th, 2012 at 15:47	and is filed under Progetti and tagged with MarcoPolo. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.


The United States pressed Europe's strongest economies on Friday to give "unequivocal" financial support to weaker euro zone states to overcome a debt crisis that threatens the world economy.
"It is completely within the capacity of the stronger members of the euro area to absorb these costs," U.S. Treasury Secretary Timothy Geithner said as G7 finance chiefs gathered in Marseille to discuss how to revive a stalling recovery.
"Those costs would be much, much greater for them and their economies if they sit here and do nothing, and they recognize that," Geithner told Bloomberg Television in comments that appeared aimed primarily at EU economic powerhouse Germany.
With markets looking to the Group of Seven major industrial economies for some sign of a policy shift to help faltering growth, a G7 source said the meeting might after all issue a communique, which G7 chair France had said was not planned.
Ministers and central bankers were under pressure to calm the biggest confidence crisis in financial markets since the 2007-8 global credit crunch.
But a shock announcement that the top German official at the European Central Bank is leaving early in disagreement with the bank's policy of buying euro zone government bonds to support the likes of Italy and Spain laid bare deep rifts over how to manage the debt crisis.
The ECB confirmed that chief economist Juergen Stark would retire nearly three years before his term is due to expire. His decision means Bank of Italy governor Mario Draghi will start his term at the ECB helm in November with a mountain to climb to restore its credibility in Germany, Europe's biggest economy.
France has called for a coordinated response from the Group of Seven industrialized nations after mounting anxiety over Europe's debt crisis and the fragility of its banks caused a big fall in world stock markets in recent weeks.
Differences between the economic problems facing the euro zone, Britain and the United States -- which unveiled a $447 billion jobs package on Thursday -- are complicating the task though, meaning one-size-fits-all solutions will not work.
IMF chief Christine Lagarde said in London before boarding a flight for Marseille that policymakers in advanced economies should use all available tools to boost growth and called for bold action to weather a "dangerous new phase" of recovery.
She also cautioned against too much fiscal consolidation in a climate of sputtering growth. But a G7 source told Reuters a unanimous agreement at the Marseille talks on coordinated monetary easing was unlikely.
A source in Brussels has said the G7 would likely agree to keep monetary policy accommodative, slow fiscal consolidation in states where that is possible, and implement structural reforms.
Fears the global economy may be in its most difficult period since the collapse of investment bank Lehman Brothers have added significance to Friday's talks but there has been little evidence of the unity of purpose shown in 2008 and 2009.
U.S. President Barack Obama's new package of tax cuts and spending could lift U.S. growth by one to three percentage points in 2012 and add more than a million jobs.
But in debt-ridden Europe, there is little scope for fiscal stimulus, and where there is some wiggle-room -- in Germany and Britain -- there is no political appetite for it.
In an indication of the conflicting positions on policy, Canadian Finance Minister Jim Flaherty told Reuters TV decisive moves were needed to restore market confidence and said slowing fiscal consolidation too much would be foolish.
"I hope we would all agree we have to stay the course, that we have to go through the pain of fiscal consolidation. It's not easy, it creates stresses in some countries, but it's necessary, we have to get through this rough patch," Flaherty said.
G7 finance ministers and central bankers trickled into the sunlit Mediterranean port city of Marseille around lunchtime and talks were due to start at 4:00 p.m. (1400 GMT).
A working dinner will be followed by briefings from around 9:15 p.m. local time (1915 GMT) by the French, German, Canadian and Japanese delegations and European Central Bank President Jean-Claude Trichet. The United States plans no briefing.
ASIA AS CONCERNED AS U.S.
With Asian economies deeply worried about the West's debt crisis and slow growth, Japan said it will voice its concern on the euro zone debt crisis and seek support for its right to unilateral action over safe-haven buying pushing up the yen.
Bank of Japan Governor Masaaki Shirakawa told reporters in Marseille he hoped the talks would share frank views on the crisis and said it was vital that G7 finance chiefs came up with a "firm stance" to stabilize the world economy.
"There are various factors behind the world economy's uncertainty but Europe's debt problem is one major factor. It is important for Europe to tackle its debt problems for its own sake but it would also indirectly bring positive effects on Japan's economy," he said.
Finance Minister Jun Azumi said Japan would ask the G7 for its understanding on its intentions to counter yen rises.
Lagarde said policymakers must act now, "and boldly," giving her blessing to more quantitative easing by central banks and saying the challenge was to find a pace of adjustment that was neither too fast nor too slow.
She said countries facing market pressures must push ahead with urgent fiscal consolidation, while there was scope for slower action in countries not at the mercy of market forces.
"If growth continues to lose momentum, balance sheet problems will worsen, fiscal sustainability will be threatened, and the scope for policies to salvage the recovery will disappear," she said.
Decisions by the European and British central banks this week to keep interest rates unchanged accentuated the gloom in Europe but neither indicated that a cut was imminent, while Federal Reserve Chairman Ben Bernanke gave no hint of new stimulus to boost the economy in a keenly awaited speech.
"Despite speculation about new coordinated forex intervention, a standard final statement remains the most likely outcome," Unicredit said in a research note.
The Organization for Economic Co-operation and Development says growth across the G7 could slow to an anemic 0.2 percent in the last quarter of 2011. Its chief economist Pier Carlo Padoan urged the G7 to send a clear signal it is ready to take action if growth slows further.


text:
Username	 Password	 Remember Me	 Forgot your password?
A Cup O' Kapeng Barako
A Voice From America
Amina Rasul
At Large, A Journal of Our Times
Bai (Cebu & Magellan) Column
Bunaoisms
Dissenting Opinion
Frank B. Quesada (Col. Ret. US)
Health@Heart
JGL Eye
Making It in the USA
Noy (Bicol Column)
Op-Ed Page
Reinventing the Philippines
San Diego Happenings
Sen. Aquilino Q. Pimentel, Jr.
Tremendous Trifles
A Poet's Corner
DrRizal.com
Filipino-Veterans' Lobby
I2D2-International Debt & Development
Insurance & Securities
Literature and Fourth Estate
MiscellaNEWS
NaFFAAgate
Obituary-Memorial Park
Philippine Immortality Project
Philippine Presidency
The Daily B.R.E.A.D.
Womens Section
Revotelution
Fil-Am Community-L.A.
RP Government Updates
User Chat
Home Sections Real Estate Is Sorsogon Turning into a Filipino Version of "Sodom & Gonorrhea" (sic)?
Is Sorsogon Turning into a Filipino Version of "Sodom & Gonorrhea" (sic)?
Sections	- Real Estate
Written by Bobby Mercado-Reyes
Perhaps Is it Time to Change the Name of the Province and the City to "Sorsodom"?
By Lolo Bobby Reyes y Mercado of <?xml:namespace prefix = st1 ns = "urn:schemas-microsoft-com:office:smarttags" />Sorsogon City and West Covina, CA
First Part of an Occasional Series called "Sorsodom?"
It was my literary mentor, Poet-pundit Fred Burce Bunao (now deceased) who punned "Sodom & Gomorrah" into an Erap joke in the late 1990s. Mr. Bunao cracked a joke that when then-Vice President Joseph Ejercito Estrada addressed the Filipino-American community of Southern Nevada, the Filipino Veep reportedly said: "I fear that Las Vegas is turning into a modern version of 'Sodom & Gonorrhea'," and all the Filipino medical practitioners in the crowd gave him a standing ovation.
Well, if Mr. Bunao, whose parents hailed from Albay Province, were alive today, he would have also asked if "Sorsogon Province, especially Sorsogon City, had become a Filipino version of "Sodom & Gonorrhea"?
This week, the DZGN-FM (AKA "Spiritfm Sorsogon") station has posted in the Facebook (FB) a report that the Sorsogon City Council was about to designate a certain area of the city as a "Red-light District," similar to certain zones in the State of Nevada or in Amsterdam, Holland. The SorCity Council did not bother to conduct a public hearing to arrive at its decision, other Sorsoganon FB members commented further. Certainly, the act of designating certain areas of Sorsogon City as a "Red-light District" will probably reduce the real-estate values in the area, if not in the entire province.
In 1988, my cousin, Antonio F. Dioneda, Sr., M.D., told me then that there were lots of cases of sexually-transmitted diseases (STD) in Sorsogon Province. He said also that quite a few Sorsoganons had died after contracting the human immunodeficiency virus (HIV) that caused the acquired immunodeficiency syndrome (AIDS). Dr. Dioneda (now deceased) said, however, that since autopsies were not conducted on the AIDS victims, his findings were only educated guesses.
Nearly a quarter of a century after my interview with Dr. Dioneda, Sorsogon's health situation is probably worse than it was in the 1980s. Proof? There is now an ongoing cholera epidemic that has so far killed more-than 18 individuals and that has afflicted hundreds more of cholera victims.
Here's an article datelined Sorsogon City, Feb. 5, 2012, by Irma A. Guhit about "Provincial health office records 18 deaths of Cholera, advises public to take extra health precautions."
URL: http://sorsogoncity.wordpress.com/2012/02/05/provincial-health-office-records-18-deaths-of-cholera-advises-public-to-take-extra-health-precautions/
If a preventable disease like cholera could run amuck in Sorsogon Province, how much more the STD, which many of its victims are ashamed to report to health authorities or their private physicians, if they could afford one?
Rising Crime Statistics
T here are many Sorsoganon victims of unsolved murders, if not extra-judicial killings, and robberies perpetrated by "Akyat Bahay" (home-invasion) gangs, the members of which commit the thefts supposedly to feed their rising illegal-drug habits.
Illegal-drug distributors and illegal-gambling lords are said to be proliferating in Sorsogon Province. Some community leaders claim that the criminal elements are being protected by corrupt politicians and even by some law-enforcement officers. There is an iota of truth in these claims that without the connivance of some crooked public officials, law and order would have prevented prohibited-drug and numbers-game syndicates from operating wantonly in the province.
Now to add insult to injury, the Sorsogon City Council is establishing a Red-light District, so as to fatten supposedly the public coffers (or line up their pockets, as some of their often men-of-the-cloth critics allege).
Perhaps is it time to change the name of the province and the city to "Sorsodom" or maybe follow Poet-pundit Bunao's punning exercise as the unofficial moniker for Luzon's southern end of a tourist destination?
(To be continued …)
02/25/2012 13:35 - Filipino Realtor, Wife, 7 Others Indicted for Mortgage-Fraud Scheme
10/18/2011 13:10 - A Priest's and his Altar Boy's Dream of an Oberammergau-like Festival in Bulan, Sorsogon
01/03/2010 10:40 - How to Make a Biblical-based Real-estate Redevelopment the Answer to Global Warming, Starting in Sorsogon
09/02/2009 20:28 - Rep. Mikey Arroyo's Real-estate Transaction in CA No Mickey-mouse Deal
03/24/2009 17:22 - Two Filipino Americans Accused of Real-Estate Investment Scam
HIV Can Be Passed on from One Lesbian to Another Woman Lover During Sex
1 Monday, 24 March 2014 18:39
http://www.latimes.com/science/sciencenow/la-sci-sn-women-hiv-sex-cdc-20140313,0,5660284.story Good heavens, here is more bad news about "same-sex marriages ('gayrriages')" or "same-sex relationship." FYI.
Comment (you may use HTML tags here):
MabuhayRadio Comments
Another Rx for Mike R. Bloomberg (Part II)
Rx for Mike R. Bloomberg's Platforms (Part I)
President Trump Ends Up As a "Butt Of Jokes" Again in China and Elsewhere
Today is DAY ONE of the Mike Bloomberg 2020 Campaign
"Baby Bloomers for Bloomberg" Spearheaded by Filipinos
A Christmas Story: An ADVENTure With a Homeless Man
ReVOTElution.com Proponents Launch an Online Constitutional-Convention Forum
The History of the Tournament of Roses' "Filipino Scandal in the Wind"
Today Is the 18th Anniversary of the Philippine Scandal at the Pasadena Tournament of Roses
Remembering the 'Boy-Scout' Filipino Tragedy of July 28, 1963
Josephine Bracken, Jose Rizal's Widow, Actually Became a Cebuana
Help Save the Philippine Languages
Fil-Am Columnist Picks Cotto by TKO in Round 11 (35)
Cats on Hot Thin Soups and Siopaos? (29)
Remembering the 'Boy-Scout' Filipino Tragedy of Ju… (19)
Lorna Dietz Fears Lynch Mob in Chicago and a New N… (18)
NaFFAA Chair Admits Making Money from Professional… (15)
We Congratulate President-elect Aquino and Pledge … (15)
The Truth About Philippine Medical Schools (14)
Josephine Bracken, Jose Rizal's Widow, Actually Be… (14)
Not Getting Mad at, But Getting Even With, Tita Co… (11)
Remember Irene Mora Who Lied that She Was the Firs… (10)
<?xml:namespace prefix = fb />
Please consider supporting the "ReVOTElution of Hope" for Sorsogon as the Pilot Province. Please see "ReVOTElution" Banner on this page for details.
Amount: USD EUR GBP CHF AUD HKD CAD JPY NZD SGD SEK DKK PLN NOK HUF CZK ILS MXN
"Every time a baseball player grabs his crotch, it makes him spit. That's why you should never date a baseball player."--Marsha Warfield
Copyright © 2006 - 2009 MabuhayRadio


text:
//
// Copyright (c) Microsoft. All rights reserved.
// Licensed under the MIT license. See LICENSE file in the project root for full license information.
//

/*=============================================================
**
** Source: test2.c (exception_handling\raiseexception\test2)
**
** Purpose: Tests that the correct arguments are passed
**          to the filter by RaiseException and tests that
**          the number of arguments never exceeds
**          EXCEPTION_MAXIMUM_PARAMETERS, even though we
**          pass a greater number of arguments
**
**
**============================================================*/

#include <palsuite.h>

BOOL bFilter = FALSE;
BOOL bTry = FALSE;
const int nValidator = 0;

ULONG_PTR lpArguments_test1[EXCEPTION_MAXIMUM_PARAMETERS];
DWORD nArguments_test1 = EXCEPTION_MAXIMUM_PARAMETERS;

ULONG_PTR lpArguments_test2[EXCEPTION_MAXIMUM_PARAMETERS+1];
DWORD nArguments_test2 = EXCEPTION_MAXIMUM_PARAMETERS+1;

/**
**
**  Filter function that checks for the parameters
**
**/
LONG Filter_test1(EXCEPTION_POINTERS* ep, VOID *pnTestInt)
{
    int i;

    /* let the main know we've hit the filter function */
    bFilter = TRUE;

    if (!bTry)
    {
        Fail("PAL_EXCEPT_FILTER_EX: ERROR -> Something weird is going on."
            " The filter was hit without PAL_TRY being hit.\n");
    }

    /* was the correct number of arguments passed */
    if (ep->ExceptionRecord->NumberParameters != (DWORD) nArguments_test1)
    {
        Fail("RaiseException: ERROR -> Number of arguments passed to filter"
            " was %d when it should have been %d",
            ep->ExceptionRecord->NumberParameters,
            nArguments_test1);

    }

    /* were the correct arguments passed */
    for( i=0; ((DWORD)i)<nArguments_test1; i++ )
    {
        if( ep->ExceptionRecord->ExceptionInformation[i]
            != lpArguments_test1[i])
        {
            Fail("RaiseException: ERROR -> Argument %d passed to filter"
                 " was %d when it should have been %d",
                 i,
                 ep->ExceptionRecord->ExceptionInformation[i],
                 lpArguments_test1[i]);
        }
    }

    return EXCEPTION_EXECUTE_HANDLER;
}

/**
**
**  Filter function that checks for the maximum parameters
**
**/
LONG Filter_test2(EXCEPTION_POINTERS* ep, VOID *pnTestInt)
{
    /* let the main know we've hit the filter function */
    bFilter = TRUE;

    if (ep->ExceptionRecord->NumberParameters > EXCEPTION_MAXIMUM_PARAMETERS)
    {
        Fail("RaiseException: ERROR -> Number of arguments passed to filter"
             " was %d which is greater than the maximum allowed of %d\n",
             ep->ExceptionRecord->NumberParameters,
             EXCEPTION_MAXIMUM_PARAMETERS);
    }

    return EXCEPTION_EXECUTE_HANDLER;
}

int __cdecl main(int argc, char *argv[])
{
    int i;
    BOOL bExcept = FALSE;

    if (0 != PAL_Initialize(argc, argv))
    {
        return FAIL;
    }

    /********************************************************
     * Test that the correct arguments are passed
     * to the filter by RaiseException
     */
    PAL_TRY
    {
        bTry = TRUE;    /* indicate we hit the PAL_TRY block */

        /* Initialize arguments to pass to filter */
        for( i = 0; ((DWORD)i) < nArguments_test1; i++ )
        {
            lpArguments_test1[i] = i;
        }

        RaiseException(0,0,nArguments_test1,lpArguments_test1);

        Fail("RaiseException: ERROR -> code was executed after the "
             "exception was raised.\n");
    }
    PAL_EXCEPT_FILTER(Filter_test1, (LPVOID)&nValidator)
    {
        if (!bTry)
        {
            Fail("RaiseException: ERROR -> Something weird is going on."
                 " PAL_EXCEPT_FILTER was hit without PAL_TRY being hit.\n");
        }
        bExcept = TRUE; /* indicate we hit the PAL_EXCEPT_FILTER_EX block */
    }
    PAL_ENDTRY;

    if (!bTry)
    {
        Trace("RaiseException: ERROR -> It appears the code in the "
              "PAL_TRY block was not executed.\n");
    }

    if (!bExcept)
    {
        Trace("RaiseException: ERROR -> It appears the code in the "
              "PAL_EXCEPT_FILTER_EX block was not executed.\n");
    }

    if (!bFilter)
    {
        Trace("RaiseException: ERROR -> It appears the code in the"
              " filter function was not executed.\n");
    }

    /* did we hit all the code blocks? */
    if(!bTry || !bExcept || !bFilter)
    {
        Fail("");
    }

    /* Reinit flags */
    bTry = bExcept = bFilter = FALSE;

    /********************************************************
     * Test that the number of arguments never
     * exceeds EXCEPTION_MAXIMUM_PARAMETERS, even though we
     * pass a greater number of arguments
     */
    PAL_TRY
    {
        bTry = TRUE;    /* indicate we hit the PAL_TRY block */

        /* Initialize arguments to pass to filter */
        for( i = 0; ((DWORD)i) < nArguments_test2; i++ )
        {
            lpArguments_test2[i] = i;
        }

        RaiseException(0,0,nArguments_test2,lpArguments_test2);

        Fail("RaiseException: ERROR -> code was executed after the "
             "exception was raised.\n");
    }
    PAL_EXCEPT_FILTER(Filter_test2, (LPVOID)&nValidator)
    {
        if (!bTry)
        {
            Fail("RaiseException: ERROR -> Something weird is going on."
                 " PAL_EXCEPT_FILTER was hit without PAL_TRY being hit.\n");
        }
        bExcept = TRUE; /* indicate we hit the PAL_EXCEPT_FILTER_EX block */
    }
    PAL_ENDTRY;

    if (!bTry)
    {
        Trace("RaiseException: ERROR -> It appears the code in the "
              "PAL_TRY block was not executed.\n");
    }

    if (!bExcept)
    {
        Trace("RaiseException: ERROR -> It appears the code in the "
              "PAL_EXCEPT_FILTER_EX block was not executed.\n");
    }

    if (!bFilter)
    {
        Trace("RaiseException: ERROR -> It appears the code in the"
              " filter function was not executed.\n");
    }

    /* did we hit all the code blocks? */
    if(!bTry || !bExcept || !bFilter)
    {
        Fail("");
    }

    PAL_Terminate();
    return PASS;

}

redpajama_set_name: RedPajamaGithub


text:
In Keynote Address, Hillary Clinton Urges Rebuilding of Strained Global Alliances
February 7, 2019 by Sana Rahman Leave a Comment
The United States must engage more with the global community to rebuild weakening alliances with regions such as Europe and Asia, former Secretary of State Hillary Clinton said at an event in Gaston Hall on Wednesday.
Throughout her speech, Clinton said that President Donald Trump's administration's decision to backtrack on previous deals and negotiations has strained relationships with other countries, putting the U.S. reputation at risk.
"At some point, as another nation, you think, 'Can we trust them?' It's not just administration to administration, it's the United States of America, it's our government. It's about giving our word, it's about following through," Clinton said. "I think we have to be smarter about how we sustain our involvements and commitments going forward."
AMBER GILLETTE/THE HOYA | Former Secretary of State Hillary Clinton said that President Donald Trump's administration must reinforce weakening relationships with European and Asian countries at an event in Gaston Hall on Wednesday.
Clinton's speech was a part of the "Future of Diplomacy" symposium hosted by the Georgetown Institute of Politics and Public Service at the McCourt School of Public Policy and the School of Foreign Service as a part of the Lloyd George Centennial Lectures, which began February 2018 to celebrate the upcoming SFS centennial anniversary. Clinton's keynote was moderated by Bill Burns, former U.S. ambassador to Russia and U.S. deputy secretary of state.
Clinton's keynote marks her third return to Georgetown since she lost her bid for the 2016 presidential election. In her 2017 and 2018 appearances, Clinton presented the annual Hillary Rodham Clinton Awards for Advancing Women in Peace and Security, which honor the peace-building work and empowerment efforts of recipients.
During a time when there is a lack of satisfaction with both the U.S. government and the governments of numerous European countries, there is a strong need for alliances between the two regions, Clinton said.
"Democracy is going through some growing pains. We are going through some growing pains, and so are the Europeans," Clinton said. "When that happens we need our friends. We need people who we have worked with and counted on, and I think the division that is being unfortunately driven between the United States and Europe is going to haunt us."
Trump threatened that the United States would withdraw from the North Atlantic Treaty Organization, a military alliance of 29 countries in North America and Europe, if allies resisted his demands for additional military spending at the NATO summit meeting last July. Over the course of 2018, Trump told senior administration officials multiple times that he wanted to withdraw from the alliance, according to The New York Times.
Clinton warned against the Trump administration's rhetoric on NATO, which puts the longstanding alliance between the United States and Europe at risk.
"When you look at what is happening in Europe, it should be a great concern to any American," Clinton said. "The fact that our current administration is almost egging on the breakup of NATO and to be in a hostile relationship with people who share a lot of our values who we have gone through a lot with, strikes me as counterproductive, and as I said, essentially dangerous."
The United States is not only neglecting its European allies, but is also allowing its allies in Asia and the developing world to falter, according to Clinton.
For the United States to repair weakening relationships in Asia, it must refocus attention to restore balance in a way that includes long-term U.S. engagement, according to Clinton.
"We are not doing everything we should to have this kind of balance that I think we need," Clinton said. "We talked about pivoting to Asia, to rebalance, but all of this means that you got to stay engaged for the long run, not just sporadically."
A lack of U.S. involvement across Asia, Latin America and Africa has allowed China to become a frontrunner in influencing these regions, Clinton said.
"We are losing the influence contest to China," Clinton said. "They are engaged in an enormous amount of cultural diplomacy, of funding projects in Latin America, in Africa and elsewhere, and we are not competing."
Clinton specifically noted that Beijing's "Belt and Road Initiative" is enjoying a growing influence in global politics. Also known as "One Belt, One Road," the initiative is a strategy by the Chinese government to develop infrastructure and investments across four continents, linking China to the nations of Southeast Asia, the Gulf countries and North Africa.
Maintaining developmental and financial aid in Central American countries could not only improve diplomatic ties with countries such as Guatemala, El Salvador and Honduras, but also help decrease the effects of drug trafficking in the region, Clinton said.
"If we spent even a relatively minor amount of money of the federal government's budget in Guatemala, El Salvador and Honduras, dealing with some of their problems and working against corruption, then all of this talk about poor, desperate people fleeing gang violence and drug traffickers would change into a different conversation," Clinton said.
The Trump administration pushed to end a humanitarian program that was enacted in 2001 following the devastating earthquakes in El Salvador and allowed Salvadorans to live and work legally in the United States. However, an injunction from the U.S. District Court for the Northern District of California in Oct. 2018 blocked the Department of Homeland Security from implementing and enforcing Trump's plan.
Clinton remains optimistic, however, that young people who engage in careers in public service can both help shape the future and reassert the importance of supporting diplomacy.
"There's a lot of hard work that has to be done, but there's also a lot of room for imagination," Clinton said. "It's one of the most exciting times to be looking at a career in public service. I would hope that what we've done today will lead to an even longer conversation so that even more students will understand the importance of supporting diplomacy."
This article was updated Feb. 12.
About Sana Rahman
I have read and accept The Hoya's official comments policy. (required)