text:
\section{Introduction}

Coreference resolution identifies mentions in a document that co-refer to the same entity. It is an important task facilitating many applications such as reading comprehension~\cite{dasigi-etal-2019-quoref} and text summarization~\cite{azzam-etal-1999-using}.

\citet{lee-etal-2017-end} proposed the first neural end-to-end architecture for coreference resolution. Most recent systems use it as a backbone and employ better scoring functions~\cite{zhang-etal-2018-neural-coreference}, pruning procedures~\cite{lee-etal-2018-higher}, or token representations~\cite{joshi-etal-2019-bert, joshi2020spanbert}.\footnote{Except \citet{wu-etal-2020-corefqa} which has not seen wide adoption.} Despite this usage, little in-depth analysis has been done to better understand the inner workings of such an influential system. \citet{xu-choi-2020-revealing} analyzed the effect of the high-order inference, while \citet{subramanian-roth-2019-improving} and \citet{zhao-etal-2018-gender} respectively examined its generalizability and gender bias. Few work has inspected the interaction between its components. \citet{lu-ng-2020-conundrums} conducted oracle experiments that are related to ours, but without fine-grained control over confounding factors affecting oracle mentions. Such an understanding is important: for example, \citet{kummerfeld-klein-2013-error}'s dissection of the then-best classical coreference systems inspired many important follow-up works \cite[\emph{inter alia}]{peng-etal-2015-solving, martschat-strube-2015-latent, wiseman-etal-2016-learning}. However, it is unknown if observations on such classical feature-based and often pipelined systems extend to current neural end-to-end models.

We consider the best instantiation of this model family, SpanBERT~\cite{joshi2020spanbert} + c2f-coref~\cite{lee-etal-2018-higher}, and investigate the interaction between its two components: mention detector and mention linker. We study how their errors independently or jointly affect the final clustering.

Using the CoNLL-2012~\cite{pradhan2012conll} and PreCo~\cite{chen-etal-2018-preco} datasets, we highlight the low-precision, high-recall nature of the detector. While traditionally only recall is emphasized for the detector as a design decision (\citealp{lee2011stanford}; \citealp{lee-etal-2017-end}), we show huge degradation from noisy mentions and, perhaps surprisingly, increasing the number of candidates considered by the baseline linker only deteriorates the performance. While some classical coreference pipelines focused on detector precision~\cite{Uryupina2009DetectingAA}, it is rarely emphasized for current end-to-end systems. We hence stress the importance of a precision-recall balance for the detector and demonstrate how pruning hyperparameters, in addition to reducing computational complexity, control this trade-off. However, we show the difficulty of obtaining a precise detector by demonstrating the importance of anaphoricity decisions and the inability of the detector to make them. Finally, we highlight the high potential of the linker and that the remaining errors mainly involve pronoun resolution. We hope this work sheds light on the internals of the mainstream coreference system and, with our proposed next steps, catalyze future research. We believe some of our findings may also transfer to other tasks with a similar joint span detection and span (pair) classification architecture, such as SRL~\cite{he-etal-2018-jointly}, IE~\cite{luan-etal-2019-general}, and entity linking~\cite{kolitsas-etal-2018-end}.  See \citet{jiang-etal-2020-generalizing} which subsumes many other tasks under such a span-based framework.

\section{Background}

\paragraph{Model} We study the coarse-to-fine coreference system (c2f-coref; \citealt{lee-etal-2018-higher}). It assigns an antecedent for every span in a document of length $T$, including a dummy that indicates non-mentions or non-anaphoric mentions. The final clustering is the transitive closure of connected spans.
The system consists of a mention detector and a mention linker. The detector scores all $O(T^2)$ spans up to length $L$ and outputs the $\lambda T$ highest-scoring spans as possibly anaphoric mentions. The linker links each mention candidate with the highest-scoring antecedent among $K$ ones. Hyperparameters $L$, $\lambda$, and $K$ control the number of considered spans and antecedents, reducing computational complexity.

\paragraph{Data} CoNLL-2012 is the most common dataset to test coreference models. However, it lacks singleton mention annotation~\cite{pradhan2012conll}.

Singleton, or non-anaphoric, mentions do not co-refer with other spans, e.g. ``The dog'' in ``[The dog] barks.'' However, they may become anaphoric in another context, e.g. ``[The dog] barks at [itself].'' Being a mention is a span's inherent property, while anaphoricity, whether or not a mention co-refers, is context-dependent. We use ``all mentions'' to refer to the union of singleton and anaphoric mentions.

To understand the effect of singleton mentions, we heuristically generate all mentions for CoNLL-12 (\S\ref{sec:heuristics}) for relevant experiments. We also experiment with PreCo, a coreference dataset with annotated singleton mentions.  We do all analyses on development sets and report dataset statistics in \S\ref{sec:dataset-statistics}.

\section{Experiments}

\paragraph{Settings} We embed tokens with SpanBERT-large, a pre-trained transformer~\cite{vaswani2017attention} with state-of-the-art performance in coreference resolution.
We choose $L = 30,\lambda = 0.4,K = 50$.
We only keep the first 110 sentences per document during training. To reduce confounding factors, we do not use speaker and genre metadata.%

\paragraph{``Original'' System} refers to a standard SpanBERT + c2f-coref trained baseline. Its F$_1$ score\footnote{We use coreference F$_1$ to refer to the average F$_1$ of MUC, B$^3$, and CEAF$_{\phi_4}$, the most common coreference metric.} is reported in Table~\ref{tab:original}, similar to the results in \citet{joshi2020spanbert} considering we disregard metadata.

\begin{table}[t]
\centering
\begin{tabular}{c|cc}

\hline
& CoNLL-12 & PreCo \\ \hline
Coref F$_1$ & 79.17 & 85.04 \\ \hline
A\textsc{na}.\xspace P & 28.37 & 39.23 \\
A\textsc{na}.\xspace R & 96.42 & 98.40 \\
A\textsc{ll}\xspace P & 82.04 & 76.55 \\
A\textsc{ll}\xspace R & 57.35 & 95.98 \\ \hline

\end{tabular}
\caption{\label{tab:original} Original system coreference F$_1$ and precision / recall for anaphoric mentions (A\textsc{na}.\xspace) and all mentions (A\textsc{ll}\xspace) on CoNLL-12 and PreCo development sets.}
\vspace{-0.3cm}
\end{table}

\paragraph{Oracles} We build oracle detectors where, starting from the original system's mention candidates (its detector output), we either remove all non-gold mentions (prefect precision), add all missing gold mentions (perfect recall), or both (perfect precision \& recall). We give the altered, rather than the original, mention candidates to the linker. We consider both anaphoric mentions and all mentions as gold mentions and modify either in a post-hoc manner or re-train the system with the altered candidates.
To control for a non-trainable detector, we train only a linker reusing the original system's mention candidates, dubbed \textbf{Fixed Detector}. We consider this baseline as the comparison target for the oracles. Besides oracle detectors, we also build an oracle linker that assigns the correct antecedent (including dummy) to each of the $\lambda T$ mention candidates.

\section{Precision-Recall Trade-Off for the Mention Detector}
\label{sec:pr-re-analysis}

\begin{table*}[t]
\centering
\begin{tabular}{c|l|c|cccccc}
\hline
\multicolumn{2}{c|}{} & Span & Conflated & Extra & Extra & Divided & Missing & Missing \\
\multicolumn{2}{c|}{} & Error & Entities & Mention & Entity & Entity & Mention & Entity \\ \hline
\multicolumn{2}{c|}{Original} & 1.6 & 3.1 & 1.3 & 3.1 & 2.6 & 2.1 & 4.5 \\ \hline
\multicolumn{2}{c|}{Fixed Detector} & 1.7 & 3.1 & 1.7 & 3.3 & 2.8 & 2.1 & 4.7 \\ \hline

A\textsc{na}.\xspace & Perfect P & 0.0 & 2.4 & 0.0 & 0.0 & 2.0 & 2.6 & 5.7 \\
Post-hoc & Perfect R & 1.6 & 3.1 & 1.4 & 3.1 & 2.6 & 2.0 & 4.4 \\
Oracle & Perfect P\&R & 0.0 & 2.3 & 0.0 & 0.0 & 1.9 & 2.5 & 5.5 \\ \hline

A\textsc{na}.\xspace & Perfect P & 0.0 & 3.4 & 0.0 & 0.0 & 0.9 & 1.5 & 1.4 \\
Re-train & Perfect R & 0.7 & 3.1 & 2.0 & 3.4 & 2.6 & 1.5 & 4.4 \\
Oracle & Perfect P\&R & 0.0 & 3.0 & 0.0 & 0.0 & 1.0 & 0.4 & 0.3 \\ \hline

\end{tabular}
\caption{\label{tab:fix-errors} The F$_1$ score improvement after fixing different types of errors on the CoNLL-12 development set. The errors are independently fixed after span errors are fixed. The categorization is from \citet{kummerfeld-klein-2013-error}.}
\vspace{-0.3cm}
\end{table*}

\begin{table}[t]
\centering
\begin{tabular}{c|l|cc}
\hline
\multicolumn{2}{c|}{} & CoNLL-12 & PreCo \\ \hline

\multicolumn{2}{c|}{Fixed Detector} & 78.28 & 84.64 \\ \hline

A\textsc{na}.\xspace & Perfect P & 86.02 & 90.31 \\
Post-hoc & Perfect R & 79.37 & 85.17 \\
Oracle & Perfect P\&R & 86.28 & 90.45 \\ \hline

A\textsc{na}.\xspace & Perfect P & 89.98 & 95.09 \\
Re-train & Perfect R & 79.65 & 85.22 \\
Oracle & Perfect P\&R & 92.39 & 96.50 \\ \hline

A\textsc{ll}\xspace & Perfect P & 79.48 & 88.37 \\
Re-train & Perfect R & 78.52 & 85.23 \\
Oracle & Perfect P\&R & 80.05 & 89.13 \\ \hline

\multicolumn{2}{c|}{Oracle Linker} & 97.07 & 98.69 \\ \hline

\end{tabular}
\caption{\label{tab:oracles} Baseline and oracle coreference F$_1$ for anaphoric mentions (A\textsc{na}.\xspace) and all mentions (A\textsc{ll}\xspace) on CoNLL-12 and PreCo development sets. ``Fixed Detector'' is the baseline with a non-trainable detector. The middle three sections are oracle detectors with perfect candidate precision/recall. The last row is an oracle linker that always makes correct antecedent decisions.}
\vspace{-0.3cm}
\end{table}

Traditionally, coreference systems heavily favor recall over precision for the detector ~\cite{lee2011stanford} as the linker cannot recover missed mentions. Similarly, our c2f-coref system gets \textgreater96\% anaphoric mention recall yet only \textless40\% precision (Table~\ref{tab:original}). We therefore explore if detector recall is always more important than its precision. If more spans are considered by increasing the max span width $L$ or the number of spans considered per word $\lambda$, will the system performance necessarily improve? In the extreme case, if we hypothetically had enough compute that allows the linker to consider all $O(T^4)$ span-antecedent pairs, should we simply remove the pruning in the detector?

\paragraph{The Aggregated Importance of Precision}
For all oracles in Table~\ref{tab:oracles}, fixing precision yields a larger improvement than recall, especially with anaphoric mentions. %
This highlights the importance of detector precision and the extent to which the linker suffers from noisy mention candidates. In Table~\ref{tab:fix-errors}, we present the F$_1$ improvement after independently fixing categorized errors following~\citet{kummerfeld-klein-2013-error}.\footnote{Span (boundary) errors are fixed before independently fixing all others. The numbers do not add up to the performance gap due to error type interactions.} Noisy candidates result in extra mention and extra entity errors, fixing which accounts for more than half of the $\approx$8 F$_1$ gap between the post-hoc perfect precision oracle and the baseline for CoNLL-12 (Table~\ref{tab:oracles}). Furthermore, re-training the system to leverage the distributional shift of the absence of noise leads to another $\approx$4 and 5 F$_1$ increase (CoNLL-12/PreCo).

To analyze how higher detector precision helps the linker, we examine the coreference score the linker assigns to every span-antecedent pair.
The anaphoric mention re-trained perfect precision oracle has an average score of --13.0 on CoNLL-12, higher than --15.1 with perfect recall. Among only correct span-antecedent pairs, these scores are 11.7 and 7.1, with the same pattern. This indicates that the noise with perfect recall prevents the linker from reliably assigning high coreference scores, even for correct links. The effect of higher coreference scores also shows in that, compared with perfect recall, the perfect precision oracle produces on average larger (4.44 vs. 4.26 entities) and longer-distance (154 vs. 152 tokens spanned) clusters.

We also see this effect by examining the amount of improvement with reduced noise in Table~\ref{tab:fix-errors}. In the anaphoric mention post-hoc oracles, as expected, fixing precision results in fewer extra mention/entity errors and more missing errors, while the perfect recall oracle behaves conversely. However, when re-trained, the perfect precision oracle has much fewer missing entities, even fewer than with perfect recall. This is surprising as the latter considers more candidates. The reason is likely that the linker learns to leverage the absence of noise and reliably assigns high coreference scores. Despite some incorrect links leading to more conflated entities, the many correct ones drastically reduce missing mention/entity errors. On the other hand, the noise in the perfect recall or the original system prevents consistent high scores, resulting in more missing mentions and entities. Hence, the improvement with perfect precision partly stems from the linker's increased confidence in assigning coreference scores when \emph{not} tasked with ignoring non-mentions (and singletons) in noisy candidates.

\begin{table}[t]
\centering
\begin{tabular}{@{\hspace{4pt}}c|c|c|c|c@{\hspace{2pt}}}
\hline
\multicolumn{1}{c|}{} & \multicolumn{2}{c|}{CoNLL-12} & \multicolumn{2}{c}{PreCo} \\ \cline{2-5}
\multicolumn{1}{c|}{} & \# Op & Op effect & \# Op & Op effect \\ \hline
A\textsc{na}.\xspace; P & 135.9 & 0.086 & 79.2 & 0.132 \\
A\textsc{na}.\xspace; R & 1.9 & \textbf{0.715} & 0.8 & \textbf{0.709} \\ \hline
A\textsc{ll}\xspace; P & 34.1 & \textbf{0.035} & 30.6 & 0.122 \\
A\textsc{ll}\xspace; R & 115.3 & 0.002 & 4.1 & \textbf{0.143} \\ \hline

\end{tabular}
\vspace{-0.05cm}
\caption{\label{tab:operation-effects} The number of addition/removal operations needed for the oracle candidates, and the oracle performance increase, in F$_1$, amortized over each operation. Boldface indicates the higher per-operation effect between perfect precision and recall in each category.}
\vspace{-0.1cm}
\end{table}

\begin{table}[t]
\centering
\fontsize{9.5}{12} \selectfont
\begin{tabular}{@{\hspace{4pt}}c|c|c|c|c|c|c@{\hspace{4pt}}}

\hline
$L$ & 30 & 32 & 34 & 36 & 38 & 40 \\ \hline
F$_1$ & \textbf{79.17} & 78.76 & 78.86 & 79.03 & 78.88 & 78.85 \\ \hline \hline
$\lambda$ & 0.4 & 0.45 & 0.5 & 0.55 & 0.6 & 0.65 \\ \hline
F$_1$ & \textbf{79.17} & 78.99 & 79.15 & 78.35 & 79.05 & 78.85 \\ \hline

\end{tabular}
\vspace{-0.05cm}
\caption{\label{tab:more-spans} CoNLL-12 development F$_1$ with increased max span width $L$ or the number of spans considered per word $\lambda$. The first column is the original setting. Boldface indicates the best performance.}
\vspace{-0.3cm}
\end{table}

\paragraph{The Average Importance of Recall}
The large improvement from fixing precision may be due to its larger original headroom than recall (Table~\ref{tab:original}). We compute the number of operations (span addition/removal) needed for each oracle and the average F$_1$ improvement per operation in Table~\ref{tab:operation-effects}. For anaphoric mentions, recall has 5-8$\times$ the average effect of precision.\footnote{CoNLL-12 with all mentions has a different pattern as we noisily generated singletons in a recall-oriented way.} If we control the number of operations by re-training an anaphoric mention (semi-)perfect precision oracle removing only as many top-scoring extra spans as the number of missing correct spans (rather than removing all extra spans), it gets 79.08 and 85.01 F$_1$ on CoNLL-12 and PreCo, lower than the perfect recall oracle with 79.65 and 85.22. It is therefore only due to the low-precision high-recall nature of the original detector that precision is more important in aggregate.

\paragraph{Precision-Recall Trade-Off}
We return to the original question: if we had more compute, is it always beneficial to consider more spans in the detector? From our results, while recall is important, an imprecise detector has substantial adverse effects by increasing the linker's learning burden. Indeed, Table~\ref{tab:more-spans} shows that increasing the max span width by up to 33\% or the spans considered per word by up to 38\% only degrades the performance. As the extra low-scoring spans are mostly noise, we slightly increase recall but more heavily decrease precision, causing more harm than benefit. Hence, besides saving computation, these hyperparameters also balance the precision-recall trade-off. Future work should hence put more emphasis on precision which is often overlooked in end-to-end systems.

\section{Difficulties Facing Each Component}
\label{sec:focus}

\subsection{The Detector's Difficulty With Anaphoricity Decisions}
\label{sec:detector-difficulty}

Despite its large aggregated improvement, i.e. $\approx$11.7 and 10.5 F$_1$ for CoNLL-12 and PreCo, perfect anaphoric mention precision requires perfectly distinguishing anaphoric from singleton mentions. These anaphoricity decisions in fact account for most of the improvement, $\approx$10.5 and 6.7 F$_1$ (Table~\ref{tab:oracles}, anaphoric v.s. all mentions perfect precision).\footnote{\citet{chen-etal-2018-preco} observed a similar pattern on an LSTM architecture that less directly receives global information which is important for anaphoricity decisions. We confirm that this still holds on transformers with a larger receptive field.} However, the detector, as a span classifier, does not explicitly model inter-span anaphoric relationships. To test this architecture's ability to distinguish anaphoric from singleton mentions, we build two span classifiers with the same structure as the detector, supervised with sigmoid loss, that recognize all mentions and anaphoric mentions in PreCo. The former achieves 79.89 classification F$_1$ while the latter only 54.32, showing the inability of a span classifier to make anaphoricity decisions.

To better understand this difficulty, we define a confusion index as singleton recall divided by anaphoric mention recall. It correlates with the classifier's inability to identify anaphoricity. Ideally, this value should be close to 0, recalling more anaphoric mentions and fewer singletons. A random classifier incapable of distinguishing between the two has an expected confusion index of 1.

\begin{table}[t]
\centering
\fontsize{9.5}{11} \selectfont
\begin{tabular}{@{\hspace{1pt}} c@{\hspace{1pt}}|@{\hspace{3pt}}p{6cm} @{\hspace{2pt}}}
\hline
Error Type & \multicolumn{1}{c}{\multirow{2}{*}{Example}} \\
(\#) \\ \hline
\multirow{4}{*}{\shortstack{Pronoun\\(109)}} & ... a cross-sea bridge connecting \textbf{Hong Kong, Zhuhai, and Macao} . \\ \cline{2-2}
& ... after \textbf{their} return, Macao, and Hong Kong, the two special administrative regions ... \\ \hline
\multirow{4}{*}{\shortstack{Exact\\Match\\(6)}} & The most important thing about \textbf{Disney} is that it is a global brand . \\ \cline{2-2}
& The subway to \textbf{Disney} has already been constructed . \\ \hline
\multirow{4}{*}{\shortstack{Head\\Match\\(11)}} & \textbf{Ten landmark buildings located on Hong Kong Island} reveal themselves ...  \\ \cline{2-2}
& ... \textbf{those private , er , buildings} , that is , the business community , ah , is willing to ... \\ \hline
Other & And \textbf{Dr. Andy Henry} notices something else \\ \cline{2-2}
Match (7) & \textbf{Dr. Mann} says they 've narrowed it down ... \\ \hline
\multirow{4}{*}{\shortstack{Semantic\\Proximity\\(12)}} & ... \textbf{Hong Kong cinema} has nurtured many internationally renowned directors ... \\ \cline{2-2}
& ... memorializing \textbf{Hong Kong 's  100 - year film history} . \\ \hline
\multirow{2}{*}{\shortstack{Others\\(5)}} & But [\textbf{Paul Kelly}] [\textbf{Steve Sodbury}] and Mel Anderson ... had no idea ... \\ \hline

\end{tabular}

\caption{\label{tab:conflated-examples} Examples of categorized conflated entity errors in the CoNLL-12 development set with a perfect detector. Following past studies \cite{kummerfeld-klein-2013-error,joshi-etal-2019-bert}, we consider all deictic terms as pronouns. Each example contains two incorrectly linked entities in bold. Square brackets are added to separate mentions.}
\vspace{-0.3cm}
\end{table}

The anaphoric mention classifier above has a confusion index of 0.81, showing its inability to make anaphoricity decisions even when explicitly trained with the signal. If we only consider text appearing as both singleton and anaphoric mentions in the same document, demanding contextual reasoning by disregarding obvious anaphoric mentions such as pronouns, the confusion index degrades to 0.997. Hence, the classifier is poor at leveraging self-attentive contextual cues to make anaphoricity decisions without explicit inter-span relational modeling. In \S\ref{sec:confusion-index-vs-span-width} we also show the degradation of the confusion index with shorter spans.

Given the importance of anaphoric mention precision (\S\ref{sec:pr-re-analysis}), more research in improving anaphoricity decisions in the detector would be fruitful, for example, by more explicitly attending to neighboring spans. Alternatively, as \citet{zhong-chen-2021-frustratingly} showed the benefit of disentangling the span representations for entity detection and relation extraction in information extraction based on the intuition that they are disparate tasks, one may split the task of anaphoricity decision from mention linking and introduce a separately parameterized anaphoricity module, similarly considering the discrepancy between the two tasks. \citet{recasens-etal-2013-life,moosavi-strube-2016-search}; \textit{inter alia} have pursued similar ideas in the pre-neural era, but it has still not yet been explored with deep models.

\subsection{The Linker's Errors}

While the detector struggles with anaphoricity decisions, the linker explicitly models anaphoricity by assigning the dummy to extra mentions. %
It is hence also viable to determine anaphoricity in the linker.
Indeed, the current detector would suffice with a stronger linker: in Table~\ref{tab:oracles}, the oracle linker gets near-perfect scores with the original mentions (not perfect since the candidates are not gold).\footnote{A modified oracle linker that only considers coarse-pruned antecedents~\cite{lee-etal-2018-higher} still gets 96.61 and 98.65 F$_1$ on CoNLL-12 and PreCo. The small difference compared to considering all antecedents also shows that, with a strong linker, coarse-to-fine pruning has only negligible performance impact while substantially reducing the decision space.}

To analyze the remaining non-anaphoricity linker errors, we assume a perfect anaphoric mention detector. Here, conflated entities is the single major error source (last row of Table~\ref{tab:fix-errors}). Table~\ref{tab:conflated-examples} shows 150 manually categorized conflated entities in the CoNLL-12 development set.\footnote{\citet{joshi-etal-2019-bert} and \citet{lu-ng-2020-conundrums} conducted similar analyses but we study in a more controlled setting by excluding detector errors and focusing on entity conflation, the largest remaining error source.} Suboptimal pronoun resolution is the biggest issue, and the linker also tends to link spans with various degrees of text match or semantic proximity.
Within pronoun errors, the most common case is a pronoun linked to an incorrect nominal (in Table~\ref{tab:conflated-examples}), occurring 43 times. Sometimes two pronouns, often identical, are incorrectly linked, a case that necessitates better higher-order inference. Third person pronouns with different referents are conflated 29 times. Errors with first or second person pronouns occur 37 times, usually due to speaker switching.

Similar to \S\ref{sec:detector-difficulty}, separately parameterizing the linker's encoder may help reduce conflation: intuitively, the span representation for mention detection may promote homogeneity. Meanwhile, the lack of discerning span-internal content for certain error types including pronoun resolution and exact match, combined with current systems' trend to rely on such cues~\cite{lu-ng-2020-conundrums}, calls for more focus on improving their contextual reasoning.

\section{Conclusion}

We analyzed the complex interaction between the mention detector and linker in the mainstream coarse-to-fine coreference system. Using oracle experiments, we showed that, while detector recall is important, higher anaphoric mention precision would lead to dramatically better linker performance, though achieving this is difficult. We also demonstrated that the oracle linker performance is near perfect and that the vast majority of remaining linker errors besides anaphoricity decisions are about pronoun resolution. We hope these discoveries will help future coreference research.%

\section{Introduction}

Coreference resolution identifies mentions in a document that co-refer to the same entity. It is an important task facilitating many applications such as reading comprehension~\cite{dasigi-etal-2019-quoref} and text summarization~\cite{azzam-etal-1999-using}.

\citet{lee-etal-2017-end} proposed the first neural end-to-end architecture for coreference resolution. Most recent systems use it as a backbone and employ better scoring functions~\cite{zhang-etal-2018-neural-coreference}, pruning procedures~\cite{lee-etal-2018-higher}, or token representations~\cite{joshi-etal-2019-bert, joshi2020spanbert}.\footnote{Except \citet{wu-etal-2020-corefqa} which has not seen wide adoption.} Despite this usage, little in-depth analysis has been done to better understand the inner workings of such an influential system. \citet{xu-choi-2020-revealing} analyzed the effect of the high-order inference, while \citet{subramanian-roth-2019-improving} and \citet{zhao-etal-2018-gender} respectively examined its generalizability and gender bias. Few work has inspected the interaction between its components. \citet{lu-ng-2020-conundrums} conducted oracle experiments that are related to ours, but without fine-grained control over confounding factors affecting oracle mentions. Such an understanding is important: for example, \citet{kummerfeld-klein-2013-error}'s dissection of the then-best classical coreference systems inspired many important follow-up works \cite[\emph{inter alia}]{peng-etal-2015-solving, martschat-strube-2015-latent, wiseman-etal-2016-learning}. However, it is unknown if observations on such classical feature-based and often pipelined systems extend to current neural end-to-end models.

We consider the best instantiation of this model family, SpanBERT~\cite{joshi2020spanbert} + c2f-coref~\cite{lee-etal-2018-higher}, and investigate the interaction between its two components: mention detector and mention linker. We study how their errors independently or jointly affect the final clustering.

Using the CoNLL-2012~\cite{pradhan2012conll} and PreCo~\cite{chen-etal-2018-preco} datasets, we highlight the low-precision, high-recall nature of the detector. While traditionally only recall is emphasized for the detector as a design decision (\citealp{lee2011stanford}; \citealp{lee-etal-2017-end}), we show huge degradation from noisy mentions and, perhaps surprisingly, increasing the number of candidates considered by the baseline linker only deteriorates the performance. While some classical coreference pipelines focused on detector precision~\cite{Uryupina2009DetectingAA}, it is rarely emphasized for current end-to-end systems. We hence stress the importance of a precision-recall balance for the detector and demonstrate how pruning hyperparameters, in addition to reducing computational complexity, control this trade-off. However, we show the difficulty of obtaining a precise detector by demonstrating the importance of anaphoricity decisions and the inability of the detector to make them. Finally, we highlight the high potential of the linker and that the remaining errors mainly involve pronoun resolution. We hope this work sheds light on the internals of the mainstream coreference system and, with our proposed next steps, catalyze future research. We believe some of our findings may also transfer to other tasks with a similar joint span detection and span (pair) classification architecture, such as SRL~\cite{he-etal-2018-jointly}, IE~\cite{luan-etal-2019-general}, and entity linking~\cite{kolitsas-etal-2018-end}.  See \citet{jiang-etal-2020-generalizing} which subsumes many other tasks under such a span-based framework.

\section{Background}

\paragraph{Model} We study the coarse-to-fine coreference system (c2f-coref; \citealt{lee-etal-2018-higher}). It assigns an antecedent for every span in a document of length $T$, including a dummy that indicates non-mentions or non-anaphoric mentions. The final clustering is the transitive closure of connected spans.
The system consists of a mention detector and a mention linker. The detector scores all $O(T^2)$ spans up to length $L$ and outputs the $\lambda T$ highest-scoring spans as possibly anaphoric mentions. The linker links each mention candidate with the highest-scoring antecedent among $K$ ones. Hyperparameters $L$, $\lambda$, and $K$ control the number of considered spans and antecedents, reducing computational complexity.

\paragraph{Data} CoNLL-2012 is the most common dataset to test coreference models. However, it lacks singleton mention annotation~\cite{pradhan2012conll}.

Singleton, or non-anaphoric, mentions do not co-refer with other spans, e.g. ``The dog'' in ``[The dog] barks.'' However, they may become anaphoric in another context, e.g. ``[The dog] barks at [itself].'' Being a mention is a span's inherent property, while anaphoricity, whether or not a mention co-refers, is context-dependent. We use ``all mentions'' to refer to the union of singleton and anaphoric mentions.

To understand the effect of singleton mentions, we heuristically generate all mentions for CoNLL-12 (\S\ref{sec:heuristics}) for relevant experiments. We also experiment with PreCo, a coreference dataset with annotated singleton mentions.  We do all analyses on development sets and report dataset statistics in \S\ref{sec:dataset-statistics}.

\section{Experiments}

\paragraph{Settings} We embed tokens with SpanBERT-large, a pre-trained transformer~\cite{vaswani2017attention} with state-of-the-art performance in coreference resolution.
We choose $L = 30,\lambda = 0.4,K = 50$.
We only keep the first 110 sentences per document during training. To reduce confounding factors, we do not use speaker and genre metadata.%

\paragraph{``Original'' System} refers to a standard SpanBERT + c2f-coref trained baseline. Its F$_1$ score\footnote{We use coreference F$_1$ to refer to the average F$_1$ of MUC, B$^3$, and CEAF$_{\phi_4}$, the most common coreference metric.} is reported in Table~\ref{tab:original}, similar to the results in \citet{joshi2020spanbert} considering we disregard metadata.

\begin{table}[t]
\centering
\begin{tabular}{c|cc}

\hline
& CoNLL-12 & PreCo \\ \hline
Coref F$_1$ & 79.17 & 85.04 \\ \hline
A\textsc{na}.\xspace P & 28.37 & 39.23 \\
A\textsc{na}.\xspace R & 96.42 & 98.40 \\
A\textsc{ll}\xspace P & 82.04 & 76.55 \\
A\textsc{ll}\xspace R & 57.35 & 95.98 \\ \hline

\end{tabular}
\caption{\label{tab:original} Original system coreference F$_1$ and precision / recall for anaphoric mentions (A\textsc{na}.\xspace) and all mentions (A\textsc{ll}\xspace) on CoNLL-12 and PreCo development sets.}
\vspace{-0.3cm}
\end{table}

\paragraph{Oracles} We build oracle detectors where, starting from the original system's mention candidates (its detector output), we either remove all non-gold mentions (prefect precision), add all missing gold mentions (perfect recall), or both (perfect precision \& recall). We give the altered, rather than the original, mention candidates to the linker. We consider both anaphoric mentions and all mentions as gold mentions and modify either in a post-hoc manner or re-train the system with the altered candidates.
To control for a non-trainable detector, we train only a linker reusing the original system's mention candidates, dubbed \textbf{Fixed Detector}. We consider this baseline as the comparison target for the oracles. Besides oracle detectors, we also build an oracle linker that assigns the correct antecedent (including dummy) to each of the $\lambda T$ mention candidates.

\section{Precision-Recall Trade-Off for the Mention Detector}
\label{sec:pr-re-analysis}

\begin{table*}[t]
\centering
\begin{tabular}{c|l|c|cccccc}
\hline
\multicolumn{2}{c|}{} & Span & Conflated & Extra & Extra & Divided & Missing & Missing \\
\multicolumn{2}{c|}{} & Error & Entities & Mention & Entity & Entity & Mention & Entity \\ \hline
\multicolumn{2}{c|}{Original} & 1.6 & 3.1 & 1.3 & 3.1 & 2.6 & 2.1 & 4.5 \\ \hline
\multicolumn{2}{c|}{Fixed Detector} & 1.7 & 3.1 & 1.7 & 3.3 & 2.8 & 2.1 & 4.7 \\ \hline

A\textsc{na}.\xspace & Perfect P & 0.0 & 2.4 & 0.0 & 0.0 & 2.0 & 2.6 & 5.7 \\
Post-hoc & Perfect R & 1.6 & 3.1 & 1.4 & 3.1 & 2.6 & 2.0 & 4.4 \\
Oracle & Perfect P\&R & 0.0 & 2.3 & 0.0 & 0.0 & 1.9 & 2.5 & 5.5 \\ \hline

A\textsc{na}.\xspace & Perfect P & 0.0 & 3.4 & 0.0 & 0.0 & 0.9 & 1.5 & 1.4 \\
Re-train & Perfect R & 0.7 & 3.1 & 2.0 & 3.4 & 2.6 & 1.5 & 4.4 \\
Oracle & Perfect P\&R & 0.0 & 3.0 & 0.0 & 0.0 & 1.0 & 0.4 & 0.3 \\ \hline

\end{tabular}
\caption{\label{tab:fix-errors} The F$_1$ score improvement after fixing different types of errors on the CoNLL-12 development set. The errors are independently fixed after span errors are fixed. The categorization is from \citet{kummerfeld-klein-2013-error}.}
\vspace{-0.3cm}
\end{table*}

\begin{table}[t]
\centering
\begin{tabular}{c|l|cc}
\hline
\multicolumn{2}{c|}{} & CoNLL-12 & PreCo \\ \hline

\multicolumn{2}{c|}{Fixed Detector} & 78.28 & 84.64 \\ \hline

A\textsc{na}.\xspace & Perfect P & 86.02 & 90.31 \\
Post-hoc & Perfect R & 79.37 & 85.17 \\
Oracle & Perfect P\&R & 86.28 & 90.45 \\ \hline

A\textsc{na}.\xspace & Perfect P & 89.98 & 95.09 \\
Re-train & Perfect R & 79.65 & 85.22 \\
Oracle & Perfect P\&R & 92.39 & 96.50 \\ \hline

A\textsc{ll}\xspace & Perfect P & 79.48 & 88.37 \\
Re-train & Perfect R & 78.52 & 85.23 \\
Oracle & Perfect P\&R & 80.05 & 89.13 \\ \hline

\multicolumn{2}{c|}{Oracle Linker} & 97.07 & 98.69 \\ \hline

\end{tabular}
\caption{\label{tab:oracles} Baseline and oracle coreference F$_1$ for anaphoric mentions (A\textsc{na}.\xspace) and all mentions (A\textsc{ll}\xspace) on CoNLL-12 and PreCo development sets. ``Fixed Detector'' is the baseline with a non-trainable detector. The middle three sections are oracle detectors with perfect candidate precision/recall. The last row is an oracle linker that always makes correct antecedent decisions.}
\vspace{-0.3cm}
\end{table}

Traditionally, coreference systems heavily favor recall over precision for the detector ~\cite{lee2011stanford} as the linker cannot recover missed mentions. Similarly, our c2f-coref system gets \textgreater96\% anaphoric mention recall yet only \textless40\% precision (Table~\ref{tab:original}). We therefore explore if detector recall is always more important than its precision. If more spans are considered by increasing the max span width $L$ or the number of spans considered per word $\lambda$, will the system performance necessarily improve? In the extreme case, if we hypothetically had enough compute that allows the linker to consider all $O(T^4)$ span-antecedent pairs, should we simply remove the pruning in the detector?

\paragraph{The Aggregated Importance of Precision}
For all oracles in Table~\ref{tab:oracles}, fixing precision yields a larger improvement than recall, especially with anaphoric mentions. %
This highlights the importance of detector precision and the extent to which the linker suffers from noisy mention candidates. In Table~\ref{tab:fix-errors}, we present the F$_1$ improvement after independently fixing categorized errors following~\citet{kummerfeld-klein-2013-error}.\footnote{Span (boundary) errors are fixed before independently fixing all others. The numbers do not add up to the performance gap due to error type interactions.} Noisy candidates result in extra mention and extra entity errors, fixing which accounts for more than half of the $\approx$8 F$_1$ gap between the post-hoc perfect precision oracle and the baseline for CoNLL-12 (Table~\ref{tab:oracles}). Furthermore, re-training the system to leverage the distributional shift of the absence of noise leads to another $\approx$4 and 5 F$_1$ increase (CoNLL-12/PreCo).

To analyze how higher detector precision helps the linker, we examine the coreference score the linker assigns to every span-antecedent pair.
The anaphoric mention re-trained perfect precision oracle has an average score of --13.0 on CoNLL-12, higher than --15.1 with perfect recall. Among only correct span-antecedent pairs, these scores are 11.7 and 7.1, with the same pattern. This indicates that the noise with perfect recall prevents the linker from reliably assigning high coreference scores, even for correct links. The effect of higher coreference scores also shows in that, compared with perfect recall, the perfect precision oracle produces on average larger (4.44 vs. 4.26 entities) and longer-distance (154 vs. 152 tokens spanned) clusters.

We also see this effect by examining the amount of improvement with reduced noise in Table~\ref{tab:fix-errors}. In the anaphoric mention post-hoc oracles, as expected, fixing precision results in fewer extra mention/entity errors and more missing errors, while the perfect recall oracle behaves conversely. However, when re-trained, the perfect precision oracle has much fewer missing entities, even fewer than with perfect recall. This is surprising as the latter considers more candidates. The reason is likely that the linker learns to leverage the absence of noise and reliably assigns high coreference scores. Despite some incorrect links leading to more conflated entities, the many correct ones drastically reduce missing mention/entity errors. On the other hand, the noise in the perfect recall or the original system prevents consistent high scores, resulting in more missing mentions and entities. Hence, the improvement with perfect precision partly stems from the linker's increased confidence in assigning coreference scores when \emph{not} tasked with ignoring non-mentions (and singletons) in noisy candidates.

\begin{table}[t]
\centering
\begin{tabular}{@{\hspace{4pt}}c|c|c|c|c@{\hspace{2pt}}}
\hline
\multicolumn{1}{c|}{} & \multicolumn{2}{c|}{CoNLL-12} & \multicolumn{2}{c}{PreCo} \\ \cline{2-5}
\multicolumn{1}{c|}{} & \# Op & Op effect & \# Op & Op effect \\ \hline
A\textsc{na}.\xspace; P & 135.9 & 0.086 & 79.2 & 0.132 \\
A\textsc{na}.\xspace; R & 1.9 & \textbf{0.715} & 0.8 & \textbf{0.709} \\ \hline
A\textsc{ll}\xspace; P & 34.1 & \textbf{0.035} & 30.6 & 0.122 \\
A\textsc{ll}\xspace; R & 115.3 & 0.002 & 4.1 & \textbf{0.143} \\ \hline

\end{tabular}
\vspace{-0.05cm}
\caption{\label{tab:operation-effects} The number of addition/removal operations needed for the oracle candidates, and the oracle performance increase, in F$_1$, amortized over each operation. Boldface indicates the higher per-operation effect between perfect precision and recall in each category.}
\vspace{-0.1cm}
\end{table}

\begin{table}[t]
\centering
\fontsize{9.5}{12} \selectfont
\begin{tabular}{@{\hspace{4pt}}c|c|c|c|c|c|c@{\hspace{4pt}}}

\hline
$L$ & 30 & 32 & 34 & 36 & 38 & 40 \\ \hline
F$_1$ & \textbf{79.17} & 78.76 & 78.86 & 79.03 & 78.88 & 78.85 \\ \hline \hline
$\lambda$ & 0.4 & 0.45 & 0.5 & 0.55 & 0.6 & 0.65 \\ \hline
F$_1$ & \textbf{79.17} & 78.99 & 79.15 & 78.35 & 79.05 & 78.85 \\ \hline

\end{tabular}
\vspace{-0.05cm}
\caption{\label{tab:more-spans} CoNLL-12 development F$_1$ with increased max span width $L$ or the number of spans considered per word $\lambda$. The first column is the original setting. Boldface indicates the best performance.}
\vspace{-0.3cm}
\end{table}

\paragraph{The Average Importance of Recall}
The large improvement from fixing precision may be due to its larger original headroom than recall (Table~\ref{tab:original}). We compute the number of operations (span addition/removal) needed for each oracle and the average F$_1$ improvement per operation in Table~\ref{tab:operation-effects}. For anaphoric mentions, recall has 5-8$\times$ the average effect of precision.\footnote{CoNLL-12 with all mentions has a different pattern as we noisily generated singletons in a recall-oriented way.} If we control the number of operations by re-training an anaphoric mention (semi-)perfect precision oracle removing only as many top-scoring extra spans as the number of missing correct spans (rather than removing all extra spans), it gets 79.08 and 85.01 F$_1$ on CoNLL-12 and PreCo, lower than the perfect recall oracle with 79.65 and 85.22. It is therefore only due to the low-precision high-recall nature of the original detector that precision is more important in aggregate.

\paragraph{Precision-Recall Trade-Off}
We return to the original question: if we had more compute, is it always beneficial to consider more spans in the detector? From our results, while recall is important, an imprecise detector has substantial adverse effects by increasing the linker's learning burden. Indeed, Table~\ref{tab:more-spans} shows that increasing the max span width by up to 33\% or the spans considered per word by up to 38\% only degrades the performance. As the extra low-scoring spans are mostly noise, we slightly increase recall but more heavily decrease precision, causing more harm than benefit. Hence, besides saving computation, these hyperparameters also balance the precision-recall trade-off. Future work should hence put more emphasis on precision which is often overlooked in end-to-end systems.

\section{Difficulties Facing Each Component}
\label{sec:focus}

\subsection{The Detector's Difficulty With Anaphoricity Decisions}
\label{sec:detector-difficulty}

Despite its large aggregated improvement, i.e. $\approx$11.7 and 10.5 F$_1$ for CoNLL-12 and PreCo, perfect anaphoric mention precision requires perfectly distinguishing anaphoric from singleton mentions. These anaphoricity decisions in fact account for most of the improvement, $\approx$10.5 and 6.7 F$_1$ (Table~\ref{tab:oracles}, anaphoric v.s. all mentions perfect precision).\footnote{\citet{chen-etal-2018-preco} observed a similar pattern on an LSTM architecture that less directly receives global information which is important for anaphoricity decisions. We confirm that this still holds on transformers with a larger receptive field.} However, the detector, as a span classifier, does not explicitly model inter-span anaphoric relationships. To test this architecture's ability to distinguish anaphoric from singleton mentions, we build two span classifiers with the same structure as the detector, supervised with sigmoid loss, that recognize all mentions and anaphoric mentions in PreCo. The former achieves 79.89 classification F$_1$ while the latter only 54.32, showing the inability of a span classifier to make anaphoricity decisions.

To better understand this difficulty, we define a confusion index as singleton recall divided by anaphoric mention recall. It correlates with the classifier's inability to identify anaphoricity. Ideally, this value should be close to 0, recalling more anaphoric mentions and fewer singletons. A random classifier incapable of distinguishing between the two has an expected confusion index of 1.

\begin{table}[t]
\centering
\fontsize{9.5}{11} \selectfont
\begin{tabular}{@{\hspace{1pt}} c@{\hspace{1pt}}|@{\hspace{3pt}}p{6cm} @{\hspace{2pt}}}
\hline
Error Type & \multicolumn{1}{c}{\multirow{2}{*}{Example}} \\
(\#) \\ \hline
\multirow{4}{*}{\shortstack{Pronoun\\(109)}} & ... a cross-sea bridge connecting \textbf{Hong Kong, Zhuhai, and Macao} . \\ \cline{2-2}
& ... after \textbf{their} return, Macao, and Hong Kong, the two special administrative regions ... \\ \hline
\multirow{4}{*}{\shortstack{Exact\\Match\\(6)}} & The most important thing about \textbf{Disney} is that it is a global brand . \\ \cline{2-2}
& The subway to \textbf{Disney} has already been constructed . \\ \hline
\multirow{4}{*}{\shortstack{Head\\Match\\(11)}} & \textbf{Ten landmark buildings located on Hong Kong Island} reveal themselves ...  \\ \cline{2-2}
& ... \textbf{those private , er , buildings} , that is , the business community , ah , is willing to ... \\ \hline
Other & And \textbf{Dr. Andy Henry} notices something else \\ \cline{2-2}
Match (7) & \textbf{Dr. Mann} says they 've narrowed it down ... \\ \hline
\multirow{4}{*}{\shortstack{Semantic\\Proximity\\(12)}} & ... \textbf{Hong Kong cinema} has nurtured many internationally renowned directors ... \\ \cline{2-2}
& ... memorializing \textbf{Hong Kong 's  100 - year film history} . \\ \hline
\multirow{2}{*}{\shortstack{Others\\(5)}} & But [\textbf{Paul Kelly}] [\textbf{Steve Sodbury}] and Mel Anderson ... had no idea ... \\ \hline

\end{tabular}

\caption{\label{tab:conflated-examples} Examples of categorized conflated entity errors in the CoNLL-12 development set with a perfect detector. Following past studies \cite{kummerfeld-klein-2013-error,joshi-etal-2019-bert}, we consider all deictic terms as pronouns. Each example contains two incorrectly linked entities in bold. Square brackets are added to separate mentions.}
\vspace{-0.3cm}
\end{table}

The anaphoric mention classifier above has a confusion index of 0.81, showing its inability to make anaphoricity decisions even when explicitly trained with the signal. If we only consider text appearing as both singleton and anaphoric mentions in the same document, demanding contextual reasoning by disregarding obvious anaphoric mentions such as pronouns, the confusion index degrades to 0.997. Hence, the classifier is poor at leveraging self-attentive contextual cues to make anaphoricity decisions without explicit inter-span relational modeling. In \S\ref{sec:confusion-index-vs-span-width} we also show the degradation of the confusion index with shorter spans.

Given the importance of anaphoric mention precision (\S\ref{sec:pr-re-analysis}), more research in improving anaphoricity decisions in the detector would be fruitful, for example, by more explicitly attending to neighboring spans. Alternatively, as \citet{zhong-chen-2021-frustratingly} showed the benefit of disentangling the span representations for entity detection and relation extraction in information extraction based on the intuition that they are disparate tasks, one may split the task of anaphoricity decision from mention linking and introduce a separately parameterized anaphoricity module, similarly considering the discrepancy between the two tasks. \citet{recasens-etal-2013-life,moosavi-strube-2016-search}; \textit{inter alia} have pursued similar ideas in the pre-neural era, but it has still not yet been explored with deep models.

\subsection{The Linker's Errors}

While the detector struggles with anaphoricity decisions, the linker explicitly models anaphoricity by assigning the dummy to extra mentions. %
It is hence also viable to determine anaphoricity in the linker.
Indeed, the current detector would suffice with a stronger linker: in Table~\ref{tab:oracles}, the oracle linker gets near-perfect scores with the original mentions (not perfect since the candidates are not gold).\footnote{A modified oracle linker that only considers coarse-pruned antecedents~\cite{lee-etal-2018-higher} still gets 96.61 and 98.65 F$_1$ on CoNLL-12 and PreCo. The small difference compared to considering all antecedents also shows that, with a strong linker, coarse-to-fine pruning has only negligible performance impact while substantially reducing the decision space.}

To analyze the remaining non-anaphoricity linker errors, we assume a perfect anaphoric mention detector. Here, conflated entities is the single major error source (last row of Table~\ref{tab:fix-errors}). Table~\ref{tab:conflated-examples} shows 150 manually categorized conflated entities in the CoNLL-12 development set.\footnote{\citet{joshi-etal-2019-bert} and \citet{lu-ng-2020-conundrums} conducted similar analyses but we study in a more controlled setting by excluding detector errors and focusing on entity conflation, the largest remaining error source.} Suboptimal pronoun resolution is the biggest issue, and the linker also tends to link spans with various degrees of text match or semantic proximity.
Within pronoun errors, the most common case is a pronoun linked to an incorrect nominal (in Table~\ref{tab:conflated-examples}), occurring 43 times. Sometimes two pronouns, often identical, are incorrectly linked, a case that necessitates better higher-order inference. Third person pronouns with different referents are conflated 29 times. Errors with first or second person pronouns occur 37 times, usually due to speaker switching.

Similar to \S\ref{sec:detector-difficulty}, separately parameterizing the linker's encoder may help reduce conflation: intuitively, the span representation for mention detection may promote homogeneity. Meanwhile, the lack of discerning span-internal content for certain error types including pronoun resolution and exact match, combined with current systems' trend to rely on such cues~\cite{lu-ng-2020-conundrums}, calls for more focus on improving their contextual reasoning.

\section{Conclusion}

We analyzed the complex interaction between the mention detector and linker in the mainstream coarse-to-fine coreference system. Using oracle experiments, we showed that, while detector recall is important, higher anaphoric mention precision would lead to dramatically better linker performance, though achieving this is difficult. We also demonstrated that the oracle linker performance is near perfect and that the vast majority of remaining linker errors besides anaphoricity decisions are about pronoun resolution. We hope these discoveries will help future coreference research.%

redpajama_set_name: RedPajamaArXiv


The puffin festival has been inspired by the colony of approximately 20,000 puffins that nest on the RSPB seabird sanctuary of Coquet Island, just a mile off Amble.
Coquet Island has been an RSPB nature reserve since 1970 and is also the sole UK nesting colony of roseate terns. Less than 90 pairs nest on Coquet Island and use special boxes constructed by the RSPB staff to create their nests and rear their chicks.
The island is a Site of Special Scientific Interest and therefore protected all year round, to safeguard the fragile habitat and to avoid disturbing the nesting birds. Consequently Coquet Island is a true sanctuary for nesting birds and landing on the island is not allowed.
Puffins are very sociable birds that like to nest in burrows in the soft sandy soil on the island – that's 12,000 burrows!
Puffins arrive on the waters around Coquet Island at the end of March and are settled on the island in plenty of time for the start of the Puffin Festival.
Puffins like to gather in vast numbers on the top of the low cliffs around the island,to take a break from nest duty and stretch their wings!
Puffins and their pufflings leave Coquet Island at the end of July and spend the winter bobbing about on the sea!
Puffins have some special neighbours on Coquet Island - eider ducks, fulmar, kittiwakes and four species of terns and lots of grey seals.
In total over 44,000 seabirds nest on Coquet Island each year.
The Atlantic Puffin is now officially endangered, after dramatic declines in their numbers over recent years.
Puffins are now facing the same level of extinction threat as African elephants and lions. Europe is home to 80% of the world population of Atlantic puffins, but breeding failures in their strongholds of Iceland, the Faroe Islands and Norway in recent years have seen numbers dramatically fall.
In France, puffin numbers have fallen to below 200 pairs, while in the Channel Islands, they have fallen to below 100 pairs.
Numbers in the UK were increasing up to 1999 but now evidence suggests that puffin numbers here are falling.
In Scotland the numbers have plummeted while on the Northumberland coast numbers are holding up, but vulnerable. Puffins are highly susceptible to extreme weather and rising sea temperatures, which affects the number and availability of sandeels, their main food source. Cooler waters along the Northumberland coast and an absence of commercial sandeel fishing means that the sandeel numbers are holding up in our region, but puffins are facing challenges here too.
Puffins lay their eggs in burrows, dug into the sandy soil on Coquet Island. A baby puffin is called a puffling.
Each spring, thousands of these iconic little birds return to the RSPB - managed bird sanctuary of Coquet Island, after a winter spent swimming in the Atlantic Ocean.
Wetter summers have increased heavy vegetation growth on the island, which threatens the puffins' access to their burrows.
The RSPB is planning to monitor the Coquet puffins, and will expand efforts to give "our" puffins and their pufflings the best chances on their Coquet Island home.


I am doing some VMware datastore shuffling and needed to move several templates to another datastore. For normal VMs, I have a script named "Move-VMThin.ps1" that moves the VMs and thins out the disks at the same time. I looked for something similar for templates, something that would allow me to migrate the template to other storage, and there was no native support.
A little googling and I found http://ict-freak.nl/2010/01/21/powercli-move-template/. It appears that you have to convert the template to a VM, move the VM, and convert it back to a template. This is fine, but a pain if you have dozens of templates.
Write-Host "Migrate $template to $datastore"
Write-Host "Converting $template to template"
allowed in the current state."


At European Wax Center Washington DC - Eastern Market, we believe that waxing hair removal does more than reveal beautiful skin - it reveals a healthier, more radiant you. For guests near Capitol Hill, Lincoln Park, and Navy Yard, our Washington DC waxing studio offers several hair waxing services, including: Brazilian waxing, eyebrow waxing, upper lip waxing, face, full-body waxing, and more. Reveal the more confident, radiant you and make your reservation today! We're conveniently located near FedEx, Starbucks, Chipotle Mexican Grill, Cafe 8, and Yes! Organic Market Capitol Hill.


Felix and Tilda seem like the perfect couple: young and in love, a financier and a beautiful up-and-coming starlet. But behind their flawless facade, not everything is as it seems.
Worried about the psychological hold that Felix seems to have over Tilda, Callie joins an Internet support group for victims of abuse and their friends. However, things spiral out of control and she starts to doubt her own judgment when one of her new acquaintances is killed by an abusive man. And then suddenly Felix dies–or was he murdered?
Disturbing, creepy, and unsettling– these are a few of the words that come to mind when I think of White Bodies. I'm usually let down by hyped up psychological thrillers, but this one was an updated take on an age old idea that we've seen time and again. I almost hate to categorize it as a mystery, because those who have read similar books in years past will likely see the who and why early on, but that isn't what makes this particular read so juicy. Robins has taken many characters who are unlikable for various reasons and worked them together in a way that is darkly delicious and easily devoured. This book is truly about the journey, the slow unraveling of the psyche in multiple characters, and the author's talent in discomforting the reader entirely.
The main reason I find myself so drawn to psychological thrillers is that, when I find a good one, it washes me with a sense of unease and terror that even a well done horror novel can't seem to manage. White Bodies was the perfect example of a story where we know something really bad is going to happen, and the feeling of niggling unease develops into full blown paranoia and you wonder if you can trust anyone in this book! Without giving away plot points, it's safe to assume that regardless of the correctness in your guesses on what happens, you'll still enjoy reading the book and arriving at the end. The smaller, less pronounced twists felt more like dropped bombs than the grand finale, which I greatly appreciated and felt really enhanced the reading experience.
If you're on the fence whether to read this book, I'll encourage you to take a chance on White Bodies. This was a fairly short read that I blew through in 24 hours; it felt like the perfect way to say goodbye to the warmth and radiance of summer and welcome in the cooler weather. With a gorgeous cover and an intriguing synopsis, I can see why this is one of the most highly anticipated reads for fall and I can't wait to hear what others think of this book. If you enjoy twisty mind games with some quirky, dark characters, I think you'll really enjoy reading this one.
*Many thanks to the publisher for providing my copy; it's a pleasure to provide my honest thoughts here.
I have been reading your blog for quite awhile now but I think this is the first comment I've left. I definitely want to read this one. It might be a definite purchase for the library where I work too. I love your blog!
As I've read more of the genre, I've actually started to not mind the who and why, especially if the thrills are legit!
You're so right! They do need to add a horror subcategory. It's definitely applicable. Wonderful review, love!
Thank you so much sweets!
Amazing review! I have just read The Couple Next Door by Shari Lapena and Sharp Objects by Gillian Flynn this weekend and i'm in a real big suspense mood right now and this one sounds awesome! I love the synopsis already but also like you I love the unsettling feeling books like this can give me! Definitely going on the TBR.
I have this on my Kindle so I'm glad to hear you thought highly of it. I've thoroughly enjoyed her non-fiction true crime stuff in the past, so I'm eager to see how she converts her style for fiction.
Ah now I'm really wanting to check out her true crime stuff!
"The main reason I find myself so drawn to psychological thrillers is that, when I find a good one, it washes me with a sense of unease and terror that even a well done horror novel can't seem to manage." I feel the same but lately I haven't read many because that didn't seem to happen!
I totally feel you. I went through a phase for about 6 months where I couldn't pick up a PT bc they all let me down!


Yancy is someone who can put a smile on anyone's face. His energy and enthusiasm is infectious, and it inspires anyone who meets him - trust us. He is a husband, a father of 2, fitness coach, motivational speaker, veteran, the owner of YancyCamp.com, and he's passionate about all things mental and physical conditioning.
When not training or competing, this husband and father of 2 enjoys building furniture & motivational speaking. Everything that Yancy takes one is guided by a single principle - Maximize your opportunities. Every. Single. Day.


Scuba diving is a team sport where underwater communication is very important, vital. Unless you dive with a very expensive voice communication mask, the only way you can talk to your partner will be through signs made with your hands, light or through sounds made, for example, by rattling with the dive knife on the tank.
When you share hundreds of dives with the same buddy you will create your own code of communication, but until that time comes you need to know some basic underwater communication signals that you'll use to warn of a danger, to show how you feel while diving, to confirm the direction to take or to warn your buddy of the presence of an animal. There are more than 200 common diving signs and there is no hurry not know all of them but you at least need to know some like the back of your hand because that can actually save your life.
The OK signal can also be done as in the image above if you're wearing gloves.
Another way of saying "I'm OK" is placing both hands above your head in an arc. This signal is used when you are far enough from your buddy so that the hand is not easily visible, for example when you are diving from a boat.
Another alternative to the above signal is doing OK with placing just one arm above the head.
With your fingers extended, palm down you will swing one hand when you want to note that you are not well, that you do not feel comfortable and that you need some time and attention of your buddy in case you need help. When your buddy makes this gesture you will stop whatever you are doing and wait until he or she makes the OK sign before continuing with the dive.
In addition to the above signal to tell your buddy that you cannot equalize, just point at your ear with a finger so he/she knows that has to wait until you get to equalize.
This is one of those signs that we hope you never have to do since it represents serious danger and an emergency. It is done by joining the fingers and taking them to the mouth asking for air from your buddy's octopus as your air tank is empty or something is wrong with it.
As in the case of the signal above, this one is also quite sensitive and represents a serious problem in a dive. It is done by placing the palm facing down and simulating a cut to the chest or neck, to say that your tank has no air. If your buddy does this sign you have to give him your spare regulator (or pass him your primary regulator and you use your spare) and prepare for a joint water exit.
Almost all diving signals related to a problem or emergency are easily identifiable, and this one, used to communicate with a boat or someone inland, is also clearly visible. Moving an arm on both sides you are asking for help because you have had a problem during the dive and need urgent assistance.
This signal does not represent an emergency but we want to locate near the image above to indicate the difference between the two. This image means "pick me up " and is done for someone on a boat to pick you up but does not involve any emergency or danger. The difference between them is the movement. With movement it means danger, motionless means no danger. But yes, if are in one of those situations that you need help your own common sense will be able to show whether there is danger, but it is worth to point it.
This signal is done by opening and closing fingers to indicate your buddy that you have had a muscle cramp (very common in the calf muscles) and either needs his/her help to stretch the muscle and relieve cramping, or you need some time to do it yourself.
Punching in a direction shows the presence of a threat of any kind to communicate to your buddies to be careful or do not go in that direction of the punch.
Another universal signal that requires no explanation. Diving cold is very nasty and uncomfortable, but when it becomes too cold it is sometimes better to stop the dive.
By pulling the thumb between the forefinger and middle finger you will show your diving buddy that you have been caught with a fishnet, a rope or similar. At that moment both of you have to be careful and analyze what the problem is and fix it with the dive knife.
This signal also easily recognizable, is done by partner when we cannot get the item we need to be cut, mainly fishing nets.
This signal is one of the most common and often used signals while diving. With the forefinger and middle finger hit in the palm of your hand to ask your buddy how much air is left in the tank.
Forming a T with both hands shows that we have reached the 100 bar limit, we have half the air in the tank and time is up, it's time to turn around (the tanks are usually filled to 200 bar).
With the fist raised to the height of the head, shows your partner you only have 50 bars left of air pressure, you have entered the reserve air tank and you need to finish the dive because you have to account for time to do the decompression stops.
With a closed fist and thumb indicating to the bottom, this will show your buddy that is time to start deflating the Buoyancy Compensator to start the dive.
Opposite to the sign above, with the thumb pointing towards the surface and moving the hand up to show your partner that the dive is over and it's time to head to the surface.
When the dive guide indicates that you are going to turn around he/she will lift the index finger and rotate it in circles. This sign is usually done when the dive starts and ends in the same place or when one of the divers has 100 bars left.
Where to go? Which direction do we take?
With a closed fist and thumb pointing sideways turn forearm 180º to ask and agree with your buddy which direction to take.
This signal is done by putting together the index finger of each hand and pointing to the front. With this sign the divemaster will tell you to get closer to your buddy. Sometimes it is because the instructor, for any issue, decides to change the dive buddies and draw two divers to later make this gesture of the image indicating that the two divers have to go together.
By putting both hands in a grip it's telling two divers (usually this sign is made by the divemaster) that they have to be secured by their hands. This signal is performed when there is any type of problem like strong currents or bad visibility and is necessary for the safety of divers to maintain full contact.
To mark who runs the dive the divers are pointed to by using the index finger in order and then placing the two fingers facing forward, one more advanced than the other. The diver who has been pointed the finger forward is the leader.
With the palm open and down and movements left and right you indicate divers that your going to keep diving at the current depth.
This signal is also universal and needs no explanation, when you see it you will know immediately what does it mean. This is a sign done usually to Open Water divers that in their eagerness to see more and discover what's happening underwater, dive very fast.
Another one of those universal signs that are not only understood by divers. For any questions that your buddy or dive master makes and you have no answer for, you then give this responding gesture, raising your shoulders and with the palms up. Mind you, this act does not apply when you are asked for your air tank level or where your partner is... you always have to know the answer to those.
Another clear sign that is commonly used to teach you how to perform certain procedures to those learning to dive or to show something worthwhile during the dive.
To show your partner that everything is OK, that everything is fine, you "draw" a circle on the bottom with the light beam, simulating the OK sign that you do with your fingers during daylight dives.
You can also make this signal with your hand, as you do under good light conditions, but illuminating the hand with the flashlight.
To draw the attention of your buddies during a night dive move the beam up and down or quickly in circles. This will help them to pay attention to you if you ask something or to show some interesting element during the dive.
Always have a dive plan before you start your dive and stick to it. We understand that sometimes circumstances arise and you have to adapt to the situation. By being able to communicate properly with your partner, you can adjust to whatever comes your way.


text:
Patriot Act Author Prepares Strong Bill to Rein in NSA Abuses…Calls for Prosecution of James Clapper
libertyblitzkrieg.com In an interesting twist of irony, one of the Congressman most instrumental in the destruction of civil liberties in these United States due to his authorship of the Patriot Act, Jim Sensenbrenner (R-Wisc.), is now leading the charge to rein in NSA abuses. His disgust with the NSA became clear back in early June when he wrote an impassioned letter to Attorney General Eric Holder criticizing the illegal NSA activity happening behind the scenes. The key point here is that Mr. Sensenbrenner strongly believes that these guys are misinterpreting the legislation he wrote to justify everything they are doing. As such, he and others are proposing legislation to make it crystal clear what is and what is not appropriate surveillance. Not only that, he is also calling for the prosecution of James Clapper the current Director of National Intelligence, who perjured himself in front of Congress earlier this year. Mr. Sensenbrenner said: Oversight only works when the agency that oversight is directed at tells the truth, and having Mr Clapper say he gave the least untruthful answer should, in my opinion, have resulted in a firing and a prosecution. This guy means business. More from the Guardian: The conservative Republican who co-authored America's Patriot Act is preparing to unveil bipartisan legislation that would dramatically curtail the domestic surveillance powers it gives to intelligence agencies. Congressman Jim Sensenbrenner, who worked with president George W Bush to give more power to US intelligence agencies after the September 11 terrorist attacks, said the intelligence community had misused those powers by collecting telephone records on all Americans, and claimed it was time "to put their metadata program out of business". His imminent bill in the House of Representatives is expected to be matched by a similar proposal from Senate judiciary committee chair Patrick Leahy, a Democrat. It pulls together existing congressional efforts to reform the National Security Agency in the wake of disclosures by whistleblower Edward Snowden. It seeks to limit the collection of phone records to known terrorist suspects; to end "secret laws" by making courts disclose surveillance policies; to create a special court advocate to represent privacy interests; and to allow companies to disclose how many requests for users' information they receive from the USA. The bill also tightens up language governing overseas surveillance to remove a loophole which it has been abused to target internet and email activities of Americans. In July, a temporary measure to defund the NSA bulk collection programme was narrowly defeated in a 217 to 205 vote in the House, but Sensenbrenner said the appetite for greater privacy protections had only grown since. to read more click here: libertyblitzkrieg.com
Categories House of Representatives News

redpajama_set_name: RedPajamaCommonCrawl


text:
by pablo doe
I share on this Newsletter exclusive
photography tips and tutorials.
Our FirmOur Team
Our ApproachOur PracticeWhat Sets Us Apart
Cases & Experience
Our WorkOur SuccessesCommunity
NewsroomContact
CLASS ACTION LAWSUIT FILED AGAINST FACEBOOK AND CAMBRIDGE ANALYTICA FOR STEALING AND IMPROPERLY USING MORE THAN 71 MILLION USERS' DATA
Lawsuit Filed on Behalf of U.S. and U.K. Facebook Users Cambridge Analytica, under Steve Bannon's Leadership, Initially Misled Facebook but Facebook Failed to Act Responsibly to Protect Users
press@fieldslawpllc.com
Lawsuit Filed on Behalf of U.S. and U.K. Facebook Users
Cambridge Analytica, under Steve Bannon's Leadership, Initially Misled Facebook but Facebook Failed to Act Responsibly to Protect Users
WASHINGTON, D.C. – Today, a class action lawsuit against Facebook, Inc., in the United States Federal District Court in Delaware, was filed on behalf of both American and British Facebook users, alleging the social media company failed to protect the personal information of 70.6 million Facebook users in the U.S. and another more than 1 million in the United Kingdom. The lawsuit was also filed against Cambridge Analytica LLC, SCL Group Limited and Global Science Research Limited, organizations that obtained the Facebook user data to develop and foster political propaganda campaigns. The suit also names Aleksandr Kogan, a founding director of Global Science Research Ltd. Former Trump campaign and White House Advisor Steve Bannon led Cambridge Analytica in 2014 at the time the data was collected and extracted. Rebekah Mercer, a prominent Trump campaign donor, was a member of the board of Cambridge Analytica at the time.
"Facebook utterly failed in its duty and promise to secure the personal information of millions of its users, and, when aware that this stolen information was aimed against its owners, it failed to take appropriate action," said Robert Ruyak, co-lead counsel in the class action suit. "Facebook must be held responsible for failing to protect its users' personal information. We must also make certain that organizations like Cambridge Analytica and their benefactors – the Mercer family and Steve Bannon –are held accountable for this egregious theft and misuse and cannot further exploit it."
"Facebook has made billions of dollars selling advertisements targeted to its customers, and in this instance made millions selling advertisements to political campaigns that developed those very ads on the back of their customers' own stolen personal information," said co-lead counsel Richard Fields. "That's unacceptable, and they must be held accountable."
"The defendants effectively abused the human right to privacy of ordinary Facebook users and, if that were not enough, then the fruits of that abuse are alleged to have undermined the democratic process. This case will go some way to ensure that neither of these things can happen in the future," said co-counsel Jason McCue, an expert on UK data privacy and human rights law.
The lawsuit alleges that in 2014, Cambridge Analytica, SCL and GSR improperly acquired the personal information of 71.6 million Facebook users, including names, phone numbers, mailing and email addresses, political and religious affiliations, and interests. This was done to accomplish Cambridge Analytica's driving principle: to build psychological profiles of voters to effect election results in the United States and The United Kingdom.
GSR was granted access by Facebook to collect data solely for academic research. Instead the data was used by Cambridge Analytica and GSR for purely political and commercial purposes. The information was obtained by using a digital application called thisisyourdigitallife, developed by Cambridge University psychologist Aleksandr Kogan. Kogan created a personality quiz that required individuals to use their Facebook login credentials to take the quiz. Approximately 270,000 Facebook users installed the application, and gave their personal information to Kogan and Cambridge Analytica. The design of the application allowed Kogan and Cambridge Analytica to obtain without consent the personal information of more than 71 million Facebook users in the U.S. and U.K. who were Friends of the original 270,000 users. When Facebook first learned of the true nature of Cambridge Analytica's work in 2015, they failed to notify Facebook users and to ensure that the stolen data was destroyed, which enabled its use through the 2016 U.S. elections and the June 2016 UK "Brexit" Referendum.
The plaintiffs are represented by co-lead counsel Robert Ruyak of RuyakCherian LLP, and Richard Fields of Fields PLLC, both in Washington, D.C., by Jason McCue and Matthew Jury of McCue & Partners LLP in London, and by Christopher P. Simon of Cross & Simon, LLC in Wilmington, Delaware.
The full class action complaint, which was filed in the United States District Court for the District of Delaware, can be found here.
Delaware sues pharmaceutical companies for contributiong to opioid epidemic
Justice Dept. Backs High-Stakes Lawsuit Against Opioid Makers


There are different types of individuals, who are understood for the life that they lead and not by their physical appearance . This is a short article that highlights this certain concept . In this short article you will get the opportunity to check out lots of people, who are leading perfectly normal lives yet there is one deformity that they have been dealing with youth. This defect is commonly called dwarfism.


Aillea is all about empowering women to make healthy choices about the products they put on their body. We offers over 50 luxury clean cosmetic brands ranging from hair care, skincare, makeup, body care, nail care, and more. We feature top brands like Kjaer Weis, Vintner's Daughter, May Lindstrom Skin, Indie Lee, Kypris, RMS, One Love Organics, Ilia, Josh Rosebrook, Rahua, Agent Nateur, Kosas, Suntegrity, Maya Chia and more. We offer a full treatment room for all of your facial and waxing needs. Aillea's friendly and knowledgeable beauty experts, makeup artists, and estheticians can help you clean your routine and find healthy swaps for all your beauty needs. Come visit us today!


text:
Q: Monitoring resumable file upload in Google Drive v3 I am writing my own wrapper around Google Drive API v3 in Python. I have a small problem with uploading files and seeing what is going on. Here is a small sample code
# create service using oauth2flow -> drive_service
# upload pdf from disk
media = MediaFileUpload('/files/test.pdf', resumable=True)

file_metadata = {'name': 'test.pdf'}

res = drive_service.files().create(
    body=file_metadata,
    media_body=media,
    fields="id"
)

Everything works fine, but I am not able to see how many bytes have been transferred so far (like in download I have next_chunk method). Also, I do not really know how to use the functionality of resumable=True in case of error. One more thing, if I put chunksize variable into MediaFileUpload I get 400 Bad Request Error. Is it possible to have similar functionality like in download method where I can upload files in pieces and monitor what is going on?
EDIT:
@Tanaike solved the problem. You can keep track of upload by using next_chunk method but the chunkize needs to be multiple of 256 KB.
Here is a sample code:
CHUNK_SIZE = 256 * 1024
DEFAULT_CHUNK_SIZE = 100 * 1024 * 1024

media = MediaFileUpload('/files/test.pdf', resumable=True, chunksize=CHUNK_SIZE)

file_metadata = {'name': 'test.pdf'}

res = drive_service.files().create(
    body=file_metadata,
    media_body=media,
    fields="id"
)

response = None
while response is None:
    status, response = res.next_chunk()

    try:
        sys.stdout.write('%d/%d Bytes transfered' % (status.resumable_progress, status.total_size))
    except:
        pass

redpajama_set_name: RedPajamaStackExchange


text:
Thinkers50 co-founders welcome the formation of the Business Ecosystem Alliance.
September 2020: The inaugural meeting of the Business Ecosystem Alliance was held on 16 September under the auspice of the annual Rendanheyi1 Conference organized by the Haier Group.
Addressing the meeting via video, Stuart Crainer, co-founder of Thinkers50 and director of the European Rendanheyi Centre, said that the aim of the Alliance as set out in its Charter was to encourage knowledge sharing about business ecosystems and to provide practical advice.
Attending virtually from around the world were Zhang Ruimin, CEO and chairman of the Haier Group, Professor Bill Fischer from IMD, and guest of honour, Professor James F. Moore, who's seminal HBR article, 'Predators and prey: a new ecology of competition,' introduced the ecosystem idea to the business world.
Professor Moore's original article states that: "A business ecosystem, like its biological counterpart, gradually moves from a random collection of elements to a more structured community. Think of a prairie grassland that is succeeded by stands of conifers, which in turn evolve into a more complex forest dominated by hardwoods. Business ecosystems condense out of the original swirl of capital, customer interest, and talent generated by a new innovation, just as successful species spring from the natural resources of sunlight, water, and soil nutrients."
The meeting was introduced by Des Dearlove, co-founder of Thinkers50 and co-director of the European Rendanheyi Centre, who said: "Thinkers50 is delighted to support the formation of the Business Ecosystem Alliance and Zero Distance Awards."
Dearlove pointed to the publication of a new book, Ecosystems Inc: Understanding, harnessing and developing organizational ecosystems, published by Thinkers50 in collaboration with ECSI Consulting and the Haier Model Research Institute, as an example of the Alliance's ability to share ideas.
"My own hope for the Business Ecosystem Alliance," he added, "is that we can be the honeybees of this evolving community, buzzing around collecting ideas and pollinating the different parts of the forest."
Also attending the inaugural meeting were Simone Cicero, CEO of Boundaryless, and Amit Kapoor, Dr Annika Steiber, Ellen Dick, and Evgeniya Serdiuk the directors of Rendanheyi Centres in India, Silicon Valley, Singapore and Russia.
More than 20 organizations and experts from 11 different countries have already signed the Charter. They include:
Michael Jacobides, London Business School (UK)
Bill Fischer, IMD (Switzerland)
Isaac Getz, ESCP (France)
Fabrizio Salvador, IE Business School (Spain)
Alessandro Di Fiore, ECSI Consulting (Italy)
Simone Cicero, Boundaryless (Italy)
Antonio Nieto Rodriguez, Strategy Implementation Institute (Belgium)
Amit Kapoor, Institute for Competitiveness (India)
Pim de Morree & Joost Minaar, Corporate Rebels (Holland)
Annika Steiber, Menlo College (US)
Alf Rehn, University of Southern Denmark (Denmark)
Nathan Furr, Insead (France)
Karolin Frankenberger, University of St Gallen (Switzerland)
Dan Toma (US)
Stuart Crainer, Thinkers50 (UK)
Des Dearlove, Thinkers50 (UK)
Zhang Ruimin, Haier (China)
1Rendanheyi is the name given to Haier's revolutionary management model. A Chinese word, it fuses the words for employee and user value. The Rendanheyi model emphasises giving people responsibility, using technology to eliminate distance to consumers and developing mutually beneficial ecosystems.


text:
Lewis Baines joins National League side Chorley permanently
Young defender departs Highbury
Fleetwood Town can announce that Lewis Baines has joined National League side Chorley on a permanent transfer.
The 20-year-old had previously spent time on loan at Victory Park last season, and will join up with Jamie Vermiglio's squad ahead of The Magpies' first game of the season against Bromley on Saturday 3 August.
The club would like to thank Lewis for his efforts whilst at the club, and wish him all the best for the future.
Lewis Baines


text:
Q: Query regarding calculation of average of some numbers program I am beginner in C# coding, and I am trying to write some code for the program where my program will show the average of numbers given by users. I have written below code, but I am thinking this would be more better and efficient. Can anyone please help me. Thanks in advance.
static void Main(string[] args)
        {
            int a=0;
            double total = 0;
            double result;
            Console.Write("For how many numbers you want to do the average calculation: ");
            a = int.Parse(Console.ReadLine());
            double[] array = new double[a];
            for (int j = 0; j < a; j++)
            {
                Console.Write("Please enter value for {0}: ", j+1);
                array[j]= double.Parse(Console.ReadLine());
            }
            foreach (var item in array)
            {
                total += item;
            }
            result = total / a;
            Console.WriteLine($"Your Calculated average value is {result}");
            Console.ReadKey();
        }

A: You dont need to take extra array for your purpose, you can do this just by adding each input value and then divide that total with total number of input items.
    double total = 0;
    double result;
    Console.Write("For how many numbers you want to do the average calculation: ");
    int a = int.Parse(Console.ReadLine());
    for (int j = 0; j < a; j++)
    {
        Console.Write("Please enter value for {0}: ", j + 1);
        total += double.Parse(Console.ReadLine());
    }

    result = total / a;
    Console.WriteLine($"Your Calculated average value is {result}");
    Console.ReadKey();


text:
Horrorific content by Yo Adrian on December 03rd, 2020 | Movie Review | Supernatural, Exorcism, Religion, Satanic, The Exorcist Series
Add Exorcist II: The Heretic (1977) to your Watchlist
It's about the continued battle to free Regan from the demon who still possesses her.
Exorcist 2: The Heretic was directed by John Boorman and stars Linda Blair (from Sorceress and Repossessed), Louise Fletcher (from Nightmare On The 13th Floor and Shadowzone) and Max Von Sydow (from Shutter Island and Sleepless).
It's four years later... what does she remember?
This infamously terrible sequel to one of the greatest horror movies of all time starts off in South America where a priest, Father Lamont, is performing an exorcism on a woman who ends up lighting herself on fire while grinning from ear to ear. Cut to New York and Regan, now a teenager, is forced to go to therapy to hash out the trauma of being possessed by a demon. Cut to the Vatican and Lamont is being forced to New York to investigate the death of Father Merrin, the original priest from The Exorcist. Cut to New York and Lamont meets Regan, her shrink Dr. Tuskin, and starts to work the case.
During Lamont's first meeting with Regan he participates in a kind of group hypnosis that allows both him and Tuskin to visit Regan's repressed memories. But here's the thing, when Tuskin visits Regan's memories she gets attacked by the demon Pazuzu that was supposed to have already been exorcised from her. Regan's memories attack Tuskin. Regan gets pulled out and Lamont goes under to help Tuskin. That's right. Thanks to some weird future machine they're both able to interact with Regan's memories without her even being involved anymore.
During the investigation Lamont learns the hard way that Pazuzu is still alive and well in Regan, he's just laying low. He's apparently a "demon of the air" who takes the form of locusts and possesses people from time to time.
Exorcist 2 is one slow ass movie. Some movies are slow burns, others have pacing issues. Exorcist 2 is just scene after scene after scene, most of them needless. The Exorcist was about a little girl possessed and a priest who had to battle evil to save her. There were a lot of crazy scenes and shocking practical effects. Exorcist 2 is mostly just a lot of talking and traveling and learning about locusts. More than an hour goes by with only two short scenes that could barely be considered horror.
I really wanted to like The Heretic. There's a few horror sequels floating around that get hated on, but that happen to be favorites of mine. This is not one of them. The hate this movie gets is warranted.
MORE HORROR Ju-on: The Grudge 2 (2003) Review
MORE HORROR The Pale Door (2020) Review
MORE HORROR The Orphanage (2007) Review
MORE HORROR Curtains Review (1983)
MORE HORROR Resolution (2012) Review
No way. Watch The Exorcist. Watch The Exorcist 3. But do yourself a favor and skip this mess.
Exorcist 2: The Heretic Review (1977) Worth Watching? - ALL HORROR Tweet it
Exorcist 2: The Heretic (1977) Review is part 2 of 4 in the Exorcist series
The Exorcist Review (1973)
Yo Adrian
Founder and Executive Director of all things Horrific at AllHorror.com (Ok, actually I'm just a guy who watches way too many horror movies. It's unhealthy, really).
All Horror - Discover new horror movies, create and share your own horror movie lists and curate watchlists from thousands of titles, new and old.
Music Corner - Monster Mash (1962 novelty song)
F*CK, SCARE, KILL: An Autopsy of Teen Horror THE CRAFT
Movies Like The Endless
15 Underrated Black Horror Movies
The Meg (2018) Review
Big Ass Spider! (2013) Review
Ringu 2 (1999) Review
Trespassers (2019) Review
The Cloverfield Paradox Review (2018)


Objective To establish a stable HEK293T cell line with c.392G>T (p.131G>V) mutation site knockout in G6PD gene using CRISPR/Cas9 technique. Methods We designed 4 pairs of small guide RNA (sgRNA) for G6PD c.392G>T(p.131G>V) mutation site, and constructed exogenous PX458 plasmids expressing Cas9-sgRNA. The plasmids were transfected into HEK293T cells, and the cells expressing GFP fluorescent protein were separated by flow cytometry for further culture. After verification of the knockout efficiency using T7 endonuclease Ⅰ, the monoclonal cells were screened by limiting dilution and DNA sequencing to confirm the knockout. We detected the expressions of G6PD mRNA and protein and examined functional changes of the genetically modified cells. Results We successfully constructed the Cas9-sgRNA exogenous PX458 plasmid based on the c.392G>T(p.131G>V) mutation site of G6PD gene. The editing efficiency of the 4 pairs of sgRNA, as detected by T7E1 enzyme digestion, was 6.74%, 12.36%, 12.54% and 2.94%. Sanger sequencing confirmed that the HEK293T cell line with stable knockout of G6PD c.392G>T(p.131G>V) was successfully constructed. The genetically modified cells expressed lower levels of G6PD mRNA and G6PD protein and showed reduced G6PD enzyme activity and proliferative capacity and increased apoptosis in response to vitamin K3 treatment. Conclusion We successfully constructed a stable HEK293T cell model with G6PD gene c.392G>T(p.131G>V) mutation site knockout to facilitate future study of gene repair.


So we havent been very active here as we had hoped to be, with so much happening in the last 12 months. Settling into the new premises and meeting new people has been an exciting experience. This coming week we are launching stage one of our website upgrade which will include a new and refreshed facelift to the home page but all the functions for now will remain the same. Over the next three months we will be working on updates to current models as well as adding some new product ranges to give CarsRus more flexibility for the consumer. Any feedback on the new look website will be much appreciated.


text:
Home Academy Awards Collection Rules, 1950 (23rd) Academy Awards
Rules, 1950 (23rd) Academy Awards
(c) Illusion of actual reality should be accomplished with such
skill that the mechanism by which the effect is achieved is
not apparent in the final result.
2. The President of the Academy shall appoint the Special Effects
Award Nominating Committee consisting of at least fifteen technicians,
representing each of the various special effects fields. This Committee
shall be charged with the responsibility of nominating from all eligible
productions, both domestic and foreign, not more than five or less than
two productions representing in their judgment the best special effects
work (visual or audible or both) of the year.
3. In making nominations for this Award, the Committee shall
decide which types of special effects in each production are worthy of
nomination.
4. The Committee shall view the special effects sequences submitted
for consideration before voting to select the productions to be nominated
for this Award. It is specified that to be qualified to participate in this
vote, a member of the Committee must view all of these sequences
during one consecutive viewing and in the same auditorium, this auditorium
to be officially selected by the Committee. A Committee member
may not vote for the production submitted by the studio which he
represents.
5. It is further specified that if, after the group of productions under
consideration is selected, any studio producing any of these productions
is not represented by at least two members on the Committee, the
President of the Academy shall increase that studio's representation by
an additional appointment or appointments.
6. Cartoons or productions containing intercut live action and cartoons
are not eligible for this Award.
7. Such other rules as may be considered necessary for the proper
conduct of this Award shall be put into effect by the Special Effects
Award Nominating Committee subject to the approval of the Academy
Awards Rules Committee.
8. Final voting for the Special Effects Award shall be restricted to
active Academy members.
Special Rules for the Irving G. Thalberg Memorial Award
1. The Board of Governors shall review the achievement records of
individuals responsible for motion picture production and verbal nominations
shall be made from the floor.
2. When the nominations are completed, a written secret ballot
shall be taken, and if one candidate receives a majority vote of the
Governors present, he shall receive the Thalberg Award.
Page Twenty-one
Title Rules, 1950 (23rd) Academy Awards
Award Year 1950 23rd Academy Awards
Full text (c) Illusion of actual reality should be accomplished with such skill that the mechanism by which the effect is achieved is not apparent in the final result. 2. The President of the Academy shall appoint the Special Effects Award Nominating Committee consisting of at least fifteen technicians, representing each of the various special effects fields. This Committee shall be charged with the responsibility of nominating from all eligible productions, both domestic and foreign, not more than five or less than two productions representing in their judgment the best special effects work (visual or audible or both) of the year. 3. In making nominations for this Award, the Committee shall decide which types of special effects in each production are worthy of nomination. 4. The Committee shall view the special effects sequences submitted for consideration before voting to select the productions to be nominated for this Award. It is specified that to be qualified to participate in this vote, a member of the Committee must view all of these sequences during one consecutive viewing and in the same auditorium, this auditorium to be officially selected by the Committee. A Committee member may not vote for the production submitted by the studio which he represents. 5. It is further specified that if, after the group of productions under consideration is selected, any studio producing any of these productions is not represented by at least two members on the Committee, the President of the Academy shall increase that studio's representation by an additional appointment or appointments. 6. Cartoons or productions containing intercut live action and cartoons are not eligible for this Award. 7. Such other rules as may be considered necessary for the proper conduct of this Award shall be put into effect by the Special Effects Award Nominating Committee subject to the approval of the Academy Awards Rules Committee. 8. Final voting for the Special Effects Award shall be restricted to active Academy members. TWENTY-ONE Special Rules for the Irving G. Thalberg Memorial Award 1. The Board of Governors shall review the achievement records of individuals responsible for motion picture production and verbal nominations shall be made from the floor. 2. When the nominations are completed, a written secret ballot shall be taken, and if one candidate receives a majority vote of the Governors present, he shall receive the Thalberg Award. Page Twenty-one


The Dolores State Bank is headquartered in Dolores and is the 28th largest bank in the state of Colorado. It is also the 2,426th largest bank in the nation. It was established in 1959 and as of December of 2018, it had grown to 46 employees at 4 locations. The Dolores State Bank has an A health rating.
Always verify rates and promotions with the bank or credit union. We are not The Dolores State Bank; we are a rate comparison website and cannot provide official rates or promotions.


The Canadian Centre for Scholarship and the Christian Faith together with Vernon Press cordially invite you to submit your proposal for contributions in the volume on "Indigenous People and the Christian Faith: A New Way Forward", edited by Bill Anderson.
The proposals may be from any of the following sub-disciplines along the theme of Indigenous People and the Christian Faith: Music, Poetry, Drama, Dance and the Visual Arts including Painting, Sculpture and Graphic Design. CCSCF, as an open and inclusive academic institution, also welcomes proposals related to Indigenous Spirituality. For guidelines and submissions go to carolina.sanchez@vernonpress.com and bill.anderson@concordia.ab.ca.
Academic Papers along the lines of the theme of Indigenous People and the Christian Faith may be approached interdisciplinarily from the Humanities and Social Sciences and from the subdisciplines of philosophy, theology, psychology, sociology, history, culture, art and ethics. There is a broad range of possible topics with some overlap. This list is neither exhaustive nor intended to be restrictive: Indigenous People and Biblical Studies, Indigenous People and Theology, Indigenous People and Education, Indigenous People and Politics, Indigenous People and Government Policy, Indigenous People and Ethics, Indigenous People and the Arts, Indigenous People and Law, Indigenous People and Literature, Indigenous People and Philosophy, Indigenous Christianity, Indigenous Peoples and Colonialism. Indigenous People and Postcolonialism. CCSCF, as an open and inclusive academic institution, also welcomes proposals related to Indigenous Spirituality.
This proposal is due at April 30th 2019.
Page last updated on March 4th 2019. All information correct at the time, but subject to change.


Pedro's popular Y-Wrenches feature heat-treated tool steel construction and durable plastic body for strength and long-lasting performance.
The Y-Wrench lineup is colour coded for quick recognition. The ergonomic shape provides excellent leverage and easy handling.
Includes 2, 2.5, and 3mm hex sizes, perfect for common hex hardware found on modern bicycles and components.