query: The query consists of users’ needs, leading to several research questions that span a paragraph. Each candidate passage is an abstract from a scientific paper. The objective of this information retrieval task is to identify the abstract that most effectively meets the user’s needs in the query : I am interested in investigating linguistics from a statistical and mathematical standpoint. My focus is on modeling linguistic patterns and phenomena as components of a complex system. I aim to compute statistical and information-theoretic features from extensive streams of natural language texts. Ideally, these features would provide insights into modeling the long-term syntactical and semantic dependencies between various linguistic elements within a lengthy text stream. Moreover, I am keen on modeling human language as a complex communication system regulated by the principles of information theory. The ultimate objective is to reveal hidden patterns and enhance or consolidate existing patterns using the language of information theory and statistics. Therefore, I need a thorough review of recent advancements at the intersection of linguistics and complex systems.

positive:
Traditional linguistic theories have largely regard language as a formal
system composed of rigid rules. However, their failures in processing real
language, the recent successes in statistical natural language processing, and
the findings of many psychological experiments have suggested that language may
be more a probabilistic system than a formal system, and thus cannot be
faithfully modeled with the either/or rules of formal linguistic theory. The
present study, based on authentic language data, confirmed that those important
linguistic issues, such as linguistic universal, diachronic drift, and language
variations can be translated into probability and frequency patterns in parole.
These findings suggest that human language may well be probabilistic systems by
nature, and that statistical may well make inherent properties of human
languages.

positive:
As is the case of many signals produced by complex systems, language presents
a statistical structure that is balanced between order and disorder. Here we
review and extend recent results from quantitative characterisations of the
degree of order in linguistic sequences that give insights into two relevant
aspects of language: the presence of statistical universals in word ordering,
and the link between semantic information and the statistical linguistic
structure. We first analyse a measure of relative entropy that assesses how
much the ordering of words contributes to the overall statistical structure of
language. This measure presents an almost constant value close to 3.5 bits/word
across several linguistic families. Then, we show that a direct application of
information theory leads to an entropy measure that can quantify and extract
semantic structures from linguistic samples, even without prior knowledge of
the underlying language.

positive:
We review computational and robotics models of early language learning and
development. We first explain why and how these models are used to understand
better how children learn language. We argue that they provide concrete
theories of language learning as a complex dynamic system, complementing
traditional methods in psychology and linguistics. We review different modeling
formalisms, grounded in techniques from machine learning and artificial
intelligence such as Bayesian and neural network approaches. We then discuss
their role in understanding several key mechanisms of language development:
cross-situational statistical learning, embodiment, situated social
interaction, intrinsically motivated learning, and cultural evolution. We
conclude by discussing future challenges for research, including modeling of
large-scale empirical data about language acquisition in real-world
environments.
  Keywords: Early language learning, Computational and robotic models, machine
learning, development, embodiment, social interaction, intrinsic motivation,
self-organization, dynamical systems, complexity.

positive:
The availability of an abundance of knowledge sources has spurred a large
amount of effort in the development and enhancement of Information Retrieval
techniques. Users information needs are expressed in natural language and
successful retrieval is very much dependent on the effective communication of
the intended purpose. Natural language queries consist of multiple linguistic
features which serve to represent the intended search goal. Linguistic
characteristics that cause semantic ambiguity and misinterpretation of queries
as well as additional factors such as the lack of familiarity with the search
environment affect the users ability to accurately represent their information
needs, coined by the concept intention gap. The latter directly affects the
relevance of the returned search results which may not be to the users
satisfaction and therefore is a major issue impacting the effectiveness of
information retrieval systems. Central to our discussion is the identification
of the significant constituents that characterize the query intent and their
enrichment through the addition of meaningful terms, phrases or even latent
representations, either manually or automatically to capture their intended
meaning. Specifically, we discuss techniques to achieve the enrichment and in
particular those utilizing the information gathered from statistical processing
of term dependencies within a document corpus or from external knowledge
sources such as ontologies. We lay down the anatomy of a generic linguistic
based query expansion framework and propose its module-based decomposition,
covering topical issues from query processing, information retrieval,
computational linguistics and ontology engineering. For each of the modules we
review state-of-the-art solutions in the literature categorized and analyzed
under the light of the techniques used.

positive:
While the use of statistical physics methods to analyze large corpora has
been useful to unveil many patterns in texts, no comprehensive investigation
has been performed investigating the properties of statistical measurements
across different languages and texts. In this study we propose a framework that
aims at determining if a text is compatible with a natural language and which
languages are closest to it, without any knowledge of the meaning of the words.
The approach is based on three types of statistical measurements, i.e. obtained
from first-order statistics of word properties in a text, from the topology of
complex networks representing text, and from intermittency concepts where text
is treated as a time series. Comparative experiments were performed with the
New Testament in 15 different languages and with distinct books in English and
Portuguese in order to quantify the dependency of the different measurements on
the language and on the story being told in the book. The metrics found to be
informative in distinguishing real texts from their shuffled versions include
assortativity, degree and selectivity of words. As an illustration, we analyze
an undeciphered medieval manuscript known as the Voynich Manuscript. We show
that it is mostly compatible with natural languages and incompatible with
random texts. We also obtain candidates for key-words of the Voynich Manuscript
which could be helpful in the effort of deciphering it. Because we were able to
identify statistical measurements that are more dependent on the syntax than on
the semantics, the framework may also serve for text analysis in
language-dependent applications.

positive:
In this article, we evaluate computational models of natural language with
respect to the universal statistical behaviors of natural language. Statistical
mechanical analyses have revealed that natural language text is characterized
by scaling properties, which quantify the global structure in the vocabulary
population and the long memory of a text. We study whether five scaling
properties (given by Zipf's law, Heaps' law, Ebeling's method, Taylor's law,
and long-range correlation analysis) can serve for evaluation of computational
models. Specifically, we test $n$-gram language models, a probabilistic
context-free grammar (PCFG), language models based on Simon/Pitman-Yor
processes, neural language models, and generative adversarial networks (GANs)
for text generation. Our analysis reveals that language models based on
recurrent neural networks (RNNs) with a gating mechanism (i.e., long short-term
memory, LSTM; a gated recurrent unit, GRU; and quasi-recurrent neural networks,
QRNNs) are the only computational models that can reproduce the long memory
behavior of natural language. Furthermore, through comparison with recently
proposed model-based evaluation methods, we find that the exponent of Taylor's
law is a good indicator of model quality.

positive:
We propose a new computational approach for tracking and detecting
statistically significant linguistic shifts in the meaning and usage of words.
Such linguistic shifts are especially prevalent on the Internet, where the
rapid exchange of ideas can quickly change a word's meaning. Our meta-analysis
approach constructs property time series of word usage, and then uses
statistically sound change point detection algorithms to identify significant
linguistic shifts.
  We consider and analyze three approaches of increasing complexity to generate
such linguistic property time series, the culmination of which uses
distributional characteristics inferred from word co-occurrences. Using
recently proposed deep neural language models, we first train vector
representations of words for each time period. Second, we warp the vector
spaces into one unified coordinate system. Finally, we construct a
distance-based distributional time series for each word to track it's
linguistic displacement over time.
  We demonstrate that our approach is scalable by tracking linguistic change
across years of micro-blogging using Twitter, a decade of product reviews using
a corpus of movie reviews from Amazon, and a century of written books using the
Google Book-ngrams. Our analysis reveals interesting patterns of language usage
change commensurate with each medium.

positive:
In this work we extend previous analyses of linguistic networks by adopting a
multi-layer network framework for modelling the human mental lexicon, i.e. an
abstract mental repository where words and concepts are stored together with
their linguistic patterns. Across a three-layer linguistic multiplex, we model
English words as nodes and connect them according to (i) phonological
similarities, (ii) synonym relationships and (iii) free word associations. Our
main aim is to exploit this multi-layered structure to explore the influence of
phonological and semantic relationships on lexicon assembly over time. We
propose a model of lexicon growth which is driven by the phonological layer:
words are suggested according to different orderings of insertion (e.g. shorter
word length, highest frequency, semantic multiplex features) and accepted or
rejected subject to constraints. We then measure times of network assembly and
compare these to empirical data about the age of acquisition of words. In
agreement with empirical studies in psycholinguistics, our results provide
quantitative evidence for the hypothesis that word acquisition is driven by
features at multiple levels of organisation within language.

positive:
A family of information theoretic models of communication was introduced more
than a decade ago to explain the origins of Zipf's law for word frequencies.
The family is a based on a combination of two information theoretic principles:
maximization of mutual information between forms and meanings and minimization
of form entropy. The family also sheds light on the origins of three other
patterns: the principle of contrast, a related vocabulary learning bias and the
meaning-frequency law. Here two important components of the family, namely the
information theoretic principles and the energy function that combines them
linearly, are reviewed from the perspective of psycholinguistics, language
learning, information theory and synergetic linguistics. The minimization of
this linear function is linked to the problem of compression of standard
information theory and might be tuned by self-organization.

positive:
Progress in pre-trained language models has led to a surge of impressive
results on downstream tasks for natural language understanding. Recent work on
probing pre-trained language models uncovered a wide range of linguistic
properties encoded in their contextualized representations. However, it is
unclear whether they encode semantic knowledge that is crucial to symbolic
inference methods. We propose a methodology for probing linguistic information
for logical inference in pre-trained language model representations. Our
probing datasets cover a list of linguistic phenomena required by major
symbolic inference systems. We find that (i) pre-trained language models do
encode several types of linguistic information for inference, but there are
also some types of information that are weakly encoded, (ii) language models
can effectively learn missing linguistic information through fine-tuning.
Overall, our findings provide insights into which aspects of linguistic
information for logical inference do language models and their pre-training
procedures capture. Moreover, we have demonstrated language models' potential
as semantic and background knowledge bases for supporting symbolic inference
methods.

positive:
Many real systems have been modelled in terms of network concepts, and
written texts are a particular example of information networks. In recent
years, the use of network methods to analyze language has allowed the discovery
of several interesting findings, including the proposition of novel models to
explain the emergence of fundamental universal patterns. While syntactical
networks, one of the most prevalent networked models of written texts, display
both scale-free and small-world properties, such representation fails in
capturing other textual features, such as the organization in topics or
subjects. In this context, we propose a novel network representation whose main
purpose is to capture the semantical relationships of words in a simple way. To
do so, we link all words co-occurring in the same semantic context, which is
defined in a threefold way. We show that the proposed representations favours
the emergence of communities of semantically related words, and this feature
may be used to identify relevant topics. The proposed methodology to detect
topics was applied to segment selected Wikipedia articles. We have found that,
in general, our methods outperform traditional bag-of-words representations,
which suggests that a high-level textual representation may be useful to study
semantical features of texts.

positive:
Recently, the focus of complex networks research has shifted from the
analysis of isolated properties of a system toward a more realistic modeling of
multiple phenomena - multilayer networks. Motivated by the prosperity of
multilayer approach in social, transport or trade systems, we propose the
introduction of multilayer networks for language. The multilayer network of
language is a unified framework for modeling linguistic subsystems and their
structural properties enabling the exploration of their mutual interactions.
Various aspects of natural language systems can be represented as complex
networks, whose vertices depict linguistic units, while links model their
relations. The multilayer network of language is defined by three aspects: the
network construction principle, the linguistic subsystem and the language of
interest. More precisely, we construct a word-level (syntax, co-occurrence and
its shuffled counterpart) and a subword level (syllables and graphemes) network
layers, from five variations of original text (in the modeled language). The
obtained results suggest that there are substantial differences between the
networks structures of different language subsystems, which are hidden during
the exploration of an isolated layer. The word-level layers share structural
properties regardless of the language (e.g. Croatian or English), while the
syllabic subword level expresses more language dependent structural properties.
The preserved weighted overlap quantifies the similarity of word-level layers
in weighted and directed networks. Moreover, the analysis of motifs reveals a
close topological structure of the syntactic and syllabic layers for both
languages. The findings corroborate that the multilayer network framework is a
powerful, consistent and systematic approach to model several linguistic
subsystems simultaneously and hence to provide a more unified view on language.

positive:
Much information available to applied researchers is contained within written
language or spoken text. Deep language models such as BERT have achieved
unprecedented success in many applications of computational linguistics.
However, much less is known about how these models can be used to analyze
existing text. We propose a novel method that combines transformer models with
network analysis to form a self-referential representation of language use
within a corpus of interest. Our approach produces linguistic relations
strongly consistent with the underlying model as well as mathematically
well-defined operations on them, while reducing the amount of discretionary
choices of representation and distance measures. It represents, to the best of
our knowledge, the first unsupervised method to extract semantic networks
directly from deep language models. We illustrate our approach in a semantic
analysis of the term "founder". Using the entire corpus of Harvard Business
Review from 1980 to 2020, we find that ties in our network track the semantics
of discourse over time, and across contexts, identifying and relating clusters
of semantic and syntactic relations. Finally, we discuss how this method can
also complement and inform analyses of the behavior of deep learning models.

positive:
Complex systems, such as life and languages, are governed by principles of
evolution. The analogy and comparison between biology and
linguistics\cite{alphafold2, RoseTTAFold, lang_virus, cell language, faculty1,
language of gene, Protein linguistics, dictionary, Grammar of pro_dom,
complexity, genomics_nlp, InterPro, language modeling, Protein language
modeling} provide a computational foundation for characterizing and analyzing
protein sequences, human corpora, and their evolution. However, no general
mathematical formula has been proposed so far to illuminate the origin of
quantitative hallmarks shared by life and language. Here we show several new
statistical relationships shared by proteins and words, which inspire us to
establish a general mechanism of evolution with explicit formulations that can
incorporate both old and new characteristics. We found natural selection can be
quantified via the entropic formulation by the principle of least effort to
determine the sequence variation that survives in evolution. Besides, the
origin of power law behavior and how changes in the environment stimulate the
emergence of new proteins and words can also be explained via the introduction
of function connection network. Our results demonstrate not only the
correspondence between genetics and linguistics over their different
hierarchies but also new fundamental physical properties for the evolution of
complex adaptive systems. We anticipate our statistical tests can function as
quantitative criteria to examine whether an evolution theory of sequence is
consistent with the regularity of real data. In the meantime, their
correspondence broadens the bridge to exchange existing knowledge, spurs new
interpretations, and opens Pandora's box to release several potentially
revolutionary challenges. For example, does linguistic arbitrariness conflict
with the dogma that structure determines function?

positive:
Statistical methods have been widely employed to study the fundamental
properties of language. In recent years, methods from complex and dynamical
systems proved useful to create several language models. Despite the large
amount of studies devoted to represent texts with physical models, only a
limited number of studies have shown how the properties of the underlying
physical systems can be employed to improve the performance of natural language
processing tasks. In this paper, I address this problem by devising complex
networks methods that are able to improve the performance of current
statistical methods. Using a fuzzy classification strategy, I show that the
topological properties extracted from texts complement the traditional textual
description. In several cases, the performance obtained with hybrid approaches
outperformed the results obtained when only traditional or networked methods
were used. Because the proposed model is generic, the framework devised here
could be straightforwardly used to study similar textual applications where the
topology plays a pivotal role in the description of the interacting agents.

positive:
Human beings possess the most sophisticated computational machinery in the
known universe. We can understand language of rich descriptive power, and
communicate in the same environment with astonishing clarity. Two of the many
contributors to the interest in natural language - the properties of
Compositionality and Structure Dependence, are well documented, and offer a
vast space to ask interesting modelling questions. The first step to begin
answering these questions is to ground verbal theory in formal terms. Drawing
on linguistics and set theory, a formalisation of these ideas is presented in
the first half of this thesis. We see how cognitive systems that process
language need to have certain functional constraints, viz. time based,
incremental operations that rely on a structurally defined domain. The
observations that result from analysing this formal setup are examined as part
of a modelling exercise. Using the advances of word embedding techniques, a
model of relational learning is simulated with a custom dataset to demonstrate
how a time based role-filler binding mechanism satisfies some of the
constraints described in the first section. The model's ability to map
structure, along with its symbolic-connectionist architecture makes for a
cognitively plausible implementation. The formalisation and simulation are
together an attempt to recognise the constraints imposed by linguistic theory,
and explore the opportunities presented by a cognitive model of relation
learning to realise these constraints.

positive:
Language is a social phenomenon and variation is inherent to its social
nature. Recently, there has been a surge of interest within the computational
linguistics (CL) community in the social dimension of language. In this article
we present a survey of the emerging field of "Computational Sociolinguistics"
that reflects this increased interest. We aim to provide a comprehensive
overview of CL research on sociolinguistic themes, featuring topics such as the
relation between language and social identity, language use in social
interaction and multilingual communication. Moreover, we demonstrate the
potential for synergy between the research communities involved, by showing how
the large-scale data-driven methods that are widely used in CL can complement
existing sociolinguistic studies, and how sociolinguistics can inform and
challenge the methods and assumptions employed in CL studies. We hope to convey
the possible benefits of a closer collaboration between the two communities and
conclude with a discussion of open challenges.

negative:
Recent research in computational linguistics has developed algorithms which
associate matrices with adjectives and verbs, based on the distribution of
words in a corpus of text. These matrices are linear operators on a vector
space of context words. They are used to construct the meaning of composite
expressions from that of the elementary constituents, forming part of a
compositional distributional approach to semantics. We propose a Matrix Theory
approach to this data, based on permutation symmetry along with Gaussian
weights and their perturbations. A simple Gaussian model is tested against word
matrices created from a large corpus of text. We characterize the cubic and
quartic departures from the model, which we propose, alongside the Gaussian
parameters, as signatures for comparison of linguistic corpora. We propose that
perturbed Gaussian models with permutation symmetry provide a promising
framework for characterizing the nature of universality in the statistical
properties of word matrices. The matrix theory framework developed here
exploits the view of statistics as zero dimensional perturbative quantum field
theory. It perceives language as a physical system realizing a universality
class of matrix statistics characterized by permutation symmetry.

negative:
Statistical methods have been widely employed in recent years to grasp many
language properties. The application of such techniques have allowed an
improvement of several linguistic applications, which encompasses machine
translation, automatic summarization and document classification. In the
latter, many approaches have emphasized the semantical content of texts, as it
is the case of bag-of-word language models. This approach has certainly yielded
reasonable performance. However, some potential features such as the structural
organization of texts have been used only on a few studies. In this context, we
probe how features derived from textual structure analysis can be effectively
employed in a classification task. More specifically, we performed a supervised
classification aiming at discriminating informative from imaginative documents.
Using a networked model that describes the local topological/dynamical
properties of function words, we achieved an accuracy rate of up to 95%, which
is much higher than similar networked approaches. A systematic analysis of
feature relevance revealed that symmetry and accessibility measurements are
among the most prominent network measurements. Our results suggest that these
measurements could be used in related language applications, as they play a
complementary role in characterizing texts.

negative:
The sequential structure of language, and the order of words in a sentence
specifically, plays a central role in human language processing. Consequently,
in designing computational models of language, the de facto approach is to
present sentences to machines with the words ordered in the same order as in
the original human-authored sentence. The very essence of this work is to
question the implicit assumption that this is desirable and inject theoretical
soundness into the consideration of word order in natural language processing.
In this thesis, we begin by uniting the disparate treatments of word order in
cognitive science, psycholinguistics, computational linguistics, and natural
language processing under a flexible algorithmic framework. We proceed to use
this heterogeneous theoretical foundation as the basis for exploring new word
orders with an undercurrent of psycholinguistic optimality. In particular, we
focus on notions of dependency length minimization given the difficulties in
human and computational language processing in handling long-distance
dependencies. We then discuss algorithms for finding optimal word orders
efficiently in spite of the combinatorial space of possibilities. We conclude
by addressing the implications of these word orders on human language and their
downstream impacts when integrated in computational models.

negative:
Many task domains require robots to interpret and act upon natural language
commands which are given by people and which refer to the robot's physical
surroundings. Such interpretation is known variously as the symbol grounding
problem, grounded semantics and grounded language acquisition. This problem is
challenging because people employ diverse vocabulary and grammar, and because
robots have substantial uncertainty about the nature and contents of their
surroundings, making it difficult to associate the constitutive language
elements (principally noun phrases and spatial relations) of the command text
to elements of those surroundings. Symbolic models capture linguistic structure
but have not scaled successfully to handle the diverse language produced by
untrained users. Existing statistical approaches can better handle diversity,
but have not to date modeled complex linguistic structure, limiting achievable
accuracy. Recent hybrid approaches have addressed limitations in scaling and
complexity, but have not effectively associated linguistic and perceptual
features. Our framework, called Generalized Grounding Graphs (G^3), addresses
these issues by defining a probabilistic graphical model dynamically according
to the linguistic parse structure of a natural language command. This approach
scales effectively, handles linguistic diversity, and enables the system to
associate parts of a command with the specific objects, places, and events in
the external world to which they refer. We show that robots can learn word
meanings and use those learned meanings to robustly follow natural language
commands produced by untrained users. We demonstrate our approach for both
mobility commands and mobile manipulation commands involving a variety of
semi-autonomous robotic platforms, including a wheelchair, a micro-air vehicle,
a forklift, and the Willow Garage PR2.

negative:
Distributional semantics provides multi-dimensional, graded, empirically
induced word representations that successfully capture many aspects of meaning
in natural languages, as shown in a large body of work in computational
linguistics; yet, its impact in theoretical linguistics has so far been
limited. This review provides a critical discussion of the literature on
distributional semantics, with an emphasis on methods and results that are of
relevance for theoretical linguistics, in three areas: semantic change,
polysemy and composition, and the grammar-semantics interface (specifically,
the interface of semantics with syntax and with derivational morphology). The
review aims at fostering greater cross-fertilization of theoretical and
computational approaches to language, as a means to advance our collective
knowledge of how it works.

negative:
In this paper, an unsupervised and cognitively driven weighted-entropy method
for embedding semantic categories in hyperbolic geometry is proposed. The model
is driven by two fields of research in cognitive linguistics: the first is the
statistical learning theory of language acquisition and the proposal of using
high-dimensional networks to represent semantic knowledge in cognition, and the
second is the domain-specific approach to semantic communication. Weighted
conditional entropy of word co-occurrence is proposed as the embedding metric,
and the two weighting parameters are collocation diversity and conditional
probability ranking in the corresponding statistical distribution. The
Boltzmann distribution is then used on the weighted-entropy metric and embedded
into a hyperbolic Poincare disk model. Testing has been in particular performed
in the domains of basic color and kinship words, which belong to the classes
that domain-specificity focused research in cognitive semantics has most
intensively investigated. Results show that this new approach can successfully
model and map the semantic relationships of popularity and similarity for most
of the basic color and kinship words in English and have potential to be
generalized to other semantic domains and different languages. Generally, this
paper contributes to both computational cognitive semantics and the research on
network and geometry-driven language embedding in computational linguistics and
NLP.

negative:
In this paper, we propose to study language modelling as a multi-task
problem, bringing together three strands of research: multi-task learning,
linguistics, and interpretability. Based on hypotheses derived from linguistic
theory, we investigate whether language models adhere to learning principles of
multi-task learning during training. To showcase the idea, we analyse the
generalisation behaviour of language models as they learn the linguistic
concept of Negative Polarity Items (NPIs). Our experiments demonstrate that a
multi-task setting naturally emerges within the objective of the more general
task of language modelling.We argue that this insight is valuable for
multi-task learning, linguistics and interpretability research and can lead to
exciting new findings in all three domains.

negative:
Neuroscientists evaluate deep neural networks for natural language processing
as possible candidate models for how language is processed in the brain. These
models are often trained without explicit linguistic supervision, but have been
shown to learn some linguistic structure in the absence of such supervision
(Manning et al., 2020), potentially questioning the relevance of symbolic
linguistic theories in modeling such cognitive processes (Warstadt and Bowman,
2020). We evaluate across two fMRI datasets whether language models align
better with brain recordings, if their attention is biased by annotations from
syntactic or semantic formalisms. Using structure from dependency or minimal
recursion semantic annotations, we find alignments improve significantly for
one of the datasets. For another dataset, we see more mixed results. We present
an extensive analysis of these results. Our proposed approach enables the
evaluation of more targeted hypotheses about the composition of meaning in the
brain, expanding the range of possible scientific inferences a neuroscientist
could make, and opens up new opportunities for cross-pollination between
computational neuroscience and linguistics.

negative:
In the last half-decade, the field of natural language processing (NLP) has
undergone two major transitions: the switch to neural networks as the primary
modeling paradigm and the homogenization of the training regime (pre-train,
then fine-tune). Amidst this process, language models have emerged as NLP's
workhorse, displaying increasingly fluent generation capabilities and proving
to be an indispensable means of knowledge transfer downstream. Due to the
otherwise opaque, black-box nature of such models, researchers have employed
aspects of linguistic theory in order to characterize their behavior. Questions
central to syntax -- the study of the hierarchical structure of language --
have factored heavily into such work, shedding invaluable insights about
models' inherent biases and their ability to make human-like generalizations.
In this paper, we attempt to take stock of this growing body of literature. In
doing so, we observe a lack of clarity across numerous dimensions, which
influences the hypotheses that researchers form, as well as the conclusions
they draw from their findings. To remedy this, we urge researchers make careful
considerations when investigating coding properties, selecting representations,
and evaluating via downstream tasks. Furthermore, we outline the implications
of the different types of research questions exhibited in studies on syntax, as
well as the inherent pitfalls of aggregate metrics. Ultimately, we hope that
our discussion adds nuance to the prospect of studying language models and
paves the way for a less monolithic perspective on syntax in this context.

negative:
With a growing interest in modeling inherent subjectivity in natural
language, we present a linguistically-motivated process to understand and
analyze the writing style of individuals from three perspectives: lexical,
syntactic, and semantic. We discuss the stylistically expressive elements
within each of these levels and use existing methods to quantify the linguistic
intuitions related to some of these elements. We show that such a multi-level
analysis is useful for developing a well-knit understanding of style - which is
independent of the natural language task at hand, and also demonstrate its
value in solving three downstream tasks: authors' style analysis, authorship
attribution, and emotion prediction. We conduct experiments on a variety of
datasets, comprising texts from social networking sites, user reviews, legal
documents, literary books, and newswire. The results on the aforementioned
tasks and datasets illustrate that such a multi-level understanding of style,
which has been largely ignored in recent works, models style-related
subjectivity in text and can be leveraged to improve performance on multiple
downstream tasks both qualitatively and quantitatively.

negative:
Nowadays, with the booming development of the Internet, people benefit from
its convenience due to its open and sharing nature. A large volume of natural
language texts is being generated by users in various forms, such as search
queries, documents, and social media posts. As the unstructured text corpus is
usually noisy and messy, it becomes imperative to correctly identify and
accurately annotate structured information in order to obtain meaningful
insights or better understand unstructured texts. On the other hand, the
existing structured information, which embodies our knowledge such as entity or
concept relations, often suffers from incompleteness or quality-related issues.
Given a gigantic collection of texts which offers rich semantic information, it
is also important to harness the massiveness of the unannotated text corpus to
expand and refine existing structured knowledge with fewer annotation efforts.
  In this dissertation, I will introduce principles, models, and algorithms for
effective structured knowledge discovery from the massive text corpus. We are
generally interested in obtaining insights and better understanding
unstructured texts with the help of structured annotations or by
structure-aware modeling. Also, given the existing structured knowledge, we are
interested in expanding its scale and improving its quality harnessing the
massiveness of the text corpus. In particular, four problems are studied in
this dissertation: Structured Intent Detection for Natural Language
Understanding, Structure-aware Natural Language Modeling, Generative Structured
Knowledge Expansion, and Synonym Refinement on Structured Knowledge.

negative:
There is an ongoing debate in the NLP community whether modern language
models contain linguistic knowledge, recovered through so-called probes. In
this paper, we study whether linguistic knowledge is a necessary condition for
the good performance of modern language models, which we call the
\textit{rediscovery hypothesis}. In the first place, we show that language
models that are significantly compressed but perform well on their pretraining
objectives retain good scores when probed for linguistic structures. This
result supports the rediscovery hypothesis and leads to the second contribution
of our paper: an information-theoretic framework that relates language modeling
objectives with linguistic information. This framework also provides a metric
to measure the impact of linguistic information on the word prediction task. We
reinforce our analytical results with various experiments, both on synthetic
and on real NLP tasks in English.

negative:
New machine translations (MT) technologies are emerging rapidly and with
them, bold claims of achieving human parity such as: (i) the results produced
approach "accuracy achieved by average bilingual human translators" (Wu et al.,
2017b) or (ii) the "translation quality is at human parity when compared to
professional human translators" (Hassan et al., 2018) have seen the light of
day (Laubli et al., 2018). Aside from the fact that many of these papers craft
their own definition of human parity, these sensational claims are often not
supported by a complete analysis of all aspects involved in translation.
Establishing the discrepancies between the strengths of statistical approaches
to MT and the way humans translate has been the starting point of our research.
By looking at MT output and linguistic theory, we were able to identify some
remaining issues. The problems range from simple number and gender agreement
errors to more complex phenomena such as the correct translation of aspectual
values and tenses. Our experiments confirm, along with other studies
(Bentivogli et al., 2016), that neural MT has surpassed statistical MT in many
aspects. However, some problems remain and others have emerged. We cover a
series of problems related to the integration of specific linguistic features
into statistical and neural MT, aiming to analyse and provide a solution to
some of them. Our work focuses on addressing three main research questions that
revolve around the complex relationship between linguistics and MT in general.
We identify linguistic information that is lacking in order for automatic
translation systems to produce more accurate translations and integrate
additional features into the existing pipelines. We identify overgeneralization
or 'algorithmic bias' as a potential drawback of neural MT and link it to many
of the remaining linguistic issues.

negative:
The learning trajectories of linguistic phenomena in humans provide insight
into linguistic representation, beyond what can be gleaned from inspecting the
behavior of an adult speaker. To apply a similar approach to analyze neural
language models (NLM), it is first necessary to establish that different models
are similar enough in the generalizations they make. In this paper, we show
that NLMs with different initialization, architecture, and training data
acquire linguistic phenomena in a similar order, despite their different end
performance. These findings suggest that there is some mutual inductive bias
that underlies these models' learning of linguistic phenomena. Taking
inspiration from psycholinguistics, we argue that studying this inductive bias
is an opportunity to study the linguistic representation implicit in NLMs.
  Leveraging these findings, we compare the relative performance on different
phenomena at varying learning stages with simpler reference models. Results
suggest that NLMs exhibit consistent "developmental" stages. Moreover, we find
the learning trajectory to be approximately one-dimensional: given an NLM with
a certain overall performance, it is possible to predict what linguistic
generalizations it has already acquired. Initial analysis of these stages
presents phenomena clusters (notably morphological ones), whose performance
progresses in unison, suggesting a potential link between the generalizations
behind them.

negative:
Statistical methods have been widely employed in many practical natural
language processing applications. More specifically, complex networks concepts
and methods from dynamical systems theory have been successfully applied to
recognize stylistic patterns in written texts. Despite the large amount of
studies devoted to represent texts with physical models, only a few studies
have assessed the relevance of attributes derived from the analysis of
stylistic fluctuations. Because fluctuations represent a pivotal factor for
characterizing a myriad of real systems, this study focused on the analysis of
the properties of stylistic fluctuations in texts via topological analysis of
complex networks and intermittency measurements. The results showed that
different authors display distinct fluctuation patterns. In particular, it was
found that it is possible to identify the authorship of books using the
intermittency of specific words. Taken together, the results described here
suggest that the patterns found in stylistic fluctuations could be used to
analyze other related complex systems. Furthermore, the discovery of novel
patterns related to textual stylistic fluctuations indicates that these
patterns could be useful to improve the state of the art of many
stylistic-based natural language processing tasks.

negative:
Words in some natural languages can have a composite structure. Elements of
this structure include the root (that could also be composite), prefixes and
suffixes with which various nuances and relations to other words can be
expressed. Thus, in order to build a proper word representation one must take
into account its internal structure. From a corpus of texts we extract a set of
frequent subwords and from the latter set we select patterns, i.e. subwords
which encapsulate information on character $n$-gram regularities. The selection
is made using the pattern-based Conditional Random Field model with $l_1$
regularization. Further, for every word we construct a new sequence over an
alphabet of patterns. The new alphabet's symbols confine a local statistical
context stronger than the characters, therefore they allow better
representations in ${\mathbb{R}}^n$ and are better building blocks for word
representation. In the task of subword-aware language modeling, pattern-based
models outperform character-based analogues by 2-20 perplexity points. Also, a
recurrent neural network in which a word is represented as a sum of embeddings
of its patterns is on par with a competitive and significantly more
sophisticated character-based convolutional architecture.

negative:
We analyze the information provided by the word embeddings about the
grammatical gender in Swedish. We wish that this paper may serve as one of the
bridges to connect the methods of computational linguistics and general
linguistics. Taking nominal classification in Swedish as a case study, we first
show how the information about grammatical gender in language can be captured
by word embedding models and artificial neural networks. Then, we match our
results with previous linguistic hypotheses on assignment and usage of
grammatical gender in Swedish and analyze the errors made by the computational
model from a linguistic perspective.

negative:
The mathematical formalism of quantum theory has been successfully used in
human cognition to model decision processes and to deliver representations of
human knowledge. As such, quantum cognition inspired tools have improved
technologies for Natural Language Processing and Information Retrieval. In this
paper, we overview the quantum cognition approach developed in our Brussels
team during the last two decades, specifically our identification of quantum
structures in human concepts and language, and the modeling of data from
psychological and corpus-text-based experiments. We discuss our
quantum-theoretic framework for concepts and their conjunctions/disjunctions in
a Fock-Hilbert space structure, adequately modeling a large amount of data
collected on concept combinations. Inspired by this modeling, we put forward
elements for a quantum contextual and meaning-based approach to information
technologies in which 'entities of meaning' are inversely reconstructed from
texts, which are considered as traces of these entities' states.

negative:
Social media platforms provide a goldmine for mining public opinion on issues
of wide societal interest and impact. Opinion mining is a problem that can be
operationalised by capturing and aggregating the stance of individual social
media posts as supporting, opposing or being neutral towards the issue at hand.
While most prior work in stance detection has investigated datasets that cover
short periods of time, interest in investigating longitudinal datasets has
recently increased. Evolving dynamics in linguistic and behavioural patterns
observed in new data require adapting stance detection systems to deal with the
changes. In this survey paper, we investigate the intersection between
computational linguistics and the temporal evolution of human communication in
digital media. We perform a critical review of emerging research considering
dynamics, exploring different semantic and pragmatic factors that impact
linguistic data in general, and stance in particular. We further discuss
current directions in capturing stance dynamics in social media. We discuss the
challenges encountered when dealing with stance dynamics, identify open
challenges and discuss future directions in three key dimensions: utterance,
context and influence.

negative:
Citation content analysis seeks to understand citations based on the language
used during the making of a citation. A key issue in citation content analysis
is looking for linguistic structures that characterize distinct classes of
citations for the purposes of understanding the intent and function of a
citation. Previous works have focused on modeling linguistic features first and
drawn conclusions on the language structures unique to each class of citation
function based on the performance of a classification task or inter-annotator
agreement. In this study, we start with a large sample of a pre-classified
citation corpus, 2 million citations from each class of the scite Smart
Citation dataset (supporting, disputing, and mentioning citations), and analyze
its corpus linguistics in order to reveal the unique and statistically
significant language structures belonging to each type of citation. By
generating comparison tables for each citation type we present a number of
interesting linguistic features that uniquely characterize citation type. What
we find is that within citation collocates, there is very low correlation
between citation type and sentiment. Additionally, we find that the
subjectivity of citation collocates across classes is very low. These findings
suggest that the sentiment of collocates is not a predictor of citation
function and that due to their low subjectivity, an opinion-expressing mode of
understanding citations, implicit in previous citation sentiment analysis
literature, is inappropriate. Instead, we suggest that citations can be better
understood as claims-making devices where the citation type can be explained by
understanding how two claims are being compared. By presenting this approach,
we hope to inspire similar corpus linguistic studies on citations that derive a
more robust theory of citation from an empirical basis using citation corpora

negative:
It is often stated that human languages, as other biological systems, are
shaped by cost-cutting pressures but, to what extent? Attempts to quantify the
degree of optimality of languages by means of an optimality score have been
scarce and focused mostly on English. Here we recast the problem of the
optimality of the word order of a sentence as an optimization problem on a
spatial network where the vertices are words, arcs indicate syntactic
dependencies and the space is defined by the linear order of the words in the
sentence. We introduce a new score to quantify the cognitive pressure to reduce
the distance between linked words in a sentence. The analysis of sentences from
93 languages representing 19 linguistic families reveals that half of languages
are optimized to a 70% or more. The score indicates that distances are not
significantly reduced in a few languages and confirms two theoretical
predictions, i.e. that longer sentences are more optimized and that distances
are more likely to be longer than expected by chance in short sentences. We
present a new hierarchical ranking of languages by their degree of
optimization. The new score has implications for various fields of language
research (dependency linguistics, typology, historical linguistics, clinical
linguistics and cognitive science). Finally, the principles behind the design
of the score have implications for network science.

negative:
Human language is recognized as a very complex domain since decades. No
computer system has been able to reach human levels of performance so far. The
only known computational system capable of proper language processing is the
human brain. While we gather more and more data about the brain, its
fundamental computational processes still remain obscure. The lack of a sound
computational brain theory also prevents the fundamental understanding of
Natural Language Processing. As always when science lacks a theoretical
foundation, statistical modeling is applied to accommodate as many sampled
real-world data as possible. An unsolved fundamental issue is the actual
representation of language (data) within the brain, denoted as the
Representational Problem. Starting with Jeff Hawkins' Hierarchical Temporal
Memory (HTM) theory, a consistent computational theory of the human cortex, we
have developed a corresponding theory of language data representation: The
Semantic Folding Theory. The process of encoding words, by using a topographic
semantic space as distributional reference frame into a sparse binary
representational vector is called Semantic Folding and is the central topic of
this document. Semantic Folding describes a method of converting language from
its symbolic representation (text) into an explicit, semantically grounded
representation that can be generically processed by Hawkins' HTM networks. As
it turned out, this change in representation, by itself, can solve many complex
NLP problems by applying Boolean operators and a generic similarity function
like the Euclidian Distance. Many practical problems of statistical NLP
systems, like the high cost of computation, the fundamental incongruity of
precision and recall , the complex tuning procedures etc., can be elegantly
overcome by applying Semantic Folding.

negative:
Formal Semantics and Distributional Semantics are two important semantic
frameworks in Natural Language Processing (NLP). Cognitive Semantics belongs to
the movement of Cognitive Linguistics, which is based on contemporary cognitive
science. Each framework could deal with some meaning phenomena, but none of
them fulfills all requirements proposed by applications. A unified semantic
theory characterizing all important language phenomena has both theoretical and
practical significance; however, although many attempts have been made in
recent years, no existing theory has achieved this goal yet.
  This article introduces a new semantic theory that has the potential to
characterize most of the important meaning phenomena of natural language and to
fulfill most of the necessary requirements for philosophical analysis and for
NLP applications. The theory is based on a unified representation of
information, and constructs a kind of mathematical model called cognitive model
to interpret natural language expressions in a compositional manner. It accepts
the empirical assumption of Cognitive Semantics, and overcomes most
shortcomings of Formal Semantics and of Distributional Semantics. The theory,
however, is not a simple combination of existing theories, but an extensive
generalization of classic logic and Formal Semantics. It inherits nearly all
advantages of Formal Semantics, and also provides descriptive contents for
objects and events as fine-gram as possible, descriptive contents which
represent the results of human cognition.

negative:
Purpose: The purpose of this paper is to identify the factors influencing the
growth of healthcare virtual communities of practice (VCoPs) through a
seven-year longitudinal study conducted using metrics from social-network and
semantic analysis. By studying online communication along the three dimensions
of social interactions (connectivity, interactivity and language use), the
authors aim to provide VCoP managers with valuable insights to improve the
success of their communities. Design/methodology/approach: Communications over
a period of seven years (April 2008 to April 2015) and between 14,000 members
of 16 different healthcare VCoPs coexisting on the same web platform were
analysed. Multilevel regression models were used to reveal the main
determinants of community growth over time. Independent variables were derived
from social network and semantic analysis measures. Findings: Results show that
structural and content-based variables predict the growth of the community.
Progressively, more people will join a community if its structure is more
centralised, leaders are more dynamic (they rotate more) and the language used
in the posts is less complex. Research limitations/implications: The available
data set included one Web platform and a limited number of control variables.
To consolidate the findings of the present study, the experiment should be
replicated on other healthcare VCoPs. Originality/value: The study provides
useful recommendations for setting up and nurturing the growth of professional
communities, considering, at the same time, the interaction patterns among the
community members, the dynamic evolution of these interactions and the use of
language. New analytical tools are presented, together with the use of
innovative interaction metrics, that can significantly influence community
growth, such as rotating leadership.

negative:
We present a study of the relationship between gender, linguistic style, and
social networks, using a novel corpus of 14,000 Twitter users. Prior
quantitative work on gender often treats this social variable as a female/male
binary; we argue for a more nuanced approach. By clustering Twitter users, we
find a natural decomposition of the dataset into various styles and topical
interests. Many clusters have strong gender orientations, but their use of
linguistic resources sometimes directly conflicts with the population-level
language statistics. We view these clusters as a more accurate reflection of
the multifaceted nature of gendered language styles. Previous corpus-based work
has also had little to say about individuals whose linguistic styles defy
population-level gender patterns. To identify such individuals, we train a
statistical classifier, and measure the classifier confidence for each
individual in the dataset. Examining individuals whose language does not match
the classifier's model for their gender, we find that they have social networks
that include significantly fewer same-gender social connections and that, in
general, social network homophily is correlated with the use of same-gender
language markers. Pairing computational methods and social theory thus offers a
new perspective on how gender emerges as individuals position themselves
relative to audiences, topics, and mainstream gender norms.

negative:
Do state-of-the-art models for language understanding already have, or can
they easily learn, abilities such as boolean coordination, quantification,
conditionals, comparatives, and monotonicity reasoning (i.e., reasoning about
word substitutions in sentential contexts)? While such phenomena are involved
in natural language inference (NLI) and go beyond basic linguistic
understanding, it is unclear the extent to which they are captured in existing
NLI benchmarks and effectively learned by models. To investigate this, we
propose the use of semantic fragments---systematically generated datasets that
each target a different semantic phenomenon---for probing, and efficiently
improving, such capabilities of linguistic models. This approach to creating
challenge datasets allows direct control over the semantic diversity and
complexity of the targeted linguistic phenomena, and results in a more precise
characterization of a model's linguistic behavior. Our experiments, using a
library of 8 such semantic fragments, reveal two remarkable findings: (a)
State-of-the-art models, including BERT, that are pre-trained on existing NLI
benchmark datasets perform poorly on these new fragments, even though the
phenomena probed here are central to the NLI task. (b) On the other hand, with
only a few minutes of additional fine-tuning---with a carefully selected
learning rate and a novel variation of "inoculation"---a BERT-based model can
master all of these logic and monotonicity fragments while retaining its
performance on established NLI benchmarks.

negative:
The use of linguistic typological resources in natural language processing
has been steadily gaining more popularity. It has been observed that the use of
typological information, often combined with distributed language
representations, leads to significantly more powerful models. While linguistic
typology representations from various resources have mostly been used for
conditioning the models, there has been relatively little attention on
predicting features from these resources from the input data. In this paper we
investigate whether the various linguistic features from World Atlas of
Language Structures (WALS) can be reliably inferred from multi-lingual text.
Such a predictor can be used to infer structural features for a language never
observed in training data. We frame this task as a multi-label classification
involving predicting the set of non-mutually exclusive and extremely sparse
multi-valued labels (WALS features). We construct a recurrent neural network
predictor based on byte embeddings and convolutional layers and test its
performance on 556 languages, providing analysis for various linguistic types,
macro-areas, language families and individual features. We show that some
features from various linguistic types can be predicted reliably.

negative:
It is widely accepted that information derived from analyzing speech (the
acoustic signal) and language production (words and sentences) serves as a
useful window into the health of an individual's cognitive ability. In fact,
most neuropsychological testing batteries have a component related to speech
and language where clinicians elicit speech from patients for subjective
evaluation across a broad set of dimensions. With advances in speech signal
processing and natural language processing, there has been recent interest in
developing tools to detect more subtle changes in cognitive-linguistic
function. This work relies on extracting a set of features from recorded and
transcribed speech for objective assessments of speech and language, early
diagnosis of neurological disease, and tracking of disease after diagnosis.
With an emphasis on cognitive and thought disorders, in this paper we provide a
review of existing speech and language features used in this domain, discuss
their clinical application, and highlight their advantages and disadvantages.
Broadly speaking, the review is split into two categories: language features
based on natural language processing and speech features based on speech signal
processing. Within each category, we consider features that aim to measure
complementary dimensions of cognitive-linguistics, including language
diversity, syntactic complexity, semantic coherence, and timing. We conclude
the review with a proposal of new research directions to further advance the
field.

negative:
The goal of language modeling techniques is to capture the statistical and
structural properties of natural languages from training corpora. This task
typically involves the learning of short range dependencies, which generally
model the syntactic properties of a language and/or long range dependencies,
which are semantic in nature. We propose in this paper a new multi-span
architecture, which separately models the short and long context information
while it dynamically merges them to perform the language modeling task. This is
done through a novel recurrent Long-Short Range Context (LSRC) network, which
explicitly models the local (short) and global (long) context using two
separate hidden states that evolve in time. This new architecture is an
adaptation of the Long-Short Term Memory network (LSTM) to take into account
the linguistic properties. Extensive experiments conducted on the Penn Treebank
(PTB) and the Large Text Compression Benchmark (LTCB) corpus showed a
significant reduction of the perplexity when compared to state-of-the-art
language modeling techniques.

negative:
Exploiting the fact that natural languages are complex systems, the present
exploratory article proposes a direct method based on frequency distributions
that may be useful when making a decision on the status of problematic
phonemes, an open problem in linguistics. The main notion is that natural
languages, which can be considered from a complex outlook as information
processing machines, and which somehow manage to set appropriate levels of
redundancy, already "made the choice" whether a linguistic unit is a phoneme or
not, and this would be reflected in a greater smoothness in a frequency versus
rank graph. For the particular case we chose to study, we conclude that it is
reasonable to consider the Spanish semiconsonant /w/ as a separate phoneme from
its vowel counterpart /u/, on the one hand, and possibly also the semiconsonant
/j/ as a separate phoneme from its vowel counterpart /i/, on the other. As
language has been so central a topic in the study of complexity, this
discussion grants us, in addition, an opportunity to gain insight into emerging
properties in the broader complex systems debate.

negative:
A short review of similarities between dolphins and humans with the help of
quantitative linguistics and information theory.

negative:
We investigated the impact of noisy linguistic features on the performance of
a Japanese speech synthesis system based on neural network that uses WaveNet
vocoder. We compared an ideal system that uses manually corrected linguistic
features including phoneme and prosodic information in training and test sets
against a few other systems that use corrupted linguistic features. Both
subjective and objective results demonstrate that corrupted linguistic
features, especially those in the test set, affected the ideal system's
performance significantly in a statistical sense due to a mismatched condition
between the training and test sets. Interestingly, while an utterance-level
Turing test showed that listeners had a difficult time differentiating
synthetic speech from natural speech, it further indicated that adding noise to
the linguistic features in the training set can partially reduce the effect of
the mismatch, regularize the model, and help the system perform better when
linguistic features of the test set are noisy.

negative:
Natural language descriptions sometimes accompany visualizations to better
communicate and contextualize their insights, and to improve their
accessibility for readers with disabilities. However, it is difficult to
evaluate the usefulness of these descriptions, and how effectively they improve
access to meaningful information, because we have little understanding of the
semantic content they convey, and how different readers receive this content.
In response, we introduce a conceptual model for the semantic content conveyed
by natural language descriptions of visualizations. Developed through a
grounded theory analysis of 2,147 sentences, our model spans four levels of
semantic content: enumerating visualization construction properties (e.g.,
marks and encodings); reporting statistical concepts and relations (e.g.,
extrema and correlations); identifying perceptual and cognitive phenomena
(e.g., complex trends and patterns); and elucidating domain-specific insights
(e.g., social and political context). To demonstrate how our model can be
applied to evaluate the effectiveness of visualization descriptions, we conduct
a mixed-methods evaluation with 30 blind and 90 sighted readers, and find that
these reader groups differ significantly on which semantic content they rank as
most useful. Together, our model and findings suggest that access to meaningful
information is strongly reader-specific, and that research in automatic
visualization captioning should orient toward descriptions that more richly
communicate overall trends and statistics, sensitive to reader preferences. Our
work further opens a space of research on natural language as a data interface
coequal with visualization.

negative:
Linguistic laws constitute one of the quantitative cornerstones of modern
cognitive sciences and have been routinely investigated in written corpora, or
in the equivalent transcription of oral corpora. This means that inferences of
statistical patterns of language in acoustics are biased by the arbitrary,
language-dependent segmentation of the signal, and virtually precludes the
possibility of making comparative studies between human voice and other animal
communication systems. Here we bridge this gap by proposing a method that
allows to measure such patterns in acoustic signals of arbitrary origin,
without needs to have access to the language corpus underneath. The method has
been applied to six different human languages, recovering successfully some
well-known laws of human communication at timescales even below the phoneme and
finding yet another link between complexity and criticality in a biological
system. These methods further pave the way for new comparative studies in
animal communication or the analysis of signals of unknown code.

negative:
This project explores the nature of language acquisition in computers, guided
by techniques similar to those used in children. While existing natural
language processing methods are limited in scope and understanding, our system
aims to gain an understanding of language from first principles and hence
minimal initial input. The first portion of our system was implemented in Java
and is focused on understanding the morphology of language using bigrams. We
use frequency distributions and differences between them to define and
distinguish languages. English and French texts were analyzed to determine a
difference threshold of 55 before the texts are considered to be in different
languages, and this threshold was verified using Spanish texts. The second
portion of our system focuses on gaining an understanding of the syntax of a
language using a recursive method. The program uses one of two possible methods
to analyze given sentences based on either sentence patterns or surrounding
words. Both methods have been implemented in C++. The program is able to
understand the structure of simple sentences and learn new words. In addition,
we have provided some suggestions regarding future work and potential
extensions of the existing program.

negative:
My doctoral research focuses on understanding semantic knowledge in neural
network models trained solely to predict natural language (referred to as
language models, or LMs), by drawing on insights from the study of concepts and
categories grounded in cognitive science. I propose a framework inspired by
'inductive reasoning,' a phenomenon that sheds light on how humans utilize
background knowledge to make inductive leaps and generalize from new pieces of
information about concepts and their properties. Drawing from experiments that
study inductive reasoning, I propose to analyze semantic inductive
generalization in LMs using phenomena observed in human-induction literature,
investigate inductive behavior on tasks such as implicit reasoning and emergent
feature recognition, and analyze and relate induction dynamics to the learned
conceptual representation space.

negative:
In recent years, automated approaches to assessing linguistic complexity in
second language (L2) writing have made significant progress in gauging learner
performance, predicting human ratings of the quality of learner productions,
and benchmarking L2 development. In contrast, there is comparatively little
work in the area of speaking, particularly with respect to fully automated
approaches to assessing L2 spontaneous speech. While the importance of a
well-performing ASR system is widely recognized, little research has been
conducted to investigate the impact of its performance on subsequent automatic
text analysis. In this paper, we focus on this issue and examine the impact of
using a state-of-the-art ASR system for subsequent automatic analysis of
linguistic complexity in spontaneously produced L2 speech. A set of 30 selected
measures were considered, falling into four categories: syntactic, lexical,
n-gram frequency, and information-theoretic measures. The agreement between the
scores for these measures obtained on the basis of ASR-generated vs. manual
transcriptions was determined through correlation analysis. A more differential
effect of ASR performance on specific types of complexity measures when
controlling for task type effects is also presented.

negative:
This paper aims to catalyze the discussions about text feature extraction
techniques using neural network architectures. The research questions discussed
in the paper focus on the state-of-the-art neural network techniques that have
proven to be useful tools for language processing, language generation, text
classification and other computational linguistics tasks.

negative:
Neural models excel at extracting statistical patterns from large amounts of
data, but struggle to learn patterns or reason about language from only a few
examples. In this paper, we ask: Can we learn explicit rules that generalize
well from only a few examples? We explore this question using program
synthesis. We develop a synthesis model to learn phonology rules as programs in
a domain-specific language. We test the ability of our models to generalize
from few training examples using our new dataset of problems from the
Linguistics Olympiad, a challenging set of tasks that require strong linguistic
reasoning ability. In addition to being highly sample-efficient, our approach
generates human-readable programs, and allows control over the generalizability
of the learnt programs.

negative:
More and more languages in the world are under study nowadays, as a result,
the traditional way of historical linguistics study is facing some challenges.
For example, the linguistic comparative research among languages needs manual
annotation, which becomes more and more impossible with the increasing amount
of language data coming out all around the world. Although it could hardly
replace linguists work, the automatic computational methods have been taken
into consideration and it can help people reduce their workload. One of the
most important work in historical linguistics is word comparison from different
languages and find the cognate words for them, which means people try to figure
out if the two languages are related to each other or not. In this paper, I am
going to use computational method to cluster the languages and use Markov Chain
Monte Carlo (MCMC) method to build the language typology relationship tree
based on the clusters.

negative:
Distributional semantics has had enormous empirical success in Computational
Linguistics and Cognitive Science in modeling various semantic phenomena, such
as semantic similarity, and distributional models are widely used in
state-of-the-art Natural Language Processing systems. However, the theoretical
status of distributional semantics within a broader theory of language and
cognition is still unclear: What does distributional semantics model? Can it
be, on its own, a fully adequate model of the meanings of linguistic
expressions? The standard answer is that distributional semantics is not fully
adequate in this regard, because it falls short on some of the central aspects
of formal semantic approaches: truth conditions, entailment, reference, and
certain aspects of compositionality. We argue that this standard answer rests
on a misconception: These aspects do not belong in a theory of expression
meaning, they are instead aspects of speaker meaning, i.e., communicative
intentions in a particular context. In a slogan: words do not refer, speakers
do. Clearing this up enables us to argue that distributional semantics on its
own is an adequate model of expression meaning. Our proposal sheds light on the
role of distributional semantics in a broader theory of language and cognition,
its relationship to formal semantics, and its place in computational models.

negative:
Linguistic typology aims to capture structural and semantic variation across
the world's languages. A large-scale typology could provide excellent guidance
for multilingual Natural Language Processing (NLP), particularly for languages
that suffer from the lack of human labeled resources. We present an extensive
literature survey on the use of typological information in the development of
NLP techniques. Our survey demonstrates that to date, the use of information in
existing typological databases has resulted in consistent but modest
improvements in system performance. We show that this is due to both intrinsic
limitations of databases (in terms of coverage and feature granularity) and
under-employment of the typological features included in them. We advocate for
a new approach that adapts the broad and discrete nature of typological
categories to the contextual and continuous nature of machine learning
algorithms used in contemporary NLP. In particular, we suggest that such
approach could be facilitated by recent developments in data-driven induction
of typological knowledge.

negative:
Qualitative analysis of verbal data is of central importance in the learning
sciences. It is labor-intensive and time-consuming, however, which limits the
amount of data researchers can include in studies. This work is a step towards
building a statistical machine learning (ML) method for achieving an automated
support for qualitative analyses of students' writing, here specifically in
score laboratory reports in introductory biology for sophistication of
argumentation and reasoning. We start with a set of lab reports from an
undergraduate biology course, scored by a four-level scheme that considers the
complexity of argument structure, the scope of evidence, and the care and
nuance of conclusions. Using this set of labeled data, we show that a popular
natural language modeling processing pipeline, namely vector representation of
words, a.k.a word embeddings, followed by Long Short Term Memory (LSTM) model
for capturing language generation as a state-space model, is able to
quantitatively capture the scoring, with a high Quadratic Weighted Kappa (QWK)
prediction score, when trained in via a novel contrastive learning set-up. We
show that the ML algorithm approached the inter-rater reliability of human
analysis. Ultimately, we conclude, that machine learning (ML) for natural
language processing (NLP) holds promise for assisting learning sciences
researchers in conducting qualitative studies at much larger scales than is
currently possible.

negative:
Modelling concept representation is a foundational problem in the study of
cognition and linguistics. This work builds on the confluence of conceptual
tools from G\"ardenfors semantic spaces, categorical compositional linguistics,
and applied category theory to present a domain-independent and categorical
formalism of 'concept'.

negative:
This research offers a new interdisciplinary approach to the field of
Linguistics by using Computational Linguistics, NLP, Bayesian Statistics and
Sociolinguistics methods. This thesis investigates word order change in
infinitival clauses from Object-Verb (OV) to Verb-Object (VO) in the history of
Latin and Old French. By applying a variationist approach, I examine a
synchronic word order variation in each stage of language change, from which I
infer the character, periodization and constraints of diachronic variation. I
also show that in discourse-configurational languages, such as Latin and Early
Old French, it is possible to identify pragmatically neutral contexts by using
information structure annotation. I further argue that by mapping pragmatic
categories into a syntactic structure, we can detect how word order change
unfolds. For this investigation, the data are extracted from annotated corpora
spanning several centuries of Latin and Old French and from additional
resources created by using computational linguistic methods. The data are then
further codified for various pragmatic, semantic, syntactic and sociolinguistic
factors. This study also evaluates previous factors proposed to account for
word order alternation and change. I show how information structure and
syntactic constraints change over time and propose a method that allows
researchers to differentiate a stable word order alternation from alternation
indicating a change. Finally, I present a three-stage probabilistic model of
word order change, which also conforms to traditional language change patterns.

negative:
Counting letters in written texts is a very ancient practice. It has
accompanied the development of Cryptology, Quantitative Linguistics, and
Statistics. In Cryptology, counting frequencies of the different characters in
an encrypted message is the basis of the so called frequency analysis method.
In Quantitative Linguistics, the proportion of vowels to consonants in
different languages was studied long before authorship attribution. In
Statistics, the alternation vowel-consonants was the only example that Markov
ever gave of his theory of chained events. A short history of letter counting
is presented. The three domains, Cryptology, Quantitative Linguistics, and
Statistics, are then examined, focusing on the interactions with the other two
fields through letter counting. As a conclusion, the eclectism of past
centuries scholars, their background in humanities, and their familiarity with
cryptograms, are identified as contributing factors to the mutual enrichment
process which is described here.

negative:
Speech-controlled user interfaces facilitate the operation of devices and
household functions to laymen. State-of-the-art language technology scans the
acoustically analyzed speech signal for relevant keywords that are subsequently
inserted into semantic slots to interpret the user's intent. In order to
develop proper cognitive information and communication technologies, simple
slot-filling should be replaced by utterance meaning transducers (UMT) that are
based on semantic parsers and a \emph{mental lexicon}, comprising syntactic,
phonetic and semantic features of the language under consideration. This
lexicon must be acquired by a cognitive agent during interaction with its
users. We outline a reinforcement learning algorithm for the acquisition of the
syntactic morphology and arithmetic semantics of English numerals, based on
minimalist grammar (MG), a recent computational implementation of generative
linguistics. Number words are presented to the agent by a teacher in form of
utterance meaning pairs (UMP) where the meanings are encoded as arithmetic
terms from a suitable term algebra. Since MG encodes universal linguistic
competence through inference rules, thereby separating innate linguistic
knowledge from the contingently acquired lexicon, our approach unifies
generative grammar and reinforcement learning, hence potentially resolving the
still pending Chomsky-Skinner controversy.

negative:
Word meaning changes over time, depending on linguistic and extra-linguistic
factors. Associating a word's correct meaning in its historical context is a
central challenge in diachronic research, and is relevant to a range of NLP
tasks, including information retrieval and semantic search in historical texts.
Bayesian models for semantic change have emerged as a powerful tool to address
this challenge, providing explicit and interpretable representations of
semantic change phenomena. However, while corpora typically come with rich
metadata, existing models are limited by their inability to exploit contextual
information (such as text genre) beyond the document time-stamp. This is
particularly critical in the case of ancient languages, where lack of data and
long diachronic span make it harder to draw a clear distinction between
polysemy (the fact that a word has several senses) and semantic change (the
process of acquiring, losing, or changing senses), and current systems perform
poorly on these languages. We develop GASC, a dynamic semantic change model
that leverages categorical metadata about the texts' genre to boost inference
and uncover the evolution of meanings in Ancient Greek corpora. In a new
evaluation framework, our model achieves improved predictive performance
compared to the state of the art.

negative:
For building question answering systems and natural language interfaces,
semantic parsing has emerged as an important and powerful paradigm. Semantic
parsers map natural language into logical forms, the classic representation for
many important linguistic phenomena. The modern twist is that we are interested
in learning semantic parsers from data, which introduces a new layer of
statistical and computational issues. This article lays out the components of a
statistical semantic parser, highlighting the key challenges. We will see that
semantic parsing is a rich fusion of the logical and the statistical world, and
that this fusion will play an integral role in the future of natural language
understanding systems.

negative:
The annotation of textual information is a fundamental activity in
Linguistics and Computational Linguistics. This article presents various
observations on annotations. It approaches the topic from several angles
including Hypertext, Computational Linguistics and Language Technology,
Artificial Intelligence and Open Science. Annotations can be examined along
different dimensions. In terms of complexity, they can range from trivial to
highly sophisticated, in terms of maturity from experimental to standardised.
Annotations can be annotated themselves using more abstract annotations.
Primary research data such as, e.g., text documents can be annotated on
different layers concurrently, which are independent but can be exploited using
multi-layer querying. Standards guarantee interoperability and reusability of
data sets. The chapter concludes with four final observations, formulated as
research questions or rather provocative remarks on the current state of
annotation research.

negative:
With the increasing number of texts made available on the Internet, many
applications have relied on text mining tools to tackle a diversity of
problems. A relevant model to represent texts is the so-called word adjacency
(co-occurrence) representation, which is known to capture mainly syntactical
features of texts.In this study, we introduce a novel network representation
that considers the semantic similarity between paragraphs. Two main properties
of paragraph networks are considered: (i) their ability to incorporate
characteristics that can discriminate real from artificial, shuffled
manuscripts and (ii) their ability to capture syntactical and semantic textual
features. Our results revealed that real texts are organized into communities,
which turned out to be an important feature for discriminating them from
artificial texts. Interestingly, we have also found that, differently from
traditional co-occurrence networks, the adopted representation is able to
capture semantic features. Additionally, the proposed framework was employed to
analyze the Voynich manuscript, which was found to be compatible with texts
written in natural languages. Taken together, our findings suggest that the
proposed methodology can be combined with traditional network models to improve
text classification tasks.

negative:
Representation of linguistic phenomena in computational language models is
typically assessed against the predictions of existing linguistic theories of
these phenomena. Using the notion of polarity as a case study, we show that
this is not always the most adequate set-up. We probe polarity via so-called
'negative polarity items' (in particular, English 'any') in two pre-trained
Transformer-based models (BERT and GPT-2). We show that - at least for polarity
- metrics derived from language models are more consistent with data from
psycholinguistic experiments than linguistic theory predictions. Establishing
this allows us to more adequately evaluate the performance of language models
and also to use language models to discover new insights into natural language
grammar beyond existing linguistic theories. This work contributes to
establishing closer ties between psycholinguistic experiments and experiments
with language models.

negative:
In recent years, central components of a new approach to linguistics, the
Minimalist Program (MP) have come closer to physics. Features of the Minimalist
Program, such as the unconstrained nature of recursive Merge, the operation of
the Labeling Algorithm that only operates at the interface of Narrow Syntax
with the Conceptual-Intentional and the Sensory-Motor interfaces, the
difference between pronounced and un-pronounced copies of elements in a
sentence and the build-up of the Fibonacci sequence in the syntactic derivation
of sentence structures, are directly accessible to representation in terms of
algebraic formalism. Although in our scheme linguistic structures are classical
ones, we find that an interesting and productive isomorphism can be established
between the MP structure, algebraic structures and many-body field theory
opening new avenues of inquiry on the dynamics underlying some central aspects
of linguistics.

negative:
This special issue is dedicated to get a better picture of the relationships
between computational linguistics and cognitive science. It specifically raises
two questions: "what is the potential contribution of computational language
modeling to cognitive science?" and conversely: "what is the influence of
cognitive science in contemporary computational linguistics?"

negative:
Significant advances have been made in artificial systems by using biological
systems as a guide. However, there is often little interaction between
computational models for emergent communication and biological models of the
emergence of language. Many researchers in language origins and emergent
communication take compositionality as their primary target for explaining how
simple communication systems can become more like natural language. However,
there is reason to think that compositionality is the wrong target on the
biological side, and so too the wrong target on the machine-learning side. As
such, the purpose of this paper is to explore this claim. This has theoretical
implications for language origins research more generally, but the focus here
will be the implications for research on emergent communication in computer
science and machine learning---specifically regarding the types of programmes
that might be expected to work and those which will not. I further suggest an
alternative approach for future research which focuses on reflexivity, rather
than compositionality, as a target for explaining how simple communication
systems may become more like natural language. I end by providing some
reference to the language origins literature that may be of some use to
researchers in machine learning.

negative:
Linguistic steganography studies how to hide secret messages in natural
language cover texts. Traditional methods aim to transform a secret message
into an innocent text via lexical substitution or syntactical modification.
Recently, advances in neural language models (LMs) enable us to directly
generate cover text conditioned on the secret message. In this study, we
present a new linguistic steganography method which encodes secret messages
using self-adjusting arithmetic coding based on a neural language model. We
formally analyze the statistical imperceptibility of this method and
empirically show it outperforms the previous state-of-the-art methods on four
datasets by 15.3% and 38.9% in terms of bits/word and KL metrics, respectively.
Finally, human evaluations show that 51% of generated cover texts can indeed
fool eavesdroppers.

negative:
Speech rhythms have been dealt with in three main ways: from the
introspective analyses of rhythm as a correlate of syllable and foot timing in
linguistics and applied linguistics, through analyses of durations of segments
of utterances associated with consonantal and vocalic properties, syllables,
feet and words, to models of rhythms in speech production and perception as
physical oscillations. The present study avoids introspection and
human-filtered annotation methods and extends the signal processing paradigm of
amplitude envelope spectrum analysis by adding an additional analytic step of
edge detection, and postulating the co-existence of multiple speech rhythms in
rhythm zones marked by identifiable edges (Rhythm Zone Theory, RZT). An
exploratory investigation of the utility of RZT is conducted, suggesting that
native and non-native readings of the same text are distinct sub-genres of read
speech: a reading by a US native speaker and non-native readings by relatively
low-performing Cantonese adult learners of English. The study concludes by
noting that with the methods used, RZT can distinguish between the speech
rhythms of well-defined sub-genres of native speaker reading vs. non-native
learner reading, but needs further refinement in order to be applied to the
paradoxically more complex speech of low-performing language learners, whose
speech rhythms are co-determined by non-fluency and disfluency factors in
addition to well-known linguistic factors of grammar, vocabulary and discourse
constraints.

negative:
Due to their inherent capability in semantic alignment of aspects and their
context words, attention mechanism and Convolutional Neural Networks (CNNs) are
widely applied for aspect-based sentiment classification. However, these models
lack a mechanism to account for relevant syntactical constraints and long-range
word dependencies, and hence may mistakenly recognize syntactically irrelevant
contextual words as clues for judging aspect sentiment. To tackle this problem,
we propose to build a Graph Convolutional Network (GCN) over the dependency
tree of a sentence to exploit syntactical information and word dependencies.
Based on it, a novel aspect-specific sentiment classification framework is
raised. Experiments on three benchmarking collections illustrate that our
proposed model has comparable effectiveness to a range of state-of-the-art
models, and further demonstrate that both syntactical information and
long-range word dependencies are properly captured by the graph convolution
structure.

negative:
Interaction between caregivers and children plays a critical role in human
language acquisition and development. Given this observation, it is remarkable
that explicit interaction plays little to no role in artificial language
modeling -- which also targets the acquisition of human language, yet by
artificial models. Moreover, an interactive approach to language modeling has
the potential to make language models substantially more versatile and to
considerably impact downstream applications. Motivated by these considerations,
we pioneer the space of interactive language modeling. As a first contribution
we present a road map in which we detail the steps that need to be taken
towards interactive language modeling. We then lead by example and take the
first steps on this road map, showing the initial feasibility of our approach.
As such, this work aims to be the start of a larger research agenda on
interactive language modeling.

negative:
Drawing causal conclusions from observational real-world data is a very much
desired but challenging task. In this paper we present mixed-method analyses to
investigate causal influences of publication trends and behavior on the
adoption, persistence, and retirement of certain research foci --
methodologies, materials, and tasks that are of interest to the computational
linguistics (CL) community. Our key findings highlight evidence of the
transition to rapidly emerging methodologies in the research community (e.g.,
adoption of bidirectional LSTMs influencing the retirement of LSTMs), the
persistent engagement with trending tasks and techniques (e.g., deep learning,
embeddings, generative, and language models), the effect of scientist location
from outside the US, e.g., China on propensity of researching languages beyond
English, and the potential impact of funding for large-scale research programs.
We anticipate this work to provide useful insights about publication trends and
behavior and raise the awareness about the potential for causal inference in
the computational linguistics and a broader scientific community.

negative:
The question of how an effective and efficient communication system can
emerge in a population of agents that need to solve a particular task attracts
more and more attention from researchers in many fields, including artificial
intelligence, linguistics and statistical physics. A common methodology for
studying this question consists of carrying out multi-agent experiments in
which a population of agents takes part in a series of scripted and
task-oriented communicative interactions, called 'language games'. While each
individual language game is typically played by two agents in the population, a
large series of games allows the population to converge on a shared
communication system. Setting up an experiment in which a rich system for
communicating about the real world emerges is a major enterprise, as it
requires a variety of software components for running multi-agent experiments,
for interacting with sensors and actuators, for conceptualising and
interpreting semantic structures, and for mapping between these semantic
structures and linguistic utterances. The aim of this paper is twofold. On the
one hand, it introduces a high-level robot interface that extends the Babel
software system, presenting for the first time a toolkit that provides flexible
modules for dealing with each subtask involved in running advanced grounded
language game experiments. On the other hand, it provides a practical guide to
using the toolkit for implementing such experiments, taking a grounded colour
naming game experiment as a didactic example.

negative:
Matrix syntax is a formal model of syntactic relations in language. The
purpose of this paper is to explain its mathematical foundations, for an
audience with some formal background. We make an axiomatic presentation,
motivating each axiom on linguistic and practical grounds. The resulting
mathematical structure resembles some aspects of quantum mechanics. Matrix
syntax allows us to describe a number of language phenomena that are otherwise
very difficult to explain, such as linguistic chains, and is arguably a more
economical theory of language than most of the theories proposed in the context
of the minimalist program in linguistics. In particular, sentences are
naturally modelled as vectors in a Hilbert space with a tensor product
structure, built from 2x2 matrices belonging to some specific group.

negative:
The development of intelligent machines is one of the biggest unsolved
challenges in computer science. In this paper, we propose some fundamental
properties these machines should have, focusing in particular on communication
and learning. We discuss a simple environment that could be used to
incrementally teach a machine the basics of natural-language-based
communication, as a prerequisite to more complex interaction with human users.
We also present some conjectures on the sort of algorithms the machine should
support in order to profitably learn from the environment.

negative:
This entry introduces the topic of machine learning and provides an overview
of its relevance for applied linguistics and language learning. The discussion
will focus on giving an introduction to the methods and applications of machine
learning in applied linguistics, and will provide references for further study.

negative:
Sorani Kurdish, also known as Central Kurdish, has a complex morphology,
particularly due to the patterns in which morphemes appear. Although several
aspects of Kurdish morphology have been studied, such as pronominal endoclitics
and Izafa constructions, Sorani Kurdish morphology has received trivial
attention in computational linguistics. Moreover, some morphemes, such as the
emphasis endoclitic =\^i\c{s}, and derivational morphemes have not been
previously studied. To tackle the complex morphology of Sorani, we provide a
thorough description of Sorani Kurdish morphological and morphophonological
constructions in a formal way such that they can be used as finite-state
transducers for morphological analysis and synthesis.

negative:
Although Natural Language Processing (NLP) is at the core of many tools young
people use in their everyday life, high school curricula (in Italy) do not
include any computational linguistics education. This lack of exposure makes
the use of such tools less responsible than it could be and makes choosing
computational linguistics as a university degree unlikely. To raise awareness,
curiosity, and longer-term interest in young people, we have developed an
interactive workshop designed to illustrate the basic principles of NLP and
computational linguistics to high school Italian students aged between 13 and
18 years. The workshop takes the form of a game in which participants play the
role of machines needing to solve some of the most common problems a computer
faces in understanding language: from voice recognition to Markov chains to
syntactic parsing. Participants are guided through the workshop with the help
of instructors, who present the activities and explain core concepts from
computational linguistics. The workshop was presented at numerous outlets in
Italy between 2019 and 2021, both face-to-face and online.

negative:
Word reordering is one of the most difficult aspects of statistical machine
translation (SMT), and an important factor of its quality and efficiency.
Despite the vast amount of research published to date, the interest of the
community in this problem has not decreased, and no single method appears to be
strongly dominant across language pairs. Instead, the choice of the optimal
approach for a new translation task still seems to be mostly driven by
empirical trials. To orientate the reader in this vast and complex research
area, we present a comprehensive survey of word reordering viewed as a
statistical modeling challenge and as a natural language phenomenon. The survey
describes in detail how word reordering is modeled within different
string-based and tree-based SMT frameworks and as a stand-alone task, including
systematic overviews of the literature in advanced reordering modeling. We then
question why some approaches are more successful than others in different
language pairs. We argue that, besides measuring the amount of reordering, it
is important to understand which kinds of reordering occur in a given language
pair. To this end, we conduct a qualitative analysis of word reordering
phenomena in a diverse sample of language pairs, based on a large collection of
linguistic knowledge. Empirical results in the SMT literature are shown to
support the hypothesis that a few linguistic facts can be very useful to
anticipate the reordering characteristics of a language pair and to select the
SMT framework that best suits them.

negative:
Neural network-based approaches have become the driven forces for Natural
Language Processing (NLP) tasks. Conventionally, there are two mainstream
neural architectures for NLP tasks: the recurrent neural network (RNN) and the
convolution neural network (ConvNet). RNNs are good at modeling long-term
dependencies over input texts, but preclude parallel computation. ConvNets do
not have memory capability and it has to model sequential data as un-ordered
features. Therefore, ConvNets fail to learn sequential dependencies over the
input texts, but it is able to carry out high-efficient parallel computation.
As each neural architecture, such as RNN and ConvNets, has its own pro and con,
integration of different architectures is assumed to be able to enrich the
semantic representation of texts, thus enhance the performance of NLP tasks.
However, few investigation explores the reconciliation of these seemingly
incompatible architectures. To address this issue, we propose a hybrid
architecture based on a novel hierarchical multi-granularity attention
mechanism, named Multi-granularity Attention-based Hybrid Neural Network
(MahNN). The attention mechanism is to assign different weights to different
parts of the input sequence to increase the computation efficiency and
performance of neural models. In MahNN, two types of attentions are introduced:
the syntactical attention and the semantical attention. The syntactical
attention computes the importance of the syntactic elements (such as words or
sentence) at the lower symbolic level and the semantical attention is used to
compute the importance of the embedded space dimension corresponding to the
upper latent semantics. We adopt the text classification as an exemplifying way
to illustrate the ability of MahNN to understand texts.

negative:
Natural language is one of the most fundamental features that distinguish
people from other living things and enable people to communicate each other.
Language is a tool that enables people to express their feelings and thoughts
and to transfers cultures through generations. Texts and audio are examples of
natural language in daily life. In the natural language, many words disappear
in time, on the other hand new words are derived. Therefore, while the process
of natural language processing (NLP) is complex even for human, it is difficult
to process in computer system. The area of linguistics examines how people use
language. NLP, which requires the collaboration of linguists and computer
scientists, plays an important role in human computer interaction. Studies in
NLP have increased with the use of artificial intelligence technologies in the
field of linguistics. With the deep learning methods which are one of the
artificial intelligence study areas, platforms close to natural language are
being developed. Developed platforms for language comprehension, machine
translation and part of speech (POS) tagging benefit from deep learning
methods. Recurrent Neural Network (RNN), one of the deep learning
architectures, is preferred for processing sequential data such as text or
audio data. In this study, Turkish POS tagging model has been proposed by using
Bidirectional Long-Short Term Memory (BLSTM) which is an RNN type. The proposed
POS tagging model is provided to natural language researchers with a platform
that allows them to perform and use their own analysis. In the development
phase of the platform developed by using BLSTM, the error rate of the POS
tagger has been reduced by taking feedback with expert opinion.

negative:
The goal of this project is to (i) accumulate annotated informal/formal
mathematical corpora suitable for training semi-automated translation between
informal and formal mathematics by statistical machine-translation methods,
(ii) to develop such methods oriented at the formalization task, and in
particular (iii) to combine such methods with learning-assisted automated
reasoning that will serve as a strong semantic component. We describe these
ideas, the initial set of corpora, and some initial experiments done over them.

negative:
Virtual Personal Assistants like Siri have great potential but such
developments hit the fundamental problem of how to make computational devices
that understand human speech. Natural language understanding is one of the more
disappointing failures of AI research and it seems there is something we
computer scientists don't get about the nature of language. Of course
philosophers and linguists think quite differently about language and this
paper describes how we have taken ideas from other disciplines and implemented
them. The background to the work is to take seriously the notion of language as
action and look at what people actually do with language using the techniques
of Conversation Analysis. The observation has been that human communication is
(behind the scenes) about the management of social relations as well as the
(foregrounded) passing of information. To claim this is one thing but to
implement it requires a mechanism. The mechanism described here is based on the
notion of language being intentional - we think intentionally, talk about them
and recognise them in others - and cooperative in that we are compelled to help
out. The way we are compelled points to a solution to the ever present problem
of keeping the human on topic. The approach has led to a recent success in
which we significantly improve user satisfaction independent of task
completion. Talk Markup Language (TalkML) is a draft alternative to VoiceXML
that, we propose, greatly simplifies the scripting of interaction by providing
default behaviours for no input and not recognised speech events.

negative:
Recent years have seen a number of proposals for performing Natural Language
Generation (NLG) based in large part on statistical techniques. Despite having
many attractive features, we argue that these existing approaches nonetheless
have some important drawbacks, sometimes because the approach in question is
not fully statistical (i.e., relies on a certain amount of handcrafting),
sometimes because the approach in question lacks transparency. Focussing on
some of the key NLG tasks (namely Content Selection, Lexical Choice, and
Linguistic Realisation), we propose a novel approach, called the Text
Reassembling approach to NLG (TRG), which approaches the ideal of a purely
statistical approach very closely, and which is at the same time highly
transparent. We evaluate the TRG approach and discuss how TRG may be extended
to deal with other NLG tasks, such as Document Structuring, and Aggregation. We
discuss the strengths and limitations of TRG, concluding that the method may
hold particular promise for domain experts who want to build an NLG system
despite having little expertise in linguistics and NLG.

negative:
With the explosive growth of online social media, the ancient problem of
information disorders interfering with news diffusion has surfaced with a
renewed intensity threatening our democracies, public health, and news outlets'
credibility. Therefore, thousands of scientific papers have been published in a
relatively short period, making researchers of different disciplines struggle
with an information overload problem. The aim of this survey is threefold: (1)
we present the results of a network-based analysis of the existing
multidisciplinary literature to support the search for relevant trends and
central publications; (2) we describe the main results and necessary background
to attack the problem under a computational perspective; (3) we review selected
contributions using network science as a unifying framework and computational
linguistics as the tool to make sense of the shared content. Despite scholars
working on computational linguistics and networks traditionally belong to
different scientific communities, we expect that those interested in the area
of fake news should be aware of crucial aspects of both disciplines.

negative:
In natural language processing, most models try to learn semantic
representations merely from texts. The learned representations encode the
distributional semantics but fail to connect to any knowledge about the
physical world. In contrast, humans learn language by grounding concepts in
perception and action and the brain encodes grounded semantics for cognition.
Inspired by this notion and recent work in vision-language learning, we design
a two-stream model for grounding language learning in vision. The model
includes a VGG-based visual stream and a Bert-based language stream. The two
streams merge into a joint representational space. Through cross-modal
contrastive learning, the model first learns to align visual and language
representations with the MS COCO dataset. The model further learns to retrieve
visual objects with language queries through a cross-modal attention module and
to infer the visual relations between the retrieved objects through a bilinear
operator with the Visual Genome dataset. After training, the language stream of
this model is a stand-alone language model capable of embedding concepts in a
visually grounded semantic space. This semantic space manifests principal
dimensions explainable with human intuition and neurobiological knowledge. Word
embeddings in this semantic space are predictive of human-defined norms of
semantic features and are segregated into perceptually distinctive clusters.
Furthermore, the visually grounded language model also enables compositional
language understanding based on visual knowledge and multimodal image search
with queries based on images, texts, or their combinations.

negative:
In vision and linguistics; the main input modalities are facial expressions,
speech patterns, and the words uttered. The issue with analysis of any one mode
of expression (Visual, Verbal or Vocal) is that lot of contextual information
can get lost. This asks researchers to inspect multiple modalities to get a
thorough understanding of the cross-modal dependencies and temporal context of
the situation to analyze the expression. This work attempts at preserving the
long-range dependencies within and across different modalities, which would be
bottle-necked by the use of recurrent networks and adds the concept of
delta-attention to focus on local differences per modality to capture the
idiosyncrasy of different people. We explore a cross-attention fusion technique
to get the global view of the emotion expressed through these
delta-self-attended modalities, in order to fuse all the local nuances and
global context together. The addition of attention is new to the multi-modal
fusion field and currently being scrutinized for on what stage the attention
mechanism should be used, this work achieves competitive accuracy for overall
and per-class classification which is close to the current state-of-the-art
with almost half number of parameters.

negative:
This paper outlines the results of sentence level linguistics based rules for
improving part-of-speech tagging. It is well known that the performance of
complex NLP systems is negatively affected if one of the preliminary stages is
less than perfect. Errors in the initial stages in the pipeline have a
snowballing effect on the pipeline's end performance. We have created a set of
linguistics based rules at the sentence level which adjust part-of-speech tags
from state-of-the-art taggers. Comparison with state-of-the-art taggers on
widely used benchmarks demonstrate significant improvements in tagging accuracy
and consequently in the quality and accuracy of NLP systems.

negative:
Linguistics holds unique characteristics of generality, stability, and
nationality, which will affect the formulation of extraction strategies and
should be incorporated into the relation extraction. Chinese open relation
extraction is not well-established, because of the complexity of Chinese
linguistics makes it harder to operate, and the methods for English are not
compatible with that for Chinese. The diversities between Chinese and English
linguistics are mainly reflected in morphology and syntax.

negative:
Ideas from forensic linguistics are now being used frequently in Natural
Language Processing (NLP), using machine learning techniques. While the role of
forensic linguistics was more benign earlier, it is now being used for purposes
which are questionable. Certain methods from forensic linguistics are employed,
without considering their scientific limitations and ethical concerns. While we
take the specific case of forensic linguistics as an example of such trends in
NLP and machine learning, the issue is a larger one and present in many other
scientific and data-driven domains. We suggest that such trends indicate that
some of the applied sciences are exceeding their legal and scientific briefs.
We highlight how carelessly implemented practices are serving to short-circuit
the due processes of law as well breach ethical codes.

negative:
This paper describes an application of reinforcement learning to the mention
detection task. We define a novel action-based formulation for the mention
detection task, in which a model can flexibly revise past labeling decisions by
grouping together tokens and assigning partial mention labels. We devise a
method to create mention-level episodes and we train a model by rewarding
correctly labeled complete mentions, irrespective of the inner structure
created. The model yields results which are on par with a competitive
supervised counterpart while being more flexible in terms of achieving targeted
behavior through reward modeling and generating internal mention structure,
especially on longer mentions.

negative:
Traditional learning-based coreference resolvers operate by training the
mention-pair model for determining whether two mentions are coreferent or not.
Though conceptually simple and easy to understand, the mention-pair model is
linguistically rather unappealing and lags far behind the heuristic-based
coreference models proposed in the pre-statistical NLP era in terms of
sophistication. Two independent lines of recent research have attempted to
improve the mention-pair model, one by acquiring the mention-ranking model to
rank preceding mentions for a given anaphor, and the other by training the
entity-mention model to determine whether a preceding cluster is coreferent
with a given mention. We propose a cluster-ranking approach to coreference
resolution, which combines the strengths of the mention-ranking model and the
entity-mention model, and is therefore theoretically more appealing than both
of these models. In addition, we seek to improve cluster rankers via two
extensions: (1) lexicalization and (2) incorporating knowledge of anaphoricity
by jointly modeling anaphoricity determination and coreference resolution.
Experimental results on the ACE data sets demonstrate the superior performance
of cluster rankers to competing approaches as well as the effectiveness of our
two extensions.

negative:
Neural sequence-to-sequence text-to-speech synthesis (TTS) can produce
high-quality speech directly from text or simple linguistic features such as
phonemes. Unlike traditional pipeline TTS, the neural sequence-to-sequence TTS
does not require manually annotated and complicated linguistic features such as
part-of-speech tags and syntactic structures for system training. However, it
must be carefully designed and well optimized so that it can implicitly extract
useful linguistic features from the input features. In this paper we
investigate under what conditions the neural sequence-to-sequence TTS can work
well in Japanese and English along with comparisons with deep neural network
(DNN) based pipeline TTS systems. Unlike past comparative studies, the pipeline
systems also use autoregressive probabilistic modeling and a neural vocoder. We
investigated systems from three aspects: a) model architecture, b) model
parameter size, and c) language. For the model architecture aspect, we adopt
modified Tacotron systems that we previously proposed and their variants using
an encoder from Tacotron or Tacotron2. For the model parameter size aspect, we
investigate two model parameter sizes. For the language aspect, we conduct
listening tests in both Japanese and English to see if our findings can be
generalized across languages. Our experiments suggest that a) a neural
sequence-to-sequence TTS system should have a sufficient number of model
parameters to produce high quality speech, b) it should also use a powerful
encoder when it takes characters as inputs, and c) the encoder still has a room
for improvement and needs to have an improved architecture to learn
supra-segmental features more appropriately.

negative:
We investigate neural techniques for end-to-end computational argumentation
mining (AM). We frame AM both as a token-based dependency parsing and as a
token-based sequence tagging problem, including a multi-task learning setup.
Contrary to models that operate on the argument component level, we find that
framing AM as dependency parsing leads to subpar performance results. In
contrast, less complex (local) tagging models based on BiLSTMs perform robustly
across classification scenarios, being able to catch long-range dependencies
inherent to the AM problem. Moreover, we find that jointly learning 'natural'
subtasks, in a multi-task learning setup, improves performance.

negative:
Named entity linking is to map an ambiguous mention in documents to an entity
in a knowledge base. The named entity linking is challenging, given the fact
that there are multiple candidate entities for a mention in a document. It is
difficult to link a mention when it appears multiple times in a document, since
there are conflicts by the contexts around the appearances of the mention. In
addition, it is difficult since the given training dataset is small due to the
reason that it is done manually to link a mention to its mapping entity. In the
literature, there are many reported studies among which the recent embedding
methods learn vectors of entities from the training dataset at document level.
To address these issues, we focus on how to link entity for mentions at a
sentence level, which reduces the noises introduced by different appearances of
the same mention in a document at the expense of insufficient information to be
used. We propose a new unified embedding method by maximizing the relationships
learned from knowledge graphs. We confirm the effectiveness of our method in
our experimental studies.