Yet, MT evaluation is a notoriously difficult topic due to the variability of the language that makes it impossible to anticipate one or more "correct" translations in advance.

a full MT development process often relies on a mix of automatic measures and human rating.


confA is the number of times a given n-gram was confirmed in sysA ("confirmed" means occurring both in sysA and in the reference for the same sentence).


We expect a metric to be able to give information about the seriousness of the errors given a task.

we are interested by generating a ranking of ASR errors according to different evaluation metrics in order to understand which metric allow a better identification of the most serious ASR errors for NER.


For a given phrase alignment, each pair of aligned phoneme symbols is converted into a single token by joining the symbols with symbol ':' as a delimiter.

the alignment of two sequences is converted into a sequence of these tokens.


As the length of both alignments is the same, only substitutions and matches are possible, and there are no deletions or insertions.

the Levenshtein distance is equal to the number of mismatches (substitutions) between these two sequences.


However, one would expect the increase of such mispronunciations in speech of non-native Russian speakers or in conversational speech where the mismatch rate between orthoepic transcription and real pronunciation is greater.

the 'V/S/C separation + /j/' setup is considered to be the best achieved solution.


The exact value of reduction is a matter of question.

a number of experiments were carried out to define the reduction cost that gives the best result.


For any given such class (say, segments or vowels) and language (say, English) in a given database and feature system (say, PP ), the system automatically selects the language's segments that belong to this class and computes their count.

for English, there are 40 seg-  What is the relationship between the two feature systems we are using, Phoible and Fonetikode?


The Jaccard index 8 of the actual segments in the classes averaged across languages varies between 0.00 and 1.00, with a mean of 0.92 and median 0.99.

for the vast majority of classes the two feature systems strongly agree producing very similar classes.


The Jaccard index of the actual segments in the classes averaged across languages varies between 0.24 and 1.00, with a mean of 0.77 and median 0.82.

the two databases tend to produce similar (but not identical) classes of segments for the languages shared between them.


For RF, the first three Princial Components (PCs) explain 39.6% of the variance (with 90% of the variance being explaind by the first 33 PCs): P C 1 (17.0%) expresss the agreement among all features, P C 2 (15.1%) distinguishes consonants from vowels, while P C 3 (7.5%) is hard to interpret.

in all three cases, the most important tendency (≈ 17% of the variance) is shared among all classes, followed by the rough distinction between vowels and consonants (≈ 13% of the variance).


The dialectometric analysis requires large databases in order to be adequately computed, and automatic segmentation and labelling can create them thanks to a procedure less costly than the manual alternative.

the integration of these tools into the SEA_AP allows to propose a distribution of geoprosodic areas by means of a quantitative method, which completes the traditional dialectological point of view.


This new approach distances itself from traditional dialectology, which used to establish a priori linguistic features that would then be used to determine the boundaries between different geographical varieties.

the tool automates repetitive and arduous tasks, enabling the researcher to focus its efforts in the analysis and interpretation of data.


The importance of studying this feature lies on intonation as it is crucial for communication, changing the intonation can completely change the meaning or even without intonation, it is impossible to understand the expressions.

this means that teaching prosody is very significant when acquiring a language different from the mother tongue.


Furthermore, the application could be used to perform more accurate prosodic division areas by including new data which will be processed more quickly by the tool.

it allows to group prosodic features into smaller areas and consequently it represents each one of the areas with greater accuracy.


Therefore, methods used to retrieve the melody in this case need to be robust to noise and inaccuracies in the singing or humming which is very challenging, and, for the system to be practical, the entire system should be very fast.

we propose a supervised machine learning based method for the query-by-humming system, which can learn the common errors associated with human humming and build a model that is unaffected by these errors.


On the other hand, related artists ought to influence each other in their musical style.

we are also extracting audio-based features to find related artists.


Note that each feature in the acoustic space has been normalized by its mean and variance.

the closer the distance is, the more similar the styles of the songs of two artists are.


For artist s i , we haveĈ i and C i , whereĈ i means calculated candidate artists set that is related to s i and C i means standard candidate artists set that is related to s i .

for each s j ∈Ĉ i , if s j ∈ C i , we can build a tuple with the flag "false".


One explanation could be the presence of noise in the speech.

95 speech events were noisy, in most cases due to the superposition of the noise made by the person during the fall to the speech.


Faster interfaces can reduce cost and increase transcriber motivation.

we analyzed how transcription time is influenced by the transcription interface, while accounting for segment length.


Before installation of the atHome system the number of devices that the user would like the homeService system to operate and the related command words were agreed.

each speaker in the database has a personalized list of commands suitable for their needs.


E.g., for M02, the duration is 4 s, but for F02, who usually introduces short pauses in her production, it had to be increased to 6 s. Participants use the system in their real home environment.

all sorts of natural house sounds are present in the recordings.


This may lead to disturbances in any of the components involved in the speech production, including respiratory, phonatory, resonatory, articulatory and prosodic elements.

this may be reflected by weakness, spasticity, incoordination, involuntary movements, or abnormal muscle tone, depending on the location of the neurological damage.


All the speech utterances were annotated by a human expert in order to identify acoustic anomalies at the phone level.

this corpus was involved in the estimate of the normal and abnormal models required in the classification step described in section 2.2.. Table 1 provides information on this corpus.


Unfortunately, the automated methods described previously don't allow to reach this necessary quality.

a web interface has been designed to manually validate the segments attributed to a given personality (see Figure 2).


We then allocate the sessions and segments in the same fashion as in (Fredouille and Charlet, 2014)), meaning that we set a minimum of 30 seconds and a maximum of 150 seconds for the training phase.

with all this constraints, the number of sessions to be added in the train base is determined as follow for a speaker i: duration seg k ≤ 150 sec.


The high agreement (kappa = 0.825) suggests that this task can be reliably performed by a single annotator.

the remaining 1,100 reviews were annotated by only one human.


Soundex is an algorithm that indexes words based on their phonetic properties.

words that sound similar have similar values.


The parser output was used to identify markables in the sentence.

an XML-like representation was created which preserved the syntactic structure of the markables (including nested markables, e.g.


Building upon the first release almost a decade ago, a considerable effort had been invested in improving the data both quantitatively and qualitatively.

we have doubled the corpus size, expanded the selection of covered phenomena to include referentiality and genericity and designed and implemented a methodology for enforcing the consistency of the manual annotation.


Since each window iteration overlaps with the previous one, any single token may have multiple field values assigned to it.

we consider each of these assignments as a "vote" and chose the majority vote as the final field assignment for the token.


In particular, we treated every structured region as being independent of all others, when in reality structuring conventions are often repeated within documents and across collections.

effort devoted to getting a single table right could be rewarded by accurate analysis across a whole collection.


The similarity of words can be computed by comparing their feature vectors.

we can predict whether two words are synonymous or similar with respect to some other semantic relation.


Since thesauri are also used for automatic indexing and for full-text retrieval, it is important to know all possible terms that refer to a certain concept.

extension of thesauri with more labels for each concept is an important task in the maintenance of these vocabularies.


After some preliminary experiments we found that including all words in the frequency range from 4 • 10 3 to 1 • 10 6 in the UkWaC Corpus is a good compromise between optimal results and acceptable storage and computing efforts.

context words which have frequency range from 4 • 10 3 to 1 • 10 6 in the UkWaC Corpus have considered to construct the context vector for each words.


Depending on the context and the speed of elocution, these inflections may correspond to a pause, which implies a silence and, normally, the end of the sentence, or a semi-pause, which implies an intonation change in the same sentence.

the combination of these types of inflection allows us to collect examples with different segmentation and observe whether people with Down syndrome recognize the difference between them.


We were interested in contrasting the behavior of these three type of classifiers as we observed in (Escudero-Mancebo et al., 2014) that they behave differently in prosodic labeling tasks.

mLP and DT offer the best results with a classification rate superior to 80%.


We are now annotating our data for all of these variables, in most cases by methods that are fully automatic (though human-checked) given an accurate time-aligned transcript.

future research, based on larger data sets as well as larger sets of behavioral variables, will offer a better basis for theoretical investigation as well as more effective methods for diagnostic classification.


Even for word families, where the typical usage of the majority of the members is not nouny, a noun can be derived.

definitions could exclusively use noun labels but still putatively cover all domains of language.


(Rijkhoff et al., 1993) include Pidgins and Creoles, but not explicitly historical or constructed languages.

we added Latin and Volapük for completeness.


The average noun percentage in the matched Wikipedia titles was highly significantly larger than in the Wiktionary according to a one sample t-test, t = 7.149, df = 16, p two−tailed < 0.001.

it appears that single-token titles or definition labels are even more probable to be nouns than loanwords are, cross-linguistically/universally.


Moreover, the existing resources often do not consider the possible contexts of words in assessing the associations between a word and a color.

the influence of context on word-color associations is not fully understood.


describe the same event; see figure 2, example 1), the temporal relations precedence and succession imply that the two propositions do not refer to the same event.

the first level of annotation, given two propositions, should be the decision, if the propositions refer to the same event.


For example, if the first event was annotated to cause the second then it might also hold that the first event precedes the second.

a chance-corrected agreement measure such as Krippendorff's α is less suitable for such data than the raw percentage agreement.


Inter-annotator agreement was comparable to related semantic annotation tasks that employ crowdsourcing.

we annotated a large set of 9856 propositions regarding well-formedness and used the wellformed propositions to generate 12403 proposition pairs for the annotation of context-sensitive semantic verb relations.


These statements (and in general, each step of the task) were formulated by the authors taking into account that the task was proposed to non expert annotators "in the crowd" with no preparation or instruction on opposition relations and whose background was not known by the authors.

in order for the task to be more intuitive, we avoid definitions, and provide simple statements with an example.


For instance, rather than laboratory conditions, data may very likely be captured by inbuilt device microphones and with unwanted background noise.

we view this as a data type that is ecologically valid and much needed for training of resources.


Even though listeners are known to excel at the task of native speaker versus non-native speaker distinction (McCullough and Clopper, 2016), we acknowledge that perception of nativelikeness remains a somewhat subjective judgement.

we were cautious in our assessment, and decided to ask for further judgements from CrowdFlower workers which we could refer to if in doubt (section 3.2).


CrowdFlower offers the facility to 'quiz' workers with pre-determined gold standard questions, but this approach does not suit our task (as T2,T3,T4 are to some degree subjective).

as CrowdFlower imposes no delay on payment, there was less facility for quality control and approval/rejection with these jobs.


Transcription quality from CrowdFlower workers has been somewhat problematic: we have rejected six-in-ten of the 3300 transcriptions received so far on the basis of their poor quality.

we have two good quality transcriptions for 1000 of our Crowdee recordings, and this is the dataset we release first.


We presented a dataset which consists of aligned images of realworld spatial scenes with accompanied referring expressions of specific objects in the images.

the dataset will be useful for research in the fields of referring expression generation, as well as language and vision.


However, there are considerable differences in the environment between crowd-sourcing workers and the traditional laboratory subjects.

trusting the work of the participant is only one of the many differences between crowdsourcing and laboratory environments.


Since we do not have a crowdsourced-based rating for each file coming from the same subject, the EWE is not applicable.

we established the Weighted Trustability Evaluator (WTE), based on the weights of the ratings derived from the trustability of the rater.


We first evaluated the gender task which is trivial, but still it is worth mentioning as an initial proof-of-concept.

we compared performances achieved using the labels obtained from the laboratory and crowdsourcing annotators.


On the contrary, online computer-mediated communication restricts the use of our communication modality, including non-verbal cues for emotional communication.

conveying our emotion in computer-mediated communication is a more effortful task than in face-to-face communication.


pleasantness under- standing in the online situation is equivalent to face-to-face communication.


The lack of physical co-presence leads to less emotional expression of bodily contact in the video chat environment.

the video chat environment obtained less emotional understanding than the faceto-face environment.


This result indicates that, pleasantness is easier to express with those alternative cues than other emotions in computer-mediated communication.

it was easy for the speaker to understand the partner's emotion.


On the other hand, arousal and dominance were considered to be difficult to express with words and emoticons.

it was not understandable for the speakers.


Indeed, conversational speech refers to an informal activity, in which participants have constantly to manage and negotiate turntaking, topic changes (among other things) "on line" without any preparation.

numerous phenomena appear such as hesitations, repeats, backchannels, etc.


Indeed, the data recorded for a certain emotional expression in a naturalistic database will be different from one another.

although being adequate for recognition and classification tasks (if the data quantity is enough with respect to the task) this kind of data would most probably not be suitable to create parametric models such as Hidden Markov Models (HMM) (Rabiner and Juang, 1986) or Deep Neural Networks (DNN) (Hinton and Osindero, 2006;Anderson, 1995) with the goal of generating descriptors.


Having three levels of each affect burst, will allow us to not only attempt to synthesize them but also to obtain intermediate levels via interpolation.

we intend, as a first step, to use the database presented here to model the AV affect bursts present in it.


This data was stored in an AVI file and recorded also at 100 fps.

11 cameras were used to collect the motion capture data and 1 was used to record the grayscale video.


These could be used in further study for audio affect bursts synthesis.

before the segmentation, session files of both modalities had to be synchronized since they were recorded on different machines.


These conversations are ideal as they are written to show common daily interaction, depicting relatable situation and emotion.

the occurring emotion is contextualized and less stereotypical.


In fact the emotions JOY, DISGUST SADNESS and SUR-PRISE reach a higher F1-score in the full model.

a future optimization might try to combine different approaches for the various emotions.


We use the words aligned in topic models and put them in a sentence to create parallel sentences for the training corpora to be used in creating the MT system.

the pseudo-parallel data generated in this case consists of 1 sentence per topic: e i1 e i2 e i3 in parallel with h i1 h i2 h i3 .


They may not appear in the same order, since these are dependent on the frequency of the word in the specific language.

our model is able to discover coarse topics underlying the datasets.


There had already been Japanese-English parallel abstracts of various scientific fields, and the Japanese-English parallel corpus was constructed by extracting the parallel sentences from the abstract pairs.

the ASPEC-JE contains parallel sentences of all the scientific fields.


The translation units which do not have another unit belonging to the same paper in the translated data are extracted.

there is no sentence pairs sharing the same paper across the training, development, development-test and test sets.


Plamada and Volk (2012) already demonstrated the difficulty in using Wikipedia categories for the extraction of domain-specific articles from Wikipedia.

in order to select articles from a given domain, we have taken advantage from the previous work carried out by Barrón-Cedeño et al.


Note that the gluegrammar (Chiang, 2005) is applied when the decoder is incapable of deriving a sound parse tree, which in our case is not an option.

the use of the glue-grammar is also prevented (i.e.


(ii) if the author does not know the target language, ProphetMT will also facilitate post-editing in the following way: Firstly, the source-and target-side parse trees will be provided to the post-editor, and the source-side parse tree can be used as a 'golden standard', since it is human-parsed.

the post-editor does not need to modify the source side.


More interestingly, it is claimed that the relationship between vector spaces that represent different language word semantics can be captured by a linear transformation.

we propose to use word embedding vectors of translation word pairs to train a supervised classifier which can predict whether a new pair of words in two different languages can be under a translation relation, according to the regularities learned from a small training set.


By contrast, other aligners such as the Microsoft Bilingual Aligner (Moore, 2002), Hunalign (Varga et al., 2005) and Gargantua (Braune and Fraser, 2010) are autonomous and automatically infer a word-based translation lexicon from the texts being aligned as they proceed.

the performance of these methods degrades when the texts to be aligned are short as there is less data to support statistical inference of a bilingual lexicon.


the range of applications of our proposal is quite large, and covers the dissemination uses of MT rather than the assimilation ones, as defined by Hovy et al.


In Statistical MT (SMT), phrases are simply chains of words that frequently cooccur and are aligned with the same source word sequences.

we experimented with three segmentation strategies: S1: Phrases from edit distance metric Our first insight in terms of segmentation was to mimic as much as possible annotators' behaviour by producing a monolingual alignment between the raw machine translation and its post-edited version.


Different from the two strategies described next, this strategy is guided by the word-level labels, rather than the types of errors, and as a side effect it produces particularly long phrases, especially for the "OK" sequences.

even though a phrase may be correctly tagged as "BAD", we lose the information on actual error boundaries in cases of multiple translation errors which happen to be consecutive, but are independent from each other.


On this basis, our motivation is to make use of linguistic information to determine the sentence segmentation.

our phrases are extracted from the shallow syntactic structure of the sentence (Constant et al., 2011), the so-called chunks, based on TreeTagger (Schmid, 1994).


So a phrase can contain words with both "BAD" and "OK" labels, whereas we need a single label for the entire phrase.

in cases of ambiguous labelling we use one of three heuristics to define a phrase-level label: • optimistic -if half or more of words have a label "OK", the phrase has the label "OK" (majority labelling).


We decided to focus on annotating "BAD" phrases only because it is much harder to define guidelines for the segmentation of correct translations into phrases.

we would have made the task very hard for humans and very prone to disagreements on segmentations of both "OK" and "BAD" phrases.


In this scenario Precision will be inevitably low.

we evaluate segmentation in terms of Recall for the "BAD" phrases.


This is not surprising since this strategy is based on the original post-editions.

this segmentation is close to the one produced by our annotators.


Part of this gap could be explained by the fact that post-editors who produced the initial post-editions in (Potet et al., 2012) had the access only to the source sentences and their automatic translations, whereas for our experiments we gave the annotators the access to all the available data: source sentences, automatic translations, post-editions and reference translations, so they could decide on the optimal labelling from a range of possibilities including existing corrections and their own knowledge.

where our phrase labelling would consider as "BAD" a phrase which has been modified in the post-edited version of the translation, a human annotator might consider the meaning as unchanged and therefore would not label this modification as an MT error.


Although human error annotation and evaluation are very time-consuming tasks, we have shown in Section 4. that this kind of data can provide interesting insights on the nature of human and machine translation in general, and subtitling in particular.

this resource is a valuable contribution for automatic error detection and MT systems developers, who can benefit from this freely available resource.


Currently, Solr 5.3 supports nearly 20 implementations of tokenization and over 100 text normalization and filtering methods 21 , covering a wide range of languages and use cases.

by selecting different components and assembling them in different orders, one can create different analyzers to achieve different extraction and normalization results.


Frequencybased methods are not effective for such terms as (1) nested term candidates will have at least the same and often higher frequency, and (2) the fact that a longer string appears n times is a lot more important than that of a shorter string appearing n times.

cValue computes a score that is based on the frequency of a candidate and its length, then adjusted by the frequency of longer candidates that contain it.


In synaesthesia, a perceptual experience in a given sensory modality is described through lexemes pertaining to another sensory modality.

our first experiment consists in finding all the sentences that contain at least two lexemes from different lists, i.e., two lexemes pertaining to two different sensory modalities (cf.


As for "V N / N V", the noun is usually the direct object of the verb: The data extracted therefore confirm what has often been observed in the literature, namely that in conventional synaesthesia the perceptual lexemes are typically in a dependency relation, which is attributive in most cases (but see also the verb-object relation mentioned above).

a way to improve the extraction method in  Table 3.


• In addition, we were interested in the effect of ambiguity (of both the modifiers and the heads) regarding the compositionality of the compounds.

we created a gold standard with a focus on two-part noun-noun compounds including • compounds and constituents from various frequency ranges; • compounds and constituents from various productivity ranges; • compounds and constituents with various numbers of senses; and • compounds with various semantic relations.


Indeed, criterion C1.3 below states that CPRE and CCONJ can be replaced by single-word PRE and CONJ.

we cannot apply it if we do not have a clear definition for these two categories.


One of the challenges in building DeQue was the fact that de and que combine with several complements, including open-class words like nouns, verbs and adverbs.

it is impossible to guarantee that our lexicon is exhaustive.


The first step models the ambiguity of each construction in theory.

it is possible to know whether a construction is potentially ambiguous and requires some special treatment.


When interpreted as a CPRE or CCONJ, the whole MWE acts as a single preposition or conjunction.

we argue that de and que should be the syntactic heads.


Compounding in particular can be seen as a way of introducing new words into the lexicon (Gagné and Spalding, 2006), and as being at the interface between morphology and syntax.

to understand and produce compounds children need to learn to combine information at different levels of linguistic description.


This means that to describe a person who pushes wagons the compound would change for example from wagon boy to push wagon, pusher wagon and finally to wagon pusher during acquisition.

compounding is an important device for children to expand their vocabularies, and several factors play a role such as productivity, transparency and simplicity (Clark, 1993), with the relation between the frequency of compounds in child-directed (CD) and in their acquisition and use in child-produced (CP) sentences in need of further investigation (Berman, 2011).


Such collocations are by their nature idiosyncratic and therefore also language-specific.

you make [a] suggestion, but you give [an] advice, you attend [a] lecture, but you assist [an] operation, rain is heavy while wind is strong.


In other words, a phrase chunk, e.g., a NP, a Verbal Phrase (VP), a Prepositional Phrase (PP), or other chunk defined by the grammar in CoreNLP, may be built from different types of PoS tags.

again from Figure 2, we see that the sentence S97 is built both from a NP and a VP chunk.


proposed the idea that certain parts of the lexicon of human languages are universal, stable over time, and rather resistant to borrowing.

he claimed that this part of the lexicon, which was later called basic vocabulary, would be very useful to address the problem of subgrouping in historical linguistics: [...] it is a well known fact that certain types of morphemes are relatively stable.


A syntactic frame is mapped to the set of thematic roles realising its syntactic arguments.

the syntax-semantics linking is made explicit, but the lexical meaning of the frame is presumed to be a composition of the meaning of the semantic arguments.


in the case of phraseological or idiomatic expressions).

the two additional properties :frameSense and :isFrameSenseOf (in analogy to lemon's :sense and :isSenseOf) were added to PDEVlemon in order to link frames with lexical senses.


More precisely, LVF does not give a traditional description of the lexical sense of the verb reading but rather the sense is suggested by semantic description(s) associated with these readings.

for a LLOD semantic representation of LVF entries we need a conceptual representation of the elements used in LVF for this semantic characterisation.


There is also, albeit not systematically, information referring to thematic roles.

it is for example indicated wether a complement or adjunct has a manner, temporal or instrumental role.


Nevertheless, the actual construction process is manual -a supervised team of lexicographers verifies automatic hints in lexicographic resources and only then introduces them into a database.

it belongs in with the so called merge approach (cf.


The linguistic patterns in sar-graphs are automatically acquired via a pattern discovery method based on distant supervision (Mintz et al., 2009;Krause et al., 2012).

sar-graphs can be directly applied to free texts for relation extraction.


Our investigations showed that 42% of the verbs in MMO are listed in multiple classes of VN.

in addition to the VN frames, the VN classes corresponding to MMO frames also had to be disambiguated.


A significant outcome of the recent editions in Reykjavik (2014) and Beijing (2015) has been to facilitate the publishing of LLOD resources and vocabularies, applications and use cases.

10 papers and at least 3 posters, including this year's edition of LDL in co-location with LREC 2016, have contributed to the community's topics of interest.


The conducted evaluation showed a different performance of the interlinking method when tested on the resources represented by generic terms (a concept label is usually a common noun or a term in a thesaurus).

it seems that it is more difficult to interlink concepts of a thesauri rather than resources corresponding to named entities.


Additionally, such resources are not widely applicable as they do not include a wide range of food data (a few dozen − hundreds of thousands of recipes) or any meal data (data on likely combinations of recipes).

a novel resource is required to promote food research.


These have been successfully applied in machine translation and synonym detection.

we have included experimental support for word2vec models (Mikolov et al., 2013), trained on the Estonian Reference Corpus 3 .


Most storage solutions provide only efficient keyword search for text whereas operations in ESTNLTK require efficient search over layers.

we have included a Database class that uses NoSQL database Elastic Search 6 to store and retrieve Text objects.


In addition, the computer hardware enables this processing, basic analysis software is available, and advanced analysis techniques, inter alia natural language processing tools and applications, often yield sufficient quality to use them.

the so-called Digital Turn creates huge opportunities for the Humanities: it can broaden the empirical basis for its research, since digital techniques can analyze data in quantities that humans never could cover.


In comparison, DC has only 15 categories, and the fine-grained MARC 21 standard has about 200 data fields, each of which can have many subfields.

the crosswalks between CMDIbased schemas and the bibliographic standards will often involve many-to-one and many-to-none mappings.


In this manner, we pave the way to easily plug together NLP pipelines without the need to worry much about UIMA technicalities.

the goal of making JCORE 2.0 tools public was, and still is, not only to provide a comfortable setting for our own work, but, first and foremost, to offer the scientific community easy to follow workflows to partake of results of cutting-edge research conducted in our lab.


lemmatization and morphological tagging.

the text processing is much easier and faster with SynKorp while retaining the present quality.


Moreover, in the EU databases are protected against unauthorized extraction and re-utilization of their contents.

proper public licensing plays a crucial role in providing access to research data.


However, the choice of a proper license is an uneasy (and often neglected) task.

tools like licentia.inria.fr or ELRA License Wizard 1 have been created.


In fact, in the EU the copyright protection of software is regulated not by the Copyright Directive (2001/29/EC), but by a special Software Directive (2009/24/EC) whose first version (adopted as early as 1991) predates the Copyright Directive.

the restricted acts, the statutory exceptions and the rules on authorship of software may differ from those concerning other categories of works.


All these particularities have to be taken into account in the licensing process.

data licenses (like Creative Commons) differ substantially from software licenses (like GNU GPL).


While a detailed comparison of all the above mentioned infrastructures would be desirable, it would fall outside the scope of the paper.

we discuss a conceptually similar infrastructure, namely German CLARIN-D infrastructure.


Also, there were open sources of Turkish vocabulary available on the web in an accessible electronic form, but that was not the case for Hausa.

we relied on various existing reference works for Hausa (Newman, 2000;Newman, 2007;Bargery, 1934).


The current end-user interface integrates a morphological web service, an extension of an open-source morpholog-ical analyzer for Latvian (Paikens et al., 2013), as a way of generating inflection tables for the lexical entries.

it also supports the inclusion of the Tezaurs wordlist (or a subset of it) as a lexicon for POS and morphological tagging and for lemmatization.


In most cases, there is a one to one mapping between graphemes and phonemes in Latvian.

the source dictionaries typically do not include information about the pronunciation of headwords, except in rare cases.


Although the coverage is greater than 80%, the lexicon is still small.

some of the random surface forms selected for evaluation had unanalysed output.


But, this is not applicable in the second singular feminine form where one would expect owNtR based on the above rule plus 2SF pronoun r, but instead one gets owNte.

one must be watchful for these irregularities and make sure to enter them into the lexicon so that the analyzer can still make sense of the data.


In more detail, the algorithm that assigns a rating score requires labelled training data.

we first collected several affective ratings for German.


Although most of such words are already present in LV as resulting from the annotation of the treebanks, 163 of them were not found yet in the texts.

we built in intuition-based fashion those that we consider to be the prototypical frame entries for these words, by filling only the frame attributes and not also the frame slots, which have to be assigned by confrontation with textual evidence.


These are the most frequent functors assigned (as frame attributes) to the third frame slot of valency-3 frame entries (the first two being ACT and PAT).

for instance, the nodes clustered around the node for ADDR are those for valency-3 frame entries whose frame attributes are ACT, PAT and ADDR (like attribuo -"to assign").


The LV frame entry of the lemma dico ("to say") linked to this occurrence reflects the active form of the clause: "[they] say wise those who arrange things correctly".

the frame entry for this occurrence of dico has a valency frame that includes three frame slots, and their corresponding frame attributes, as follows: (1) a generic Actor (ACT); (2) a Patient (PAT) expressed by a pronoun: "those (who arrange things correctly)"; (3) an Effect (EFF: functor assigned to obligatory predicative complements): "wise".


Figure 1 illustrates the node-wise alignment task, where one concept in the source LSR is chosen as a query (henceforth, a query concept), and the task is to discover a set of related concept nodes (including synonymous concepts) in the target LSR.

as in ordinal information retrieval tasks, the problem is to achieve a higher precision while ƐŽƵƌĐĞ >^Z ;ZͿ ƚĂƌŐĞƚ >^Z ;WtEͿ keeping a level of recall.


While extending Walenty with the semantic layer, we had semantic parsing and textual entailment in mind.

unified representation of utterances bearing similar information is crucial.


Exemplary sentences are originally assigned to syntactic schemata, but they are connected to lexical units as well.

they also illustrate semantic frames connected to LUs.


an archaic or wrong schema can be connected to a certain frame, if that schema represents an archaic/incorrect use of a syntax for some meaning.


The FrameNet roles are too granulated and their number is too large, whereas the Vallex roles are bound too closely with syntax.


We want to be much more precise, hence we use PLWORDNET synsets (represented by LUs) and relations to represent selectional preferences.

only dogs can BARK, we DRINK only beverages (not all liquids), and we only use bandages to BANDAGE (not every cloth).


Classic selectional preferences tell us nothing about what can be used by those objects for milling, but we can clearly see that they have to have to be internal parts of original object.

we introduced selectional preferences determined by means of relations to another argument.


It may concern concrete objects, e.g., various devices, groups of concrete people, events having their phases, etc.

we consider a symbol RELAT meaning any close PLWORDNET relation between LUs occurring in text.


Contrary, in (8b), TRAWA appears in instrumental, whereas ZBOCZE appears in nominative (an accusative object is impossible here; furthermore, there is no way to include information about Attribute).

the same phrase type may be used in text in two different ways.


For example, the class of UCCIDERE ("to kill") verbs was associated with the frame of KILLING, in which "a Killer or Cause causes the death of the Victim with an Instrument".

we provide the following definition of verb class: verb classes are sets distinct senses of verbs that share common argument realization patterns (i.e., they share the same constructions) and that profile the same template of event (i.e., they evoke the same conceptual frame).


: administer, contribute, disburse, distribute, etc).

levin describes this subclass as lacking the Dative alternation, because these "latinate" verbs do not admit the Double Object construction (levin, 1993:139).it is not surprising that a satisfactory Italian counterpart for CONTRIBUTE verbs cannot be found.


In psycholinguistics, specific research questions are studied and some of them can be related to those studied in linguistics: • Knowledge of components within complex words and their decomposition, which main purpose is to study how the complex words are processed and recorded in brain and then to make hypotheses on how these words are decoded and produced by speakers.

it has been demonstrated that several factors can facilitate reading and production of complex words: hyphenation (Bertram et al., 2011), space character (Frisson et al., 2008), presence of other morphologically related words (Lüttmann et al., 2011), use of primes (Bozic et al., 2007;Beyersmann et al., 2012), of pictures (Dohmes et al., 2004;Koester and Schiller, 2011) and of supportive contexts (Cain et al., 2009); • Order of components and variety of morphological patterns, which focus on the position of the components.


Word sense embeddings used in our experiments are on the countrary coarsegrained with at most five senses per word (this is tunable by the α parameter).

the corpus-based model cannot learn a fine-grained polysemic inventory featuring tens of senses.


Yet, both these senses are strongly related.

a major problem with recall is caused by sparseness of the bag-of-words representation.


Unfortunately, these semantic resources are developed independently and they are not fully integrated in a common framework.

a common semantic infraestructure would allow the interoperability among all these lexicons.


However, in the new multilingual Predicate Matrix this Spanish synset also have associated the Spanish word senses parpadear and guiñar.

the integration of additional senses for Spanish, Catalan and Basque also produces as a side effect the enrishment of the Spanish, Catalan and Basque wordnets integrated into the MCR.


Now, as PropBank is integrated into the Predicate Matrix, our SRL module can also obtain the corresponding predicate classes and roles for the rest of the predicate resources.

steve Jobs identified as A0 role of the nominal predicate speach.01 corresponds to the Communicator role of a Communication frame according to FrameNet.


Thus, Steve Jobs identified as A0 role of the nominal predicate speach.01 corresponds to the Communicator role of a Communication frame according to FrameNet.

thanks to the Predicate Matrix, predicates and roles from PropBank appear also aligned to the rest of resources.


This also applies across languages thanks to the multilingual Predicate Matrix.

when processing the Spanish sentence by our SRL module, we obtain the same languageindependent semantic representation as the one obtained from the English sentence.


a common semantic infraestructure would allow the interoperability among all these resources.


For example, it is difficult to think of a single word that would describe a scale containing big and small but not tall or wide.

we chose prompt words that could possibly be on the scales from lists of synonym and antonyms as provided by dictionaries and thesauruses as a proxy to naming the scale.


Note that we framed the problem as a multi-label task, and considered two annotations to be identical if and only if their sets of assigned labels are identical.

the inter-annotator agreement estimate is probably somewhat pessimistic.


This dictionary include synonyms aligned with each word sense, which means that it is possible to extract, from the same dictionary entry, different synonyms for different senses.

when creating sets of synonyms for each sense we obtain something very similar to the synsets of WordNet.


As PULO is aligned with the other languages' wordnets available in the MCR through their InterLingual Index (ILI), it allows the exploitation of these wordnets for our experiments.

our idea is based on the assumption that translating DLPC entries will allow the application of the approach described in the previous section, computing intersections between the translated DLPC sets and other languages synsets (in our experiment, Galician, Spanish and English).


These can vary wildly in terms of size, class distribution, and various linguistic properties.

the collected data is likely to be imbalanced and to suffer from high sample selection bias, as it is not representative of the whole population (Zadrozny, 2004;Bareinboim, Tian, & Pearl, 2014).


The effects of different stemmers on classifier accuracies are far from the same, yet they all shrink the vocabulary to a similar extent.

these differences must be caused by the quality of the stemming rules used in each algorithm.


Conveniently, a three-fold cross-validation on the English MR 1.0 dataset yields a similar number of tokens for training as a five-fold cross-validation on the polar portion of SerbMR (661K vs 633K).

we average 20 runs of five-fold cross-validation on SerbMR-2C and consider two types of settings.


However, as words in ANTUSD were collected from various sources, not all of them were included in E-HowNet.

through the integration with E-HowNet, E-HowNet can provide both the sentiment information and the lexical semantic information for 12,995 Chinese words, which cover 47.74 percentage of ANTUSD.


However, the annotators labeled the sentiment words they found at the time they read the candidate sentence, but we cannot guarantee all annotators will label the same words in one sentence.

all labels of words were collected from these two corpora, and as the word labels may depend on the content of the current sentences, these labels were context-dependent.


Words labeled as NEU (neural words) were also dropped since there are only 16 such words.

we only use words with POS, NEG, NONOP labels in our experiments.


Processing different Arabic dialects is crucial for understanding what is going on social media and most online platforms.

it is useless to develop systems which are only MSA-based, unless the unique purpose is to process news, some blogs or forums content.


", the shifter word "not" before the positive word "like" changes the text polarity to negative.

ignoring sentiment shifters can lead to noticeable decline in overall accuracy of opinion mining systems.


In the above examples, the valence shifters "eliminated", "reduced", "no" and "less" invert the polarity of the corresponding polar terms.

capturing such shifters will improve the performance of polarity classification.


Sometimes different parts of a compound sentence have different polarities.

to achieve more accurate shifter patterns, compound sentences were broken down into simple units.


Table 1: Examples of shifter patterns In addition, as can be seen in Table 1, most of the extracted patterns are not domain-specific.

although the shifter identification method was tested on drug review domain, the extracted patterns can be used in any other domain as well.


However, in order to have a more general pattern set, we can extract shifter patterns from several domains for those polarity-tagged corpora are available.

as future work, we would like to address the issues of extracting general patterns (i.e., domain-independent patterns) for shifter identification and provide an efficient method to alleviate these issues.


If the occurrences of BCs are not appropriate in terms of timing, people receive negative impression.

we created a model in order to determine relevant BC placement.


Since users will not have had   previous exposure to the person represented by the system, they may find it difficult to glean information regarding the characters if the calculated probability is used as is.

we exaggerated the difference between source speakers.


The candidate closest to the last matched term is more likely to be the right answer.

we impose a window for short-distance searching.


Automatic evaluation of Machine Translation (MT) is based on the idea that the closer the MT output is to a human reference translation, the higher its quality.

the task is typically approached by measuring some kind of similarity between the MT (also called candidate translation) and a reference translation.


In this work we propose to use local context to discriminate between acceptable and non-acceptable differences.

variation between the MT and a human translation can be considered meaning-preserving if they contain semantically similar words and the words occur in syntactically equivalent contexts.


A number of successful approaches, therefore, combine different types information.

giménez and Màrquez (2010) propose a combination of specialized similarity measures operating at various linguistic levels (lexical, syntactic and semantic).


lexically similar) but play different roles in the sentences and therefore should contribute less to the sentence-level evaluation score.

for each pair of aligned words, the words that constitute their syntactic contexts are compared.


changes the context to a greater extent than dropping a determiner or an adjunct.

context words are assigned different weights depending on the relative importance of their syntactic functions.


We use the natural logarithm of the weighted count of context words, since this impact saturates above a threshold.

a context difference receives a higher value when the number of context words is high (it is not the same translating zero words out of one and zero words out of ten), while limiting the increase if the number of context words continues to grow (the difference between translating six words out of eight and eight words out of ten is less relevant).


A representative sample of the language as a whole shows the opposite tendency-that is, a tendency towards infinite growth, or what one might call "openness," on an analogy to open-class words.

the same functionality in SubCAT that indicates the extent of a corpus's fit to the sublanguage model indicates equally well the opposite-its representativeness.


Additionally, this study improves on previous works by using actor model, which enables us to take advantage of all available cores by highly parallelizing the corpus building process.

languageCrawl scales well with the increase in the number of worker nodes.


After creating the N-gram model, duplicated N-grams have been removed.

data set size has been reduced by approximately 99.4%.


Apart from learnability problems concerning some cQLs, the real problem is that a number of linguistic resources are accessible through drastically different corpus management systems which provide their own cQLs.

mastering one or two cQLs is not enough to benefit from the entire palette of existing corpus linguistic software packages.


Dependency tree representations of sentences allow us to utilize the syntactic dependency relations among words.

it is a more powerful approach than the bag of words representation for modeling the syntactic and semantic information in sentences.


So the recall value is: In order to get the values for the "bad requirement" perspective, we inverse the perspective of "good requirement".

for "bad requirement" the true positive value is 94, there are 2 false positives and 0 false negatives.


The disadvantage of manually created lexica is that they have low language coverage, since they contain only a few thousand words.

computational methods are used to create or expand an already existing lexicon.


These opinions become important resources for those who need information about products, as well as manufacturers who wish to improve their productivity (Ahn et al., 2012).

the demand for efficient automatic extraction of opinions from the web is growing day by day.


On the one hand, the document-and the sentence-level sentiment classifications are based on the assumption that each document or each sentence expresses only one definite opinion on a single target (Turney, 2002;Liu, 2012).

these methods of analysis are not applicable to documents or sentences which evaluate more than one entity (see Section 3.1.).


This method of analysis is based on the assumption that each document expresses only one definite opinion on a single target.

the document-level approach is not applicable to documents which evaluate more than one entity.


The differentiation of entities and their aspects is an important task of sentiment analysis, with regard to the fact that in many applications, opinion targets are described by entities and/or their different aspects (Liu, 2012).

the goal of this level of analysis is to produce a structured summary of opinions about entities and their aspects.


We regarded an expression as a sentiment fragment if it contained only one polarity item connected to only one target (or more than one target, coordinated to each other) (Szabó et al., 2016).

most of the fragments of the corpus annotated are whole sentences and phrases.


From the point of view of computational linguistics, the automatic detection and handling of sentiments in texts encounter difficulties since the syntactic, semantic and pragmatic realization of opinions is far from uniform.

sentiment analysis poses challenges for computer processing.


Unlike in (+/-Effect)WordNet, each example sentence comes with a syntactic subcategorization frame which lists the set of arguments and adjuncts occurring with the lemma being exemplified.

in the example sentence in Figure 1,übergeben 'turn sth over to sb' is realized with a noun phrase in nominative case (NN), another in dative case (DN), and a third in accusative case (AN).


However, in the case of the accidental finding, nobody need have acted volitionally.

although finding and stealing both have a possession entailment, the former has no causally responsible participant.


On the other hand, in The minister forces the president to cheat both are factual.

if the factuality of the subclause is licenced by the matrix verb, then the algorithm proceeds to determine the status of the subclause.


Moreover, users manually tagged the post according to the polarity and nature of the comment, by using one of the following predefined categories: 'what I like', 'what I don't like', 'what is missing', and 'new integration'.

we can consider such texts as already annotated both for what concerns the polarity of the opinion on the reform, and for what concerns the finer-grained information about the specific topics (or aspects of the reform) that were addressed in their comments.


The smartphone reviews were annotated in close collaboration with the SemEval 2016 shared task organizers.

the Dutch guidelines form an integral part of the ABSA 2016 guidelines 3 .


But it only labels a part of the nodes in the constituency tree.

it cannot clearly represent the relation between clauses or the relation between various arguments (e.g., the semantic relation in phrase "everyday [Modifier] life [Head word]" or between clauses "I come back [Result], because of the rain)[Reason]" are not represented ).


Besides, it is also found that the work of those early stages (e.g., syntactic tree and word alignment annotation) notably influences our work.

the annotation quality management should be set up in each step of this kind of project.


In addition to this, it is worth noting that many websites are available in several languages and this translated content is another useful source of parallel data.

a considerable scientific effort has been put during the last years in order to exploit the web as a source to automatically acquire new parallel data (see Section 2).


criteria for evaluating the resources directly, whereas the extrinsic ones are connected to the system's function.

in order to do an intrinsic evaluation, it should suffice to manually evaluate the accuracy of a random sample of the corpora obtained.


Unfortunately, Croatian was not one of these languages.

in order to build a suitable test set, the first 1, 000 segments were manually translated into Croatian by a native speaker.


The number of tweets about programme differs a lot between the different episodes.

the correlation of the episodes of a single programme is low in general (table 3), or even negative for two drama (police) seriesthe latter may be due to special episodes such as cliffhanger episodes or season finales, which draw roughly the same viewers as other episodes, but trigger more reactions on Twitter.


Understanding what tweets are saying is getting more dependent on understanding hashtags.

identifying the individual words that constitute a hashtag is an important, yet a challenging task due to the abrupt nature of the language used in tweets.


for 10-CV evaluation), training/testing splits were defined with respect to the overall and per class proportions of tweets indicated in Table 4 (%All column).

10-CV training and testing splits comprised 986 and 109 tweets per run respectively.


Variants (1) and (2) yield clusters of average size 6.6 and 6.0, respectively.

the first two variants may be considered fine-grained clusters, while the third one is more coarse-grained.


From a machine translation (MT) perspective, the deeper the processing of utterances, the less language-specific differences will remain between the representations of the meaning of the source and target texts.

chances of success are expected to increase considerably by MT systems that are based on deeper semantic engineering approaches.


The results show a pattern which has also been shown in other results reported with other evaluation sets (Raghunathan et al., 2010), namely, the fact that a large part of the performance of the multi-pass sieve system is based on a set of significant sieves.

this module so far focuses on a subset of sieves only, namely, Speaker Match, Exact Match, Precise Constructs, Strict Head Match and Pronoun Match (Lee et al., 2013).


These clausal arguments can be successfully analyzed with existing roles, most often Theme.

we obtain the same role set for 'He knew the answer' and 'He knew he was right'.


Even though we provide preannotated targets, the annotators must be able to verifyand if needed, correct -the proposed targets.

the introductory part of the guidelines declares the theoretical background and technical instructions of the annotation, including special cases, e.g.


For instance, the query B-52 can relate to a bomber, a hairstyle, a cocktail or a rock band.

search engines strive to diversify their output and to present results that are related to as many query senses as possible.


Vice versa, results belonging to different senses can overlap lexically (by chance or because of some frequent words).

in order to successfully cluster search results, the semantic relatedness or similarity of the words should be taken into account: the algorithm must 'understand' which words are semantically close to each other, even if their surface forms differ.


It allows to tune all the important settings so that the difference in performance could be attributed only to the difference in algorithms themselves, not to some peculiarities in preprocessing.

both types of models used symmetric window of 1 (only immediate left and right neighbors of words were considered) and no downsampling.


In the KBK-1M dataset there is no guarantee the captions will refer to anything depicted in the image.

it is not possible to directly learn literal description models.


This suggests that there is at least one image with at least one pair of identical descriptions; we have manually verified this fact.

meteor is the only metric covered in this paper that can capture such cases.


Although some fingerspelled signs are in frequent usage, fingerspelling is often used for proper names and spoken language borrowings.

there is no fixed fingerspelled vocabulary: many fingerspelled productions would not be included in any ASL dictionary.


Hausdorff distance is not necessarily symmetric.

there is a difference between the nearest neighbors of the given frame and the frames that would consider the given frame as their nearest neighbor.


These utterances consisted of 3,085 relevant signs (42,947 relevant frames plus 21,535 frames of movement epenthesis or sign classes not considered here).

our test and training datasets were significantly different in temporal structure.


Based on the previous work of Teufel and Liakata, our annotation aim is to detect characteristic features of the scientific discourse that may appear at any point in a research paper.

the annotation scheme includes the following 7 categories: Advantage and Disadvantage not only limited to the author's approach but also to any reference to an advantage or disadvantage in the documents of our corpus.


The annotation process is characterized by its collaborative approach between the developers of the methodology, experts in annotation and text mining, and the 12 annotators who are experts in the domain of Computer Graphics.

a web-based collaborative annotation tool (Annote) was developed, enabling users to easily annotate textual contents by exploiting a web browser (see Section 6.).


We focus on evaluating algorithms that produce a segmentation for each word in the corpus, e.g., (Bernhard, 2008), Morfessor Categories-MAP (Creutz and Lagus, 2007), ParaMor (Monson et al., 2009), RALI-ANA (Lavallée and Langlais, 2010), MetaMorph (Tchoukalov et al., 2010), Morfessor Baseline , and others.

we assume that if we concatenate the segmentations, the result is identical to the input word.


Additionally, the ambiguities in words do not overlap, and each boundary in the gold-standard words can be marked with a maximum of one label.

we can maximize the sum of true positives and true negatives separately for each dilemma, and solve the problem by finding the preferred theory for one dilemma at a time.


An example of such an alignment is shown in Figure 2.

the overall schema of the evaluation algorithm can be summarized as follows: tek.e.m.i.s+i+s+sä tek e+m i s i+s sä Figure 2: An example of a gold standard to response alignment.


For our work, we focus on neoclassical compounds and prefixed words, and compounds such as hidden compounds which are at the border between native and neoclassical compounds.

the morphemes under consideration for the distributional analysis are neoclassical elements and prefixes, including elements that are not purely neoclassical elements but look like them, such as the element radio in radiology.


Since our extraction method is deterministic, all lemmas with the same signature will be parsed in the same way.

it is sufficient for a human to verify and correct a single lemma with a particular signature to determine that all lemmas with that same signature are correctly extracted.


Although the language may vary even for similar tasks, it is possible to use clustering techniques on top of semantic relatedness to group related tasks together (see Table 4).

new speech input can be associated with one of the learned virtual apps.


In a negotiation situation two or more parties have interests in reaching one or several possible agreements, but their preferences over these agreements are not completely identical (Raiffa et al., 2002).

negotiators may have partially competitive and partially cooperative goals.


They also may revise their past offers, accept or decline any standing offer, make counter-offers, etc.

there is always a latent risk that bargaining may breakdown.


Information about a Negotiation Move allows the system to interpret partners and to generate adequate communicative behaviour, to interpret partners negotiation strategies, and to take correct decisions in negotiation.

we propose to have an additional set of acts which will be assigned to negotiators actions by extending the ISO 24617-2 tag set with negotiation moves as Task domain-related communicative functions.


Similarly, defining their inverses F −1 MultiTab and F −1 TabSW , shows their unambiguity.

the composition of functions such as MultiTab defines a conversion from annotations, represented in the DiAML-MultiTab format to representations in the DiAML-XML format.


students in a psychological research methods course are often encouraged or required to be research participants).

participants in lab-based behavioral experiments often experience a task that is already somewhat routine.


The relative paucity of studies that report on the differences between friends and strangers may be due to the fact that the dialogues are quite short (an average of about 10 minutes) and very focused on a straightforward task.

they may be less likely to reflect the characteristics that distinguish the conversations of friends and strangers.


Directors used fewer Descriptors in Round 2 (M = 4.17, SD = 1.77) than in Round 1 (M = 12.23, SD = 3.75), t(47) = 14.06, p<.0001, 95% CI [6.91, 9.21], d = 2.03 (Liu et al., under review).

on this measure, we replicated effects found in the laboratory "in the wild": Participant pairs became more efficient at referring to their targets in Round 2 than they had been in Round 1.


Our annotation schema should be able to cope with all feedback phenomena that appear in different interactional environments, including face-to-face interaction.

we chose material from corpora that involve different recording situations and have high quality audio recordings and videos.


Moreover, aside couple exceptions, it is always the same two raters that are excluded.

what we call 'best-pair kappa' is actually simply the removal from the data set of the annotation of the worse two raters (which is a relatively standard practice).


However, both are composed of texts written by young adults.

it is impossible to achieve significant and realistic studies regarding German L2 learning by children of elementary school age on a small or a large scale.


However, the form was frequently incomplete since it was filled in paper format and no strong requirement was applied.

we established a set of 7 fields that were required for the productions of the informant to be included in COPLE2: name, age, nationality, gender, mother tongue, knowledge of other second or foreign languages, period of time studying Portuguese.


The authors consider a MWE to be a grammatically complete sequence of words that suffer some process of lexicalization.

the sequence shows some lexical, syntactic and semantic cohesion (even though they don't necessarily have an idiomatic meaning).


GG speakers are used to read in German some of them privileged a fast reading in order to finish quickly others chose to read aloud carefully in order to interpret the stories.

in reading, speech rate varies a lot in GG speakers what leads to varying segment durations.


On the other hand, there is a visible difference between the percentage of spelling errors of distance 1 and 2.

we perform another chi-square test with the following null hypothesis: the number of spelling errors with distance 1 and distance 2, and the clinical status of subjects are independent.


Indeed, their proposals seem to fail to improve linguists' per-sonal data management environments.

in order to make the language resources in personal environments publically available, to be used in language processing research, we suggest the need to seek a way to establish a consistent data management model from entering data through personal data management to storing it in public archives.


However, it is also true that our proposals do not seem to be perfectly in tune with our purposes of supporting personal data management activities that ensures long-life data, or diachronic data sharing, because any ID-resolution system needed for a proposal is difficult to be regarded as a lifelong service.

we decided to reconstruct our research approaches from scratch.


As an example, consider the following url (from ALT-Web) to display the data with respect to the semantic field of 'Agriculture': http://www.gabmap.nl/~alt/bin/accountALT?action=ALT &username=_tmpALT&description=Agriculture&kmlurl =http://www.gabmap.nl/~alt/resources/Tuscany-all.kml& dataurl=http://dbtvm1.ilc.cnr.it/proveALT/MAT/Matrix_f on_agr_ALL.txt The third change involved the extension of the Gabmap application by allowing direct (url) access to the distribution maps.

when the user supplies both the question item and the variant as url parameters, the corresponding distribution map is shown visualizing the relative frequency of use of the variant throughout Tuscany.


According to the Ethnologue (Lewis et al., 2015), the three dialects to the South of the Danube were developed between the 5th and the 10th century, while according to Rosetti (1966), this process took place after the 10th century.

according to Rosetti (1966), Aromanian and Megleno-Romanian developed in the 11th century, while Istro-Romanian developed in the 13th century.


However, this one-to-one assumption is too strong to suit languages which have the one-to-many translation characteristic.

we aim at extending the constraint-based bilingual lexicon induction to support any other closely related languages.


Languages are classified into low-resource and highresource languages based on their number/amount of resources, including size of dictionaries.

we identify characteristics of language that affect the performance and further identify the best constraint set for different language characteristics.


When we connect words in non-pivot language A and C via pivot words B, we can get translation results from language A to C. In this example w A 1 is symmetrical with w C 1 , w C 2 , and 1 − w C 3 are correct translations.


When we use Ω 3 , the recall remains high as we remove edges while the form of the transgraphs is maintained.

f-score remains highest and most stable as we remove edges when we use Ω 3 .


As a result, F-score remains highest and most stable as we remove edges when we use Ω 3 .

when only small input dictionaries are available, as in the case of lowresource languages, by using Ω 3 , our framework can infer and add the new edges based on our semantic constraint assumption.


Because both function and city name are given, that person's identity could easily be revealed.

it is necessary to consider even shortened institutional names as candidates for anonymization.


However, some experiments showed that completely blackening that timespan invalidates the whole sentence for further linguistic analysis as suprasegmental signals are disturbed.

we defined several options how to manipulate a stretch of video sufficient to make the sign or mouthing component unrecognizable: • In the case of mouthing, only the mouth including cheeks and the chin is to be hidden.


In the long run, we not only want to publish front view camera perspectives, but profile views as well.

we need to make sure that the blackening can be applied to corresponding regions in the different perspectives.


We observe that the type-equivalent configuration yields much better Precision with only a slight tradeoff in terms of Recall.

we decided to select for the final ELMD dataset only those URIs stemming from a typeequivalent setting where agreement score is equal or greater to 1.


Following the official Wikipedia guidelines on article style and writing 1 , the first sentence usually tries to put the article in a larger context.

wikipedia abstracts can be understood as "Is-A"-corpus, defining an entity in relation to a broader class of related entities or an overarching topic.


To our knowledge, no tool exists that implements all templates and LUA scrips used in Wiki markup, making Mediawiki the only tool available to produce high-quality text from Wiki markup.

a local Mediawiki instance as mirror of the Wikipedia was installed and configured as first part of the extraction pipeline, tasked with rendering the Wiki markup to plain text.


For example, kbd elements were not skipped, because they are used to style the text they contain as if it was a keyboard letter.

10 they contain important content easily representable as text.


For each link, another resource was created, containing the surface form in the nif:anchorOf property, its position in the string of the referenced nif:Context resource, as well as the URL of the linked resource via the property itsrdf:taIdentRef.

the identity of the link's surface form and the linked resource is made explicit, leading to the disambiguation of the link text by URL.


the link annotations) equals the part of the respective abstract text indicated by begin and end offset of said string.

the correctness of link positions and surface forms have been tested and verified for each link.


Besides, western and eastern European languages are quite well covered while African, Semitic and Asian languages have less annotated material.

if the degree of multilingualism of NE-annotated corpora is not catastrophic, it still remains an issue as many languages are not covered at all.


However, it is worth noting that the single tokens which form complex mentions (i.e., "Lennon" in "John Lennon"; "Saramago" in "José Saramago") may occur in other mentions belonging to different entites ("Alfred Lennon PER ", "Fundação José Saramago ORG ").

this should be taken into account when developing both NER and CR systems.


As shown in Section 4., the heuristics presented in this paper partially rely on the semantic labels predicted by the NER.

some of the errors produced by the system derive from some frequent NER mislabelings.


However, due to the license restriction of redistribution of the RTE 4 corpus, we only can release the annotation with two tab-separated columns pair ID, relatedness score.

users need to acquire the original RTE-4 corpus by themselves.


However adding this complexity comes with a price on accuracy.

is better to have a more robust way to ensure that the structural analysis does not deviate due to errors.


In order to classify the speeches we repurpose a set of features previously used for gender and affect detection.

we establish a first baseline for ICNALE on both speech and text and provide a comparative analysis between these two distinct aspects of interlanguage.


In this subsection we address this question.

we make use of the Pearson correlation as a measure of similarity between the vectors corresponding to each native language in each row.


ASF is endemic in sub-Saharan Africa and Sardinia (island Figure 2: Method for identification of terms for monitoring the web in Italy); however, in 2007 ASF appeared for the first time in the Caucasus region in Europe and in the following years infected several eastern European countries, including in 2014 several Baltic countries.

there is a risk of spread of ASF towards other western and southern European countries (Sánchez-Vizcaíno et al., 2013).


We believe that both clinical IR research innovation and legitimate privacy concerns can be served by the creation of intra-institutional, fully protected resources.

we have begun creating such resources at two sites: Oregon Health & Science University (OHSU) and Mayo Clinic.


Since a test collection is a collection of patients, it represents a patient population.

in patient-level IR, the choice of which patient records to include is of potential medical and public health significance.


Furthermore, chart reviews require specialized medical knowledge to be meaningful, and patient privacy concerns prevent the sharing of documents from a patient's EHR.

in our work, crowd-sourcing relevance judgments was not an option; rather, we have employed intra-institutional medical experts to take on the costly step of chart review to make patient-level relevance judgments.


One nice property of this annotation schema is that the structure for an event is guaranteed to be a tree because each tuple field has at most one parent and there are no loops due to the total order between fields.

we can treat the event detection task as a dependency parsing problem, as discussed in Section 4.2.


Nowadays, attaining communicative competence in the target language is considered the main goal of language pedagogy and an effective communicative setting is considered key to a successful acquisition process (Richards & Rodgers, 2001;Savignon, 2006).

the ideal learning environment should provide a setting and materials whereby the learner can communicatei.e.


However, for the majority of schools, which are English-medium schools, Irish is simply taught as one of a number of school subjects.

despite a small number of native speakers, there are large numbers of learners and a diversity of school-types, and the State is committed to delivering a nationwide programme of Irish language education (Rialtas na hÉireann, 2010).


The timeline for the audio is then used to annotate the beginning timestamps for the gestures: as shown in Figures 9 and 10, in front of every gesture, a time is shown inside a pair of square brackets to indicate the beginning time of this gesture stroke.

any change in the wording of the dialogic telling requires regenerating the audio and relabelling the gesture placement.


Research has shown that people prefer gestures occurring earlier than the accompanying speech (Wang and Neff, 2013).

in this annotation, a gesture stroke is positioned 0.2 seconds before the beginning of the gesture's following word.


during the action tasks in the rest of data acquisition scenarios.

informants were instructed to incorporate the predefined audio and gestural commands also within scenarios, while performing the action tasks foreseen.


For example, for the DBpedia property spouse the English label spouse is given, but variants such as wife of, husband of or to marry are not covered.

in order to make resources of conceptual information such as DBpedia accessible to language technology systems, usually further external resources of linguis-  tic knowledge are necessary.


A particular challenge when trying to crowdsource ontology lexicons is finding a task design that is suitable for workers with no knowledge about ontologies or lexical resources: Obviously, it would not make much sense to simply present the workers with an ontology element and then asking them to come up with a good verbalization, or even a whole lexicon entry.

we instead turn the lexicon generation task into a translation task.


For example, to write cannot only be used to verbalize the relationship holding between an author and a book, but may link authors to all kinds of written work, and there may be target languages in which these different kinds of relationships are verbalized in different ways.

we need to ensure that the English verbalizations are understood in the right sense, which can be accomplished by presenting them to the workers embedded in some kind of context.


be done through test sentences for which we already know in advance which translations are correct and which are not.

if in the evaluation stage we received the information that the sentence provided by worker number one is a good translation, while the other two are not, we could be rather confident that this information is correct.


However, as on Mechanical Turk only workers who are based either in the US or India can be paid in cash 4 , well over 90 percent of workers come from either of these countries 5 , which would make finding a sufficient amount of workers with a native language other than English or an Indian language rather difficult.

mechanicalTurk was not a good option for us.


As for a task such as ours one cannot really formulate test questions, which form CrowdFlower's main mechanism of quality control, we made use of a number of alternative control mechanisms that are commonly used with translationrelated tasks (Irvine and Klementiev, 2010; Zaidan and Callison-Burch, 2011): Generally, a good strategy for discouraging people from cheating is to design one's task in a way that makes cheating as laborious and time-consuming as actually working on the job at hand.

we showed the English seed sentences -and, in case of the later evaluation stage, the Japanese candidate translations -to the workers in the form of images rather than text, so as to make it more difficult for people to make use of automatic translation services.


A possible explanation would be that many people may cheat on CrowdFlower's language proficiency tests and pass them even though they do not speak the respective language at all: As mentioned before, the English sentences are presented to the workers as images, so for someone with a sufficient knowledge of both Japanese and English simply translating a sentence themselves should actually be less work than presumably entering it into some automated translation system by hand, or even looking up every single word on its own.

we did a second run of the translation task, which only differed from the first one in that now only workers from Japan were allowed to work on it.


concerning the appropriate amount of payment during the translation stage or the effect the language of instruction has, would require more data to answer with confidence.


We additionally annotated coreference chains between NPs.

the process resulted in three layers of annotation: event types, participant types and coreference annotation.


While more than a third of the turkers mentioned the event of fastening the seat belts in the plane (BUCKLE SEAT BELT), no person wrote about undoing their seat belts again, although in reality both events appear equally often.

we added an event type label for buckling up, but no label for undoing the seat belts.


However, the difference in the variation of lexical variance of scenarios is larger for DeScript than for InScript.

the properties of a scenario apparently influence the lexical variance of the event descriptions more than the variance of the narrative texts.


Informative seeds are expected to have the strongest corrective effect on the clustering algorithm.

the event descriptions to be aligned should be the borderline cases that are the most difficult for the algorithm.


And this in turn cre-ates another confound-deception is a mental process impossible for other humans to directly observe so that corpus creators are forced to rely on judgments of cooperative subjects.

of the difficulty in obtaining labeled data, the data used in previous studies is susceptible to bias arising from various sources.


Literature in deception detection demonstrates that humans perform at chance or even below chance when trying to detect deception.

in a valid corpus of texts it should be difficult for humans to distinguish truthful from deceptive documents.


Overall, STACC provides the best balance between the number of alignments it can identify and the precision in identifying correct alignments.

this method proved to be the optimal choice to retrieve the largest amount of precise alignments in the EITB corpus.


One of the most challenging issues for current state-ofthe-art word aligners is that they show poor generalization capability and are prone to errors when infrequent or unknown words (with respect to the training data) occur in new sentence pairs to be aligned (Farajian et al., 2014).

wA research would highly benefit from gold standard data specifically tailored to assess wA systems on this is-sue.


Still, XPath querying enhances the query flexibility compared to the example-based approach.

another approach of querying the corpora in GrETEL consists of directly formulating an XPath query describing the syntactic pattern the user is looking for.


The purpose of NorGramBank is to capture the variety of grammatical constructions that characterize the language, and their distribution in texts.

providing the best possible analyses of grammatical sentences or sentence parts takes priority over providing ad hoc analyses of ungrammatical or marginally grammatical sentences.


The dominance operator may be labelled, which is relevant in f-structure search, where the f-structure attributes can be seen as labelling the dominance relation between f-structures.

the expression #x >(SUBJ PRED) 'jeg' will find all sentences whose f-structures at some level contain an attribute path <SUBJ PRED> whose value is 'jeg' (cp.


Another argument against the use of prepositions for denoting semantic relations is that in our framework, lexicon, syntax and semantics are considered autonomous levels of representation/analysis of the language.

the use of lexical items for the typification of the more universal semantic types should be avoided.


Although it uses Lucene as well, it applies different indexing strategies.

it can be a good candidate for performance comparison in the future and may serve as an alternative search backend.


The frequency per million words (fpmw) measure did not yield a normal distribution of frequency counts; the distribution was highly skewed.

a log10 transformation was applied to the fpmw values which is generally regarded to be adequate to standardize the frequency counts.


The ideal way for this would be to use spoken language corpora of adults and children but this was not possible due to scarcity of available resources on Turkish.

the CLC was created to represent children's language.


In addition, a better understanding of such procedures has also been recognised as a bottleneck for machine translation (Helmreich et al., 2004).

modelling how humans translate may result not only in a theory of translation better capable of describing, explaining, and predicting translational phenomena but also in a better understanding of what can be done to improve machine translation.


By separating the notion of 1-gram from that of word spelling, we were able to consider any sequence of morphemes a separate grammatical word according to our needs.

we could treat both the 1-gram Papierballs 'paper ball' and the 2-gram Balls Papier 'ball of paper' as a sequence of two grammatical words.


The final object is to provide automatic links between each reference and its article or book in OpenEdition site 2 , which is composed of three different sub-platforms, Revues.org, Hypotheses.org and Calenda.

the automatic recognition of references' zone is essential as a first step.


On the testing level, we are not capable of detecting long patterns such as bibliographical references' zones using Unitex 3.0.

we suggest the use of machine learning technique for the annotation of references, so we can treat each reference apart and not a large amount of data at once.


Additionally, some documents were too long, which would probably make takers give up on answering the questionnaires.

we removed parts of such documents that were not important for answering the questions and did not affect document coherence.


In a second scenario (Scenario 2), the order of the MT systems was different: Mixed, Systran, oracle, Moses, Bing and Google.

our experiments include two scenarios, varying the order of the MT systems that translated the documents.


Human translation errors only partially overlap with errors made by MT and display greater variability, which makes human translations less predictable, see section 4.2 our pilot experiment for reference.

this non-uniformity of the creative human choices and translations further complicates the task of automated estimation of human translation quality, including automated scoring of trainee translators' work, and so far this area remains under-researched.


Secondly, MT output is often below what is acceptable, for example, for post-editing tasks, while human translations usually have reasonable quality, at least acceptable for post-editing.

tQA distinguishes between translations of usable quality that would in any case require only a minimum amount of post-editing, while Mt evaluation aims at distinguishing low-quality from reasonable quality Mt output (Babych & Hartley, 2008).


Further work is need to predict the content quality of human translations in a more stable manner.

we will investigate more semantic-related features that approximate sentential and inter-sentential structures for transmission of meanings between the source and the target languages at different levels.


In addition, models can be evaluated within the toolkit.

its pipelines allow users to go from raw training data to trained and evaluated models with minimal configuration.


Each stage in the pipeline for an experiment is specified in the configuration file as the path to a Python class or function.

the framework is agnostic about the actual functions that it calls, allowing the user to create a custom pipeline without writing any code, simply by specifying which parsers, feature extractors, and learning method(s) they wish to use.


In addition, language-dependent resources such as stemmers, the lists of stopwords, lemmatizers, etc, may also be required to improve the quality of a generated summary.

analyzing pre-processing settings for Urdu, which is indeed morphologically rich, is an important research question that this paper tries to answer.


• Some studies stress the use of customized stopword lists that should be extracted from the domain corpus of the task in hand (Lo et al., 2005;Blanchard, 2007).

the third list is built by calculating the term frequency on the documents of Urdu summary Corpus (Humayoun et al., 2016).


However, it is worth note that the resources (stopwords lists, lemmatizer, and stemmer) are built on space segmented words.

it may be inferred that if the resources are built on properly-segmented words, we might have better results for PS summary corpus.


In Experiment 1, we observed that the pre-processing setting S3 has outperformed 7 .

in this experiment, first, we remove the stopwords using this customized stopwords list (of the preprocessing setting S3), and then, apply the stemming approaches of Experiment 2.


In experiment 2, the last preprocessing setting was S25.

the preprocessing settings starts from S26 in this experiment.


It is observed that incorrect stemming done by the Assas-Band stemmer and limited coverage of the Humayoun's lemmatizer undermines the results.


These methods may rely on different kinds of information: sequential patterns, distributional vectors, but also on the semantic properties of the entities, as extracted from the ontology.

entities representing concepts were annotated based on available resources.


Consequently, specialized resources are usually less available and more restricted in size: this is why there is currently significant research aiming to enrich ontologies with concept and relation candidates from corpora (Petasis et al., 2011).

we experimented with combining domain-specific and generic resources to achieve a satisfying balance between annotation density and precision.


The experiments confirmed that data-driven term extraction methods can be efficiently combined with large-coverage external resources, and allow to expand coverage while maintaining the same level of precision that is achieved using domain-specific resources.

the entire corpus was automatically annotated for domain concepts.


Using the standalone version, linguistic preprocessing is performed directly.

a simple solution to develop a HeidelTime GATE wrapper would have been to build a black box wrapper that calls HeidelTime with all its preprocessing capabilities (sentence splitting, tokenization, part-of-speech tagging).


Neither replacements of linguistic preprocessing components nor the use of gold standard annotations for sentence, token, and partof-speech information would be possible.

a solution allowing for flexible linguistic preprocessing is required.


Direct assessment is appealing for its simplicity, but it raises the question of the adequacy of lexicons used as references.

several studies have proposed indirect assessments through a task requiring the generated thesauri.


Other tasks, such as lexical substitution, more focused on exact synonyms, might give different results.

an interesting perspective would be to measure the correlation between the intrinsic and extrinsic evaluation scores in different tasks and scenarios to better help choose the most suitable distributional method for a given task.


Below, in the Light ERE framework, a title must resolve to a specific position to be taggable.

a chief on the English side was considered too generic to support a title mention, whereas on the Chinese side 处长 chúchàng always denotes a specific position and so was tagged.


Mental disorders are one of the main contributors to the contemporary burden of disease and one of the major groups of disorders that cause disability (Murray and Lopez, 2013).

it is important for research in the medical domain to advance in this field.


The subdictionaries contain an average of µ = 31.8 words or stems.

in theory, the extent to which a news article uses the words in each sub-dictionary captures its author's concern for each moral foundation.


In practice coreference was problematic, in query responses comprised complex predications and were very rarely (if ever) truly coreferential.

coreference of citations was eliminated in Phase 2 and beyond.


Contrastively, the "B" branch of the decision tree did not present assessors with Question 4, since the only object of assessment in this branch of the decision tree is the English citation.

regardless of whether the assessor responded "Yes" or "No" to Q3B, they moved directly to Q5 and indicated whether any assessments were made generously, and then moved to the next citation (without being presented with the tier 4/Q4 question).


For example, does synset sunday.n.01-first day of the week have a predominant temporal class?

in addition to the four classes proposed by (Dias et al., 2014), we introduce another class as 'neutral'.


Such synset would be marked as 'neutral' in our annotation scheme.

one might consider our 'neutral' class to encompass past, present and future class.


So, the value of information or its quality is intrinsically time-dependent.

a new research field called Temporal Information Retrieval (T-IR) has emerged and deals with all classical IR tasks such as crawling (Kulkarni et al., 2011), indexing (Anand et al., 2012) or ranking (Kanhabua et al., 2011) from the viewpoint of time.


the concepts would have been C0007635 ("Cultured cells") and C0445623 ("Microorganism"), which will present difficulty generating query to find positive microbiology test results in the end-goal application.

c2242979 ("Microbial culture") appears to be a much better mapping for this question.


They gain familiarity among the members of a society as they store their culture, conceptual thinking and accumulated experiences (Emrich, 1972).

proverbs have high communicative and crosscultural value and they can be an essential tool for gaining insight into the way a language group experiences and conceptualises the world (Wilson, 2009;Moreno, 2010).


As also indicated by the Conceptual Metaphor Theory (Lakoff and Johnson, 1980), the mapping across domains stands at the core of the metaphorical connection and this kind of connection is very commonly established in proverbs as linguistic metaphors.

metaphors can resolve a significant amount of the figurative meaning in many proverbial utterances (Faycel, 2012).


As already mentioned before, the English proverbs were annotated before the ones in Italian.

the annotators had a better understanding of the task during the annotation of the Italian proverbs.


As it is obvious from the examples (3) and (4), the syntactic realizations of the both sentences are the same (||subject, prepositional object (to)||) but the assigned numbered arguments are different and so there are semantically two different verbs.

the semantic analysis of the sentences is essential to achieve propositional content.


These changes in the approach of the verb or complex predicate definition resulted in an increase in the number of the verbs from more than 4500 to more than 9200 unique verbs while the sentences in the syntactic corpus did not change.

the mentioned differences of the two approaches are represented below by the means of dependency graphs: Sentence (6) shows the syntactic approach in which to the word "" promise the label object with OBJ abbreviation has been assigned and the clause "beavad" with the label 1 Accusative case marker clause of noun (NCL) is the dependent of "".in the syntactic approach,""is not a complex predicate and it is a combination of an object and a simple verb.


The proposed method is evaluated for POS tagging tasks).

as several layers of linguistic structure must be investigated in order to answer such queries, we aim to establish a framework for uniformly representing semantic structure involving the roles of entities described in technical documents, represented across those layers.


In addition, because computers and computational methods can be applied to a wide and ever-widening range of topics, formalizing the frameset for events in the computer science/technology domain is an extremely difficult task.

instead of precisely defining a frameset for in-domain events, we describe the roles of entities in the form of their mutual relations using a set of general relationships such as method-purpose, system-output, and evaluationresult.


Their focus is on determining how things such as systems and data are developed and used.

in the annotated corpora used for establishing the systems for these purposes, things described in a document are labeled and classified according to their role in a certain context, such as application domain, method, and product.


As such information can be an answer to different search requests, and there could be yet other information to be searched, we believe annotating the relationship described in the sentences as completely as possible would be useful for a variety of purposes.

we have decided to annotate relations other than the ones for roles, including discourse-oriented relations such as cause-result and ontological relations such as hypernym-hyponym.


The contribution of semantic features in finding TECHNIQUE is related to the nature of TECHNIQUE and DOMAIN: A technology can be a DOMAIN when another technology is applied to achieve it, and can be a TECHNIQUE when it is applied to achieve another technology.

they can be distinguished only in relation to other entities mentioned in the text.


News content is diachronic in nature and is characterised by different topics which change with time, leading to a change in the linguistic content and vocabulary.

a typical problem faced by Large Vocabulary Continuous Speech Recognition (LVCSR) systems processing diachronic audio news is Out-Of-Vocabulary (OOV) words.


I-vector-based DNN-HMM system takes the I-vector features plus GMM-SAT features as DNN input for I-vector based training and decoding as shown in Figure 1 in red boxes.

the input feature dimension is expanded by 100 tuned in the experiments for a trade-off RTF, particularly in our on-line STT system.


By definition, it is the language used in all the Institutions of the government like administrations and all administrative papers, courts, schools, etc.


However, since these results were obtained on balanced datasets, they do not reflect real situations, where hyper-articulation is rare.

we also performed experiments on highly unbalanced versions of the datasets, using the same classifiers.


Identifying the dialog acts is important for spoken dialogue systems, since they reveal the intention of the speaker.

an inconsistent sequence of dialog acts is a cue for the presence of communication problems in the dialog.