Competence errors are attributable to a lack of knowledge in the language, while performance errors are due to external factors, such as lack of attention, stress or fatigue (Corder, 1967).

researchers have highlighted the fact that even though this distinction is theoretically relevant, it is practically impossible to distinguish competence errors from performance errors (Thouësny, 2011).


In the case of missing articles, we notice that results in the two comparison corpora are similar, and are much lower than in the requirement corpus.

determination errors in general account for 24.1 % (student essays) and 15.8 % (scientific papers) of all errors in the two comparison corpora, indicating that authors produce a more varied range of determination errors in these types of writing than in requirements, where determination errors other than missing articles are marginal to non-existent.


English and Norwegian for the past 2-3 decades, resources for this kind of studies have been largely lacking for L2 Swedish.

researchers of L2 vocabulary and grammar acquisition are in great demand of digitized L2 corpora of Swedish, that can help verify hypotheses generated by experimental studies and/or smaller scaled empirical studies.


in dubious cases, we applied a principle of positive assumption, i.e.


Kelly is a frequency-based wordlist generated from web corpora, and translated into and compared between nine languages for identification of core vocabulary across these languages (Kilgarriff et al., 2014).

kelly has shortcomings, namely that (1) frequency statistics are collected from web texts aimed at L1 speakers of Swedish, which can be misleading since the vocabulary used for L1 speakers may differ from what beginner L2 speakers need to concentrate on; (2) the division into the CEFR levels is based on frequency and L1 text coverage, which needs explicit validation to confirm its relevance for a CEFR-based curriculum; and (3) kelly lacks some vocabulary useful in the L2 context, such as table, alphabet, toothpaste -i.e.


In the context of second language learning it means that a learner who has acquired the knowledge of these words can read and understand most of the modern Swedish texts.

as was the case for the coverage approach of Laufer and Ravenhorst-Kalovski (2010), this list only describes a global learning goal and does not contain any indications for appropriateness at different levels of learner proficiency.


The methodology applied has the great advantage to be able to assign different difficulty levels to the different senses of a word.

it is also a long and costly process to repeat for other languages.


As a result, the lack of available corpora calls for the use of word frequency lists in the lexical substitution task (Devlin and Tait, 1998;Shardlow, 2014).

as the commonly used frequency lists are mainly L1-focused, we believe that they do not fit the L2 context very well.


On the one hand, we automatically annotated each lexical unit in the global corpus with its level of difficulty based on FLELex (see Section 2.2.1.).

we asked a group of Dutch-speaking learners of French to manually annotate the 51-text sample according to their vocabulary knowledge (see Section 2.2.2.).


In order to detect a significant lexical simplification effect, we compared each pair of original and simplified texts with respect to the number of lexical units that were attributed to each of the six difficulty levels in FLELex (Table 3).

as the texts included in both the Tales and the Wiki corpus differed greatly in terms of text length, we normalised the counts to a length of 1,000.


The core vocabulary per level includes those words that are observed in most textbooks.

the peripheral vocabulary per level includes those words that tend to appear in only few textbooks and which are hence not indicative of the vocabulary commonly targeted at a given proficiency level.


In contrast, the peripheral vocabulary per level includes those words that tend to appear in only few textbooks and which are hence not indicative of the vocabulary commonly targeted at a given proficiency level.

we could collect more extensive learner annotations for all six proficiency levels in order to automatically learn a new transformation rule that defines vocabulary knowledge similarly to the learners' annotations.


On the one hand, we observed that defining the word's CEFR level on the basis of its first occurrence in FLELex enabled us to detect a significant simplification effect in manually simplified texts.

we observed that an expert model predicting the learner's actual vocabulary knowledge based on the word's CEFR level was relatively accurate, but that it presented some errors with respect to the recall of unknown words.


Since the baseline score on the F metric is harder to beat, its value is correspondingly less informative.

the D metric, which measures discriminative ability rather than error rate, works equally well for both groups of systems.


This is a counterintuitive result, since it would suggest that the system is less useful to students producing higher proportions of correct responses; in fact, the results presented in chapter 7 of (Baur, 2015) suggest the opposite pattern.

the D metric returns the same value irrespective of the balance between correct and incorrect answers, as long as the relative reject rate on each group stays the same.


If the RALL systems cover a wide range of topics, learners can train their conversational skill in L2 on the topics of their interest (Wilcock & Yamamoto, 2015).

aSR of L2 speech is still a challenge because the speech is often made with poor pronunciation and contains grammatical errors.


Spearman's scores showed very low correlation between OSMAN scores of the two Arabic versions (0.035) which indicates the importance of the presence of diacritics, which plays a vital role in determining the text ease of reading.

the OS-MAN scores (0.329) for the diacriticised Arabic showed a higher positive correlation with the English Flesch scores.


Spearman's scores showed very low correlation between OSMAN scores of the two Arabic versions which indicates the importance of the presence of diacritics, which plays a vital role in determining the text ease of reading.

the OSMAN scores for the diacriticised Arabic showed a higher positive correlation with the English Flesch scores.


Enabling users of intelligent systems to enhance the system performance by providing feedback on their errors is an important need.

the ability of systems to learn from user feedback is difficult to evaluate in an objective and comparative way.


Still another one is interactive information retrieval, where the user provides feedback on the relevance of retrieved documents to improve the overall search results, be it for textual (Salton and Buckley, 1990) or multimedia documents (Nguyen and Worring, 2008).

in general, evaluating the ability of systems to learn from user feedback in an objective and comparative way is difficult.


As mentioned above, the goal of system adaptation is to maximize the system performance while minimizing the amount of feedback information needed to get the improved performance.

since the feedback provided, and in particular the amount of information therein, is under the control of the user, if the task under study does not naturally lend itself to have this amount of information also controlled by the evaluator, no meaningful comparison between systems can be drawn.


Furthermore, if the oracle is designed using knowledge of the internal functioning of the system with which it interacts, the evaluation can be biased.

these issues Error rate (E) Figure 2: Computation of a Minimum Supervision Rate (MSR): Starting from the initial system error rate (E T0 ), some amount of information is provided as feedback (F T1 ), leading the system to update its models and yield a new error rate (E T1 ).


In such situations, which are common in information retrieval and entity recognition, metrics like precision and recall are typically used to describe performance.

precision and recall fail to describe the differences between sets of objects selected by different decision strategies, instead just describing the proportional amount of correct and incorrect objects selected.


If one feature group's removal keeps overall performance similar, but selects completely different entities, this indicates a potentially interesting analysis candidate.

this is not possible using the conventional measures.


They are based on precision and recall, thus using conventional solutions to evaluation problems that come from the typical skew toward negatives.

they also take into account complementarity, giving more insight into differences than F-score, precision or recall.


Also, from the high complementary recall that ANNIE offers over OpenNLP, adding ANNIE's results is likely to offer strong recall improvements.

from P Comp (OpenN LP, AN N IE) being only 20.00 we can see that ANNIE and OpenNLP select quite different entities, and that precision gains are not likely to be made.


That topic modelling does well at identifying differences in topic is perhaps not so surprising, and this could be seen as consistent with the good performance of topic modelling on the nyt june KSC, where news articles from different time periods would be expected to be on somewhat different topics.

(this does not provide an account for why perplexity does so well for nyt 5678 where we would also expect differences in topic.)


The best score for adequacy is 5 and this means that the whole meaning of the reference is also expressed in the translation, while 1 means that none of the original meaning was kept.

it seems that people have difficulty in evaluating these two aspects of language separately.


To address this issue, research has been focusing on developing automatic methods for readability evaluation, which could account for the specific reading difficulties of various reader populations (Dubay, 2004;Benjamin, 2012), including people with cognitive disabilities (Feng et al., 2010;Yaneva and Evans, 2015).

relying on readability formulae to measure text accessibility is an approach that has many drawbacks (Benjamin, 2012; Dubay, 2004;Siddharthan, 2004), mainly re-lated to the fact that readability formulae only employ surface text features such as word or sentence length.


F leschReadingEase = 206.835− 1.015 × words sentences -84.6 × syllables words (Flesch, 1948) Subtler text characteristics could be accounted for by readability models based on machine learning algorithms.

developing these models for people with cognitive disabilities is currently not feasible, due to the lack of large enough corpora for model training and evaluation.


Considering the simple embeddings, we observe that Skipgrams performs significantly better than CBOW and GloVe on POS and MENT tasks.

for the other tasks CBOW achieves the best results.


Tukey post-hoc comparisons indicate that the mean score for the EN condition (M=198.9, SD=22.0) was significantly different to the DE_PE condition (M=305.8, SD=62.2).

the DE_MT (M=255, SD=43.9) condition did not significantly differ from the EN and DE_PE conditions.


AQL is a powerful language that implements an IE algebra (Reiss et al., 2008).

in our opinion, this loses some of the simplicity that Odin's Runes enjoys.


For example, the syntaxbased grammar correctly finds two ubiquitination events and two negative regulations in the sentence "CYLD inhibits the ubiquitination of both TRAF2 and TRAF6" because the dependency graph correctly connects "ubiquitination" to "TRAF2" and "TRAF6", as seen in Figure 2.

the surface-based grammar misses the ubiquitination event involving "TRAF6" (and the negative regulation of this ubiquitination), because the last two tokens of the sentence are not explicitly handled by the rules.


On the other hand, the surface-based grammar misses the ubiquitination event involving "TRAF6" (and the negative regulation of this ubiquitination), because the last two tokens of the sentence are not explicitly handled by the rules.

syntax-based grammars assume that a syntactic parser is available and produces robust output.


Since one of the points of the Living Lexicon is to investigate how an unsupervised DSM will develop as it continuously reads uncontrolled data from online sources, such qualitative differences are to be expected; the lexicon is nothing more than a representation of the current state of online language use.

this makes it slightly challenging to perform quality assurance and evaluation of the lexicon.


For example, they assume that annotation is performed in the token level.

korean is an agglutinative language whose words are formed by joining morphemes together, so it can not be annotated properly in the token level.


One may argue that using the different attribute names to denote IDs will make the various kinds of tags easier to recognize.

in terms of further applications that make use of temporal information, it is not necessary to use various attribute names to denote tag IDs the kind of tag is already known when its attributes are parsed.


Most of the existing wide-coverage knowledge bases have no problems in identifying, e.g., major cities ("New York is a city") or celebrities ("Madonna is a singer"), but they show limitations with respect to small villages and less known people.

the potential of web-scale intelligent, data-intensive applications can only be unlocked if they are capable of dealing with the most prominent entities, as well as the long tail.


The dataset contains types for DBpedia entities that have been extracted from the corresponding Wikipedia articles using Hearst patterns.

the work presented in this paper uses the whole Web as a corpus, not only Wikipedia, and hence, it is not limited to find hypernymy relations to entities that are represented by a Wikipedia page, but between arbitrary entities.


For example, the eleventh query (OHSU11) had an F-measure of 0.16 for KantanMT and 0.19 for Moses.

the original English query managed to yield an F-measure of only 0.08, which is half the performance of the machine translated queries.


Most of the previous studies on English spelling-error extraction collected English spelling errors from web services such as Twitter by using the edit distance or from input logs utilizing crowdsourcing.

in the former approach, it is not clear which word corresponds to the spelling error, and the latter approach requires an annotation cost for the crowdsourcing.


Their approach saves the cost of crowdsourcing, and guarantees an exact alignment between the word and the spelling error.

they did not assert whether the extracted spelling error corpora reflect the usual writing process such as writing a document.


This game includes the intended word and does not require the cost of crowdsourcing.

the writing process of a word-typing game may differ from the usual writing process (e.g., writing a document).


We suppose it was because they extracted uncorrected spelling errors using a set of common   We implemented a correctable typing game to extract corrected spelling errors.

the difference in the two games may affect the spelling errors.


Google definition boxes 1 (that appear sometimes when you search on Google) are very well-known result of "automatic" definition finding.

they are probably (to the best of our knowledge, the exact algorithm has not been published yet) based on a few reliable sources of definitions, such as first paragraphs of Wikipedia articles or particular dictionaries.


Overall, systems appeared to perform better over the generalization and drifting sessions than the specification ones (Kanoulas et al., 2010).

only one team achieved a significant statistical improvement.


Hand inspection showed that, in some cases, producing better translation does not necessarily mean that the information need is expressed better.

this fact should be contrasted with a more extended topic-set including this type of reformulations.


For instance, Figure 2 shows the growth of the topic "Statistical Machine Translation" in the NLP community using the ACL anthology corpus 4 .

saffron does not provide any topic trend forecast for the coming years.


In this approach, we individually model the time series data for every keyword.

as we use regression, we assume that the previous year values are independent of each other, which is not true for time series data.


Also, more sophisticated approaches for modelling the time series data such as using Fourier transforms or Recurrent Neural Networks can be investigated in context of modelling the growth of scientific topics in this domain.


The purpose of a POS tag is to describe the grammatical structure of words in an utterance, and a corresponding resource aims to help computational methods reduce ambiguity in what is communicated by the text.

domain-specific annotations for NLP are often centered on real-world applications of text.


In order to respect both perspectives (namely, the crosslinguistic one and the one associated to grammar traditions in each language), the ontology put forward here is to be used for the overall classification and linking of linguistic information across languages.

each specific application resorting to such content (e.g., dictionaries, computational lexicons, automatic tools for language processing, etc.)


There are 6 tags that are only present in our ontology: affix, contraction, and idiomatic (all of them needed here because they are elements present in dictionary and grammar content); article and predeterminer (both subsumed under determiner in UD); and ideophone, not accounted for in that ontology.

the UD POS tagset has categories not available in our proposal: tween the OGL Ontology and each of our bilingual dictionaries.


For example, assuming English to isiZulu and an English to Northern Sotho datasets exist, we are able to extract isiZulu to Northern Sotho translations.

this strategy might not always be accurate; as, a word might have multiple senses, thus isolating a sense to sense translation without some post-processing on data may not be possible; or even in the case of having only one sense in both mentioned dictionaries, the senses might not represent the same meaning.


We will next explore supervised and ensemble methods.

(ensemble methods may not work for this task as captions may be funny in different ways; for example, of two equally funny captions, one may be funnyabsurd and the other funny-ironic.)


The assumption behind eye tracking is that the longer the eye gaze fixation on a certain word is, the more difficult it is for cognitive processing (Just et al., 1996).

until now, eye tracking has not been used to investigate reading in autism, possibly due to the number of procedural difficulties related to this kind of research with autistic participants (Section 3), and thus there is no reliable information about the particular types of phrases which need simplification for readers with autism.


increase eye contact with the audience), a full green bar would indicate to the participant that his/her performance is very good.

the virtual characters only adopted a neutral posture and did not provide additional feedback.


The interactive virtual audience condition producing nonverbal feedback was not significantly better than the control condition after the investigated minimal training of only two short presentations.

we found using participants' questionnaires that the interactive virtual audience was perceived as more engaging (µ IV A = 4.50, µ N on−IV A = 3.44; t(30) = 0.86, p = 0.001, g = 1.211) and challenging (µ IV A = 2.94, µ N on−IV A = 2.00; t(28) = 1.02, p = 0.025, g = 0.801) than the control condition, which could prove pivotal in the long run and keep the learner engaged and present a more challenging task.


Out of the single modalities the acoustic information seems to be most promising for the assessment of performance improvement.

we are confident that with the development of more complex and tailored visual features similar success can be achieved.


For instance, the goal of Schone and Jurafsky (2000) was "to identify an automatic, knowledge-free algorithm that finds all and only those collocations where it is necessary to supply a definition."

in order to accomplish this complex task authors made little if any effort to normalize extracted MWT candidates: "Prior to applying the algorithms, we lemmatize using a weakly-informed tokenizer that knows only that white space and punctuation separate words."


However, in order to accomplish this complex task authors made little if any effort to normalize extracted MWT candidates: "Prior to applying the algorithms, we lemmatize using a weakly-informed tokenizer that knows only that white space and punctuation separate words."

for highly-inflected languages, such as Serbian and other Slavic languages, this task can hardly be avoided as each nominal MWT can have many inflected forms (from five to ten or even more) and many of these forms (but usually not all) can in general be extracted from a corpus.


But if obloge trake is extracted, then two different structures -N2X and NXNcan be associated with various interpretations for both constituents.

if some other form is extracted besides it, e.g.


argument merger (Grimshaw and Mester, 1988), argument fusion (Butt, 2010), argument composition (Hinrichs et al., 1998), and also Alonso Ramos (2007).

to our best knowledge, none of these mechanisms have been verified on corpus data.


To obtain more data, we made use of the Czech part of the PCEDT, see above Table 3.

the annotation of CPs in the PCEDT is not as rich as in the PDT.


Al-Haj (2009) presented a systematic linguistic characterization of MWEs in Hebrew, and provided in a full picture of the diverse properties that Hebrew MWEs exhibit.

al-Haj and Wintner (2010) limited their investigation to noun compounds.


Tsvetkov and Wintner (2012) proposed an algorithm for identifying MWEs in bilingual corpora, using automatic word alignment as their main source of information.

automatic construction of parallel corpora is a time consuming task.


"(he) ate with all his mouth") can inflect for number, gender, person, and all tenses.

the noun ph does not inflect for number.


This representation can also benefit from the fact that it is possible to find multi-word entities of a given language in texts in another language (especially with names of international organisations such as European Space Agency which can be found in German text).

translated tokens many entities have different written forms across languages so that a string-based comparison of tokens is not successful.


This knowledge enables an entity linking system to model and perform reasoning over the semantic context of the real-world entities it links to.

the heterogeneity of the LOD knowledge also represents a barrier towards its automatic use, integration and manipulation.


General Motors Corporation ( GM ) 3 even specifically introduces an abbreviation "GM", allowing a human reader to understand the further mentions of this abbreviation in the remainder of the article.

the first step of our approach is not able to correctly disambiguate the other 4 entity mentions from this snippet.


After that, the projection is given to the non-linear hidden layer and then the output is given to softmax in order to receive a probability distribution over all the words in the vocabulary.


Due to the fact that each tweet is treated as a single document with only 140 characters, it is difficult to make use of non-local features such as context aggregation and prediction history for the NER task on tweets.

local features are mostly related to the previous and next tokens of the current token.


Finally, the total score for the node e is score(e) = coh(e) + P P R avg • iSim(e) where total coherence coh(e) of node e to the graph is computed with respect to aggregation constraints, and initial similarity score iSim(e) is weighted by an average value of P P R weights used in coherence computation.

this approach often ranks higher a popular candidate connected to many nodes in a graph over the correct but less popular one.


As we can see the number of Company and Product entities in the Synthesio corpus is almost four times more than in the Ritter Corpus.

the number of Person, Geo-loc and job-title entities in the Synthesio corpus is only about one fifth of the same entities in the Ritter Corpus.


Improvement has been reported when translated from French (+1.6 BLEU), German (+1.95 BLEU) or Hungarian (+1 BLEU) into English.

application of similar approach for English-Latvian MT has resulted in insignificant improvement by only +0.12 BLEU points (Rikters, 2015).


In the presented approach, the chunker splits sentences in top-level chunks without analysis of sub-chunks or cases when a chunk is single token.

the larger chunks should be split in smaller sub-chunks and the single-word chunks should be combined with the neighbouring longer chunks.


In Hiragana TIMES, NE abstraction methods were effective.

in BTEC, NE abstraction methods were not effective because most of the sentences are typical dialogues that are easy to translate with the baseline, e.g., "I must arrive in Tokyo by tomorrow morning ."


In contrast, in BTEC, NE abstraction methods were not effective because most of the sentences are typical dialogues that are easy to translate with the baseline, e.g., "I must arrive in Tokyo by tomorrow morning ."

manual NER (indicated as "man-NER" in Table 5) achieved a better result than automatic NER methods ("auto-NER") and the baseline method, even in the BTEC corpus.


(1991) obtained above 95% precision on the Hansards).

for literary bitexts, alignment quality could be much less satisfactory.


For example the inclusion of "this/that" which like "it" can be used as anaphoric or event reference pronouns, or "your" which requires a similar deictic/generic disambiguation approach as for "you" (included).

they represent similar translation problems to those posed by "it" and "you" and in order to keep the number of pronoun tokens manageable when it comes to manual evaluation, certain exclusions must also be made in terms of pronoun forms.


For example, the IDIAP system (Luong et al., 2015) has fewer reference translation matches for intra-sentential anaphoric "they" than the baseline.

it produces some pronoun translations that are better than those produced by the baseline.


The "Briggs" does not exist in the corpus either.

word2vec retrofitted by PPDB incorrectly paraphrased it to "Smith", leading to this incorrect translation.


Generally the text transcript generated by automatic speech recognition (ASR) systems is unpunctuated and unsegmented.

the readability of the transcript can be largely improved by the existence of punctuation marks, and the segmentation of the transcript based on punctuation positions will also increase the efficiency of many downstream natural languages processing (NLP) tasks, such as semantic parsing, question answering or machine translation (Matusov et al., 2007;Wang et al., 2010).


In previous research efforts, frequently used lexical features include the language model (LM) score, token, part-ofspeech (POS) tag, chunk tag and so on.

we would like attempt another possibility by using the Word Vector.


Interestingly enough, no single parser in Table 6 outperforms the others by achieving the highest LAS for all the selected dependency relations.

each parser seems to be best performing on specific relations.


This helps the grammar writer to arrange the rules in appropriate sections, with safest and most effective rules coming first.

this method will not notice a missed opportunity or a grammar-internal conflict, nor suggest ways to improve.


In order to find the set of readings, we expand a morphological lexicon 1 , ignore the word forms and lemmas, and take all distinct analyses.

many grammar rules target a specific lemma or word form.


The readings in a grammar can be underspecified: for example, the rule REMOVE (verb sg) IF (-1 det) gives us "verb sg" and "det".

the lexicon only gives us fully specified readings, such as "verb pres p2 sg".


If we accept those lists as readings, we will generate symbolic sentences that are impossible, and not discover the bug in the grammar.

if we are primarily interested in rule interaction, then using the underspecified readings from the grammar may be an adequate solution.


Since VISL CG-3 already points out unused sets, we did not add such feature in our tool.

we noticed an unexpected benefit when we tried to use the set definitions from the grammar directly as our readings: this way, we can discover inconsistencies even in set definitions that are not used in any rule.


Despite the inaccuracies, we can see that increasing the number of readings and adding the ambiguity class constraints slow the program down significantly.

many of the use cases do not require running the whole grammar.


A third of the transcripts (35 %, 145 hours) has been created very carefully using a systematic transcription approach and manual quality cross-checks, resulting in an orthographic basic transcription (Wendelstein, 2016).

the majority of the transcripts (65 %, 236 hours) does not follow the criteria of linguistic transcripts, was created without any postchecks and includes hardly any transcribed vocables (hesitations, back-channeling, disfluencies).


In order to have a number of reliable short segments for training and evaluation, a part of the transcribed data is currently also being segmented manually.

as the manual segmentation takes about 5 times real-time, it will not be able to replace the long audio alignment based segmentation.


A more rigorous definition of prosodic coverage is the ability to synthesise speech from the speech corpus with as many possible prosodic states (declarative intonation or interrogative intonation for example).

to define prosodic coverage more rigorously, one should define prosody.


Many errors occur because the pronunciation of inflections, compound words and words of foreign origin as well as so-called "non-standard words" (Sproat et al., 2001) often cannot be handled by rules.

there are too many of these for inclusion in exception dictionaries.


In order to collect real system interaction data, one should choose elicitation methods which do not bias the participants.

a high number of valid utterances should be generated.


Beside the benefits, crowdsourcing is often criticized as producing poor quality because it is difficult to control the work quality and the status of the workers (Eskenazi et al., 2013).

in the past years, the speech processing communities have realized that crowdsourcing is a possible solution to their strong need for speech data (Eskenazi et al., 2013).


The imperative and the command style & infinitive were by far the most frequently used sentence constructions over all tasks and methods.

it is striking that the command style & infinitive is the most preferred sentence construction in the semantics method, whereas the imperative is the most preferred sentence construction in the pictures and in the text method.


The PTDB-TUG (Pirker et al., 2011) and the Keele corpus (Plante et al., 1995) contain single-channel and close-talking speech recordings, which cannot be used for multi-channel experiments.

there are corpora containing multi-room and multi-channel recordings: the ATHENA corpus (Tsiami et al., 2014), the DIRHA-GRID corpus (Matassoni et al., 2014), and the GRASS corpus (Schuppler et al., 2014a).


Second, the audio quality exceeds that of most other work in the field of clinical voice research.

two limitations exist and should be taken into account when the data are analysed and conclusions are drawn.


Automatic speech recognition (ASR) technologies for Latvian have a relatively short history because even three years ago (i.e., in 2013 and before) there was no orthographically annotated speech corpus, which could be used for ASR purposes, available.

there have been attempts to develop ASR systems for broadcast speech recognition (Oparin et al, 2013) in the Quaero project (Lamel, 2012) using acoustic model bootstrapping.


and provided a list of possible spoken commands that they could use during the recording session.

speakers were instructed to not limit themselves to the list if they thought that a different spoken command was necessary.


There are some children's speech databases for EP, such as Speecon with rich sentences (Speecon Consortium, 2005); ChildCAST (Lopes, 2012;Lopes et al., 2012) with picture naming; the Contents for Next Generation (CNG) Corpus targeting interactive games (Hämäläinen et al., 2013) and (Santos, 2014;Santos et al., 2014) with childadult interaction.

these databases do not present the required samples of disfluent reading speech.


With regards to Urdu Summary Corpus, we did not perform such experiment.

we found the token difference of 9.8% between both versions, as shown in Table 3.


ROUGE functions based on the assumption that in order for a summary to be of high quality, it has to share many words or phrases with a human gold summary.

different terminology may be used to refer to the same concepts and thus relying only on lexical overlaps may underrate content quality scores.


In the case of Keywords (KW) query reformulation, without using discounting, we can see that there is no positive gain in correlation.

keywords when applied on the discounted variant of SERA, result in higher correlations.


For instance, comment sentences linked to the same article sentence can be seen as forming a "cluster" of sentences on a specific point or topic.

having labels capturing argument structure enables computing statistics within such topic clusters on how many readers are in favour or against the point raised by the article sentence.


In this paper, we make use of ConceptNet (Speer and Havasi, 2012), that is a semantic graph that has been directly created from it.

with linguistic resources such the above-mentioned WordNet, ConceptNet contains semantics which is more related to common-sense facts.


Given that the end of the candidate triple c 1 is contained in P conceptnet (pain), the triple is added to synset S burn,burning .

the triple c 2 is not added to S burn,burning since relatedto − melt is not contained neither in P conceptnet (pain) and P conceptnet (hurting).


On the contrary, is-a and related-to relations have shown a lower performance.

this  Table 3: Accuracy of some WordNet semantic enrichments obtained by the manual evaluation.


Also, we were using a 7-item Likert scale, whereas the WSsim/USim team used a 5-item Likert scale, and, intuitively, increased granularity ought to increase the risk of interannotator disagreement.

wSsim/Usim were purposefully annotated by non-experts, whereas all our annotators are linguists (with high non-native proficiency in English) and had been working with PDEV before.


About half the verbs, two thirds of the nouns, and nearly all the adjectives have only a single sense listed in GermaNet.

the average number of senses per lemma, 1.40, is still higher than GermaNet's overall average of 1.31.


Conceptually, our work is more in the spirit of Birke and Sarkar (2006), who treated "literal" and "idiomatic" as two different senses of the target word and applied (unsupervised) word-sense disambiguation techniques.

to this work, however, we use a simple supervised approach to distinguish literal from idiomatic uses.


Accordingly, we see an increase of the frequency of variants written as one word for idiomatic uses.

we also observe that literal uses start to be more frequently written as one word, though not to the same extent as idiomatic uses.


The average agreement score for full senses is 0.52, compared to an average agreement on 0.56 for clustered senses.

each ambiguous noun tells its own very individual story, some being fairly easy to annotate with or without clusters and others being more or less impossible, agreement scores spanning from 0.048 for plade (plate, sheet, disc, etc.)


The annotation of such multiword expressions that are present in the provided MWE-list of the word achieves the highest agreement because the MWEs are not individually interpreted by the annotators, as they just have to mark them up according to the list.

occasional syntactic and lexical co-incidence between a free construction and a fixed expression makes up a difficulty for an appropriate identification of the lexical vs. phraseological sense.


Here we can conclude that a clustered annotation scheme based on an ontologically driven collapsing of subsenses performs substantially better than a fully fine-grained scheme (disregarding here the better chance of agreeing on few tags than on many).

it is remarkable how each individual noun exposes its own pattern, and how some very ambiguous nouns prove almost impossible to annotate -with or without clusters.


Our corpus is about one fifth of the size of SemCor.

as mentioned, a large part of the data has been doubly annotated and later adjudicated.


Although segmentation errors can affect the quality, it has been shown that such word embeddings can improve Chinese NLP tasks such as parsing (Wu et al., 2013).

character information can also be used for NLP (Zhang et al., 2014a).


By setting the number of senses according to the cilin corpus, the multi-prototype embeddings give the best results.

there is not a universal standard on the number of senses for each character, due to variation in semantic granularity.


While the pair "abduct-abductor" can be used in the muscular sense in Portuguese -{01449427-v abduct, abduzir -pull away from the body; "this muscle abducts"}, it seems more commonly used in its kidnapping sense, {01471043-v snatch, kidnap, abduct, nobble, abduzir, sequestrar, raptar -take away to an undisclosed location against their will and usually in order to extract a ransom; "The industrialist's son was kidnapped"}, where the role would be agent.

looking up the pair "dilate-dilator", we see one of the first applications of the work morphosemantic links do for us.


These resources are mostly produced manually by domain experts and contain high quality data including segmented inflectional and derivational morphemes even for under-resourced languages.

this kind of morpheme data is not machine-processable and, therefore, hardly reusable and hence remains isolated on the Web.


We did not find a way to reliably identify the source language from which a specific translation was created.

we observed that each source document is translated (potentially via a pivot document) into 2.77 languages on average, thus generating combinations of sentence alignments as in the examples of Table 3.


This has entailed encoding roots as Lexical Entries with type root (see LE kbr in the figure).

the choice of encoding roots as LE is somewhat problematic for theoretical and practical reasons.


It is apparent that we need to be cautious in conclusions, as different data are of different sizes which may cause errors in estimations (Baayen, 2001;Kilgariff, 2001).

we believe that our analyses have shed considerable light into quality of the Digi collection and our procedure can be used for quality approximation also after possible improvements in the data.


For this purpose, Optical Character Recognition (OCR) systems have been developed to transform scanned digital text into editable computer text.

different kinds of errors in the OCR system output text can be found, but Automatic Error Correction tools can help in performing the quality of electronic texts by cleaning and removing noises.


An automatic classifier is then applied to determine the character code that most likely corresponds to each character image.


It can be seen that in our annotations we have tended to provide the contemporary correct form far more often than a historical form (of which there may be several, which might obfuscate results).

the results also show that the LD from a historical variant to its historical correct form is often lower than to its contemporary correct counterpart.


In order to solve this problem, TICCL will need to be equipped with a special module geared at picking up elongated stretches of very short word tokens.

the full book evaluation told us nothing about the extent of the foreign language problem in our supposedly Dutch corpus.


Our prior example of 'vossestaarten' already showed that confusion of the long 's' by 'f' is not necessarily the case.

it is the feature of digitized diachronic texts most often commented on by researchers working on digitized text (cf.


(3) Argument A2 attacks the controversial issue, which says that vaccines protect the population (with no exception).

since the number of deaths is very limited, this statement may be interpreted either as a weak attack or as a concession: Vaccine protects the population HOWEVER there are a few cases where it does not work.


On the one hand, we are able to validate the interoperable scheme in an application.

the successful application of the scheme indicates possible interoperability in existing resources.


In this pilot we measured the durations of the filled pauses that could be identified on the basis of manual transcriptions of the interviews.

these transcriptions are not always accurate.


Previous research works have proven that discourse information is beneficial to these NLP tasks.

the current discourse annotations concentrating only on monolingual languages have some insufficiency that limits its power in many multilingual NLP tasks.


In RST, the temporal reading similarly corresponds to "circumstance".

it is not at face value clear in which way während is contrastive: Does it express a mere (same-level) contrast, or is it concessive by indicating one argument as more central than the other?


The RST corpus also identifies an "antithesis" use which is a nucleus/satellite (subordinating) relation.

according to the (semantic) PDTB definition of concession (a denial of expectation), we could not find any concessive uses of während in 100 instances from German newspaper text, leaving only general "contrast".


This has resulted in the creation of discourse-annotated corpora, such as the Penn Discourse Treebank (PDTB) (Prasad et al., 2008), the Rhetorical Structure Theory (RST) Treebank (Carlson et al., 2002), and the Annodis corpus (Segmented Discourse Representation Theory, SDRT) (Afantenos et al., 2012), to name a few.

as of yet, the different frameworks are not inter-operable, nor is there a unified scheme for discourse annotation (but see the proposals by Benamara and Taboada (2015), Chiarcos (2014), Bunt et al.


With that aim in view, the labels in the different frameworks are mapped onto each other.

the mapping does not include the structures (i.e.


The authors claim that their taxonomy is robust across theoretical frameworks and can be applied to corpora from different languages.

they identify some problems for the mapping, e.g.


Other studies that have used the PDTB framework to annotate spoken data are Demirşahin and Zeyrek (2014) and Stoyanchev and Bangalore (2015).

they have not made any further changes to the PDTB framework.


This corpus is already available on line at http://sldr.org/sldr000847/fr in its first version.

many enhancements have been made in order to allow the development of automatic techniques to transcript conversations and to capture their meaning.


The goal of the DECODA project is to enable the development of these strategies and these applications while allowing the annotation effort to be less important.

the type of speech that is found in the conversations of call-centers is quite far from what usually treat ASR Systems.


It consists of two parts: On the one hand, 23 texts were written by one of the authors and have been used as examples in teaching and testing argumentation analysis with students.

90 texts have been collected in a controlled text generation experiment, where 23 subjects wrote short texts of controlled linguistic and rhetoric complexity, discussing one of the issues they chose from a pre-defined list of controversial issues.


Argumentative SUP-PORTs cooccur with ELABORATION, EXPLANATION, and RESULT.

48% of the supports cannot be mapped to SDRT edges, which is more than in the alignment of argumentation and RST.


fuĝ|into [fugitive]), dropped by a similar margin, from 2% to 0.8%.

most new/unknown tokens are CWs 8 , and the effect is even bigger for multiple (3+) CWs, which were 10 times as frequent among new words.


Moreover, many descriptions can be deduced from other relations because most relations are symmetrical or could be recovered transitively.

we consider this redundancy as beneficial because it eases the creation of the resource, its update, and its use.


the lemma accuracy reported by Pasha et al.


The morphological part comprises deep-structure morphological analyses of German.

as it was developed in the Nineties, both encoding and spelling are outdated.


Incorporating this feedback in the development process may have influence on the success of the app.

one challenge for the developers is to deal with the overwhelming amount of reviews.


The analysis of opinions in reviews has been widely investigated within the last decade (Pang and Lee, 2008) and is 1 https://itunes.apple.com/us/genre/ios/ id36?mt=8 2 https://play.google.com/store/ 3 https://appworld.blackberry.com/ webstore/ 4 https://www.microsoft.com/en-us/windows/ apps-and-games typically referred to as sentiment analysis or opinion mining (Liu, 2012).

previous work in the area of analyzing user reviews in app stores is limited.


The number of subjective phrases within the different application categories is relatively constant.

the number of application aspects (from 149 to 287) and relations (104 to 237) varies much more.


The out-of-domain Subtask 3 had no participants.

the data developed for this subtask remains available and can be used for future research.


implies the fear emotion, brought forward because of the possible diseases which germs can transmit.

for a software system to detect that, it should be able to conclude that germs are capable of spreading diseases and people are afraid of things which can spread diseases.


we have learnt the smallest number of property-value pairs with respect to the other emotions.


Concepts can be connected with several types of conceptual relations (Loukachevitch & Dobrov, 2014).

to SentiWordNet and other sense-based sentiment resources, we found that a lot of Russian ambiguous words have the same polarity in all available senses.


Several pro-Iraq demonstrations have taken place in the last week.

to TimeML, in the Richer Event Description 3 (Styler et al., 2016, RED) framework the event/non-event distinction is not based on morpho-syntactic information.


12 In FactBank, the factuality value assigned to blowing up in this sentence is PR+ ("did probably happen") for both relevant sources, namely the AUTHOR and the GEN AUTHOR.

13 the uncertaint y expressed by suspected is not so much directed towards the proposition as a whole as it is towards a specific aspect of the event: the two Libyans being the ones who did it.


There is comparable little work with respect to visualisation in lexical semantics: Kievit-Kylar and Jones (2012) present a tool that allows to identify relations between words in a graph structure, where words are visualised as nodes, and word similarities are shown as directed edges.

to our proposed tool, the actual position of a word is however not determined by its underlying feature distribution, and can even be changed using click and drag.


In previous work by Siegel and McKeown (2000), Zarcone and Lenci (2008) and Friedrich and Palmer (2014), the event type of corpus utterances is predicted on the basis of syntactic and semantic features of the main predicate and the linguistic context thereof.

in this study, we investigate lexical (inner) aspect through a French lexical resource, "Les Verbes Français" (François et al., 2007), a valence lexicon of French verbs providing a detailed morpho-syntactic and semantic description for each reading (use) of a verb.


This is justified by the fact that under reading 01, remplir is more natural with an in-adverbial than with a for-adverbial.

it is nevertheless still acceptable with for-adverbial under its 'partitive' reinterpretation (described e.g.


7 Again, these results confirm the importance and effectiveness of the syn/sem features as well as the influence of the agentivity features.

they again do not provide further insights about the direction of this influence.


This confirms in some way our hypothesis that agentivity positively correlates with aspectual flexibility (or, in other words, that predicates are aspectually more flexible under their agentive than under their nonagentive reading).

features indicative of concreteness are characteristic both of the VARiable and TELic classes.


DSMs have been around for a while (Landauer and Dumais, 1997).

the recent enthusiasm about neural networks and word embeddings has made DSMs more accurate and faster to build using very large corpora.


The comparisons with the individual monolingual tasks confirmed that the "monolingual similarity after translation" approach works reasonably well, in the light of common insights gained in the field of cross-language information retrieval.

as expected, it is highlighted that the quality of translation would impact the end-to-end performances.


For instance, for an application involving a simple interface to a robot, quantifiers may not be needed.

there is little point in providing a single 'Simplified MRS' as an alternative to the standard ERS.


First, the subgraph S to be replaced has to be identified in the DMRS graph G. This is done via a form of exact subgraph matching, where nodes in S are associated with corresponding nodes in G and all links in this subgraph of G match the ones in S (see §4.1).

matching nodes exactly would be very inflexible and would require writing all possible concrete instantiations of a mapping pattern.


Initial experiments with DMRS mapping for paraphrase using automatically acquired rules are described by MacKinlay (2012, ch7).

when the source and target languages of the subgraphs are different, mapping becomes a method of machine translation, analogous to the use of MRS transfer rules in MT (compare e.g., Bond et al.


How-to websites like wikiHow 1 and eHow 2 help how to do things in daily life.

the quantity of wikiHow and the quality of eHow cannot compete with those of wikipedia.


Of the 30 sampled tasks, "buy movie tickets early", "buy a car", and "write a cover letter for a recruitment consultant", which achieve accuracies of 0.3000, 0.3200, and 0.2800, respectively, perform worse than the other tasks.

"download music safely" and "play experimental music", which have accuracies of 0.8200  and 0.7000, perform better.


Indeed, semantic vectors built for a single language were able to predict in an unsupervised setting typological patterns that hold across languages.

the evaluation showed that our test set is a valid semantic benchmark.


Consider the following example: RT @NewAppleDevice: Apple's smartwatch can be a games platform and here's why http://t.co/uIMGDyw08I It contains factual information that can be understood even without visiting the link.

the following tweet is not an argument, given that it does not convey an independent message while excluding the link: For all #business students discussing #Apple-Watch this morning.


The majority of works in author profiling use Supervised Learning which requires a sufficiently large corpus of clean, correctly annotated training data.

in many author profiling tasks, such data is not available.


Syntactic features are less often used; cf., e.g., (Cheng et al., 2009).

from linguistics and philology we know that syntactic idiosyncrasies are also distinctive features of the writing style of individuals and groups of individuals who share demographic, social, cultural or gender characteristics; see, e.g., (Crystal and Davy, 1969;Biber, 1989;Strunk and White, 1999;Tufte, 2006).


We believe that this is due to the composition of the feature set we use.

adding more instances in a semi-supervised fashion lets the classification further improve.


Building on this automated measurement of edit types, we use a graphical model analogous to LDA topic modeling to identify the latent roles editors occupy, much as document comprise topics.

to studies that employed either clustering analysis or principle component analysis to extract user roles (Liu and Ram, 2011), our role modeling treats an editor as comprising multiple roles at the same time.


Most written Arabic has been Classical Arabic or Modern Standard Arabic (MSA), and therefore most natural language processing (NLP) tools address these two variants of Arabic.

this situation is currently undergoing change.


dyAl has a higher degree of grammaticalization than similar forms in other dialects, such as Levantine tabaς.

to tabaς, can reduce to d and cliticize, e.g.


In other words, we expected that the less frequent a lemma is, the more regularly it is derived from the viewpoint of derivational patterns.

this was a working hypothesis and had to be proved empirically first.


The ocamorph analyzer is also implemented in OCaml.


We evaluated the task of semantic categorization by manually counting the number of correct and incorrect words in the given category.

in order to be able to perform this validation efficiently, the result lists were clustered automatically so that these groups could be reviewed at once.


Some of these false rejections were recovered during listening by the auditor noticing and transcribing the sentence manually.

separating all false from correct rejections would yield diminishing returns.


Human auditors may comb through the many hours of data in order to verify the read speech, but this task would be quite unwieldy.

a pure automatic approach that aligns the transcript and excludes imperfect alignments may be unnecessarily conservative.


The motivation for setting up this corpus was that there is no phonetically annotated and segmented corpus for this language pair of comparable of size and coverage.

to most learner corpora, the IFCASL corpus incorporate data for a language pair in both directions, i.e.


As expected, there were more frequent pauses and more frequent disfluencies in L2, as well as longer durations of pauses filled with breath noise than those without.

the analysis also reveals that in fluent phases the vast majority of pauses contains an audible inhalation -which suggests a reinterpretation of the terms "unfilled" and "silent" pauses.


We are exploiting our manually annotated corpus to design more robust forced speech-text alignment algorithms by anticipating possible errors made by learners.

the non-native realizations are studied in order to investigate efficient acoustic feedback provided to learners, and how feedback can robustly interact with automatic segmentation provided by automatic speech recognition.


Most of the observations made for English will probably remain identical for other European languages.

some analysis elements may differ due to the specific syntactic and morphological features.


We then observe many redundancies and fuzzy or vague uses.

even with a limited number of verbs, fuzzy uses are observed, which are due to the arguments the verb is combined with.


They are full of mention-to-entity mappings from which we might be able to train a wikifier.

we decide to annotate corpora by ourselves primarily because Wikipedia is too inconsistent to be used directly as training data.


If two LUs correspond to the same frame, they are both ascribed to it.

exemplary sentences are linked to LUs appropriate for the meaning of the predicate.


The super-lexicographers can make particular portions of entries available for particular lexicographers, which en-ables a sort of specialisations (e.g., for verb, noun and adverb subdictionaries).

the entries finished in one phase constitute an input for the next one.


Statistical machine translation algorithms tend to disregard lexicons.

when no parallel corpora are available, algorithms may resort to monolingual corpora and bilingual lexicon induction (Klementiev et al., 2012).


In recent years, several datasets have been released that include images and text, giving impulse to new methods that combine natural language processing and computer vision.

there is a need for datasets of images in their natural textual context.


Decades ago Stokoe defined sign language glosses as combinations of movements, hand shapes and location (Stokoe, 1980).

many studies developed their own gloss based annotations.


On the other hand, HOME ADDRESS and SCHOOL ADDRESS are considered as phrases because the words in the phrases can be modified.

for ADDICTED, the meaning is not preserved when the words are modified.


In particular, regarding elderly living alone at home, the detection of distress situation after a fall is very important to reassure this kind of population.

many studies do not include tests in real settings, because data collection in this domain is very expensive and challenging and because of the few available data sets.


Many effective SRL systems have been developed to work with written data.

when applying popular SRL systems such as ASSERT (Pradhan et al., 2005), Lund SRL (Johansson and Nugues, 2008), SWIRL (Surdeanu and Turmo, 2005), and Illinois SRL (Punyakanok et al., 2008) on transcribed speech, which was generated by an automatic speech recognizer (ASR), many errors are made due to the specific nature of the ASR transcribed data.


These attempts at connecting language expressions to real world objects such as images are often called symbol grounding (Harnad, 1990), an exciting new area in natural language processing.

images, videos, and many other forms of media have ambiguities that make symbol grounding difficult.


For example, two clusters that contain segments with the same name are merged.

a cluster which contains segments with different labels is divided.


This seems to be partly due to the underlying being designed for sitting subjects, while in our study participants were standing so that distance reference information like knee position is not applicable.

the remaining annotations proved to be highly consistent, with percentage agreements ranging from over 80 % to about 98 %.


Those rules can then be assembled in a system that is aimed to support inhabitants of a smart home for example by suggesting a certain configuration preset to the user.

to existing home automation technology, such a system is able to infer more general rules than remembering the status of a single device at a certain time, e.g.


The number of verbs from Italian-English annotated is 515, translated is 473 out of the total 521.

annotated and translated verbs from English-Italian are 546 and 497 in number respectively out of the total 550.


But in Urdu they are semantically treated as referring to the same action verb "moving" /ghumanA/.

in the English counterpart of the sentence ID: b65d7431ghumAnA refers to two verbs: 'to turn' and 'to rotate'.


Domain ontology describes knowledge of a certain domain and has been used in many applications such as QA systems.

constructing ontologies by hand is a tedious and time-consuming work.


We therefore added a Difference measure, which shows the difference between the coverage for each list in the two corpora.


While the numbers indicate that the trilingual NN produces more (reliable) triples than the German NN, this is not generally the case (e.g., for triple f-score 0.6).

our evaluation is biased towards features from the German annotation as we use the German-based MLG gold annotations for bootstrapping the annotation scheme.


1 For other languages, several corpora of spoken language exist.

similarly to the situation for the German language, only a few of them are manually annotated with part-of-speech tags or have tagsets adapted to their needs.


This can be inferred from the high accuracy of state-of-the-art POS tagging not only for English, but also most other languages such as Arabic, which reaches 97% for Arabic and English being at 97.32% (Gadde et al., 2011).

the performance of standard POS taggers for English is severely degraded on Tweets due to their noisiness and sparseness (Ritter et al., 2011).


There is a growing interest within the NLP community to build Arabic social media corpora by harvesting the web such as (Refaee and Rieser, 2014;Abdul-Mageed et al., 2012).

none of these resources are publicly available yet.


AMIRA also breaks urls into parts since they contain punctuation marks.

mADA and Stanford do not break all Twitter-specific elements into parts since they use the space as an indicator for a new token.


Based on that, we decide to further improve the accuracy of Stanford tagger while preserving its speed by using Arabic tweets training data.

there is no labelled training data available to do so.


Another approach to utilizing state-of-the art taggers is proposed by (Eger et al., 2016) mostly based on MarMoT (Müller and Schütze, 2015) and, thus, on higherorder CRFs.

to this, we focus on 1st order CRFs by asking about the payout of such a simpler model.


If we consider the inflectional categories in isolation, TLT-CRF performs worse than several of its competitors.

if we consider joint learning of these inflectional categories, TLT-CRF performs second best again (see the line ALL in Table  2).


Two other corpora exist: MULTEXT-East, a multilingual corpus extracted from the novel 1984, sentence-aligned and with morphosyntactic annotations; a corpus of 36,150 tokens has also been annotated with dependencies in the context of the BALRIC project.

1 these corpora can not be used for our purpose: the former does not contain information about dependencies and, as reported by (Calacean and Nivre, 2009), the latter consists in simple sentences of only 8.9 tokens in average, and no diacritics, 2 therefore not representing actual Romanian language.


This is probably to be explained by the size of the lexicon as only ∼2% of the words in the whole 500ktoken corpus are not covered by the lexicon and the fact that including suffix features already deals with tokens not covered by the lexicon.

given that in the next section we perform experiments in which we drastically decrease the size of the lexicon used, we decided to keep for the the remainder of the experiments the more complex feature of MSD hypotheses given the longest suffix.


For example, some linguists consider the expression cá f ish rô anabas {anabas} as a compound word but bệnh illness sởi measles {measles} as two words (Hoang, 1998;Diep, 2005).

these expressions have a similar construction: the combination of a categorization noun 4 and a specific noun.


For example, adjuncts indicating degree and tenses modify adjectives and verbs, respectively.

this method does not necessarily work sufficiently with real texts.


and they are expressed through the adjunct.

the adjunct is dropped frequently in the text.


By applying these rules to a treebank, the method corrects errors.

this method does not achieve wide coverage of error correction.


There are several methods of detecting annotation errors in a phrase structure treebank (Dickinson and Meurers, 2003;Ule and Simov, 2004;Dickinson and Meurers, 2005;Boyd et al., 2007;Dickinson, 2009;Przepiórkowski and Lenart, 2012;Kulick et al., 2013;Faria, 2014).

there is little work on treebank error correction.


So the treebank is constituency-based and syntactic relations are represented by means of trees.

the FTB contains several specific annotations, aiming at reducing the embedding of the trees (limitation of the projections, suppression of the VP level, affixation of clitics, etc.).


The amount of data needed to establish subtle discourse constraints is too large 5 given the costly discourse annotation process.

a solution of manipulating control 'original' texts annotated to form hypotheses-driven modified text from an intermediate size corpora seems to be a very promising option.


In the EDT version, the word-form trükikoja 'printing house-gen' is governed by the postposition eest 'of', which in turn is governed by the main verb of the clause, which appears to be a particle verb kandis hoolt 'took care'.

the word-form trükikoja 'printing house-gen' itself is modified by the main verb asus 'was located' of the relative clause, resulting in crossing arcs.


The comparison also shows that written Slovenian is significantly more nominal than spoken Slovenian with a higher share of nominal and prepositional phrases (the nmod and case relations) that are also more complex in terms of the number of adjectival (amod) or clausal (acl) attributives.

the spoken treebank contains notably more adverbial modifiers (advmod), which mostly include expressions of deixis, discourse cohesion and modality.


Loan words were segmented based on how they are segmented in the English corpus such as "ဂရိ တ် တား မန် ချက် စတာ" for "Greater Manchester", "အင် တာနက် ဖို ရမ် " for "internet forum", "ဝီ ကီ နယူ းစ် " for "Wikinews".

segmentation of some Myanmar particles and post positional markers were heuristically segmented depending on the context.


Furthermore, the building, transformation or annotation of the (sometimes huge) language resources implies to set up and drive large scale projects involving human resources, thus generating potential issues with regards to working conditions.

nLP is called for when it comes to providing tools for de-identifying documents, discovering plagiarism or performing fact checking, all applications that can be considered as ethically positive.


Here again, the questions reflected our bias in favor of a better concern for ethics, but we are glad that people did not hesitate to express some disagreements in their answers and comments.

it has to be noted that out of the 43 comments added to this poll, the large majority is surprisingly positive and pro-active (much more than the French ones), while a few criticize the form of the survey, i.e.


This offers in one part copyright protection over the design of the database schema.

as the structure of translation memories and the process of segmentation are widely understood and even subject to standardisation, e.g.


Revised Translations provided by a human post-editor could be subject to copyright protection as for translated content in general.

if the machine translated output is of a high quality, such that very little post-editing is required, then the claim of the post-editor to providing creative input to the generation of translation may be challenged, weakening the claim to copyright protection over the revised translation.


This is advantageous since the legal aspects of assuring the correct configuration of ODRL files may make their management and quality assurance an expensive task.

this means it is complex to express differing right for different attributes in a CSV resource recorded in different columns.


for a revised translation CSV file en-t-fr-m0-postedit.csv (using the schema from figure 2) the translation data can be referenced as en-t-fr-m0-postedit.csv#col=posteditedText and the post-editing timing information can be referenced as en-t-fr-m0-postedit.csv#col=timeToPostedit.

as the reference from the odrl:target attribute is a URL, one such declaration needs to be added for each CSV table which references that ODRL file, thereby complicating the management of the ODRL file.


using term extraction, named entity recognition, entity linking, part of speech tagging and word sense disambiguation techniques.

if the assertion and assignment of rights over these term-in-context annotations is not easily captured, then this inhibits attribution or compensation for use of this data between parties and disincentivises its capture as reusable datasets.


4 https://ec.europa.eu/jrc/en/language-technologies 5 http://rma.nwu.ac.za Figure 2: ISLRN submission process -During the edition process of the metadata, multiselection fields were presented in drop-down lists.

some of those lists, like the selection of languages were too painful to fill in as such.


Challenges derive from the highly international nature of an early career applicant pool.

a number of successes and even a few testimonials justify the expense of the program.


On the one hand, we approached the task of the annotation of figurative devices in order to evaluate in a real data set the suitability of a task known as very hard (Reyes and Rosso, 2014;Filatova, 2012;Reyes et al., 2013;Maynard and Greenwood, 2014).

we propose a semantic-oriented annotation assuming that it can give more precise hints about the conversational context, considering that often the meaning of a text varies according to the topic and, in the case of political debates, also according to the specific aspects discussed or to the author (e.g.


This suggests that INTROVERT-EXTRAVERT as well as THINKING-FEELING are predictable from linguistic input alone, while this is much less the case for the other two dimensions.

for languages for which we have fewer than 500 authors, namely Italian and German, the model usually does not outperform the majority baseline.


Creating large corpora for training supervised machinelearning models is hard because it requires time and money that may not be available.

since our dataset was used for disaster relief efforts, volunteers were willing to annotate it; this work can now be leveraged to improve text classification and language processing tasks.


Previous linguistic studies have looked at the structural and functional aspects of spoken and hence, small scale code-switched data.

with the huge amount of text available on social-media there is now an opportunity to study different aspects of this phenomenon on a large scale.


This is because, in other language, the former functions as a determiner.

the Japanese language does not have articles and the traditional Japanese grammar does not have the determiner word class.


Syntactic dependency types in Japanese are defined in order to be as in conformance with the principles of UD as possible.

the definition of Japanese syntax under UD involves several issues that should be discussed.


Coordination We take the first conjunct as the head in the coordinating construction in the fashion of the UD scheme.

because Japanese is a head final language, the last conjunct tends to be the head.


One of the most famous examples is zou wa hana ga nagai "For elephants, trunks are long."

7 this type is also used for fronted or postposed elements that do not fulfill the usual core grammatical relations; for example, the relation between "office" and "me" in the sentence "This is our office, me and Sam."


ADV PUNCT  DET  NOUN  VERB  DET  NOUN  ADP  DET  NOUN Figure 2: UD annotation for a French sentence.

(Translation: girls love chocolate desserts.)


Moreover, it should be possible to refine the analysis by adding language-specific subtypes of universal categories.

figure 2 uses the french sentence Toutefois, les filles adorent les desserts au chocolat (the girls love chocolate desserts) to exemplify the different UD annotation layers, which are described in more detail in the following sections.


If we try to directly convert word-based dependency to MWE-aware dependency, we need to combine nodes in an MWE into a single node.

this naive approach often leads to the following problem: A node derived from an MWE could have multiple heads and the whole dependency structure including MWE might be cyclic.


When the Taishō Revised Edition was produced in the 19th century, only 10,000 characters were available to the publishers and thus many substitutions of similar characters had to be made.

the digital version of the Tripiṭaka Koreana reproduced every glyph found in the blocks, making it more accurate for our purposes.


In this last annotation round, annotators were instructed to re-evaluate their sense labels.

they were asked to do so only for the instances on which they disagreed with the other annotator assigned to the same instance.


The average number of NOTA labels per word is 25, which seems reasonable taking into account the dataset size.


Even though we expected a high agreement on this word, due to its few and very well-distinguishable senses, we still find the perfect agreement rather surprising.

word pronaći (to find) was the most difficult one to annotate (IAA of 0.638).


Another disambiguation task focused on WordNet glosses was presented as part of the SensEval-3 workshop (Litkowski, 2004).

the best reported system obtained precision and recall figures below 70%, which arguably is not enough to provide high-quality senseannotated data for current state-of-the-art NLP systems.


We propose an unsupervised system for a variant of cross-lingual lexical substitution (CLLS) to be used in a reading scenario in computer-assisted language learning (CALL), in which single-word translations provided by a dictionary are ranked according to their appropriateness in context.

to most alternative systems, ours does not rely on either parallel corpora or machine translation systems, making it suitable for low-resource languages as the language to be learned.


As the similarity measure for the edge weights, we use simple co-occurrences based on the Spanish part of the annotated Wikicorpus (Reese et al., 2010), of less than 120 million tokens in size.

to the Nynorsk dataset, the Sem-Eval dataset consists of only one context sentence per test instance, which means we cannot make use of a large context window.


In a post-hoc analysis, we constructed our own ranked dictionary baseline, but cannot even come close to it in the best measure.

we see an improvement of 14.3% compared to this dictionary for mode 3, which, however, we cannot directly compare to the Sem-Eval systems.


The task consists in recommending relevant papers to be cited at a specific point in a draft scientific paper, and is universally framed as an information retrieval scenario.

in order to make these suggestions as useful and relevant as possible, we argue here that we need to apply a measure of understanding to the text of the draft paper.


Categorising the information status of noun phrases (NPs) or resolving anaphoric relations automatically requires annotated data for training and testing.

for scientific text, annotated data are scarce: to the best of our knowledge, there is no full-text scientific corpus annotated with both information status and anaphoric relations.


Since the possible set of connectives has been generated using a fixed set of known discourse connectives, it might seem redundant to include connective string as a feature.

including the connective string would allow the model to better understand the distribution of features with respect to each connective.


We argue that in case where no referential ambiguity is present in the context of an information seeking query sessions, the progression of discourse topic can be identified (and also annotated) with a set of simple heuristic rules.

in the case of referential ambiguity, which may be introduced by anaphora in followup queries, disambiguation can be achieved through automated anaphora resolution.


Our annotation layers cover syntactic cues, semantic relations, discourse entities and discourse topic development.

in this paper we only present the method to annotate discourse entities and topic development, and leave out syntactic and semantic categories, which have been commonly discussed in the literature.


Consequently, Topic can be characterized as given, whereas Focus as new discourse information.

what we identify as Topic or Focus changes whether we consider referential givenness/newness (e.g.


Due to the non-entailed character of modal auxiliaries, questions that include these elements do not primarily ask about when and how an activity took place, but whether it took/can take place at all.

from the point of view of the issuer's intention, which is of our primary interest here, the Topic of Q 1 shifts from the identity of the manufacturer to the activity of buying in Q 2 .


This is because the knowledge of the annotators increased and the guidelines for annotators were improved.

in the present study there are some tags where even expert annotators have struggled to detect them compared to others, such as the CLARIFYING (CLA) tag for both disciplines and, the ARGUING (ARG) tag for the economics discipline.


DAMSL is a de facto standard in dialogue analysis, due to its theoretical foundation (acts are annotated as context update operations), its genericity (high level classes allow for the annotation of a wide range of conversations types) and multidimensionality (each utterance can be annotated with several labels).

its dimensions are not discussed and lack conceptual significance.


"here's my question") is rare, unlike in forums and especially emails.

the EVAL-UATION ("OK let me see"), ATTENTION-PERCEPTION-INTERPRETATION ("I understand") and PSYCHOLOGICAL STATE ("I'm feeling good") dimensions are much more prevalent there, which shows that grounding as well as informational and emotional synchronization between participants is more important in synchronous conversations than in asynchronous ones.


As mentioned earlier, our definition does not distinguish between irony and sarcasm.

the annotation scheme allows to signal variants of verbal irony that are particularly harsh (i.e., carrying a mocking or ridiculing tone with the intention to hurt someone), as shown in Figure 4.


is the only group containing three languages.

we also noted that classifiers showed very high degree of confusion when discriminating between Bosnian and Croatian texts.


Compilation (and later annotation) of the ACC is an ongoing project.

version 1.0 of the corpus, with some 1,877,615 word tokens, and drawn exclusively from the Internet, already attempts to classify a wider range of children's genres than any of the above datasets.


Based on the tests conducted in WebBootCat, and the problems outlined above, we found both tool options for automatic corpus collection unsuitable for our purposes.

the 'Create Corpus' tool in SketchEngine (Kilgarriff et al 2014) enables researchers to gather texts in a range of different formats and then upload them into SketchEngine format.


Version 1.0 of our corpus classifies texts via two overarching categories: Fiction or Non-Fiction, and further specifies genres according to the following types (Figure 2).

figure 2: Genre classification scheme used in the ACC classifying texts in terms of their primary genre does not preclude further sub-categorisation.


for the RCV1 corpus contains valuable background information about the editorial processes at Reuters at the time the corpus was created, and most of their findings appear to be applicable to RCV2 as well.

while the RCV1 documentation is helpful, there are still many undocumented features in the corpus.


For example, one coder might well decide that the direction of the sales change is no longer sufficiently clear, as is the impact of the weather on the sales change, and, as a result, code this statement as neutral, that is as NEU, EXT, NEU under Sent_Tone, Attr, Attr_Tone.

another coder might well decide to make the ambiguity inherent in the statement explicit by marking this sentence as UNSURE, EXT, UNSURE.


Assessing OSS software quality has traditionally focused on analysing the source code behind the software to calculate quality indicators and metrics.

complimentary information about OSS quality can be extracted by analysing messages posted to communication channels (newsgroups, forums, mailing lists), and issue trackers supporting OSS projects.


Since the set of types is very relevant to the topic of this paper, it was taken into account when developing our novel hierarchy.

our novel hierarchy of content types is more detailed and contains content types that specifically describe the content of online messages related to OSS.


Assessing the quality of discussion threads has also been attempted (Kim and Beal, 2006).

very simple metrics, such as the number of messages, the length of messages and the number of responses, have been employed.


P, R and F can be computed as follows: For non-final types, we could average the F-measure scores associated with its descendants.

this would not take into account the actual number of annotations of each descendant class.


The best performing unigram frequency threshold is 1 or 2, achieving 70% accuracy.

using other thresholds can achieve similar accuracy.


Many supervised learning methods have been employed to solve this problem.

supervised methods require a large amount of labelled training data.


The classifier trained on selected hashtag dataset achieves comparable result with the manually annotated NLP&CC2013 dataset.

the recall the selected lexicon is lower.


Debole and Sebastiani (2005) analysed the complexity of the different subsets of the Reuters-21578 corpus in terms of the relative hardness of learning classifiers on the subcorpora, a strategy which does not assume monolinguality in the corpora.

they were only interested in the relative difficulty and give no measure of the complexity as such.


There are many ways to combine two (or several) information sources, in particular if they are independent; see, e.g., Genest and McConway (1990) for an overview.

p partially depends on max{t Li }, 4 which, for example, rules out the common logarithmic opinion poll: Instead we will use the linear opinion poll: Combining f m (x) and f p (x) gives a revised utterance level measure for N (x) > 0: where w m and w p are weights (w m + w p = 1).


In the summer term, an active usage of the LT in all seven lectures could be observed.

in three lectures this was only the case at the day of the presentation of the system and for a short amount of time (about 10 minutes).


We can conclude that the lists obtained with WER and ATENE are the most consistent with respect to the different systems, having a mean correlation above 0.8.

the metric In is the one that seems to be less consistent with a mean correlation lower than 0.7.


In order to avoid any misunderstanding, we position our work as an extrinsic detection, the aim of which is to find near-matches between texts, as opposed to intrinsic detection whose aim is to show that different parts of a presumably single-author text could not have been written by the same author [Stamatatos et al 2011a], [Stein et al 2011], [Bensalem et al 2014].

our main objective is to deal with the entry level of the detection.


In Table 1, we observe that simply using all terms of a story as a query to retrieve a ranked list of images does not produce satisfactory results, as can be seen from the low MAP and P @k values.

even a very simple approach of weighting the terms in the text of the story by their tfidf weights can produce a significant improvement in the results.


These methods are useful in order to arrive at an overview of the content of large corpora.

these techniques do not identify which predicates relate co-occurring elements with each other.


In the open RE task there is a great diversity of relations, which makes their classification more difficult.

for the extraction of pre-defined relations (closed RE), the CRF classifier only needs to learn specific types of relations.


Each output keyphrase is considered correct if it matches one of the reference keyphrases.

the choice of the appropriate textual unit (keyphrase) for a topic is sometimes subjective and evaluating by exact matching underestimates the performance.


High-quality corpora are extremely important for conducting humanities research in areas such as history, cultural studies, literary studies or linguistics.

to build such corpora, usually high manual effort is involved.


Previous work on event schema induction was evaluated on the MUC-4 corpus (Grishman and Sundheim, 1996).

this corpus raises two main issues: • It was annotated with templates describing all events with the same set of slots.


1 We also annotated entity coreference chains in documents.

only the entities appearing at least once as an event argument were annotated with coreference chains.


The additional experiments showed that in less than 2 % of cases a speaker does not pronounce the assimilated consonant instead of the original one, or vice versa pronounces it in the context where there is no assimilation process.

one would expect the increase of such mispronunciations in speech of non-native Russian speakers or in conversational speech where the mismatch rate between orthoepic transcription and real pronunciation is greater.


due to very frequent clipping in some speakers' recordings, defective fragments shorter than 100 ms were allowed.


Fortunately, the last several years have seen the release of phonetic/phonological databases that are much richer and have better coverage, such as Phoible 1 (Moran et al., 2014) and the dataset accompanying a recent comparison of phonological and genetics patterns of diversity (Creanza et al., 2015) curated by Merritt Ruhlen 2 .

such segment-level databases have a major drawback intrinsic to their design in that they cannot be directly used for analyses that require generalizations over classes of segments that share theoretically interesting features, such as "front rounded vowels", "retroflex stops" or "clicks".


Actually, we can not find similar artists information for artists with low degree on the last.fm's website.


a straightforward alternative would have been to type the correct transcription for each segment from-scratch.


Taking a closer look at the random fit for each transcriber ( Figure 5) reveals that the poor performance of FS − was largely attributed to the non-expert group CROWD, who were not able to beat the ASR output with this method.

note that even for several top transcribers, iterative interface design increased quality.


Given that from-scratch transcription required typing of all the words, its slower speed is not surprising.

in Section 2. we hypothesized an inherent overhead for post-editing due to verification and navigation between errors that need correction.


Moreover, FS + slightly but statistically significantly improves speed over FS − , possibly because displaying the ASR hypothesis helps the transcriber recall the words uttered in the audio faster.

bear in mind that most segments will have an edit rate much lower than 67%, meaning that the reduced typing effort of post-editing will usually outweigh its inherent overhead disadvantage.


(2013) show that respeaking by non-experts can be an option when ASR transcripts are to be improved, but not made perfect.

respeaking requires recording equipment, a quiet environment, and a clear speaker.


Even though only ER data can be released at the time of writing, for most of the users, the process to collect ID data for M03 and F02 is ongoing.

m02 ID data alone allows us to demonstrate the quality and the amount of data that are going to be recorded with the homeService system.


In these papers, spontaneous speech was found to be less intelligible and contained more disfluencies than read speech.

patients suffering from ALS present similar and even lower anomaly rates on spontaneous speech compared to reading (32% and 36% respectively).


At the end of the validation process, a total of 2,290 personalities constitute our speaker dictionary (Table 1 shows the detail).

it has to be kept in mind that, despite our efforts to reduce it, such dictionaries are by nature greatly imbalanced.


For instance by generating new queries: "find media where A speaks with B" or "get me contents where C talks about D", etc.

a boost of performances could be obtained by using multimodality to confirm, correct or invalidate the identity of detected speakers.


In chats, we observe no difference between the number of words by B-messages and the number of words by I-messages.

i-messages are significantly longer in the phone corpus.


Bavaria is one of 16 federal states in Germany.

it is important to distinguish the state of Bavaria ("Freistaat Bayern") and the Bavarian dialect, which is not per se identical: Not all inhabitants of Bavaria speak Bavarian dialect, and there are also speakers of Bavarian dialect outside of Bavaria, e.g.


These results suggest that DN is an easier task and as W1 has more true DN markables it could be expected that the W1 corpus would be annotated to a higher quality.

this is not the case due to the poor performance of interpretations of DO markables in the W1 corpus.


16  The nature of multiple coreference anaphora resolution that involves reflexive/reciprocal pronouns implies the use of syntactic parsing for finding the multiple antecedents, given that antecedents and pronoun should depend on the same head.

as the parser performance is not optimal in patents (e.g., it does not always detect correctly coordinations or the scope of the clause) (Burga et al., 2013), instead of relying on the resulting dependency tree, we use other criteria to establish the clause and, therefore, restrict the search: the system only evaluates cases in which there is at most one content verb between the pronoun and the antecedent candidate (with the candidates having the same structure as NP 1 and NP 2 in (12) and (13) above), and where there are no determined punctuation marks (":" and ";") between them.


Again, restricting the annotation scope allows for reducing the manual effort per document and thus for increasing the corpus size.

a dataset with all the nominal markables annotated provides material for training mention detection systems.


(2010b; are notable in this regard, proposing an approach using probabilistic finite automata and promising completely unsupervised extraction.

extraction is an exercise in semantic interpretation, which imposes a penalty in the form of domain-specificity.


We find that a SVM learned on the pairwise features on one type for all 6 data sets gives better results than the cosine similarity and different types of features classification.

feature addition, as used by Hagiwara, seems not always to be the best choice and only gives best results when the feature vectors are normalized before addition.


In 5 the arrival event is annotated as POS as it may still occur (conditioned on the accomplishment of the if clause).

in 6 it is annotated as IND because the event has eventually happened (or not happened), but that information is not deductible from the text.


This result evidences that the users in the target group are less fluent in their turns.

there are differences between fundamental frequency but their standard deviation is high.


1 We assume, that titles as found in the Wikipedia are often definition labels, which would help explain such a bias.

the term definition must be used with caution since Wikipedia articles are not all necessarily definitions in the strict sense.


12 This data is an approximation as other word classes are being dismissed partly due to set incompatibilities in the word class labels.

they present the main classes of content words, which would make them most numerous as opposed to function words.


Exemplarily, looking at both ends of the continuum, see Table 3, within the unknown words present as Wikipedia titles (UW) of the unknown nouns therefrom (N), the Stanford Tagger mistagged only few nouns (SMT).

uW contained some non-nouns (NN), the majority of which led to redirects, disambiguation pages or deleted pages leaving some errors through the method (WE) but excluding relatively many assignments.


We were not able to confirm that the participants did not simply choose color associations in a random fashion.

because the annotation results fit our hypothesis (see also Tables 2 and 3 in the next section), we think they are somewhat reliable.


He reported that the order of the most frequently associated colors was identical to the Berlin and Kay order (i.e., white, black, red, ...).

from the second row in Table 1, it is apparent that the order is different in Japanese (i.e., white, red, blue, ...).


To try to attract more annotators we increased the rewarding but this did not help in rising the number of contributors.

we decided not to decrease the threshold of accuracy in the TQs filter in order not to affect quality of the data.


Concerns have been expressed as to the quality of crowdsourced data, which some assess as part of a trade-off for speed and economy (Snow et al., 2008;Madnani et al., 2011;Ball, 2014), with others describing methods to filter out errors (Gadiraju et al., 2015;Schmidt et al., 2015), or indeed encouraging researchers to 'embrace' error (Jamison and Gurevych, 2015;Krishna et al., 2016).

as the old adage goes, you only get what you pay for.


We address this issue by assessing a sample of crowdsourced soundfiles in section 4.

aSR needs to, and does already deal with speech data captured in less-than-ideal recording environments (e.g.


Furthermore, we used the interface to calibrate our task for changing parameters such as the amount of time required to complete a rating task, and the desired accuracy level to derive the payment.

crowdsourcing is prone to spammers trying to get paid without performing the task.


The caveat has to be made that this is a pilot study, with a limited number of samples per class; the results will be reviewed and verified with larger databases and crowdsourced ratings collected in the future.

the results lend further weight to the assumption that crowdsourcing can be applied as a reliable annotation source for computational paralinguistics given a sufficient number of raters and suited measurements of their reliability.


Harada (1997) supported this theory and reported that the three different communication devices, i.e., video conference, telephone and text message (e-mail and chat), changed the speaker's subjective evaluation on online communication using these devices.

dennis and Kinney (1998) concluded that the new media (i.e.


A positive correlation implies that the listener accurately understood the speaker's emotions throughout the dialog.

a negative correlation implies that the listener understood the speaker's emotions to be the opposite of what they actually were.


This implies that they did not experience difficulty in delivering their message in either method of communication.

when the binomial test was performed using the data grouped by chat type, the test revealed a marginally significant difference between TX and FaceToFace (p < 0.08, the right panel in Fig.


Those emotional words and emoticons can be helpful for the speaker's understanding of the partner's pleasantness.

arousal and dominance were considered to be difficult to express with words and emoticons.


Laughter can be conceived of as a non-linguistic or paralinguistic event.

with the amount of interest for the conversational speech data, laughter appears as an essential component of human spoken interaction (Glenn, 2003).


Laughter can appear in earnest context and function as nervous social or polite cue among others.

we can suppose that laughter related to humorous context will be frequent in our corpus.


These instructions have been specially selected to elicit humor in the dialogs.

they were not exhaustive and participants often spoke very freely about other topics, in a conversational speaking style.


For disgust, the mean RMS energy values seem to be higher for higher levels.

this is less obvious in the case of startle and surprise.


With the monologue, we gather prominent, simple, and isolated emotional speech, useful for preliminary multimodal emotion research.

the dialogues are to provide less stereotypical, more subtle, and contextualized emotion, counteracting the shortcomings of typical emotion portrayal.


the feeling of joy is indicated by positive valence while fear is negative.

arousal measures the activity of emotion; e.g.


ja: Then it was reported in India, Kenya, Tanzania, Malawi, Uganda, and     Current lexicon-based algorithms also require the input to be segmented into words (Ma, 2006).

like sentence segmentation, word segmentation is not a trivial issue especially for Asian languages like Chinese or Japanese with no explicit word boundaries (spaces), and matching rate between the input words and lexicon entries decreases if different segmentation rules are used.


Many SMT based systems are evaluated in terms of the information gained from the word alignment results.

there is not a lot of parallel data available for these languages making it necessary for specialized techniques that improve alignment quality has been felt (Sanchis and Sánchez, 2008;Lee et al., 2006;Koehn et al., 2007).


Their results show that this method improved precision without loss of recall in English to German alignments.

if the same unit is aligned to two different target units, this method is unlikely to make a selection.


The line representing the Cartesian product approach clearly shows the degradation of MT output for English -Hindi.

the sentential approach shown minor improvements for a varied number of topic models.


since there was no pre-existing resource for Japanese-Chinese, ASPEC-JC was constructed by manually translating the Japanese documents into Chinese from scratch.


The basic idea is that indomain training data can be exploited to adapt all components of an already developed system.

previous work showed small performance gains after adaptation using limited in-domain bilingual data (Bertoldi and Federico (2009), Daumé III and Jagarlamudi (2011)).


This way, articles are grouped by categories and the category hierarchy forms a graph.

many articles are not associated to the categories they should belong to.


We also checked whether among the correctly classified pairs there were similar low frequent words, and indeed it was not the case.

for some other errors the explanation is less obvious.


(1991) and Gale and Church (1993) (the former measures sentence length in terms of tokens and the latter in terms of characters).

for texts available in less-friendly formats, such as PDF, from which we cannot avoid extracting some noise intermixed with the text (such as figure and table captions, page headers and footers, etc) we need more robust aligners that take into account the actual text within sentences and not only their lengths.