This process can be speeded up with the use of an automatic speech recognizer.

cORILGA repository has been enhanced with an automatic alignment tool, available to the administrator of the software repository that aligns speech with an orthographic or a phonetic transcription.


Defining the partial transcription as the target text, the perplexity reached for a mixture model would be similar if the objective would be the rest of the file.

the process for obtaining the best mixtures starts giving initial weights for the input models, and using and iterative algorithm to find the best perplexity and the best weights combination.


The interface was designed to allow searches for targeted groups of speakers, separated by age, sex, education level, or L1 (Galician or Spanish), targeted genres and types of text (formal or informal, conversation, talk, media, reading, among others) and the year (or group of years) when the file was recorded.

that will facilitate not only researches on language variation between speakers (by urban/rural context, sex, age or education level) and between language varieties (by place, formal/informal, oral/written language, genre and type of text), but studies on language change as well, both in apparent (comparing different generations at a given time) and in real time (comparing the same age groups recorded with a distance of decades).


For that, we need to account for the number of supports provided for each of the nodes.

given The two concepts of divisiveness presented here allow for automated identification of divisive issues in the corpus of argument networks.


Even in the cases where some kind of annotation was performed, they are not freely available and/or are comprised of a limited number of examples used only to support the category organization decisions.

it is also important to look at approaches that represent extensive annotation efforts.


The assumption here is that the more information a country exports to another the more influence it has over that country.

the important geolocation information is which country the Twitter users were from: we needed the location that the users consider themselves to either be living in, or to be their origin, instead of the location they happened to occupy when their tweet or retweet was sent.


An example of neutral stance is shown below: Target: Hillary Clinton Tweet: Hillary Clinton has some strengths and some weaknesses, I could vote either way come election day.

in our work we obtain manual annotations for 'favor', 'against', 'neutral', and 'no stance', but later combine 'neutral' and 'no stance' into one category 'neither' (neither favor nor against) since less than 0.1% of the data received the 'neutral' label.


As mentioned above in reference to Property 1, one of the challenges with creating good data for stance detection is that human annotators can find it difficult to determine stance if they do not understand the domain or the relationships between relevant entities.

we chose the targeted search option as this allowed us to focus on only those targets that are widely known.


Stance can be expressed in many different ways, for example by explicitly supporting or opposing the target, by supporting an entity aligned with or opposed to the target, by re-tweeting somebody else's tweet, etc.

after a few rounds of internal development and pilot annotations, we presented the questionnaire shown below to the annotators.


For example, knowing that entity X is an adversary of entity Y can be useful in detecting stance towards Y in tweets that mention X (and not Y ).

the stance dataset and the associated classification task can be used for developing and evaluating automatic approaches for relationship extraction.


Stance detection can be thought of as a textual inference or textual entailment task, where the goal is to determine whether the favorability of the target is entailed by the tweet.

the dataset can be used for developing textual inference engines and open domain reasoning.


However, relevant information can be redundant as well.

we search for novel information in terms of new content and new subjective perspectives such as sentiments in order to track the way the sentiment changes across a given mention.


The theory of priming applies to any perceptual entities regardless of modality.

the use of lexical and visual features as priming cues is cognitively valid (Stenberg et al., 1995).


Annotators would consider the roles laid out in make-01 or offer-01, and decide which were more fitting of the The verb annotation in (9) does not capture any of the semantics of the offer event, instead focusing on make as if it were a heavy, semantically rich verb.

the argument "to acquire PS of New Hampshire" lacks an appropriate tag in the roleset.


The word 'stuff' has different meanings depending on the context.

it can be labeled correctly with multiple supersenses.


Annotation procedures could be greatly improved if the annotators were shown all other instances of the same type that have already been annotated, warning of a possible inconsistency.

it would be possible to increase consistency during the annotation process and not only on completely annotated corpora.


nodes with in-degree zero; in this sense, the majority of graphs are multi-rooted in all SDP graph banks.

our format designates one (or more) top node(s) per graph to reflect notions like semantic headedness, top-level focus, or generally the most central semantic entities in the graph.


For example, if a third singular (3SG) person and number inflection is marked by an ascending tone in SJQ, here annotated as 42 (see table 1), the same 3SG will fit into the range of tone 32 in another Eastern Chatino dialect (as noted in section 2.3. below, table 1 shows the tone registers with 1 as the label for the relatively highest tone).

phonetic tonal differences may lead to contradictory interpretation of meaning across different dialects.


Given that Chatino is an extremely tonal language, no research could be done without a good understanding of these tone patterns (Cruz and Woodbury, 2014).

a large part of the research on SJQ Chatino has been on phonological patterns of tones.


The availability of relevant corpora characterizing home environments is of fundamental importance to support research on the area of multi-channel processing for smarthome applications.

some related data collection efforts have been conducted in the past.


However, the recall clearly decreases for the more challenging acoustic environments.

it seems that the distant noisy and reverberant acoustic environment smooths the speech events in such a way that the probability of matching the background model increases for more challenging acoustic conditions.


Nevertheless, it is worth noting that, although inaccurate in some acoustic conditions, most of the read commands have been retrieved by the SAD module: more than 90% and 98% are detected in the simulated and real test sets, respectively.

the impact of missed detected commands in the aggregated WER performance is very limited in the Re-alCorpus test set (from 17.46% to 18.49%), while it is more relevant in the SimCorpus case (from 13.69% to 21.03%).


One of the most remarkable characteristics of the complete segmentation, channel selection and speech recognition pipeline is that it is considerably robust to different home environment characteristics.

the system performed remarkably well in real data collected in a totally different acoustic environment to that used for the development of the systems in most of the acoustic conditions.


Keeping a number of copies of the same form would not only be inefficient, but also hard to maintain: if there is an error in one, all of them would have to be corrected.

in TEITOK there is an inheritance hierarchy for the different forms, where (for instance) in those cases where there is no explicit normalized form, it is assumed to be identical to the written form.


But doing so is very slow and labour intensive.

it is also possible in TEITOK to run a CQP query, and correct all results directly.


Uysal and Gunal (2014) show that choosing the appropriate preprocessing tasks significantly improves classification accuracy.

in any NLP research project, experimenting with different preprocessing schemes is an important component of the research space.


It was found that all the repetitions in them were reduced into a single letter.

we decided to put the 29 (7%) cases in a lookup table and reduce all the other repetitions into a single letter.


It is worth highlighting that in reality the training data is incomplete, in the sense that it lacks alignment information.

we introduce alignments between syntactic patterns in adjacent sentences as a latent variable.


We also expect the bias towards certain CoreSC concepts to be low as the priority ranking was chosen according to their occurrence in text, meaning that rarely occurring CoreSC concepts possess a high priority.

using the label priority in the case of disagreement, favours the selection of rarely occurring CoreSC concept, but there will only be a small number where this rule needs to be applied due to the low occurrence of these concepts.


In this case, the perceiver is considering "could the agent have been forced to carry out an action" or "was the agent tricked into carrying out an action".

coercion covers not only cases where the agent was forced but also where the agent was tricked to execute an action.


On the off chance that an entity does not have these requirements, it is exempted from blame (Tognazzini et al, 2014).

we use named entity recognition focusing on entities of persons, organisation and country.


The agent "I" in this case did not willingly quite the job and the sentence does not mention who forced the agent.

the sentence is classified as "Others" (i.e., no blame or praise).


Our approach relies on results generated from a series of NLP tasks such as POS tagging, word-sense disambiguation, dependency parsing and polarity detection in order to be able to assign values to a set of variables for blame/praise detection.

any error that occurs will be propagated down the pipeline process.


Word embeddings-being an unsupervised technique-cannot be easily evaluated without comparing performance in some downstream task.

we present two novel benchmarking tasks of our own making: a relation identification task analogous to previous evaluations on English, in which the quality of different kinds of word embeddings is measured, and a dialect identification task which measures the usefulness of word embeddings as a linguistic resource for Dutch in particular.


This is because it only measures predefined textual features found in literature on irony and humor detection.

one does not have to worry as much to have perfectly clean data, which is beneficial when working with web scraped data.


The properties of the Russian subset limited the comparison to hyper-, hypo-, holo-, and meronyms.

of those 850 relations, 596 relations could be compared due to their relation type.


It could be assumed that WordNet is the most representative manual knowledge base, based on its pioneering role and superiority in size.

we further assume that it most representatively shows the gaps in this kind of knowledge base.


As discussed earlier, classic semantic relations play a role in the linguistic encoding of knowledge.

tasks that have the aim to extract knowledge would benefit from an automation of the annotation effort discussed herein.


(Benajiba and Diab, 2010;Diab et al., 2010), DATOOL (Tratz et al., 2013), DIWAN (Al-Shargi and Rambow, 2015), among others, and have been successful in serving different annotation tasks, yet they were either not available or did not suit our needs exactly.

we have built a custom webbased annotation tool.


We chose to use vocabulary lists instead of parallel sentences, as the latter allow for too many degrees of freedom.

we collected and aligned parallel Slavic word lists for the two language pairs.


Note that description lengths are, by Shannon's source coding theorem (Shannon, 2001), log-likelihoods.

two-part MDL can be thought of as a regularized maximum likelihood approach.


An alphabet here is simply a collection of enumerated symbols, the exact identities of which are irrelevant.

alphabets are specified simply via their lengths.


In our previous experiment, we originally formulated 103 unique diachronically-based orthographic correspondences for CS-PL and 77 unique correspondences for BG-RU, including equal-to-equal correspondences (e.g., (y:y), (u:u) ).

only 48 correspondences were applied on parallel word lists for the BG-RU mapping, and 81 for the CS-PL mapping.


However, derivational and inflectional aspects may interfere with each other on the surface of words, and even orthography may play a role in the exact manifestation of morphological elements.

oftentimes, inflectional and derivational aspects have to be considered jointly in order to formulate correspondences based on morphological features.


Part-of-speech tagging is an important preprocessing step in natural language processing.

there are many implementations available such as the Stanford tagger (Toutanova et al., 2003), the TreeTagger (Schmid, 1994), or the ClearNLP tagger (Choi and Palmer, 2012).


In 2015 we set the bar to 500 thousand tokens, mostly because of our results on morphosyntactic annotation of Slovene (Ljubešić and Erjavec, 2016) which showed corpus supervision to be of much greater importance than lexicon supervision.

the second phase of corpus construction consisted of manually selecting 320k tokens of suitable documents from the hrWaC web corpus and having experts do correction of automated morphosyntactic annotation learned from the 190k-sized corpus.


One characteristic of TGermaCorp is the composition of its primary sources: TGermaCorp is designed in view of capturing the lexical and morpho-syntactic varieties of written German as exhibited in German-speaking literature.

tGermaCorp complements corpora that address the homogeneous style of mainly contemporary German (e.g., newspaper texts) like tIGER (tIGER project, 2003), (Brants et al., 2004), DeReKo (DEREKO project, 1999), (Dipper et al., 2002), or the Huge German Corpus (Stuttgart, 2010), (Schiller et al., 1999), as well as individual language resources like the Kant Korpus (Schmitz and Stark, 2008) (Lenders and Schmitz, 2007).


For syntactical analysis, we converted the TreeTagger output of the corpora to the CoNLL 2009 format 12 and parsed the result using the latest version of the MALT parser 13 .

if there is some noise induced by the preprocessing procedure utilising the specific tools mentioned, all corpora should be affected in a similar way.


As a basis of our implementation, we chose Pylearn2 (Goodfellow et al., 2013), which is a machine learning library for conducting scientific experiments built upon Theano (Bergstra et al., 2010;Bastien et al., 2012).

we get all the benefits of Theano; the automatic calculation of the gradients will be optimized and stabilized, and the same code can be compiled to GPU if preferred to CPU.


Note that recognizing multiword tokens is quite different from tokenization -the multiword token is usually not just a simple string concatenation.

the described tokenizer does not handle multiword tokens.


Usually E can be considerably smaller than G, because the most common form analyses are likely present in the dictionary itself.

most forms having E instead of G E analyses speeds up the decoding substantially, by a factor of (G/E) 3 , because Viterbi decoder of order 3 is used (each state consist of a pair of tags).


5, the words for dog and computer can be elided if the context and the semantic features of the CLs are enough to resolve the anaphoric reference.

though often redundant, CLs add to the semantic content of noun-phrases they are quantifying.


While the exemplar sentences are annotated with syntactic information, we do not rely on this structure at the semantic annotation stage, as this information is added automatically, and is not reliable enough.

our annotators must select spans of text to fill the various roles encoded in the semantic frame definition, as opposed to annotating syntactic structures, a method employed by some other projects (Erk et al., 2003;Candito et al., 2014).


For example, the sentences "I come from London", "I come fro London" and "I come frcm London" all have different words preceding the location "London" for the tool, although they are all (made up) variants of the word "from".

the tool would benefit if these variations would be diminished, and this is what the spelling variation reduction module intends to do.


Some occurrences are terminological and other are not.

a global decision at the corpus level about the terminological status of all occurrences of a lexical unit would then be erroneous.


Indeed, ATA tools extract T C considering criteria and indices computed over a whole corpus.

they take a global decision for the T C. If a string is identified as a T C, all its occurrences are considered as T Os.


For each occurrence of a T C, the expert should answer if: (1) it is syntactically well-formed, (2) it belongs to the scientific lexicon (3) it belongs to the domain lexicon (here linguistics), (4) it is a T O.


Formally, this theory is based on the triple K = (G, M, I) called formal context, where G is a set of objects, M is a set of attributes and I is the the binary relation I ⊆ G×M between objects and attributes.

(g, m) ∈ I means that g has the attribute m. For instance, occurrences of the introductory examples with the T C Aspect are encoded in the formal context given by Table 1.


This may affect computation and analysis of the recall scores.

we propose two definitions of False Negatives (F N ).


We observed that a combination of our three methods of annotation that roughly favours T O annotations will pull down precision very close to the worst precision of the three methods and will provide a very low improvement of recall.

association rules could probably suggest a better combination of these three methods.


Objectivity: the metrical pattern represents only metrical information that could be derived from the text objectively.

on one hand, the pattern is formed only by two kinds of syllables: stressed and unstressed.


While it is rather straightforward to think of linguistic usage scenarios with regard to linguistic TEI encoding, the benefit of layout and conceptual text structure annotation for linguistic research may not always be evident.

with the DTA corpus at hand, this paper provides examples of linguistic research questions where TEI encoding is a either involved or even a crucial factor.


It was reduced to the time span 1600 to 1900, because this has been the scope of the DTA project and there are only sporadic texts older or younger than that in the DTA corpus.

the current study is based on a total number of 2,649 works (183,377,535 tokens) dating back to the 17 th to 19 th century.


5 Interoperability in this context does not only comprise homogeneity of quality and transcription practice but also homogeneity of linguistic, conceptual and layout tagging.

the creation of an annotation format for the DTA which is fully compliant to the TEI Guidelines but reduces the TEI tagset and adds specifications on attribute and value level in order to prevent not only ambiguous tagging but also slips and lapses, was an important task.


it is possible to annotate proper nouns as well as foreign terms in a DTABf conformant way even though the DTA core corpus does not contain consistent annotation of these phenomena.


for the name 'berlin', the resource dbpedia:Berlin (Germany) is much more 'talked about' than Berlin, New Hampshire (USA).

we also take into account estimates of prominence and dominance as: Prominence Let the true prominence of a resource r i be the percentage of other resources r k ∈ R, which are less known than r i .


11 The figure illustrates that all investigated benchmarks favor entities that are much more popular than average entities in DBpedia.

the benchmarks show a considerable bias towards head entities.


SCT is also faster than LSH in clustering the tweets mainly because it operates on words rather than messages.

sCT represents an effective approach to perform real time event detection in Twitter.


We observe that upon increasing the number of clusters, the precision metric improves while recall degrades.

one can trade off recall for a better precision by choosing a larger number of clusters.


Many AL strategies that may yield theoretically nearoptimal sample selection are in practice infeasible for use because of excessively high computation times (Cohn et al., 1996).

aL-based annotation should be based on a computationally tractable selection strategy (even if this may result in a less than optimal reduction of annotation costs) (Tomanek and Wermter, 2007).


Through reported speech, one gives voice to different characters both in a piece of news or in fiction.

both the presence and the absence of particular characters indicate choices by the text author.


We revised the new verb list, looking at each verb with a frequency greater than one.

we produced a list with patterns and 285 reported speech verbs associated to it.


The annotation was completed in batches of two.

to remove the differences, we calculated agreement in those batches.


As pointed out in Section 3.1., most conversations are conducted at homes (35%) or workplaces/schools (30%).

we classify places into three categories: i) private places, ii) public places, and iii) the others.


Because data that would be otherwise stored together in an XML file is scattered across several tables, some straightforward queries may involve several joins.

for instance, to find sentences in a topic area, one must join a clutch of tables together (posts, discussions, topics, texts, and sentences).


While it is tempting to use some of the rich resources currently available, it is unclear that results obtained from them would apply to structurally different casual talk.

we have undertaken a programme of collection, annotation, and analysis of multiparty casual talk data.


Discussion referred to stretches of talk shared more or less evenly among two or more participants throughout the bout, while dominated referred to bouts largely dominated by one participant.

discussion and dominated correspond to Slade and Eggins' concepts of chat and chunk as discussed above.


As we have seen in the previous sections, MT services perform a series of automatic operations on input data which are far from being a simple word-for-word recoding.

it is beyond doubt that MT qualifies as 'processing' of data.


:133075818 rdf:type owl:Class ; owl:equivalentClass :133075826 , :133075850 ; rdfs:subClassOf :MorphosyntacticCategory .

• The BLL Linking Models (currently a Linking Model to the OLiA Reference Model using the rdfs:subClassOf property is under construction) we preserve the original BLL structure (skos:broader), and its ontological model (BLL Ontology), and clearly separate both from their interpretation in terms of OLiA (etc.).


These functions cannot be replaced by cloud services.

the use of resources and services on mobile devices as well as resources and services in the cloud is essential.


These services run continuously to detect events and have their own life cycles whereas normal services work once when invoked.

the composite services that contain the monitoring function must be run constantly.


Moreover, the text-to-speech service on Android devices does not need the output parameter (because the speech can go directly through the speaker) whereas the text-to-speech service in Language Grid requires the speech as the output parameter.

service Adapter has to deal with two differences (the difference in service names and the difference in service input and output parameters).


1 For these reasons, the methodological and technological boundaries existing in each linguistic project need to be overcome in order to find common grounds where linguistic material can be shared and re-used over a long period of time.

a possibly standardized methodology for designing linguistic databases is necessary to develop linguistic resources that fully meet the desiderata of the Strategic Agenda.


This might be due to little or no real interaction between these sub-disciplines.

we assume that most language material from documentation projects ends up in digital language archives without transcription or any kind of linguistic analysis or annotation.


This can only be understood as an estimate that provides a basic transcription and maybe also part-of-speech tagging and translation.

it is prohibitively expensive to transcribe these resources and thus most of the already collected recordings are not transcribed or analyzed at all, while more and more raw audio and video material is being collected or made available via various channels.


Often enough documentary linguists restricted access to their collections themselves.

changes and negotiations of access restrictions for archived language resources are most often impossible to achieve.


XML-based annotation standards or common open formats like Praat's TextGrid, RDF, TEI) and freely available online.

the resources 17 See http://www.indiana.edu/~libarchm/.


For all languages that we tested, our goal was to create initial corpora and language models that can be potentially used in the transcription and annotation process of larger resource collections.

most corpora were not created using existing data from fieldwork or documentation projects.


The original recordings in the AHEYM collection (Kerler, 2014), (Cavar et al., 2016b) are only partially useful to serve as training corpora for speech technologies due to limitations of the recordings done in the field, as described above.

we used a small sub-set of this material together with various freely available audio-books read in Yiddish.


In the process of exploring common technologies for tokenization and morphological analysis we also evaluated the processes to generate models for under-resourced languages that would enable us to increase the quantity of corpora.

in addition to the audio material and speech corpora, we developed Finite State Tools using the Foma toolkit and libraries (Hulden, 2009).


Additionally, all tools have been built to be able to integrate existing solutions: Pepper together with Salt facilitates not only the integration of existing corpus tools with corpus-tools.org software, it also assists collaboration among different third-party tools; both Atomic and ANNIS have been built to be extensible via plugins.

tried and tested annotation editor types or NLP components can be added to Atomic, established visualization types to ANNIS.


Syntactic annotations, for example, will usually affect only one sentence.

each sentence can be a discrete partition.


Pepper utilises an intermediate-model approach, with Salt as its intermediate model: Instead of implementing a direct conversion solution from format X to format Y, X is mapped onto Salt, manipulated where necessary, and subsequently mapped onto format Y.

the number of mappings to convert n into m formats is reduced to 2n, as compared to n 2 −n mappings for direct conversion routes.


Atomic is basically a set of plugins, adding to the set of plugins making up the Eclipse RCP.

new functionality can be implemented as yet another plugin and added to Atomic.


Any corpus processing step, e.g., parsers, taggers, etc., can thus be implemented as an Eclipse plugin and added to Atomic dynamically via the respective extension point.

atomic is in principle an annotation platform rather than simply an annotation tool.


A tool is deployed which automates the process of downloading both the COCOA files and the original CommonCrawl data and recreating a full corpus from both sources on the end user side.

no copyrighted data are actually redistributed by the COW/ COCO corpus creators.


The process of reconstructing a COCO corpus requires some effort, and the end users referred to in this paper are most likely predominantly institutions and research groups with well equipped machines and a high download bandwidth rather than individuals.

i hope that providers in countries with more permissive legislation use the tools described here to reconstruct COCO corpora and make them accessible (ideally also in convenient interfaces) while respecting the CC-BY license for the annotations and giving the COCO/COW project the required credit by referencing this paper.


In order to ensure robustness and integrity of the annotation data, in cases such as internet connectivity failures or Web browser crashes, the annotation tool is able to recover the annotation status just before the connection failure or the crash: When a user begins an annotation session on a document, the current annotated statues of the document is also maintained on the server automatically, even between saves of the document (on a separate, "shadow" cache of the document).

the server always maintains the last saved state of the document, but also the current state, shared among all annotators currently annotating the document.


Being an important element of human communication, its use is very frequent in texts, as a means to convey meaning to the reader.

argumentation has attracted significant research focus from many disciplines, ranging from philosophy to artificial intelligence.


In batch processing all the cluster nodes are busy processing a large set of documents, whereas in a streaming processing scenario resources may be idle, waiting for documents to arrive.

streaming computing focuses on maximizing resource usage for each document minimizing document latency.


While the Alveo API provides some facilities for accessing larger collections of documents it does not yet provide a good interface for large scale data processing.

doing any computational work on the entire Trove corpus or even on significant subsets is not efficient using the Alveo (or Trove) web APIs.


In this paper we describe the new developments brought to LRE Map, especially in terms of the user interface of the Web application, of the searching of the information therein, and of the data model updates.

users now have several new search facilities, such as faceted search and fuzzy textual search, they can now register, log in and store search bookmarks for further perusal.


Now, fuzzy full-text search looks through the normalized data first, backing off to original data when no results are retrieved based on the normalized data, and retrieves the results sorted by their relevance, computed according to the TF-IDF of the terms of the query with respect to the documents being searched.

full-text search works at the same time for people typing e.g.


Backed by the Lucene 4 text search engine library, Elasticsearch is one of the most powerful and customizable (via a JSON RESTful API) search engines available.

in order to harness its power, a special NLP-oriented configuration has been chosen for Elasticsearch.


The interesting part of the Elasticsearch analyzer configuration resides in the ability to define a pipeline of processing elements (called filters in the Elasticsearch parlance).

several filters have been piped together in the Elasticsearch LRE Map fuzzy-search configuration: 1.


Last but not least, Elasticsearch allows us to control the fuzzy search robustness in a fine-grained manner.

also by trial and error, we have been able to set the tolerance threshold on the Levenshtein-Damerau similarity distance to 2, so that the distance between two "identical" strings is 2 at most, i.e.


Meanwhile, the French Ministry of Culture continues its actions related to linguistic diversity and technology, in particular through a project with Wikimedia France related to contributions to Wikipedia in regional languages, the upcoming new version of the "Corpus de la Parole" and the reinforcement of the DGLFLF's Observatory of Linguistic Practices.

of a partnership with the Délégation générale à la langue française et aux langues de France (DGLFLF, French Ministry of Culture and Communication) 1 , the Evaluations and Language resources Distribution Agency (ELDA) 2 conducted an inventory of language resources currently existing for the regional languages of France throughout mainland France and French overseas departments and territories (Leixa et al., 2014).


Data are typically segments of broadcast and telephone conversations audited for the linguistic variety spoken, speaker number and sex, and sound quality.

the 'success' criteria deal with the availability, in the variety of interest, of the desired data types and native speakers capable of the annotation.


83.33% = 0.833) and considered for our task only sentences for which the difference between L1 and L2 was less than 0.30 5 .

11 sentences were eliminated from the dataset (49 sentences left).


Sinica Corpus -which is the base of CWN -was constructed from Taiwanese Mandarin language sources.

variances in terminology such as, 北市 (beishi, North city) vs. 台北市 (taibei shi, Taipei city) did not fit the background of our raters, whom were all from Mainland China.


This problem was encountered in both Hindi and Urdu data-sets as well.

we adopted a 2-stage approach of first identifying the arguments and subsequently classifying the identified semantic arguments into one of the semantic labels.


The high percentage of precision and recall in the Argument Classification task can be accounted from the fact that dependency structure and predicate-argument structure are somewhat similar structures and dependency labels and propbank labels are similar in orientation.

using dependency structures as features to predict and classify the arguments of a predicate of Hindi and Urdu is very influential.


Aspect encoding through lexical derivation is thus highly ambiguous and inconsistent, presenting numerous challenges for generalisation.

slavic verb aspect is usually considered to be highly complex, involving an interaction of semantic and formal categories that has not yet been fully understood.


That is, because we are only discussing objects when situated in a minimal embedding space (even an otherwise empty one) wherein all coordinates are given Cartesian values, the axis of rotational symmetry or plane of reflectional symmetry are those denoted in the world, not of the object.

a tetrahedron-which in isolation has seven axes of rotational symmetry, no two of which are orthogonal-when placed in the MES such that it cognitively satisfies all "real-world" constraints, must be situated with one base downward (a tetrahedron placed any other way will fall over).


This method was evaluated by comparing it with two baseline methods based on previous studies on the Japanese language that used case frames and co-occurrence information.

the proposed method exhibited significantly better accuracy (0.85) of determining words as metonymic or literal expressions than the baselines.


'Far' means that the sum of linked nodes was more than 8.

in this learning phase, the proposed method extracts the attributes of decision tree learning in Table 5 and determines each noun as 'Metonymic' or 'Literal' with the decision tree.


On the other hand, the two baselines correctly determined 66 and 69 cases, respectively.

the proposed method exhibited higher accuracy (0.85) than the two baselines.


In other words, each is an accuracy of the first correct word in the top N. As shown on the table, the proposed method expressed higher accuracies than the baselines.

the proposed method had a higher possibility of being used to interpret metonymic expressions than the baseline methods.


These narratives reflect domain-specific medical knowledge communicated by physicians for the interpretation of other medical professionals.

they reflect highly-specialized medical knowledge, both in explicit and implicit ways.


Because we considered lexical relations between concepts, rather than individual words, the resulting knowledge graph was quite large, containing 32,522,807 vertices and 52,209,411 edges.

we relied on the Apache Spark (Zaharia et al., 2010) web-scale data processing engine to generate a distributed knowledge graph.


Evaluating the quality of any common-sense knowledge base is known to be difficult (Singh et al., 2002).

we measured two aspects of our knowledge embedding: (1) the quality of the relations in the knowledge graph, and (2) the quality of any new inferred relations in the embedded space.


The part of the formulation of the problem for finding word embeddings is very similar to the continuous bag of words model (Mikolov et al., 2013), with the main difference that a context document is added.

the resulting word vector representations u can naturally be used for distance computations by computing the cosine similarity or Euclidean distance between a pair of embeddings.


Secondly, despite the rather simple rules, open world video games provide enough liberty for players, and their popularity attracts people to ask many questions about them, including creative and fun questions that can be solved only by completely understanding the rules and logically combining them.

we expect the domain to be interesting and challenging as well.


One of the main challenges is the constant evolution of Wikipedia: articles (or their spoken versions) are added, article text is revised, meta-information is updated or erroneously breaks for a multitude of reasons.

our solution must not rely on manual corrections of the ex-tracted data.


Some words in the data yields dialect specific phenomena concerning the utilization of an alternative pronunciation.

different official orthographic forms of a certain word were applied.


As a result, it has been reported that deterioration in speech recognition accuracy with elderly users is caused by the mismatch between acoustic models and the acoustic characteristics of elderly speech (Anderson, 1999;Baba, 2001;Vipperla, 2008).

it is important to construct an acoustic model which takes into account the characteristics of elderly speech in order to improve the speech recognition accuracy of speech applications for the elderly.


However, in the case of the Japanese language, although duration of one mora is almost constant, the duration of speech changes slightly depending on the position of the phoneme (Ota, 2003).

in order to accurately calculate speech duration, it will be necessary to precisely calculate the duration of each mora.


Additionally, consecutive V or C segments are merged into one V or C interval.

a string of CVCVCV... intervals with time-stamps is created for each recording, and can be used as input for the Category PVI and Duration Intervals plugins.


Our technologies and resources can potentially be used to make these resources accessible and searchable.

the project will lead to valorization of archives' existing holdings and rectifying the problem of these resources being under-used.


Belonging to the public domain, most of the texts are old and not available publicly in foreign language.

the novels that were collected in foreign language are mostly novels from world's classics.


It is worth noting that the SMT results were obtained with domain adaptation techniques that have been extensively researched and time-tested over the years.

it is unlikely that different domain adaptation methods for statistical machine translation would provide significant gains overall, which in turn places domainadapted NMT as the currently optimal approach in terms of automated metrics.


Examining post-editing edit distances, shown in Figure 2(a), confirms the ranking of the systems in this domain.

in the case of GT more than 37% of the segments had an edit distance of 0 and 22% an edit distance of 1.


Finally, for the ELEV domain all positive attitude indicators increased, except for quality of post-edited texts, which maintained the same score, and boredom, which also increased.

after completion of the task, the perception of quality of machine translation increased, as did its perceived usefulness and the interest in using it as input.


As shown in Table 7, there are relevant correlations between segment-level BLEU and the human assessments for fluency, ranking and WORK, particularly in the ELEV domain.

the higher the fluency and ranking results and the lower the WORK, the higher the BLEU metric obtained.


Considering the different domains selected for the experiments, their specificity as well as the amount of available in-domain data, the results were not unexpected but the experiments performed provide a quantified view of the impact of domain adaptation.

for the two domains that were more specific, domain-adapted systems in one form or another provided clear advantages that are reflected in all metrics, automated, based on human subjective evaluation, or based on objective post-editing metrics.


In NMT, as described in Equation 3, computing the probability involves mapping the hidden state vector to a vector with the dimension of the vocabulary size.

to make computation tractable, the NMT vocabulary size is limited.


The compositionality assumption leads to literal translation errors, the word-byword translation of idioms, resulting in a translation that is confusing and not understandable.

idiom translation is a hard problem in MT and has attracted considerable research interest (Cap et al., 2015;Salton et al., 2014b;Anastasiou, 2010).


However, these metrics only provide global evaluation and are unable to evaluate MT systems' performance on specific aspects.

they are unsatisfactory in evaluating idiom translation performance.


Advantages and Disadvantages According to the classification criteria introduced in section 2.2., our blacklist method is a manual-construction, automatic-evaluation test suite method.

the main advantage of the blacklist method is that, after creating the blacklist, large-scale evaluation is inexpensive and reproducible.


Meanwhile, we cannot expect a system to learn to translate idioms with too low frequency.

we selected idioms appearing between 7 and 1000 times in the training data.


Table 4: Example for correctly detected errors.

in the example shown in Table 4, the word "three" is the literal translation of 三, but should not appear in the correct idiomatic translation the occurrence of "three" in the translation triggers our blacklist, correctly indicating a literal translation error.


The goal of this research was to create a tool that is able to assign semantic features to words, even if the target word is not included in any semantic lexicon or if such a lexicon does not even exist in the given language.

two problems had to be handled: assigning features and, if needed, bridging the language gap.


In the names section of the  : educate institution science group study student degree society *sleeve @Catholic_Church 4L2: educate study institution science knowledge group religion *system job HAS.purpose RL: College School_Adj Education Occupations School Table 5: Examples of features returned for proper names and abbreviations of names of institutions from the models that each person is assigned features that provide information about them.

the model can be queried even for names one is not familiar with, and relevant features will be provided.


In the new version, we focus on one subevent as being the cause of another.

something an agent does (e.g., do(e 2 , Agent)) causes a state change or another event (e.g., motion(e 3 , Theme)), which would be indicated with cause(e 2 , e 3 ).


Events that include some sort of change from one location to another or one state to another compose the majority of classes in VerbNet and include some of the more complex event types.

we will use examples from the change of state and change of location classes to illustrate the new VerbNet representations.


In their experiments, they find that semantic attributes are more conducive to predicting the similarity of fonts than geometrical features.

making use of a set of semantic attributes, they learn a font similarity metric based on crowdsourced comparisons, in which users need to assess which of two presented fonts is more similar to a provided reference font.


Hence, we first define µ(e, a) = 1 − a if a is assessed as an antonym of e a otherwise, where 1 is an |F|-dimensional vector of ones.

for those words that are assessed as antonyms, we do not use the regular font vector a, but instead consider an inverted vector, in which we subtract each value from the maximum value of 1.


To obtain font vectors e for emotions e ∈ E, we compute where the a i are the k most similar attributes, as described above.

the font vectors are a weighted average of the vectors for related attributes, after possibly inverting their respective vectors.


The VA annotation was created in a linguistic research context, but not as a building block of an NLP pipeline.

parts of the case semantic information are not encoded explicitely, but need to be supplemented by a human reader.


The original motivation for the annotation was its use in personal linguistic research.

(1) the used inventory of relation types was independently developed and does not follow a standard dependency-grammar or role-semantic annotation schema, and (2) the labelling is selective, i.e., relations that are evident and can be easily supplemented by the reader are not labelled.


4. and 5., the verb-argument annotation is selective.

we designed a basic argument identification algorithm that supports the re-annotation of non-oblique cases.


However, performing reliable dialect identification to properly route text to the appropriate POS tagger may be problematic, because conventional dialectal identification may lead to results lower than 90% (Darwish et al., 2014).

building a POS tagger that performs reliably across multiple dialects without the need for dialect identification is desirable.


Each relation is annotated by coding a relation-type and an antecedent for each utterance after the first in a transaction.

each transaction can be viewed as a tree structure, with the first utterance as root (having no relation-type or antecedent annotations).


For translations, there is one subtype for each source and target floor combination.

a dialogue with two floors would have two translate relations, while one with three floors would have up to six.


In human-human interaction, the likability of the social interaction partner can be judged within seconds and this judgment has been found to be a strong predictor for interaction with a robot (Clark and Rutter, 1985;Robbins and DeNisi, 1994).

besides trust likability of the robot acting as a social interaction partner (Bartneck et al., 2008) seemed to be worthwhile considering in our study as a dependent variable.


The three-way interaction of all independent variables did not yield promising results (all p > .05).

it is not further considered in the following.


Additionally, likability and acceptance do not seem to interdepend gender-personality wise.

robots of all genders are accepted but may show differences in likability depending on the personality.


Furthermore, a more sophisticated approach for modeling the female domain is necessary.

the future participants should not work on scripted tasks, but make their own decisions in a more realistic environment.


In speech research community, utterance definition is a slightly different; it refers to a complete unit of speech bounded by the speaker's silence while we refer to the complete unit of speech as a turn.

a single turn can be composed of many utterances.


During annotation process, it is being noted that; the Egyptian turns are almost long and contains many utterances as noticed during data collection.

turn segmentation into utterances for Egyptian Arabic dialogues model namely 'USeg' (Elmadany et al., 2015d) has been built, which a machine learning approach based on context without relying on punctuation, text diacritization or lexical cues.


Within the field of computational linguistics, recent work, closely linked to the development and deployment of spoken language dialogue systems, has focused on some of the conversational roles such acts can perform.

dialogue act recognition is considered an important component of most spoken dialogue systems.


The proposed approach is mainly constructed using two models, one for each layer.

we think that our approach model is faster than binary classification and it can be more an efficient dialogue act classification model in real-time systems.


For instance, the act "Service-Question" is related to the customer because he connected to service support service to asking about a provided service, but the act "Other-Question" and "Choice-Question" are related to operator because the operator asking the client about his name or choosing the client to select one of the provided services.

the sentential features represent the relationship between the dialogue acts and the surface utterances.


It is therefore expressly stated that identifiability may change over timedata that do not allow to identify the data subject today, may identify him tomorrow.

it is necessary to periodically review the results of anonymization processes.


This is particularly important to ensure that the principle of data minimization (see above) is respected.

the data controller is obliged to implement "data protection by design and by default" (art.


While the primary concern of this section is the estimation and coverage of financial costs of processing, it should also "clearly identify responsibilities for data management in [the] project" (H2020, 2016).

this is where the data controller should be clearly identified (the entity that defines the means and purposes of data processing), and data processors (entities processing the data on behalf of the controller) should be named.


In this process, researchers are supported by the staff of the library and computing centre.

a Submission Information Package (SIP) is created that contains all research data and its metadata in the EAD/METS format (Encoded Archival Description / Metadata Encoding Transmission Standard), see [U3].


Readers who managed to migrate from, say the Fedora 3 repository system to its Fedora 4 successor will be aware of the many technical subtleties and intricacies involved, even if such migration is taking place within a single institution.

1 many design and migration decisions may well differ across institutions, which makes it hard to generalize.


For example, an article maybe older but has fewer views than a newer 11 https://en.wikipedia.org/wiki/Category: Wikipedia_articles_with_possible_conflicts_ of_interest 12 https://en.wikipedia.org/wiki/Wikipedia: article OR articles from a particular category may have more views than other categories.

we chose the random selection approach as long as the article was from the same category as a CoI article and did not belong to CoI disputes category.


Psychologists have argued that some emotions are more basic than others (Ekman, 1992;Plutchik, 1980;Frijda, 1988;Parrot, 2001).

3 most work on capturing wordemotion associations has focused on a handful of emotions, especially since manually annotating for a large number of emotions is arduous.


Thus, emotion analysis of tweets is particularly compelling.

apart from common English terms, we also chose to annotate terms common in tweets.


Since most words do not convey a particular emotion to a marked degree, annotating all words for all emotions is sub-optimal.

for each of the eight emotions, we created separate lists of terms that satisfied either one of the two properties listed below: • The word is already known to be associated with the emotion (although the intensity of emotion it conveys is unknown).


Thus, a large amount of representationally incompatible emotion lexicons has been developed by various research groups adopting one or the other emotion representation format.

the reusability of these resources decreases as does the comparability of systems using them.


Conversely, values for BE5 categories range in the interval [1,5] where '1' means "absence" and '5' means "most extreme" expression of the respective emotion.

1 the VAD and BE5 formats are conceptually different from one another insofar as VAD dimensions are bi-polar, whereas BE5 categories are uni-polar.


To our surprise, all of them performed equally well (with only negligible differences).


Concerning the other affective dimensions and categories, we are not aware of any other system predicting numerical scores for them.

we will restrict our comparison to Valence and Arousal.


In the two crosslingual set-ups (training and test data drawn from different languages, respectively data sets), we make use of the fact that our models do not rely on any languagespecific information since the categories/dimensions describe (supposedly universal) affective states rather than linguistic entities.

models trained on one language could, in theory, be applied to another without any adaptation.


Compared to the monolingual set-up (relative to the target language), the drop of the average performance amounts to only a few percentage points (< 5 in most cases).

the predictions for BE5 are much more robust compared to the VAD predictions which might be an effect of the respective source representation.


Concerning the average over all emotions, the combination of Polish and German, once again, performed worst (r = .755), whereas the combination of English and Spanish worked best (r = .794; i.e., for testing on English, Spanish was used for training and vice versa, while for testing on the remaining languages, training was done on English and Spanish).

this combination of gold data was used for creating novel lexicons in the crosslingual set-up (see Section 5.).


Team members in highly-demanding operational tasks do not often notice triggers that cause them to be emotionally and mentally stressed (Murphy, 1996;Stein, 2001).

they might communicate with their teammates and express their emotions in ways that may not be noticed in observable audio-visual cues.


In addition to knowing a fine-grained score indicating degree of intensity, it is also useful to qualitatively ground the information on whether the intensity is high, medium, low, etc.

we manually identify ranges in intensity scores that correspond to these coarse classes.


In 100 randomly chosen tweets from each emotion set (joy, anger, fear, and sadness), we removed the trailing query term (emotion-word hashtag, emoticon, or emoji) so that our dataset also includes some tweets with no clearly emotion-indicative terms.

the EI-reg dataset included 1400 new tweets for each of the four emotions.


Our eventual goal is to study how valence, arousal, and dominance (VAD) are related to joy, fear, sadness, and anger intensity.

we created a single common dataset to be annotated for valence, arousal, and dominance, such that it includes tweets from the EI-reg datasets as described below.


To study valence in sarcastic tweets, we also included 200 tweets that had hashtags #sarcastic, #sarcasm, #irony, or #ironic (tweets that are likely to be sarcastic).

the Vreg set included 2,600 tweets in total.


It has been shown that non-native language and translations are closer to each other than they are to the native language (Rabinovich et al., 2016).

this difference in accuracy cannot necessarily be attributed to the embeddings but more likely to the input labels.


However, no free corpus has been released for the similar, varieties or dialects in Mandarin Chinese.

in this paper, we focus on the corpus building and its computational model design for the closely related parallel monolingual Gan Chinese languages.


It is well know that pronunciation is vital for any language.

we annotate Chinese Pinyin into our corpus.


Compared with English, no space exists between words in Chinese sentence.

we use character uni-grams, bi-grams and tri-grams in Chinese Pinyin as features.


The picture postcards often contained private information, such as the name and address of the receiver, the telephone number or even the bank account number of the sender.

the transcribers did not include such sensitive information but coded as [Vertraulich] (i.e.


We observed that frequently used verbs, such as give, be and have often occurred with the reduced pronoun s without an apostrophe.

we created a list of these verbs and some wh question words in order to separate s from them.


Furthermore, it contains fewer orthographical deviations.

we experimented with features and training data to determine the best method for optimising the accuracy of the tagger applied to the postcard text in this study.


However, the wider context window size slightly improved the accuracy of the test set of TüBa.

we conducted further experiments using the combination of the feature sets (A), (B) and (C) in context windows 0 to 5.


However, the selected sentences did not include in-domain interpersonal and fragmental sentence patterns typically used in private communication.

we did not find any sentences whose subject was in the first or second person, such as Danke für Deine Karte.


Although there are standards with regard to the classification of scientific fields (UNESCO, 1978), these standards do not seem to have gained acceptance when it comes to compile corpora of academic discourse.

the French SCIENTEXT corpus is divided in ten scientific domains (Tutin and Kraif, 2016), Coxhead (2000) distinguishes four big scientific areas (Arts, Commerce, Law, and Science) in turn subdivided into 27 subareas, Paquot (2010) draws a big divide between hard and soft sciences and further breaks down the former into four sections and the latter into six, etc.


Predictably, not all four categories will be equally productive in this respect.

nouns are much more interesting than the other three categories as collocational bases.


Then, we manually validated sentence segmentation in both languages in order to create a reference corpus that may be used to train and evaluate sentence segmentation tools.

updated versions of the corpus reflect the manual sentence segmentation.


Semantically idiosyncratic MWEs, or idiomatic expressions, are non-compositional in that their meanings cannot be predicted from their parts; these expressions are used frequently to make language more fluent (Jackendoff, 1997), and often contain word senses not found in other contexts.

identifying non-compositional MWEs presents a clear challenge for fields such as automatic machine translation (MT), information retrieval, natural language understanding, natural language generation, question answering, text summarisation, and word sense disambiguation (McCarthy et al., 2007).


The isomorphicity assumption built into TectoMT generally works well, but is problematic when the source language contains non-compositional MWEs.

we integrate lexical information about MWEs in the form of our compositionality-ranked list into the TectoMT pipeline, by collapsing multiple t-nodes in a t-tree which represent a single MWE into a single composite t-node.


Many previous studies annotate "events" that express situations that happen or occur, which are defined in the guideline of TimeML (Sauri et al., 2006).

ex-pressions as in the following example are not annotated.


The Foreign Minister has laid out the basic policy of "privatization and re-nationalization."

the major cause of the disagreements among the annotators is that there are multiple interpretations depending on the context and common sense, closely related to the writing style and theme of newspaper.


Another ambitious project is the PFC corpus, which aims at gathering data of spoken French all over the world, using the same methodology.

the PFC project is based on spoken tasks only (there are no written tasks), with reading tasks (words, short texts) and also spontaneous speech (mainly interviews).


The /word tier creates an oejeol unit, which can be a lexical word or a combination of a lexical word with a grammatical morpheme in Korean grammar.

when two lexical words are aligned as one unit in the first interval tier, the /word boundary of the two tiers between the words should be corrected also.


However, academic shared tasks have a very specific scenario, which does not necessarily match the real-world settings in which a system like FreeLing is required to operate.

considerable efforts must be devoted to the integration of a module successful in the laboratory, such as Re-laxCor, in a production pipeline.


The above mentioned suites largely differ with respect to the used programming language, offered APIs, processing speed, supported languages, customization or retraining capabilities, whether they are more developer-oriented or end-user oriented, etc.

a detailed comparison is out of the scope of this paper.


One option would be retraining the module using a corpus adapted to the new criteria, which would require a costly re-annotation effort.

we opted by developing a set of hand written constraints, inspired on the approach proposed in (Lee et al., 2013), the winner system in CoNLL-2011, that defines ten sieves of decreasing precision rules, applied in a cascaded schema.


This has been found to be quicker and easier than the building of chains via the schema-like strategy and is the method currently used in the Democrat project (Landragin et al., 2017).

the tool used to annotate must only allow marking referring expressions and recording referent names.


Input features described above are concatenated into a single vector and act as input to our neural network.

the network takes an input vector of 1147 units and is passed through a fully connected network with a single output (a value between 0 and 1).


Dominance of negative examples over positive is a typical situation in real texts, where most pairs are not coreferent.

our new training set should correspond better to a real test scenario.


From the viewpoint of coreference resolution, all this entails that the anaphor might not be present in the sentence as a separate token, only as a zero pronoun (pro).

before the annotation process started, they had had to be inserted into the text.


However, this evaluation metric does not align with our goal of ranking antecedents for a given target mention.

we evaluate the performance by Mean Average Precision (MAP), Precision at k (P @k), Recall at k (R@k).


The cluster-ranking model has a very complex structure and modifying it to take into account the merging features was impractical, such that we decided to leave this as future work.

we decided to create a new threedimensional vector, one dimension for each semantic feature: i) if the entities of the two mentions are the same, ii) if the type of the two mentions are the same, and iii) if the type of one mention is included into the hierarchy of the other one.


However, the de facto standard in the industry is to use sizeable training corpora, which total durations are measured in hundreds or even thousands of hours (Amodei and others, December 2015; Xiong and others, January 2017).

we expect to achieve better model generalisation and word error rates (WER) with more data, and thus aim in training SARMATA on at least 1000 hours of speech.


Current version of the tool provides a user-friendly approach towards recording continuous speech.

of our work, we were able to obtain 63 hours of quality recordings, which we use both for training of the acoustic models and for ASR evaluation.


They also converted text in the common language into their dialect, and read the dialect text in the web-based recording platform available on personal computers and smartphones.

the resultant resources consist of texts (one common language and 21 dialects), nine hours of speech data, and their geographic contexts (i.e., areas of the dialect).


For this model to make sense, h needs to be an S-shaped curve so that the estimated probability of a reader knowing a word increases monotonically as the difference between the annotator's level (denoted L a ) and the word's complexity (denoted C w ) increases (see Figure 6).

we choose to define h as follows: where a is the annotator, w is the word, and φ is the sigmoid function φ(x) = 1/(1 + e −x ).


Based on existing data (timestamps of users' actions) on the platform, we can approximate the average time spent by one of the experts on each information: 23.5 seconds for marking, and 13 seconds for transcription.

an expert would spent approximately 26 minutes per page.


Figure 5 illustrates the issue caused by displaying direct access to documents.

we added a condition on the workflow based on an arbitrary  order.


Additionally to the dialectal variation and although the Orthal spelling guidelines (Crévenat-Werner and Zeidler, 2008) have existed since 2006, no orthographic convention is recognized yet as the legitimate standard among the speakers.

elided forms such as m'r for mir ("we"), or d' for die ("the") coexist depending on the spelling habits of the writer and independently of the variant.


Our simplification is aimed at vocabulary compression of sentences as we usually use everyday.

we do not consider other simplification parameters such as difficult grammar and sentence length (E.g.


It is many of the sentence pairs have been derived from textbooks.

we consider that many the sentences of the daily conversation level are included.


These data sets do not cover all the languages we are interested in and, additionally, do not target the educational domain.


The number of words can be different in the two synsets.

the same synset will be assigned to several workers who may end up suggesting many candidate words for the target synset.


Vocabulary tests as a high-stakes test are common as placement tests, but are not common in language-support systems because users know that they may be provided with incorrect support if they pretend.

our dataset is more similar to a realistic environment in which language learners do not have motivation to exaggerate their knowledge of second language vocabulary.


For example, an elementary school math test cannot reliably measure the abilities of computer-science math students.

the informative-ness of the test results can be given as a function of the ability of test takers.


To predict the parameters from the word embeddings, we applied support vector regression (SVR) with a linear and radial basis function (RBF) kernel.

the difficulty parameters were predicted with a root mean squared error (RMSE) of 3.632, and the discrimination parameters were predicted with an RMSE of 1.020 using SVR with a linear kernel.


As a result, the quality of information in social media has become a primary concern for many social media oriented tasks, such as rumor detection (Qazvinian et al., 2011) and credibility evaluations (Jaworski et al., 2014).

uncertainty identification is significant for users to synthesize information to derive reliable interpretation.


Extraction of such justifications and assessment of their validity would be necessary to conclude on absence or presence of spin, but it falls outside scope of our work.

there are several difficulties that we should take into account when developing annotation guidelines: 1) Some of the tasks require at least some level of special medical knowledge, so it is likely that the annotators will not be linguists and will not have experience in corpus creation/annotation.


However, demonstration of text annotation with Glozz to a medical expert showed that it does not meet the requirements of non-linguist annotators: ease of installation of the tool, amount of time needed for training for a person without previous experience in corpus annotation, complexity of guidelines describing the task.

we decided to replace the task of linguistic annotation of texts by a set of simpler tasks (in the form of questions) such as the following: -validation/correction of primary outcomes found at the pre-annotation stage; -validation/correction of reported outcomes found at the pre-annotation stage; -establishing if two given (extracted at previous stages) outcomes refer to the same concept; -identification of similarity statements in the Results and Conclusions of abstracts; -identification of within-group comparisons in the Results and Conclusions of abstracts; -identification of other positive evaluation of the studied treatment in the Results and Conclusions of abstracts.


Our future tasks include running a pilot survey to validate usability of survey format for our task and evaluate the adequacy and clarity of the questions for annotators.

we will proceed to a full-scale survey project.


Yarowsky (1995); Yarowsky (1993) show that the trend for words to exhibit only one sense in a collocation is much less pronounced in collocations with function words.

in his context it was also a crucial feature of word pair relations whether they were function or content words.


The choice was motivated not only by the fact that these guidelines are very well-defined and detailed, but also by the fact that in Italian, as in German, it is also possible to contract articles and prepositions.

the Italian guidelines are a more natural choice than the English ones when adapting annotation guidelines to German.


In the FREME terminology, such services are called e-Service.s Every e-Service in FREME uses NIF or plaintext as input format and produces NIF as output format.

it is possible to feed the output of one e-Service in the next e-Service to form pipelines.


User level 2 usually has no expertise in using LD and SPARQL.

these technologies are often considered challenging.


This is especially true for technically less-skilled users, where a complex system is likely to act more as deterrent than encouragement.

it is not uncommon for researchers to document their workflows manually by means of a local Word or Excel file.


In parallel, researchers looked at ways to combine knowledge graphs of this kind with distributional semantics to include human-readable meaning representations based on semantic vector spaces within the Semantic Web ecosystem (Faralli et al., 2016).

in this paper we bring these two lines of research together and present a new resource that combines a wide-coverage symbolic repository of frames with humanreadable distributional semantic representations from text.


In this paper, we bridge the gap between distributional and frame semantics by linking distributional semantic representations to Framester, a knowledge graph that acts as a hub between resources like FrameNet, BabelNet and DBpedia, among others.

of this, we introduce a new lexical resource that enriches the Framester knowledge graph with distributional features extracted from text, and show how this hybrid resource yields better results on the task of recognizing frames in running text.


Negation is one of the most central linguistic phenomena.

negation modeling is essential to various common tasks in natural language processing, such as relation extraction (Sanchez-Graillet and Poesio, 2007), recognition of textual entailment (Harabagiu et al., 2006) and particularly sentiment analysis (Wiegand et al., 2010).


the loved ones still remain to be loved.

in this sentence the verb does not function as a shifter despite carrying an intrinsic polarity.


(Sentences contain considerable fewer words than larger units of texts which are usually considered for text classification, such as paragraphs or documents.)

we examine different types of word generalization methods.


we count the number of polar expressions in a sentence to be classified.


Based on our definition of support-based entailment, a specific premise supports a more generalised premise.

we ignore rules based on subsumption relation that look into hypothesis containing non-related premises.


Moreover, implicit opinions (text) cannot have any inter-subsumption relation and hence three of those combinations are ruled out.

we have a total of six different rules based on each of inclusion and subsumption.


The challenges of this task can be summarized in the following points: • There is a large variety of expressions to denote a range of sentiments.

a sentiment dictionary, no matter how large it could be, cannot list all the possible ways people can express their attitudes.


Of interest is also the quality of the text in the corpus and its applicability to other NLP tasks.

in this section we evaluate error in the raw corpus and provide comparison with other corpora used to construct sarcasm datasets.


In the most general case, we use the provided raw files to construct datapoints for systems to learn the following task: given a post and a sequence of comments, determine which comments among the responses to the last comment in the sequence are sarcastic.

each datapoint consists of a conversation thread followed by a series of responses and sarcasm labels.


Even humans will struggle with sarcastic comments drawn from unfamiliar topics, for instance, obscure hobbies or art forms.

we also test human and machine performance on comments drawn solely from the politics subreddit, a topic for which all evaluators had sufficient background information.


For example, the bold words "morning", "enjoy", "woke", "son", and "great" (excluding "went"), which are words we use in our positive examples, appear highly among the top 20 words.

we experimented with an instruction set without the positive examples (I 2 ) and collected another 300 happy moments.


Booking.com allows you to search for reviews in Catalan, but it does not include Basque.

for Basque we crawled reviews from a number of other websites that allow users to comment on their stay 1 Many of the reviews that we found through crawling are either 1) in Spanish, 2) include a mix of Spanish and the target language, or 3) do not contain any sentiment phrases.


STTS IBK, however, is an extension of the original STTS and not of the TIGER variant.

we automatically replace all instances of the tag PROAV with the original STTS tag PAV.


7 As noted in the previous section, there are no sequences of graphical emoticons in the training data.

sequences of emoticons are frequently misclassified.


Roelli and Macé (2015, p.129) state that their method is: [...] a subcategory of distance-based methods.

the traditional scholarly concept of Leitfehler is taken to be a quantitative one: a variant's usefulness as Leitfehler may be assigned a number or weight.


Such a comparison can be made for all combinations of potential Leitfehler while both Leitfehler in pairs with only three combinations get their score increased.

the algorithm produces a list, where every variant in the textual tradition gets a value that indicates how good a leitfehler it is (obtained through the comparison of the distributions of this variant in conjunction with each other variant).


The reason for this is, that any manuscript m can share t or more % with another manuscript a but at the same time t or more % with yet another manuscript b, where the variants both share with m are not the same ones.


For example, (GU 7 , to eat) is composed of (KA, mouth) and (GAR, bread).

we'll focus on most frequently appeared and probably most basic classes.


When we tackle to OCR from the handcopies, we have to treat that variations.

we decided to include sev-  We firstly derived candidate character images from given resource.


We found 31 classes in which almost all candidate images were (Type-1) ones.

we obtained 118,801 roughly filtered candidate images.


On the other hand, some characters such as IN consist of multi parts which tend to be separated by simple Gausian somoothing based character detection.

it is also needed to detect these "complex" characters as one character precisely.


The candidates are too many to conduct the collection of characters by hand.

it is needed to combine automatic method and manual correction to achive more efficient and precise image collection.


The important and typical glyphs are ones used in Neo-Assyrian and Hittite in addition to the collected glyphs in this paper.

the future collection targets are those two glyphs.


It does not guarantee that the order of the encoded text resembles the order in which it is intended to be read.

extracting text from PDF documents is not a straightforward task.


Our use case is primarily to enable the automatic processing of text extracted from PDFs, with emphasis on text presented tabularly, such as IGT.

it is important to maintain the visual alignment of text elements when projecting coordinate-positioned text in a variable-width font to columns in a monospaced plain-text file.


As speech is the modality used by the surgeon to communicate with their staff, using it to control the technical devices does not pose an additional mental burden.

we claim that the surgical environment presents a potential field of application for Spoken Dialogue Systems.


An intelligent OR also needs an intelligent human-computer interface as the surgeon as well as the whole surgery team must not be disturbed by the usage of complex computer applications and numerous devices.

the interface has to be designed so as to be simple and intuitive (Feußner et al., 2014).


During a procedure, the surgeon needs his hands to operate on the patient and his eyes for being aware of what he is doing.

any graphical and gesture-based systems are not well suited for this purpose.


Moreover, it is the modality used by the surgeon to communicate with their staff.

using speech to control the technical devices does not pose an additional mental burden.


The surgeon can focus on the surgery and control the technical environment at the same time without taking care of how to interact with the system.


For keeping track of the procedure and automatically controlling surgical devices, the system needs to know when to perform which action on which device and when to stay in the background.

it has to be aware of the whole context of the surgery, i.e.


Keeping track of the surgery is thereby done by tracking the tool usage as described by Padoy (2010).

we introduced variables for all kinds of instruments which are used and all assistance actions which are performed during this specific surgery.


Furthermore, the IDACO Database provides an interface in order to control the surgical devices in the OR.

the database contains a list of all existing devices as well as the corresponding device parameters.


Then, the data converter serves as interface to the specific device protocols of the peripheral an installed devices and changes the current parameters of the devices accord-ingly.

iDACO is able to automatically control several devices at the right time of the procedure.


Speaker P2 agrees with P1, but does not show any sympathy with the fact said by P1.


The current system extracts the features from the previous (long) utterance, but most of them are irrelevant to sympathy identification.

too many irrelevant features cause classification errors.


This approach comes with two disadvantages: first, in small corpora sentences might only occur rarely in exactly the same wording.

only little context information is available for each sentence.


A good vector representation of dialogue utterances implicitly contains the information about dialogue acts.

the clusters constituted by DVMs achieve a comparable accuracy in grouping utterances according to the dialogue act.


Identifying the second illocutionary act and reacting accordingly is essential in a cooperative dialogue.

a lot of research goes into the automatic detection of user intent (e.g.


Despite having a multichannel customer service and also providing support through web forms and email, the majority of the clients still prefer calling.

the corpus provided consisted of raw audio recordings of such calls.


Last but not least, since speeches are held in public and part of official debate, they are very often not subject to copyright issues.

their copyright status makes them highly relevant for replication studies as well as a wide range of purposes.


Documents from four different archives were used: Gerhard Schröder's terms (1998)(1999)(2000)(2001)(2002)(2003)(2004)(2005), Angela Merkel's 1st (2005Merkel's 1st ( -2009Merkel's 1st ( ), 2nd (2009Merkel's 1st ( -2013, and 3rd term (2013-2017).

the online archives are not homogeneous, there was no real classification for texts anterior to 2005, where a few unrelated speeches from other politicians are to be found.


Nominal and prepositional phrases are in focus, the results are often comparable to text chunks, but the approach is closer to grammatical rules and to the linguistic understanding of a phrase.

the potential keywords are head nouns of nominal and prepositional phrases, they are then counted and filtered using term frequencies and inverse document frequency (TF-IDF).


Each Thiessen polygon defines an area of influence around its sample point, so that any location inside the polygon (Euclidian distance-wise) is closer to that point than any of the other sample points.

we no longer utilize the points to represent the language data, but instead the derived discrete polygons.


Our approach also lacks a straightforward way to encode language relatedness beyond using point symbology or adding genealogical markers, such as coloring or border effects to the Thiessen polygon layer proposed here.

the problem of visualizing genealogical relatedness and areal contact between languages on a global scale is an area that needs more research.


Preparing corpora of plenary protocols is certainly an obvious idea.

corpora of plenary protocols are not new in computational linguistics (Koehn, 2005) (Vinokourov et al., 2003).


10 The German Bundestag is an important parliament, but it is one parliament among others.

a workflow needs to be designed that is sufficiently generic to allow for a simple preparation of a corpus from any plenary protocol.


For many users in the humanities and the social sciences, XML files in general and TEI files in particular will not be overly accessible: The technical barriers to entry may be considerable for the greatest potential user group of the corpus.

the GermaParl Corpus is also disseminated as an R data package.


However there are barely evaluation sets with large enough amount of data for Chinese word embeddings.

in this paper, we create several evaluation sets for Chinese word embedding on both word similarity task and analogical task via translating some existing popular evaluation sets from English to Chinese.


The two most popular kinds of benchmarks to evaluate word embedding are word similarity task and analogical reasoning task.

we create some benchmarks for Chinese word embedding on both kinds of tasks through translating some existing popular evaluation sets from English to Chinese and remains the scores of the original benchmarks.


Also, we look up the word "脚踏车" in E-Hownet to get its synonyms in Chinese.

(bicycle, bike) is translated as (脚 踏车 (bicycle), 自行车 (bike)).


In this list, some words are similar to the entry, but others (those which are written in bold) have different meanings that should be discarded.

a cleaning process is done on each entry: removing the words (bold examples in Table 6) that are not related to the entry, removing the words (in italic in Table 6) which are similar, but they do not have the exact meaning as the entry and adding the missed words (examples in blue in Table 7).


A true neologism may have a large number of queries concentrated in a recent period of time, whereas very common misspellings would have consistently high queries.

we also include two percent change features: • Over mean: comparing the number of queries of the item this month against its mean monthly queries in past months • Year-over-year: comparing this month's queries against the queries of this item in this month of last year (to account for items of seasonal interest) Furthermore, we include the percentage of months for which we have volume for user query data that the item.


While many languages do not have the same complexities of orthography, they may be influenced by external pressures, so maybe more subject to the adoptions of calques.

our approach may require adjustments to the orthographic features to best suit the given language.


However, the French form is feminine, while the Italian one is masculine, so the correct embedding space should have mapped postale in Italian with postal in French.

the cross-lingual embedding space needs to be built in a way which takes into account the similarity across the full set of forms with respect to their grammatical functions.


Having good language resources in the form of a dictionary would benefit these languages, because they can be used in various tasks such as machine translation (Brown, 1997) and pedagogy (Antonsen;Huhmarniemi & Trosterud, 2009) when teaching the language to non-native speakers.

their impact on the revitalization of endangered languages would be high.


By combining these translations, we can improve the translation coverage of both dictionaries.

of the combination, we will first of all have, more translations for lemmas in the four languages, and new target languages that haven't existed before in the original dictionaries.


Mg is short for meaning group, this means that all the translations under the same meaning group refer to the same semantic concept.

when combining the translations from different dictionaries, it's important to do it based on the meaning groups.


The alignment tools we used do not guarantee full alignment coverage for each word in a sentence.

some target or frame elements are not projected from English to Chinese if no alignment information is provided.


Textbook designers therefore have quite a latitude to decide which topics will be included in their book.

it is likely that the importance of some low frequency words, related to specific topics, will be overestimated using raw frequencies, especially when a topic generally encompasses several texts within the same lesson.


Moreover, only 3,051 entries have a total frequency higher than 10 occurrences per million words, which is a clear limitation of the resource.

conclusions on the real usage of infrequent entries in EFLLex should be drawn with caution.


This means that the word laugh, for example, is assigned the same frequency, whether it be a verb or a noun.

the MRC does not report reliable frequency information for specific token and tag combinations.


The IT-domain training set consists of 89,983 samples, but only 2,000 of them are proper sentences; the rest are translations of IT terms from Wikipedia and localization PO files.

the amount of "good quality" data in the training set is very limited.


This feature is very convenient for phenomena or segments that are more complex.

the database is constantly expanding and covers an increasing amount of possible MT outputs.


Especially the number of segments featuring negative tokens are increasing with every report when new erroneous translations are evaluated.

the database is constantly growing and the more reports are carried out, the less manual work of inspecting segments that are neither covered by a regular expression nor by a positive/negative token is needed.


From these statistics, we can see that the under-translation in NMT is more serious than the overtranslation.

further analysis for the under-translation is made and Table 2 shows the results.


That is to say the words in the similar positions between the source and target sentences are more likely to be translation pairs.

we further enhance the pre-ordering model with the position embedding to encourage monotone translation.


As a comparison, the system only using the position embedding (Row 4) improves the baseline with 0.37 BLEU.

we find an interesting result that when using pre-ordering and position embedding separately, the respective improvement is quite small (0.32 BLEU and 0.37 BLEU, respectively), but using them together can significantly boost the performance (1.13 BLEU), suggesting that pre-ordering and position embedding can enhance each other.


Besides the translation quality, our approaches also aim to reduce the under-translation and over-translation cases in NMT.

we randomly select 500 source sentences and analyze the translation results produced by different systems to evaluate their performances on the undertranslation and over-translation.


Some previous works attribute the problems of the undertranslation and over-translation to the lack of coverage mechanism.

they introduce coverage mechanism to NMT.


Different from the above methods, we treat this problem with another perspective, as we observe that the words need to be reordered during translation are more likely to be ignored by NMT.

we exploit the pre-ordering for NMT to alleviate this problem.


To compare fairly with the multi-source model (2) in Table 1, it is necessary to make the size of a training corpus equal.

we used 450,000 sentences of the synthetic corpus to make 600,000 sentences.


Since Arabic is a rich-morphological language, its performance would be underestimated because nontokenized BLEU evaluates units separated by whitespaces.

in this study, Arabic sentences were evaluated based on the results separated by morphemes.


At the beginning of training, the model is not well-trained and thus selecting more often the true previous token should be helpful; whereas at the end of training the sampling strategy should favor sampling from the model more often, as this corresponds to the true inference situation.

the probability of using y t−1 will be initially high and then decrease during training.


To illustrate, once we decide to use an estimated token coming from the model itself, our oracle can help us select the next suitable word and feed it into the model.

even during training time the truth have been completely altered, our method will always provide the model with the best suitable token in the following time-steps.


If we just use Scheduled Sampling, then at fourth time the model will make a mistake and generate "talks" instead of "border".

the next time if we still choose the original true previous token "talks", the sentence will not be accurate.


Figure 1 visualizes the process of constructing the new training and test sets.

for each language direction, we obtain a targeted test set for idiom translation and the corresponding training corpus representing a natural distribution of sentences with and without idioms.


However, the maintenance of these resources is a very time-consuming and expensive process.

the work described in this contribution aims to automate the maintenance process of such resources.


To further enhance the IATE terminological resource, we believe that the identified disambiguated sentences can be beneficial for the users selecting a term due to the contextual information of the targeted domain.

we append the identified relevant sentences to the IATE RDF resource.


In the case of low-resource languages, there is often little training data with which to train transliteration models.

one major obstacle for low-resource languages is the problem of out of vocabulary words (in the transliteration setting, these would be unknown characters).


However, evaluating distributed representation in languages that do not have such resources (e.g., Japanese) is difficult.

as a first step toward evaluating distributed representations in Japanese, we constructed a Japanese word similarity dataset.


The difference in the similarity of word pairs originates from each annotator's mind, resulting in different scales of a word.

we propose to use an examplebased approach (Table 2) to control the variance of the similarity ratings.


of the pairs during annotation (Table 2).


At each time, we only tag those instances with a i = 1.

by gradually increase λ, we can fetch more less-confident instances for our sense tagging.


In Flickr, each image is annotated with a variety of tags, including the name of a depicted object, the place where the picture is taken, and the emotional feeling expressed by the contributor.

the amount of data made possible by the popularity of Flickr is also a crucial factor; meaningful co-occurrence statistics can be collectively obtained from this huge dataset.


1-1  1,356  1-2  41  2-1  60  1-3  3  3-1 10 1-4 1 4-1 1 1-5 1 1-0 6 :ARG1 (c / carneiro)) Figure 6: Annotation of parallel sentences In addition to the subject omission, there are some other differences in the translation into Portuguese.

the annotation for Portuguese sometimes becomes different from English.


Therefore, we searched and collected Welsh names, including person names and place names, from various sources 6 , including the Language Technologies Unit of Bangor University 7 , UK and the websites of "Behind The Name" 8 , "Think Baby Names" 9 , and "Wales UK" 10 .

we collected 6,553 Welsh names to expand the semantic lexicon.


For instance, the CyTag POS tagset has 59 fine-grained sub-categories for Welsh verbs.

the mapping of the tagsets is not straightforward for some POS categories such as abbreviation and foreign words etc.


CyTag produced 13,220 words (excluding punctuations) of which 3,716 (28.11%) are function words; WNLT produced 14,435 words (excluding punctuations) of which 5,314 words (36.81%) are function words.

the text coverages are based on different word numbers, but they are comparable in terms of tool performance.


Selecting the most frequent verbs and nouns from the 1.3M words CALOR corpus as frame triggers is a guarantee that the average number of occurrences per frame is high.

even if the list of frames annotated in the CALOR corpus is small compared to the Framenet corpus, we have a large variety of occurrences for each of them, allowing us to build robust parsers for encyclopedic texts, which is the goal of the CALOR corpus.


This approach has the advantage of keeping the number of possible labels to predict for each CRF relatively small, limited to the frames that can be triggered by the word considered.

the ambiguity is limited and CRFs can be trained efficiently even with a large number of features.


When the number of surrounding words was smaller than the window size, we used a zero vector.

the dimensionality of the surrounding word vectors was 800 when they were created using only the word embeddings, and 1,000 when both the word and concept embeddings were used.


In order to provide the overall value of ARI, we follow the addition approach used in (Lopukhin et al., 2017).

since the quality measure is computed for each lemma individually, the total value is a weighted sum, namely where w is the lemma, I(w) is the set of the instances for the lemma w, ARI w is the adjusted Rand index computed for the lemma w. the contribution of each lemma to the total score is proportional to the number of instances of this lemma.


The primary reason for that is the implicit handling of similar words due to the averaging of dense word vectors for semantically related words.

we recommend using the dense approach in further studies.


It is for this purpose that the SemCor (Miller et al., 1993) was annotated.

the senses in WordNet are, since version 1.7, sorted by this distribution of senses estimated on the SemCor.


English, with more than 2 million words sense annotated ranks first, before Dutch with nearly 300,000 annotations and Bulgarian with 100,000 annotations.

it is unsurprising that most of researches in WSD focus on English.


For example, looking at the data from the SemEval 2007 campaign (Navigli et al., 2007), which most of the recent systems were evaluated on, we observe that systems that did not use sense annotated data obtain a precision score up to 78 − 79% 2 (Schwab et al., 2013) (Chen et al., 2014) whereas those which use a lot of annotated data reach a score up to 82% (Chan et al., 2007) (Navigli, 2012) (Vial et al., 2016) and even 84% (Yuan et al., 2016).

having all existing corpora in a unique format and using the same sense inventory offers several advantages: it allows to easily expand the quantity of data available for improving WSD systems, it allows to better estimate the distribution of senses in English, and finally, this format can help creating more robust WSD systems.


The merging phase identifies identical sentences with annotations on different words, and creates a single sentence containing all annotations.

this steps adds a crucial information for some WSD systems.


D(s(w k )), every other words of the sentence, i.e.

w i ∀i ∈ [0, n]i = k. every sense's definition in the dictionary will be extended with words that are related to this sense, in the same manner than (Banerjee and Pedersen, 2002)'s extended Lesk, but with words taken from sense annotated corpora.


Then, using a Shapiro-Wilk test (Shapiro and Wilk, 1965), we determined that none of the result distribution follow a normal distribution.

we used a non-parametric Wilcoxon/Mann-Whitney-U (Wilcoxon, 1945) (Mann and Whitney, 1947) test in order to check the pairwise significance (p < 0.01) of all pairs of result distributions.


As we can see, our expansion of the definitions with words taken from the UFSAC corpora improves considerably the original Lesk measure, even more than the Extended Lesk measure.

this experiment demonstrates how much the addition of the UFSAC resource can improve a similarity-based WSD sytem.


In addition, the formats of these corpora are often very different from each other: different file hierarchy, different syntax, and different sense inventory are used.

most WSD systems are trained and evaluated on few corpora comparing to the amount of existing corpora.


Arrows add dynamism to images that portray how certain processes stem from others and how they affect one another.

arrows as visual knowledge patterns (VKPs) most often convey meronymy in the case of entities (part_of) and (phase_of), and the result_of relation in the case of processes.


At the moment, the function of each VKP is a free-text box.

each annotator can freely describe the function the VKP represents in this image.


The automatic identification of hand gestural units from videos has been addressed by numerous studies, but identifying gestures from all kinds of videos is still not possible, and for this reason many studies use tracking devices or pose particular restrictions to lighting and settings of the videos (Keskin et al., 2011).

there is still a need to identify the function of gestures automatically, and distinguishing their semiotic type is a first step toward their interpretation which should also include the context in which the gestures are performed.