Definition: In this task, you are given a part of an article. Your task is to generate headline (title) for this text. Preferred headlines are under fifteen words.
Input: Sequence model learning algorithms typically maximize log-likelihood minus the norm of the model (or minimize Hamming loss + norm). In cross-lingual part-ofspeech (POS) tagging, our target language training data consists of sequences of sentences with word-by-word labels projected from translations in k languages for which we have labeled data, via word alignments. Our training data is therefore very noisy, and if Rademacher complexity is high, learning algorithms are prone to overfit. Norm-based regularization assumes a constant width and zero mean prior. We instead propose to use the k source language models to estimate the parameters of a Gaussian prior for learning new POS taggers. This leads to significantly better performance in multi-source transfer set-ups. We also present a drop-out version that injects (empirical) Gaussian noise during online learning. Finally, we note that using empirical Gaussian priors leads to much lower Rademacher complexity, and is superior to optimally weighted model interpolation. 1 Cross-lingual transfer learning of sequence models The people of the world speak about 6,900 different languages. Open-source off-the-shelf natural language processing (NLP) toolboxes like OpenNLP1 and CoreNLP2 cover only 6–7 languages, and we have sufficient labeled training data for inducing models for about 20–30 languages. In other words, supervised sequence learning algorithms are not sufficient to induce POS models for but a small minority of the world’s languages. What can we do for all the languages for which no training data is available? Unsupervised POS induction algorithms have methodological problems (in-sample evaluation, community-wide hyperparameter tuning, etc.), and performance is prohibitive of downstream applications. Some work on unsupervised POS tagging has assumed other resources such as tag dictionaries [Li et al., 2012], but such resources are also only available for a limited number of languages. In our experiments, we assume that no training data or tag dictionaries are available. Our only assumption is a bit of text translated into multiple languages, specifically, fragments of the Bible. We will use Bible data for annotation projection, as well as for learning cross-lingual word embeddings (§3). Unsupervised learning with typologically informed priors [Naseem et al., 2010] is an interesting approach to unsupervised POS induction that is more applicable to low-resource languages. Our work is related to this work, but we learn informed priors rather than stipulate them and combine these priors with annotation projection (learning from noisy labels) rather than unsupervised learning. https://opennlp.apache.org/ http://nlp.stanford.edu/software/corenlp.shtml

Empirical Gaussian priors for cross-lingual transfer learning